Investor emotion index verification method based on machine learning

Abstract

The invention discloses an investor emotion index verification method based on machine learning, and particularly relates to the technical field of emotion index verification. Multi-source stock related texts from social platforms, financial news, brokerage reports and the like are collected, vectorization processing is performed by using a semantic representation model, and emotion recognition accuracy is improved by combining a multi-level fusion label generation mechanism and a weak supervision classification model with an attention mechanism. Dividing the annotation data into a training set, a verification set and a test set to construct a steady emotion classification model; and finally, a daily investor emotion index is constructed based on model output, the effectiveness of the daily investor emotion index is verified through the correlation with market variables and the predictive ability, a label system and model parameters are iteratively optimized based on a verification result, a sustainable emotional analysis closed loop is formed, and the emotion recognition precision and the explanatory power of the index are remarkably improved through the method.

Description

Technical Field

[0001] The present invention relates to the technical field of sentiment index verification, and in particular to an investor sentiment index verification method based on machine learning. Background Art

[0002] In the study of investor sentiment measurement, traditional methods mostly rely on structured data and classical econometric models, which are difficult to fully capture the nonlinear characteristics and complex semantic relationships implicit in investor sentiment. With the development of natural language processing (NLP) and machine learning technology, researchers have begun to try to use large-scale text data to construct investor sentiment indicators, and use machine learning methods to improve the accuracy and timeliness of measurement. Machine learning models (such as deep neural networks DNN, convolutional neural networks CNN and recurrent neural networks RNN) can automatically extract high-dimensional semantic features from unstructured texts, and perform well in tasks such as investor sentiment classification and sentiment intensity judgment. Compared with traditional regression analysis, the verification paradigm in machine learning (such as training set-validation set-test set division, cross-validation, etc.) is more helpful to avoid overfitting and improve the generalization ability of the model. Introducing such methods into the construction and verification process of investor sentiment indicators can not only improve the model's explanatory power for market variables, but also provide a new paradigm for empirical financial research.

[0003] The prior art has the following deficiencies: In China's A-share market, due to the complexity of information release entities, diverse language expressions, and the tendency of emotional expressions to be obscure, there is a lack of unified and authoritative labeling standards. Research often relies on keyword dictionaries, rule engines, or social feedback sentiment libraries to automatically label sentiment tags. However, this weakly supervised labeling method is very likely to misjudge sentiment words (such as "limit up" may be used in ironic situations) or ignore contextual irony and reverse emotional expressions, resulting in repeated reinforcement of wrong labels in model training, which in turn causes the model to continuously deviate from the emotional judgment in a specific context. Taking a certain financial social platform as an example, users often use slang or emoticons to convey complex investment psychology, but the existing NLP model cannot accurately capture such unstructured emotional signals, resulting in a serious deviation between actual market behavior and model predictions, further affecting the application effect of sentiment indicators in quantitative trading strategies or market early warning systems. This problem not only weakens the predictive ability of investor sentiment indicators, but also poses a potential risk to emotion-based decision-making systems. Summary of the invention

[0004] The purpose of the present invention is to provide an investor sentiment indicator verification method based on machine learning to address the deficiencies in the background technology.

[0005] In order to achieve the above object, the present invention provides the following technical solution: an investor sentiment index verification method based on machine learning, comprising: Collect unstructured text data from stock-related content from several data sources and vectorize the text based on a semantic representation model; A multi-level fusion label generation mechanism is used to generate initial labels and a weakly supervised sentiment classification model with an attention mechanism is used to optimize label accuracy. The labeled dataset is divided into training set, validation set and test set, which are used to train the deep learning model, adjust the hyperparameters and finally verify the model performance. Based on the sentiment classification results output by the deep learning model, a daily investor sentiment index is constructed to verify the effectiveness of the constructed sentiment indicator. Based on the verification results, the sentiment labeling system and deep learning model parameters are iteratively updated.

[0006] Preferably, the attention mechanism and weakly supervised learning model are used to optimize the training of labels, specifically including: Construct training data set: input is structured text corpus and labels generated by multi-level fusion mechanism; adopt Bi-LSTM or Transformer structure with attention mechanism, the model structure includes: embedding layer: convert text into word vector sequence; encoding layer: use Bi-LSTM or Transformer to capture contextual semantic relationship; attention layer: assign attention weight to each word to improve the model's attention to emotional words; output layer: predict emotional category and confidence distribution; Manually annotated standard data is used as the validation set to correct the labels during the model training process. Noise label regularization terms are added to the model training to reduce the negative impact of incorrect labels. The final output emotion labels are optimized and corrected according to the model confidence and consistency strategy.

[0007] Preferably, a daily investor sentiment index is constructed to verify the validity of the constructed sentiment index, specifically including: de-dimensionalizing and normalizing the sentiment density and sentiment bias so that they are both between [0,1], and calculating the investor sentiment index based on the normalized sentiment density and sentiment bias.

[0008] Preferably, through the investor sentiment index Correlation analysis with market variables, including: obtaining daily investor sentiment index , select representative market variables as control indicators, including yield and volatility; Calculate the correlation coefficient between the investor sentiment index and the market variable using the Pearson correlation coefficient or the Spearman rank correlation coefficient; Rank the data by size and calculate the correlation between the ranks; Perform a t test on the correlation coefficient r to determine whether it is significant: ; Where Z is the sample size; Look up the table or use statistical software to obtain the p value; p < 0.05 is considered a significant correlation; If r>0.3 and p<0.05, it means there is a linear positive correlation, and sentiment and market variables fluctuate in the same direction; if r<0 and p<0.05, it means there is a linear negative correlation, which is a reverse indicator; If the correlation coefficient with at least one market variable is significant, that is, p<0.05, and |r|>0.3, then it is judged that the investor sentiment index has the ability to reflect the market during the same period.

[0009] Preferably, the emotion density XC is calculated based on the intensity concentration of the emotion words in the unit text. The calculation method is: Chinese word segmentation and part-of-speech tagging are performed on the original text, and the emotion word set is extracted using the financial field emotion dictionary ; Each word Associate an initial emotion intensity value ; Construct an undirected weighted graph G=(V,E), where: V is the set of emotion word nodes; E is the co-occurrence edge between words, and the edge weight is defined as , calculate the semantic influence weight, use the TextRank / PageRank algorithm to perform weighted sorting on each node in the graph, and calculate the emotional density by taking the weighted average sum of the semantic influence weight and the emotional polarity value.

[0010] Preferably, the emotion bias is used to measure the asymmetry between positive and negative emotions. The emotion bias is obtained by dividing all emotion texts in a fixed time window into S sets, directly taking the emotion corresponding to the maximum value in each set as the classification result, and counting the proportion of each type of emotion: is the proportion of positive emotions, The proportion of neutral emotions, is the proportion of negative emotions; and ; Calculate the discrete probability of emotion , the expression is: ; k is the number of emotion categories, is the proportion of the i-th category of emotion; the emotional bias is obtained by standardizing the discrete probability of emotion, and the expression is: ; In the formula, VB is the emotional bias.

[0011] Preferably, the sentiment labeling system and deep learning model parameters are iteratively updated, specifically including: collecting misclassified samples or high-error sentiment index samples, updating the sentiment dictionary, semantic rules and confidence adjustment mechanism; performing partial layer adjustment training on the sentiment classification model or retraining the entire model, and updating the neural network weights; regenerating the sentiment classification results after the update, and constructing a new investor sentiment index to enter the next round of verification.

[0012] In the above technical solution, the technical effects and advantages provided by the present invention are: This invention breaks through the traditional emotion recognition method that relies on keyword matching and static dictionaries by introducing multi-source unstructured text data and integrating deep learning and semantic graph analysis technology. By adopting a multi-level fusion label generation mechanism and a weakly supervised emotion classification model with an attention mechanism, this invention can more accurately identify the real emotional expressions of investors in different contexts, improve the robustness and interpretability of emotion labels, and lay the foundation for building a high-quality investor sentiment index.

[0013] The present invention also achieves closed-loop verification and adaptive optimization of the emotion recognition system and market performance by constructing a daily investor sentiment index with emotion density and emotion bias as core elements, and combining correlation analysis, statistical testing and feedback iteration mechanism. This method not only improves the effectiveness and stability of the emotion index in predicting market fluctuations, but also continuously improves the model performance and labeling system quality, with significant technical innovation and wide application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0015] Figure 1 It is an overview schematic diagram of the present invention. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] For examples, see Figure 1 As shown, the investor sentiment index verification method based on machine learning described in this embodiment includes: Collect unstructured text data from stock-related content from several data sources and vectorize the text based on a semantic representation model; A multi-level fusion label generation mechanism is used to generate initial labels and a weakly supervised sentiment classification model with an attention mechanism is used to optimize label accuracy. The labeled dataset is divided into training set, validation set and test set, which are used to train the deep learning model, adjust the hyperparameters and finally verify the model performance. Based on the sentiment classification results output by the deep learning model, a daily investor sentiment index is constructed to verify the effectiveness of the constructed sentiment indicator. Based on the verification results, the sentiment labeling system and deep learning model parameters are iteratively updated.

[0018] Efficiently and purposefully collect text information related to the stock market from multiple types of text data sources to support subsequent investor sentiment analysis tasks.

[0019] The text data sources collected in this embodiment include at least the following four categories: Financial news texts: including daily news bulletins, in-depth reports and headlines from mainstream financial portals (such as Sina Finance, NetEase Finance, Securities Times, Jiemian News, etc.); We-media commentary texts: including WeChat public account articles, Zhihu columns, Xueqiu column content, B station financial UP main commentary texts, etc. Brokerage research report texts: including brokerage analysis reports, research report summaries, rating instructions, etc. obtained through financial databases such as Wind, Tonghuashun, and Choice, which are semi-structured documents; Social media platform texts: including user comments, posts, replies, etc. related to specific stocks or sectors on platforms such as Eastmoney Stock Forum, Weibo, Xueqiu Community, and Baidu Tieba.

[0020] In order to efficiently perform the above-mentioned collection tasks, a customized web crawler system was developed using the Python development language combined with the Scrapy open source framework. The specific modules include: Scheduler module: Dynamically request the preset URL list or keywords, set crawler frequency control and anti-crawling mechanism processing; Crawler module: Responsible for initiating HTTP requests and parsing page content in HTML, JSON, and RSS formats; Parser module: Use tools such as XPath, regular expressions, and BeautifulSoup to extract structural fields such as text, timestamp, author, label, and emotional words from the source code; Data storage module: Store the collection results in a unified manner in the MongoDB database, and mark metadata such as data source type, source platform, release time, and data unique ID.

[0021] Collection frequency and real-time performance: Financial news and self-media comment data are updated once a day; brokerage research report data are captured and updated every evening of trading days; social platform comment data are dynamically collected at a frequency of minutes, and support is provided for WebSocket monitoring of feedback to trigger crawlers based on hot events.

[0022] In order to improve the accuracy and stability of the subsequent text analysis model, the collected original text needs to undergo systematic preprocessing and structural transformation, which includes the following sub-steps: Text cleaning: remove HTML tags, advertising content, script code, special symbols, invalid blank characters, etc.; unify the encoding format to UTF-8, remove garbled characters; remove non-financial related text (such as irrelevant titles, empty documents, greeting comments, etc.); word segmentation: use Jieba Chinese word segmentation tool for basic word segmentation; add industry terms, stock codes, company abbreviations (such as "CATL", "601318", "Zhong Zi Tou") and other custom words to the word segmentation dictionary to improve the accuracy of word segmentation; use phrase recognition algorithm to merge financial special phrases (such as "bad news", "bottom-fishing", "cut meat") Part-of-speech tagging and named entity recognition (NER): use NLTK or HanLP natural language processing library to tag text with parts of speech; perform NER recognition and classification on entities such as company names, stock codes, product names, people, time, and places, and extract structured meta-information; bind entity tags to original text indexes as input factors for subsequent sentiment attribution and topic mining.

[0023] Deduplication and denoising: Use the SimHash algorithm to calculate the similarity of semantically repeated texts (such as copy and paste, forwarded texts) and remove them; delete texts that are too short (such as less than 5 words) or emotionally neutral (no emotional words or subject-predicate structure).

[0024] Semantic vectorization representation: Use the Word2Vec model to generate word vectors for the processed text data, or use the BERT pre-trained model to extract sentence vectors / paragraph vectors; map each text into a vector matrix of fixed length for use as input for subsequent machine learning models; initialize the vectors of "new words" that are not in the dictionary and dynamically update them through fine-tuning training.

[0025] Emotional dictionary matching labels (primary labels): Use the constructed financial emotional dictionary, including positive words (such as "good news", "bullish", "overweight"), negative words (such as "bad news", "plunge", "cut losses") and neutral words; if a text contains a high-confidence emotional word, directly assign the corresponding emotional value of the word (such as: positive +1, negative -1); if positive and negative emotional words coexist in the text, calculate the preliminary label according to their weight and quantity. For example, "Today's market rebound is not strong enough, many stocks have risen and fallen, it is recommended to wait and see." → Contains "rebound" (positive) and "high and fall", "wait and see" (negative), and is initially labeled as weak negative (-0.3).

[0026] Emotional rule engine adjusts labels (secondary labels): Introduce emotion modification rules (emotional intensity, adverbs, negative words) for semantic enhancement: such as "very optimistic" > "optimistic"; "not optimistic" ≈ negative; multiple negative structures: "It is possible to rise" → actual meaning is positive; use regular expressions or dependency syntax analysis tools to identify grammatical structures that affect emotional polarity for secondary correction; use emotional intensity function: , where wi is the modifier weight and ei is the sentiment value. For example: "I don't think this stock will fall a lot" → "I don't think + fall + a lot" → corrected to weak positive (+0.2).

[0027] Contextual semantic disambiguation (three-pole label): For texts with potential irony, metaphors, slang or abbreviations, context-aware models (such as BERT) are introduced to combine semantic vectors and context to disambiguate; if the user's historical text sentiment is positive, but the current sentence contains "the price has reached its daily limit, I want to cry", it can be judged as a negative expression in an ironic context; the co-occurrence semantic word graph is introduced to assist in contextual semantic understanding, and the label direction is predicted in combination with historical corpus. For example: "The price has reached its daily limit again, it really makes me laugh, I didn't buy it at all" → It appears positive, but is actually negative, and is corrected to a strong negative (−0.8).

[0028] The above three processing results are integrated and weighted by confidence to generate a preliminary sentiment label for a text (for example: [positive, confidence 0.6]) as the input of the subsequent weak supervision model.

[0029] When there is some noise in the preliminary labels, the attention mechanism and weakly supervised learning model are used to optimize the labels and enhance the generalization and accuracy of the model. The specific steps include: Step 1: Construct a training dataset (including weak labels): The input is: structured text corpus + labels generated by a multi-level fusion mechanism; since the labels are not manually annotated, there are errors, so they are constructed as a weakly supervised dataset.

[0030] Step 2: Model selection and structural design: Use Bi-LSTM or Transformer structure with attention mechanism. The model structure includes: Embedding layer: convert text into word vector sequence (using pre-trained BERT or Word2Vec); Encoding layer: use Bi-LSTM or Transformer to capture contextual semantic relationships; Attention layer: assign attention weights to each word to increase the model's attention to key emotional words (such as "panic" and "bad news"); Output layer: predict emotional categories (positive, negative, neutral) and confidence distribution.

[0031] Step 3: Model training and label optimization: Use a small amount of manually annotated "gold standard" data as the validation set to correct the labels during the model training process; through multiple rounds of fine-tuning, the model gradually corrects the deviation caused by the noisy labels; add noisy label regularization terms to the model training to reduce the negative impact of incorrect labels, such as using Bootstrapping loss or Noisy student training strategies; the final output emotion labels are optimized and corrected according to the model confidence and consistency strategies (such as historical consistency and multi-model voting).

[0032] Step 4: Continuous optimization and feedback iteration: The system supports automatic labeling of new input samples, and sends samples with confidence levels below the threshold to the manual labeling queue; the model output is used to reversely update the sentiment dictionary and semantic rules to achieve a two-way feedback loop between the label generation mechanism and model training.

[0033] The obtained sentiment labels are divided into three categories: positive, negative, and neutral. Each text has been optimized through a weakly supervised model, including semantic vectors, sentiment labels, and confidence levels. To achieve high-quality training of deep learning models, the dataset is divided into the following three subsets: The training set (Train) is used for fitting and learning the model weights, with a division ratio of 70%; the validation set (Validation) is used for parameter adjustment, overfitting control and early stopping strategy, with a division ratio of 15%; the test set (Test) is used to finally evaluate the generalization ability of the model on new samples, with a division ratio of 15%. The division method is stratified sampling (StratifiedSampling) to ensure that the distribution of emotion categories in each subset is consistent and prevent model bias due to class imbalance.

[0034] Input the training set into a deep neural network model (such as CNN, Bi-LSTM, Transformer); Model structure example: Embedding layer: import BERT or Word2Vec word vectors; Encoding layer: bidirectional LSTM or multi-layer Transformer encoder; Attention layer: identify the position and weight of emotional keywords; Fully connected layer + Softmax output classification results (positive / negative / neutral); Loss function: use cross-entropy loss (Cross-Entropy Loss); Optimizer: Adam optimizer, learning rate is 1e-4.

[0035] After each training epoch, the model is evaluated on the validation set. The evaluation indicators include: Accuracy, Macro-F1, and Recall. If the performance of the validation set decreases or fluctuates dramatically, the following steps should be performed: adjust the learning rate; modify the model depth or hidden layer width; adjust the regularization parameters (such as L2, dropout rate); enable the Early Stopping mechanism: if the accuracy of the validation set does not improve for N consecutive rounds, terminate the training early to avoid overfitting.

[0036] A one-time evaluation is performed on the test set to measure the generalization ability of the final model on unseen samples; indicators such as confusion matrix, Precision-Recall curve, F1-score, ROC curve, etc. are output; if the test set performs well, the model is the final deployment version; otherwise, it is necessary to return to the training set for re-fine-tuning or optimize data annotation.

[0037] Construct a daily investor sentiment index, taking into account the number and intensity of positive and negative sentiment texts. The investor sentiment index is obtained by de-dimensionalizing the sentiment density and sentiment bias so that they are both between [0,1], and calculating the investor sentiment index based on the normalized sentiment density and sentiment bias.

[0038] For example, the present invention can use the following formula to calculate the investor sentiment index, and the calculation expression is: ; In the formula, is the investor sentiment index, is the density of emotion, is the emotional bias, is the weight coefficient of emotion density and emotion bias (which can be optimized based on experimental experience or machine learning), and Both are greater than 0.

[0039] Among them, the emotional density XC can be calculated according to the intensity concentration of emotional words in the unit text. The calculation method is: Chinese word segmentation and part-of-speech tagging of the original text; Extract sentiment word sets using the financial sentiment dictionary (including positive / negative sentiment words) ; Each word Associate an initial emotion intensity value , for example: "Good news" → =+0.7; “Panic” → =−0.9.

[0040] Construct an undirected weighted graph G=(V,E), where: V is the set of emotion word nodes (extracted from the text); E is the co-occurrence edge between words; the edge weight is defined as ; Use a sliding window (e.g., size 4) to scan the text and count the co-occurrence of sentiment words; the closer the distance, the greater the edge weight, indicating a stronger semantic connection between words.

[0041] Calculate the semantic influence weight and use the TextRank / PageRank algorithm to weighted sort each node (emotional word) in the graph. The formula is: ;in: Emotional words The semantic weight of ; d is the damping factor, usually 0.85; For all pointing Neighbor nodes; iterative update until convergence, Represents a slave node Points to all adjacent nodes The outbound edge weights are used for normalization so that the influence is distributed proportionally among neighbors.

[0042] Calculate the sentiment density XC according to the semantic influence weight and sentiment polarity value , calculate the sentiment density of the text: ; Where: L is the total length of the text (can be the number of characters, words or sentences); is the semantic weight of the i-th sentiment word, It represents the emotional polarity value of the i-th emotional word, n is the total number of emotional words, and the positive and negative emotions will offset each other, showing the overall emotional tendency and activity.

[0043] Emotional bias is used to measure the asymmetry between positive and negative emotions. The method for obtaining emotional bias is as follows: divide all emotional texts within a fixed time window (such as a single day) into S sets, directly take the emotion corresponding to the maximum value in each set as the classification result, and count the proportion of each type of emotion: is the proportion of positive emotions, The proportion of neutral emotions, is the proportion of negative emotions; and ; Calculate the discrete probability of emotion , the expression is: ; k is the number of emotion categories, usually 3 ; is the proportion of the i-th category of emotion; the emotional bias is obtained by standardizing the discrete probability of emotion, and the expression is: ; In the formula, VB is the emotional bias.

[0044] Investor Sentiment Index Correlation analysis with market variables to verify Whether it can synchronously reflect the changing trends of market status or investor behavior.

[0045] Obtain the daily investor sentiment index calculated by the present invention The time span is recommended to be no less than 6 months to 1 year; each row of data includes: date (such as 2025-03-01); ISI index (range 0~1).

[0046] Select representative market variables as control indicators, such as: yield: CSI 300, Shanghai Composite Index, A-share overall return (calculated by closing price); volatility: intraday amplitude, Bollinger bandwidth, historical volatility, etc.

[0047] Ensure that sentiment indexes and market variables correspond to each other by date, delete missing values ​​and data on non-trading days; perform necessary standardization or logarithmic processing on market variables (such as taking the logarithm of trading volume, or removing extreme values ​​from yields); Calculate the correlation coefficient between the investor sentiment index and market variables, usually using the Pearson correlation coefficient (Pearson) or the Spearman rank correlation coefficient (Spearman). Rank the data by size and calculate the correlation between the levels.

[0048] Perform a t test on the correlation coefficient r to determine whether it is significant: ; Where Z is the sample size; Look up the table or use statistical software to obtain the p value (two-tailed); generally p < 0.05 is considered a significant correlation (95% confidence level).

[0049] If r>0.3 and is significant, it means there is a linear positive correlation, and sentiment and market variables fluctuate in the same direction; if r<0 and is significant, it means there is a linear negative correlation, which may be a reverse indicator.

[0050] If the correlation coefficient with at least one market variable is significant, that is, p<0.05, and |r|>0.3, it can be judged that the investor sentiment index has the ability to reflect the market during the same period.

[0051] The invention obtains feedback signals through empirical verification of the sentiment index (such as correlation analysis, prediction test, and strategy backtesting). If the sentiment index performs poorly in certain time periods and specific situations, it is necessary to optimize the sentiment labeling system (such as dictionary, labeling strategy), sentiment classification model parameters (such as learning rate, network structure), and achieve system adaptive learning and evolution.

[0052] Extract the signal of "model failure" from the following scenarios: The correlation between sentiment index and yield rate has significantly decreased (r<0.2), sentiment fluctuates extremely, but the market is actually stable (or reversed), strategy backtest returns deviate greatly or the drawdown is too large, and the accuracy of text classification in specific industries / platforms is significantly low. Mark it as a sentiment misjudgment sample and enter the iterative analysis pool.

[0053] For misjudged samples, the generation mechanism of sentiment labels is optimized, including: adding uncovered sentiment expressions (such as hot topics, slang, and financial jargon), re-assigning higher accuracy or confidence to high-error words, and adding industry-specific sentiment words (such as "mergers and acquisitions" and "delisting").

[0054] Count the commonalities among the misjudged samples (e.g. “It’s amazing, it hit the limit down again” was misjudged as positive); introduce rules or models to label and correct typical ironic structures (e.g. “positive words + negative situations”).

[0055] For samples with low model confidence (softmax value < 0.6), they are set to "neutral" or enter the "weakly supervised learning pool" for re-evaluation.

[0056] Partially retrain or fine-tune the model to improve classification performance: The identified misclassified samples (and their manually corrected labels) are added to the training set, and pseudo-label enhancement (semi-supervised) can be performed using high-confidence verification samples; Dynamically adjust the learning rate, dropout ratio, and batch size; adjust the model structure (such as adding an Attention layer and using a deeper BERT encoder); Without destroying the learned capabilities, only some layers (such as the output layer) are fine-tuned; this speeds up the iteration cycle and improves the system's adaptability.

[0057] The criteria for determining whether to update (control mechanism) include: If the verification indicators (such as correlation and accuracy) decrease for several consecutive days, the automatic update process will be triggered; or experts will manually review the results to determine whether to perform corrections (for high-risk strategy scenarios).

[0058] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. The investor sentiment indicator verification method based on machine learning is characterized by: include: Collect unstructured text data from stock-related content from several data sources and vectorize the text based on a semantic representation model; A multi-level fusion label generation mechanism is used to generate initial labels and a weakly supervised sentiment classification model with an attention mechanism is used to optimize label accuracy. The labeled dataset is divided into training set, validation set and test set, which are used to train the deep learning model, adjust the hyperparameters and finally verify the model performance. Based on the sentiment classification results output by the deep learning model, a daily investor sentiment index is constructed to verify the effectiveness of the constructed sentiment indicator. Based on the verification results, the sentiment labeling system and deep learning model parameters are iteratively updated.

2. The method for verifying investor sentiment indicators based on machine learning according to claim 1, characterized in that: The attention mechanism and weakly supervised learning model are used to optimize the training of labels, including: Construct training data set: input is structured text corpus and labels generated by multi-level fusion mechanism; adopt Bi-LSTM or Transformer structure with attention mechanism, the model structure includes: embedding layer: convert text into word vector sequence; encoding layer: use Bi-LSTM or Transformer to capture contextual semantic relationship; attention layer: assign attention weight to each word to improve the model's attention to emotional words; output layer: predict emotional category and confidence distribution; Manually annotated standard data is used as the validation set to correct the labels during the model training process. Noise label regularization terms are added to the model training to reduce the negative impact of incorrect labels. The final output emotion labels are optimized and corrected according to the model confidence and consistency strategy.

3. The method for verifying investor sentiment indicators based on machine learning according to claim 1, characterized in that: A daily investor sentiment index is constructed to verify the effectiveness of the constructed sentiment indicator, specifically including: de-dimensionalizing and normalizing the sentiment density and sentiment bias so that they are both between [0,1], and calculating the investor sentiment index based on the normalized sentiment density and sentiment bias.

4. The method for verifying investor sentiment indicators based on machine learning according to claim 3, characterized in that: Investor Sentiment Index Correlation analysis with market variables, including: obtaining daily investor sentiment index , select representative market variables as control indicators, including yield and volatility; Calculate the correlation coefficient between the investor sentiment index and the market variable using the Pearson correlation coefficient or the Spearman rank correlation coefficient; Rank the data by size and calculate the correlation between the ranks; Perform a t test on the correlation coefficient r to determine whether it is significant: ; Where Z is the sample size; Look up the table or use statistical software to obtain the p value; p < 0.05 is considered a significant correlation; If r>0.3 and p<0.05, it means there is a linear positive correlation, and sentiment and market variables fluctuate in the same direction; if r<0 and p<0.05, it means there is a linear negative correlation, which is a reverse indicator; If the correlation coefficient with at least one market variable is significant, that is, p<0.05, and |r|>0.3, then it is judged that the investor sentiment index has the ability to reflect the market during the same period.

5. The method for verifying investor sentiment indicators based on machine learning according to claim 4, characterized in that: in, The emotional density XC is calculated based on the intensity concentration of emotional words in the unit text. The calculation method is: Chinese word segmentation and part-of-speech tagging are performed on the original text, and the emotional word set is extracted using the financial field emotional dictionary. ; Each word Associate an initial emotion intensity value ; Construct an undirected weighted graph G=(V,E), where: V is the set of emotion word nodes; E is the co-occurrence edge between words, and the edge weight is defined as , calculate the semantic influence weight, use the TextRank / PageRank algorithm to perform weighted sorting on each node in the graph, and calculate the emotional density by taking the weighted average sum of the semantic influence weight and the emotional polarity value.

6. The method for verifying investor sentiment indicators based on machine learning according to claim 5, characterized in that: Emotional bias is used to measure the asymmetry between positive and negative emotions. The method for obtaining emotional bias is as follows: divide all emotional texts in a fixed time window into S sets, directly take the emotion corresponding to the maximum value in each set as the classification result, and count the proportion of each type of emotion: is the proportion of positive emotions, The proportion of neutral emotions, is the proportion of negative emotions; and ; Calculate discrete probability of emotion , the expression is: ; k is the number of emotion categories, is the proportion of the i-th category of emotion; the emotional bias is obtained by standardizing the discrete probability of emotion, and the expression is: ; In the formula, VB is the emotional bias.

7. The method for verifying investor sentiment indicators based on machine learning according to claim 6, characterized in that: Iteratively update the sentiment labeling system and deep learning model parameters, including: collecting misclassified samples or high-error sentiment index samples, updating the sentiment dictionary, semantic rules and confidence adjustment mechanism; performing partial layer adjustment training on the sentiment classification model or retraining the entire model, and updating the neural network weights; regenerating the sentiment classification results after the update, and constructing a new investor sentiment index to enter the next round of verification.

Images