Financial text emotion analysis method and system based on double-layer low-rank adaptation

Abstract

The invention provides a financial text emotion analysis method and system based on double-layer low-rank adaptation, belongs to the field of natural language processing and financial science and technology, and aims at solving the problems that existing financial emotion analysis is single in dimension and high in model fine adjustment cost. According to the method, an emotion label system with five dimensions of emotion category, emotion intensity, duration, layered influence and policy response is constructed, a financial text is converted into a structured five-dimensional emotion label through an instruction fine tuning framework to be output, and a double-layer low-rank adaptation module is injected into a pre-trained large language model to perform efficient parameter fine tuning. The system comprises a data annotation module and a model fine adjustment module, and automatic annotation of emotion tags, expert verification and efficient training of a large model are achieved. By applying the method, the emotional characteristics of the financial event can be accurately extracted, the precision of emotional analysis is improved, and reliable decision support is provided for stock market trend prediction and risk control.

Description

Technical Field

[0001] The present invention relates to the intersection of natural language processing and financial technology, and in particular to a financial text sentiment analysis method and system based on double-layer low-rank adaptation. Background Art

[0002] Sentiment analysis in financial text is a key technology in quantitative investment and risk management. Its core goal is to extract market sentiment signals from unstructured text such as announcements and news, providing auxiliary information for trading strategies and risk control. In recent years, with the development of large-scale language models, sentiment analysis has made significant progress in natural language processing tasks. However, its application in the financial sector still faces the following challenges: Existing representative technologies include the BERT-based sentiment classification model and the parameter-efficient fine-tuning method LoRA (Low-Rank Adaptation), but both methods have some shortcomings in different aspects.

[0003] BERT-based sentiment classification models typically use general pre-trained language models like BERT, fine-tuned on financial text, to identify positive and negative sentiment. They achieve high accuracy on some public datasets and exhibit good transfer and generalization capabilities. Their advantages include not requiring extensive domain-labeled data and possessing strong semantic modeling capabilities. However, their disadvantages are that training on general corpus leads to a weak understanding of financial terminology and numerical variations (e.g., "5% increase" versus "0.5% increase"), making it difficult to parse the implicit emotional differences in specialized contexts. Furthermore, sentiment labels are limited in dimensionality, supporting only polarity classification and failing to capture the complex impact of financial events.

[0004] The parameter efficient fine-tuning method LoRA introduces two low-rank matrices without modifying the pre-trained model structure and , achieving parameter updates. This method has been applied in multiple NLP tasks and is publicly available as the HuggingFace PEFT standard interface. The advantage of this method is that it allows for efficient migration based on frozen original parameters, saving video memory and training time. In some tasks, its performance approaches that of full-parameter fine-tuning. However, its disadvantages are that the LoRA architecture only supports a single-layer low-rank mapping, which has limited expressive power and cannot model complex interactions between multidimensional features. It also lacks adaptability to structured tasks such as sentiment label generation, making it difficult to meet the requirements of multidimensional financial modeling scenarios.

[0005] In addition to the above specific technologies, the following technical deficiencies are also common in the current field of financial sentiment analysis: Poor domain adaptability: General models cannot handle the rich numerical descriptions and professional terms in financial texts, resulting in insufficient model generalization ability and the F1 indicator being far below the usable level.

[0006] High fine-tuning cost: Fine-tuning all parameters requires a lot of video memory and time, which is not conducive to large-scale deployment.

[0007] Single labeling system: Current methods generally use sentiment polarity labels (positive / negative / neutral), ignoring key dimensions such as sentiment intensity, duration, hierarchical impact, and policy response.

[0008] Limited expressive power: Even with the introduction of parameter-efficient methods such as LoRA, their expressive power is still insufficient to model the nonlinear interactions between multidimensional emotions in financial events.

[0009] In summary, while existing technologies have achieved some success in sentiment recognition and model fine-tuning, they still face significant deficiencies in fine-grained label modeling, efficient parameter adaptation, and complex semantic parsing. Therefore, an improved solution is urgently needed that can integrate structured label design, enhance model expressiveness, and maintain training efficiency, providing a practical and feasible technical path for financial sentiment analysis tasks. Summary of the Invention

[0010] In response to the aforementioned technical issues, a method and system for sentiment analysis of financial text based on dual-layer low-rank adaptation is provided. This invention constructs a sentiment labeling system based on five dimensions: sentiment category, sentiment intensity, duration, hierarchical impact, and policy response. Using a command fine-tuning framework, financial text is converted into structured five-dimensional sentiment label outputs. A dual-layer low-rank adaptation module is then injected into a pre-trained large language model for efficient parameter fine-tuning. This invention is used to perform multi-dimensional sentiment analysis on financial text to assist in stock market trend forecasting and risk control.

[0011] The technical means adopted in the present invention are as follows: A financial text sentiment analysis method based on two-layer low-rank adaptation, including: Based on the definition of financial texts, we construct a multi-dimensional emotion labeling system by defining dimensional labels including emotion categories, emotion intensity, duration, hierarchical impact, and policy response. Using the instruction-input-output triplet format, we construct an instruction fine-tuning dataset, embedding financial text into structured instruction templates, and outputting standardized text containing multi-dimensional sentiment labels. Inject a two-layer low-rank adaptation matrix into the target module of the pre-trained large language model and select the adaptation module based on the model architecture; Establish evaluation metrics for large language model training and implement sentiment analysis of financial texts based on two-layer low-rank adaptation.

[0012] Furthermore, the emotion category is determined as positive, neutral and negative based on the stock price fluctuation on the day of the event; the emotion intensity is divided into strong, medium and weak based on the absolute value of the fluctuation; the duration is classified into short-term, medium-term and long-term through explicit time description in the text and implicit semantic analysis; the hierarchical impact is generated by weighted calculation of the frequency of corporate entity mentions, industry keyword density and the number of policy verb occurrences to generate a comprehensive score; the policy response is based on the correlation score calculated based on the matching of the policy keyword library and the weighted calculation of the situation.

[0013] Furthermore, the construction of the multi-dimensional emotion labeling system includes an automated labeling stage and a manual review stage, wherein: The automated labeling phase generates initial labels by matching keywords through a rule engine, and dynamically modifies sentiment categories and intensities based on stock price fluctuations. The manual review stage: cross-validate the labels of the five-dimensional emotions to ensure that the consistency coefficient Kappa is ≥ 0.85.

[0014] Furthermore, the instruction fine-tuning dataset is enhanced through synonym replacement, sentence reconstruction and noise insertion to generate diversified training samples.

[0015] Furthermore, the step of injecting a two-layer low-rank adaptation matrix into the target module of the pre-trained large language model specifically includes: By updating the weight matrix Perform two low-rank decompositions and express them as the product of four matrices. The weight update formula is:

[0016] in, , , , , , , and is an additionally introduced trainable transformation matrix, Represents the low-rank bottleneck dimension, which controls the degree of compression of the model in parameter adaptation. Represents the input dimension, that is, the dimension of the input feature of the current layer, matrix Compress the input from high-dimensional to low-rank space; Represents the output dimension, that is, the dimension of the output feature of the current layer, the matrix Map the low-rank features back to the original output space.

[0017] Furthermore, the evaluation indicators in the large language model training include: classification task indicators and generation task indicators, where: The classification task indicators include precision, recall and F1 score, and the calculation formula is:

[0018]

[0019]

[0020] in, represents the accuracy, represents the recall rate, represents the F1 score; TP represents the number of events whose predicted categories are correct, FP represents the number of events whose predicted categories are incorrect, and FN represents the number of events whose true categories are positive but the model incorrectly predicts them as negative. The generation task indicators include ROUGE-L, BLEU-4 and structured output accuracy, where the structured output accuracy is defined as the ratio of model-generated labels to true labels that fully match.

[0021] The present invention also provides a financial text sentiment analysis system based on double-layer low-rank adaptation, including: a data annotation module, a model fine-tuning module and a visual training monitoring panel, wherein: The data annotation module is used to configure the rule engine and manual review interface, supporting the automatic generation and manual correction of five-dimensional labels; The model fine-tuning module supports the loading of Qwen2.5-7B, InternLM2-7B, and ChatGLM3-6B base models and the injection of DNLoRA adaptation layers; The visual training monitoring panel is used to display the loss curve, F1 score and video memory usage in real time.

[0022] Compared with the prior art, the present invention has the following advantages: The financial text sentiment analysis method and system based on dual-layer low-rank adaptation provided by this invention significantly improves sentiment analysis accuracy. Fine-tuning a large model using the DNLoRA strategy effectively improves metrics such as precision and recall of the sentiment classification model. In tests, the DNLoRA fine-tuned model achieved significant improvements in metrics such as the F1 score compared to the basic LoRA method, demonstrating more accurate sentiment recognition.

[0023] The financial text sentiment analysis method and system based on dual-layer low-rank adaptation provided by this invention significantly reduces training costs. This method only requires training a minimal number of parameters to complete model adaptation, significantly reducing video memory usage and computational overhead. On a 7B parameter-scale model, DNLoRA fine-tuning shortens training time to approximately 3 hours, far less than the time required for full parameter fine-tuning, significantly improving training efficiency.

[0024] The financial text sentiment analysis method and system based on dual-layer low-rank adaptation provided by this invention provides a more comprehensive portrayal of sentiment. The introduction of a five-dimensional sentiment labeling system overcomes the limitation of traditional sentiment analysis, which relies solely on the polarity dimension. By modeling event sentiment from multiple perspectives, including intensity, timeliness, scope of impact, and policy relevance, this invention can provide more granular and richer sentiment information, providing a more comprehensive reference for investment decisions.

[0025] The financial text sentiment analysis method and system based on dual-layer low-rank adaptation provided by this invention enhances predictive decision-making capabilities. The multidimensional sentiment information extracted by this invention can be better utilized for financial market forecasting and risk monitoring. In experiments, when sentiment analysis results were used to predict stock price trends, the model's closing price prediction error was significantly reduced. This demonstrates that this invention can provide more reliable decision-making support for stock market trend analysis and risk control.

[0026] Based on the above reasons, the present invention can be widely promoted in the intersection of natural language processing and financial technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0028] Figure 1 This is a flow chart of the financial text sentiment analysis method based on double-layer low-rank adaptation in the present invention.

[0029] Figure 2 This is a flowchart of a financial sentiment analysis method for fine-tuning large-scale model instructions according to an embodiment of the present invention.

[0030] Figure 3 Schematic diagram of the five-dimensional emotion labeling system according to an embodiment of the present invention.

[0031] Figure 4 2 is a schematic diagram of the model fine-tuning framework structure of an embodiment of the present invention. DETAILED DESCRIPTION

[0032] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0033] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is in no way intended to limit the present invention and its application or use. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0034] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0035] Unless otherwise specified, the relative arrangement of the parts and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention. At the same time, it should be clear that, for ease of description, the sizes of the various parts shown in the accompanying drawings are not drawn according to actual proportional relationships. The technology, methods and equipment known to those of ordinary skill in the relevant art may not be discussed in detail, but in appropriate cases, the technology, methods and equipment should be considered as a part of the specification. In all examples shown and discussed here, any specific value should be interpreted as being merely exemplary, rather than as a limitation. Therefore, other examples of the exemplary embodiments can have different values. It should be noted that similar numbers and letters represent similar items in the following drawings, and therefore, once an item is defined in an accompanying drawing, it does not need to be further discussed in subsequent drawings.

[0036] like Figure 1 As shown, the present invention provides a financial text sentiment analysis method based on double-layer low-rank adaptation, comprising: Based on the definition of financial texts, we construct a multidimensional emotion labeling system with the dimension labels including emotion category, emotion intensity, duration, hierarchical impact and policy response, as shown in Table 1. In specific implementation, as a preferred embodiment of the present invention, the emotion category is determined as positive, neutral and negative based on the stock price fluctuation on the day of the event; the emotion intensity is divided into strong, medium and weak according to the absolute value of the fluctuation; the duration is classified into short-term, medium-term and long-term through explicit time description in the text and implicit semantic analysis; the hierarchical impact is generated by weighted calculation of the frequency of corporate entity mentions, industry keyword density and the number of policy verb occurrences to generate a comprehensive score; the policy response is based on the correlation score calculated by matching the policy keyword library and weighted calculation of the situation.

[0037] Table 1 Five-dimensional emotion label settings

[0038] The calculation rules for the hierarchical impact dimension are as follows: the enterprise-level impact score is set as the proportion of the enterprise entity's appearance in the text multiplied by 0.7; the industry-level impact score is set as the TF-IDF weight of the industry keyword in the text multiplied by 0.2; and the macro-level impact score is set as the frequency of policy-related verbs multiplied by 0.1. Finally, the scores at each level are summed to obtain a comprehensive impact score. The corresponding relationship between the hierarchical impact scores and the actual impact of the event is shown in Table 2.

[0039] Table 2 Relationship between stratified impact scores and actual impact of events

[0040] This paper uses policy response to measure the relevance of an event to current policy direction. This dimension can help identify whether an event is supported or suppressed by policy, thereby assisting in determining the potential impact of the event on the market. The correlation between the policy response score and the event and policy is shown in Table 3.

[0041] Table 3 Correlation between policy response scores and events and policies

[0042] In specific implementation, as a preferred embodiment of the present invention, a multi-dimensional emotion labeling system is constructed, including an automated labeling stage and a manual review stage, wherein: The automated tagging phase uses a rule engine to match keywords to generate initial tags, and dynamically adjusts sentiment categories and intensities based on stock price fluctuations. Manual review stage: Cross-validate the labels of the five-dimensional emotions to ensure that the consistency coefficient Kappa ≥ 0.85.

[0043] Using the instruction-input-output triplet format, we construct an instruction fine-tuning dataset, embedding financial text into structured instruction templates, and outputting standardized text containing multi-dimensional sentiment labels. In specific implementation, as a preferred embodiment of the present invention, the instruction fine-tuning dataset is enhanced through synonym replacement, sentence reconstruction and noise insertion to generate diversified training samples.

[0044] When implementing, the instruction template is designed as follows: Instruction: Analyze the sentiment categories, intensity, duration, layered impact, and policy responses of the following event texts.

[0045] Input: The company's annual report shows that net profit increased by 25% year-on-year, but cash flow decreased by 8% year-on-year.

[0046] Output: Sentiment category: neutral; Intensity: medium; Duration: medium; Stratified impact: 6.2; Policy response: general correlation.

[0047] The input is the text of a specific financial event, and the output of the model is required to be a structured result text containing five emotional dimensions. By converting the sentiment analysis problem into a generation task that follows a predetermined format, the large language model is forced to output sentiment labels in a standardized format, which facilitates the analysis of the results and subsequent application processing.

[0048] Inject a two-layer low-rank adaptation matrix into the target module of the pre-trained large language model, and select the adaptation module according to the model architecture; in specific implementation, as a preferred embodiment of the present invention, inject a two-layer low-rank adaptation matrix into the target module of the pre-trained large language model, specifically including: By updating the weight matrix Perform two low-rank decompositions and express them as the product of four matrices. The weight update formula is:

[0049] in, , , , , , , and For the additionally introduced trainable transformation matrix, both Gaussian random initialization and LoRA are used. Represents the low-rank bottleneck dimension, which controls the degree of compression of the model in parameter adaptation. Setting it to 8 means compressing the weight update model through the low-rank matrix; Represents the input dimension, that is, the dimension of the input feature of the current layer, matrix Compress the input from high-dimensional to low-rank space; Represents the output dimension, that is, the dimension of the output feature of the current layer, the matrix The low-rank features are mapped back to the original output space. This adaptation scheme allows for flexible injection locations based on the model architecture. For example, for the Qwen2.5-7B model, the DNLoRA module is injected into the q_proj, v_proj, and up_proj matrices; for the InternLM2-7B model, the DNLoRA module is injected into the gate_proj and k_proj matrices; and for the ChatGLM3-6B model, the DNLoRA module is injected into the encoder.q_proj, encoder.v_proj, and decoder.v_proj matrices.

[0050] Train and optimize the model using mixed precision training, setting the batch size to 2, the number of gradient accumulation steps to 8, and the learning rate to . Establish evaluation metrics for large language model training and implement sentiment analysis of financial texts based on two-layer low-rank adaptation.

[0051] In specific implementation, as a preferred embodiment of the present invention, the evaluation indicators in the large language model training include: classification task indicators and generation task indicators, where: Classification task indicators include precision, recall and F1 score, and the calculation formula is:

[0052]

[0053]

[0054] in, represents the accuracy, represents the recall rate, represents the F1 score; TP represents the number of events whose predicted categories are correct, FP represents the number of events whose predicted categories are incorrect, and FN represents the number of events whose true categories are positive but the model incorrectly predicts them as negative. The generation task indicators include ROUGE-L, BLEU-4 and structured output accuracy, where the structured output accuracy is defined as the ratio of model-generated labels to true labels that completely match.

[0055] The present invention also provides a financial text sentiment analysis system based on a double-layer low-rank adaptation method, comprising: a data annotation module, a model fine-tuning module, and a visual training monitoring panel, wherein: The data annotation module is used to configure the rule engine and manual review interface, supporting the automatic generation and manual correction of five-dimensional labels; The model fine-tuning module supports the loading of Qwen2.5-7B, InternLM2-7B, and ChatGLM3-6B base models and the injection of DNLoRA adaptation layers; The visual training monitoring panel is used to display the loss curve, F1 score and video memory usage in real time.

[0056] Example This example obtains historical data from public financial information platforms, selecting announcements and related news from listed companies that are components of the CSI 300 Index as the data source. For example, announcement texts and news briefs from relevant companies between 2021 and 2023 were collected through channels such as the China Securities Journal official website, Sina Finance, and the Xueqiu community. In addition, the open interface BaoStock was used to obtain stock trading market data for the corresponding time period. Approximately 12,500 raw texts were initially collected. After data cleaning and deduplication, 8,360 valid raw text data items were obtained.

[0057] The collected raw text data is normalized. First, regular expressions are used to remove irrelevant content such as HTML tags, special symbols, and redundant whitespace. Then, invalid text data with a length of less than 50 characters is filtered out, and duplicate text is removed. This preprocessing ensures the cleanliness and standardization of the input data.

[0058] According to the five-dimensional emotion labeling system proposed by the present invention, each text is manually labeled and verified for emotional information. Figure 2 As shown in Figure 2, the emotion labeling system includes the following five dimensions: 1) Sentiment Category: Sentiment polarity is determined based on the price fluctuation of the relevant stock on the day the event is released. A significant increase in the price fluctuation, for example, ≥+1.2%, is marked as positive; a small change, for example, between -1.2% and +1.2%, is marked as neutral; and a significant decrease in the stock price, for example, ≤-1.2%, is marked as negative. In practice, these thresholds can be dynamically adjusted based on overall market volatility to more accurately reflect changes in market sentiment.

[0059] 2) Sentiment Intensity: This measure assesses the intensity of the sentiment impact based on the magnitude of the stock price fluctuation on the day of the event. The absolute magnitude of the fluctuation is used as the basis: a change of 5% or greater is considered strong sentiment, between 2% and 5% is considered moderate sentiment, and less than 2% is considered weak sentiment. This classification reflects the magnitude of the event's impact on the market.

[0060] 3) Duration: The duration of the emotional impact of an event is determined by combining temporal descriptions in the text. By analyzing explicit temporal terms (such as "short-term" and "long-term") in the text and the implicit temporal information identified by the model, emotional impacts are categorized as short-term, medium-term, or long-term. For example, emotional fluctuations caused by a one-time event are considered short-term, while events related to industry trends may have medium- to long-term emotional impacts. In this example, an LSTM+CRF model is used to extract temporal information from the text to assist in determining the duration dimension of the emotional impact.

[0061] 4) Layered Impact: Quantify the impact of an event at the enterprise, industry, and macro levels. Specifically, for each article, the enterprise-level impact score, industry-level impact score, and macro-level impact score are calculated separately, and then the total layered impact score is calculated. For example, the enterprise-level impact score can be calculated as the percentage of occurrences of the enterprise entity in the article multiplied by 0.7, the industry-level impact score can be calculated as the TF-IDF weight of the industry keyword in the article multiplied by 0.2, and the macro-level impact score can be calculated as the frequency of occurrence of policy-related verbs multiplied by 0.1. Finally, the scores at each level are summed to obtain a comprehensive impact score.

[0062] 5) Policy Reaction: This measures the relevance of an event to current policy trends. First, a policy keyword library is constructed, containing common policy terms and their weights. The policy sensitivity index of the text is then calculated by counting the occurrences of policy keywords within the text and weighting each keyword based on its preset weight and the sentiment polarity it carries. This yields a score representing the degree of relevance between the event and policy. The policy keyword library can be dynamically updated to reflect the latest policy trends. The policy reaction dimension can help identify whether an event is supported or suppressed by policy, thereby assisting in determining the potential impact of the event on the market.

[0063] After the rules engine initially generates the five-dimensional emotion labels, they undergo manual verification to ensure accuracy. In this example, a "rule-based automatic labeling + expert review" process is employed: The system first automatically generates initial labels based on pre-set rules, and then modifies the emotion category and intensity based on the current market conditions. A team of financial experts then reviews the labeling results, with at least three experts cross-checking each data point's labeling and only validating the labels upon unanimous agreement. This process builds a high-quality dataset with five-dimensional emotion labels, providing reliable training samples for subsequent model training.

[0064] Large model loading and double-layer low-rank adaptation (DNLoRA) fine-tuning.

[0065] like Figure 3As shown, the system provided in this embodiment includes a model fine-tuning module for loading the pre-trained model and injecting it into the DNLoRA adaptation layer to perform instruction fine-tuning training on the above-mentioned labeled dataset. The specific steps are as follows: Large open source language models are selected as base models, such as Qwen2.5-7B, InternLM2-7B, and ChatGLM3-6B. During the model loading phase, the pre-trained model weights are imported through the HuggingFace Transformers library. Subsequently, all the original parameters of the model are frozen, and trainable LoRA adaptation modules are inserted only in some layers of the model, as shown in Table 4, to ensure that only a very small number of parameters need to be updated during fine-tuning. This embodiment freezes all the original Transformer layer weights and only trains the newly added DNLoRA layer parameters.

[0066] Table 4 Fine-tuning hyperparameter settings for different large language models

[0067] The DNLoRA low-rank adaptation structure of the present invention is injected into the pre-trained model to enable the model to efficiently learn financial sentiment features. DNLoRA adopts a weight update path of a double-layer nonlinear transformation, namely ,in , , , , , , N1 and N2 are additional trainable transformation matrices. Compared with traditional LoRA, DNLoRA introduces nonlinear expression capabilities through two-level transformation. In implementation, the insertion position of the adaptation layer is selected according to different model architectures. For example: For the Qwen2.5-7B model (Mixture of Experts architecture), a DNLoRA module is inserted into the query projection layer q_proj, the value projection layer v_proj of the self-attention module, and the up-projection layer up_proj of the feedforward network to enhance the adaptability of attention and feedforward calculations. For the InternLM2-7B model (SwiGLU gated architecture), the DNLoRA module is inserted at the gate_proj and key projection layer k_proj of its gate unit, so that the model can incorporate the adaptation strategy of the present invention into the gating mechanism; For the ChatGLM3-6B model (Encoder-Decoder architecture), the DNLoRA module is inserted into the encoder's q_proj layer, the encoder's v_proj layer, and the decoder's v_proj layer, respectively, thereby improving the ability of both the encoder and decoder to represent emotional features.

[0068] Other common hyperparameter configurations are shown in Table 5 below: Table 5 General hyperparameter settings for fine-tuning large language models

[0069] Through the above-mentioned adaptation and insertion for different model structures, the method of the present invention has wide applicability and can achieve efficient sentiment analysis fine-tuning on a variety of mainstream large models.

[0070] After the model and adaptation layer are prepared, the model is fine-tuned. This example trains the model in parallel on three NVIDIA RTX 4090 GPUs, using the DeepSpeed ​​framework combined with the AdamW optimizer to accelerate the training convergence of the large model.

[0071] The initial learning rate is set to , using a cosine annealing learning rate scheduling strategy; the batch size per GPU is 2, with 8 gradient accumulation steps; the maximum sequence length is set to 4096 to ensure the model's ability to handle long text; the number of training epochs is set to 45; and half-precision floating-point training is enabled to reduce GPU memory usage. During training, an early stopping mechanism can be introduced: if the F1 metric on the validation set stops improving for five consecutive epochs, training is terminated early to prevent overfitting.

[0072] After model training is complete, the performance of the fine-tuned model is evaluated. Evaluation metrics include classification and generation. The former uses precision, recall, and F1 scores to measure the model's performance in classifying emotion categories. The latter uses ROUGE-L and BLEU-4 to assess how closely the model's generated output matches the standard answer. The accuracy of the structured output is calculated (i.e., the proportion of complete emotion label sequences output by the model that are completely consistent with the manually labeled sequences).

[0073] The results in Table 6 show that the large model fine-tuned with DNLoRA outperforms the model fine-tuned solely with LoRA across all of the aforementioned metrics. Precision, Recall, and F1 scores for the classification task all improved by approximately 2-3 percentage points. The ROUGE-L and BLEU-4 scores for the generation task also improved, and the proportion of output sentiment labels that fully matched the manual ground truth increased. These results demonstrate that our method significantly improves the model's sentiment analysis accuracy while maintaining efficient training.

[0074] Table 6 Experimental results of sentiment analysis of Qwen2.5-7B with different fine-tuning methods

[0075] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A financial text sentiment analysis method based on double-layer low-rank adaptation, characterized by: include: Based on the definition of financial texts, we construct a multi-dimensional emotion labeling system by defining dimensional labels including emotion categories, emotion intensity, duration, hierarchical impact, and policy response. The sentiment category is determined as positive, neutral, or negative based on the stock price fluctuation on the day of the event; the sentiment intensity is divided into strong, medium, and weak based on the absolute value of the fluctuation; the duration is classified as short-term, medium-term, and long-term through explicit time description in the text and implicit semantic analysis; the layered impact is calculated by weighting the frequency of corporate entity mentions, industry keyword density, and the number of policy verbs; the policy response is scored based on the relevance calculation of the policy keyword library matching and contextual weighting; Using the instruction-input-output triplet format, we construct an instruction fine-tuning dataset, embedding financial text into structured instruction templates, and outputting standardized text containing multi-dimensional sentiment labels. Inject a two-layer low-rank adaptation matrix into the target module of the pre-trained large language model and select the adaptation module based on the model architecture; The method of injecting a double-layer low-rank adaptation matrix into the target module of the pre-trained large language model specifically includes: updating the weight matrix Perform two low-rank decompositions and express them as the product of four matrices. The weight update formula is: ;in, , , , , , , and is an additionally introduced trainable transformation matrix, Represents the low-rank bottleneck dimension, which controls the degree of compression of the model in parameter adaptation. Represents the input dimension, which is used to represent the dimension of the input feature of the current layer, matrix Compress the input from high-dimensional to low-rank space; Represents the output dimension, that is, the dimension of the output feature of the current layer, the matrix Map low-rank features back to the original output space; establish evaluation indicators in large language model training to implement financial text sentiment analysis based on double-layer low-rank adaptation.

2. The financial text sentiment analysis method based on double-layer low-rank adaptation according to claim 1 is characterized in that The construction of the multi-dimensional emotion labeling system includes an automated labeling stage and a manual review stage. In the automated labeling stage, initial labels are generated by matching keywords through a rule engine, and the emotion category and intensity are dynamically modified based on the stock price fluctuation. In the manual review stage, the labels of the five-dimensional emotions are cross-validated to ensure that the consistency coefficient Kappa is ≥0.

85.

3. The financial text sentiment analysis method based on double-layer low-rank adaptation according to claim 1 is characterized in that The instruction fine-tuning dataset is enhanced through synonym replacement, sentence reconstruction, and noise insertion to generate diverse training samples.

4. The financial text sentiment analysis method based on double-layer low-rank adaptation according to claim 1 is characterized in that The evaluation indicators in the large language model training include: classification task indicators and generation task indicators, where: The classification task indicators include precision, recall and F1 score, and the calculation formula is: in, represents the accuracy, represents the recall rate, represents the F1 score; TP represents the number of events whose predicted categories are correct, FP represents the number of events whose predicted categories are incorrect, and FN represents the number of events whose true categories are positive but the model incorrectly predicts them as negative. The generation task indicators include ROUGE-L, BLEU-4 and structured output accuracy, where the structured output accuracy is defined as the ratio of model-generated labels to true labels that fully match.

5. A financial text sentiment analysis system based on double-layer low-rank adaptation implemented based on the financial text sentiment analysis method based on double-layer low-rank adaptation according to any one of claims 1 to 4, characterized in that: include: Data labeling module, model fine-tuning module and visual training monitoring panel, including: The data annotation module is used to configure the rule engine and manual review interface, supporting the automatic generation and manual correction of five-dimensional labels; The model fine-tuning module supports the loading of Qwen2.5-7B, InternLM2-7B, and ChatGLM3-6B base models and the injection of DNLoRA adaptation layers; The visual training monitoring panel is used to display the loss curve, F1 score and video memory usage in real time.

Images