Aspect-level emotion triple information extraction method and related device

Abstract

The embodiment of the invention provides an aspect-level emotion triple information extraction method and a related device, and relates to the technical field of computers. Obtaining a to-be-analyzed statement; inputting the to-be-analyzed statement into a pre-trained information extraction model for processing to obtain triple information of the to-be-analyzed statement; the triple information comprises aspect words, opinion words and sentiment polarity of the opinion words in the to-be-analyzed statement; in the embodiment of the invention, the information extraction model comprises a plurality of Transform layers, and the information extraction model is obtained by setting a low-rank matrix for each Transform layer and performing parameter updating on the low-rank matrixes corresponding to the plurality of Transform layers in a training process, so that the recognition precision of triple information extraction can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a method and related apparatus for extracting aspect-level emotion triplet information. Background Technology

[0002] Aspect Sentiment Triplet Extraction (ASTE) is a key task in the field of sentiment analysis. Its core objective is to extract triplets consisting of aspect terms, corresponding opinion terms, and sentiment polarity, and it is widely used in content recommendation, public opinion analysis, and other fields.

[0003] Current research on aspect-level sentiment triple extraction often involves building models to extract triple information from text. However, related technologies often suffer from poor recognition accuracy when extracting triple information using models. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a method and related apparatus for extracting aspect-level emotion triplet information, so as to improve the recognition accuracy of triplet information extraction.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows: Firstly, this application provides a method for extracting aspect-level sentiment triplet information, the method comprising: Obtain the statement to be analyzed; The statement to be analyzed is input into a pre-trained information extraction model for processing to obtain the triple information of the statement to be analyzed. The triple information includes aspect words, opinion words, and the sentiment polarity of the opinion words in the statement to be analyzed. The information extraction model includes multiple Transformer layers, and the information extraction model is obtained by setting a low-rank matrix for each Transformer layer and updating the parameters of the low-rank matrices corresponding to the multiple Transformer layers during training.

[0006] Secondly, this application provides an aspect-level emotion triplet information extraction device, the device comprising: The acquisition module is used to acquire the statements to be analyzed. The processing module is used to input the statement to be analyzed into a pre-trained information extraction model for processing to obtain the triple information of the statement to be analyzed; the triple information includes aspect words, opinion words, and the sentiment polarity of the opinion words in the statement to be analyzed; the information extraction model includes multiple Transformer layers, and the information extraction model is obtained by setting a low-rank matrix for each Transformer layer and updating the parameters of the low-rank matrices corresponding to the multiple Transformer layers during training.

[0007] Thirdly, this application provides an electronic device including a processor and a memory, wherein the memory stores a computer program executable by the processor, and the processor can execute the computer program to implement the method described in any of the foregoing embodiments.

[0008] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any of the foregoing embodiments.

[0009] The aspect-level sentiment triple information extraction method and related apparatus provided in this application involve inputting the statement to be analyzed into a pre-trained information extraction model for processing, thereby obtaining the triple information of the statement. The information extraction model includes multiple Transformer layers, and this model is obtained by setting a low-rank matrix for each Transformer layer and updating the parameters of only the corresponding low-rank matrix of each Transformer layer during training. Since only the parameters in the smaller part of the low-rank matrix are updated during the training of the information extraction model, rather than the entire original weight matrix, the degrees of freedom involved in each update can be limited, avoiding optimization fluctuations caused by large-scale synchronous parameter changes. This reduces the risk of gradient oscillation during training, ensures the stability of model convergence, and effectively alleviates overfitting. This allows the model to better learn general language patterns and semantic rules, improves recognition accuracy, and enables the information extraction model to achieve good generalization performance, thereby improving the accuracy and robustness of triple information extraction in practical applications.

[0010] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 An example diagram of the ASTE task is shown; Figure 2 A block diagram of an electronic device provided in an embodiment of this application is shown; Figure 3 This paper illustrates a flowchart of an aspect-level sentiment triplet information extraction method provided in an embodiment of this application. Figure 4 This paper illustrates another flowchart of the aspect-level sentiment triplet information extraction method provided in an embodiment of this application. Figure 5 A diagram illustrating the thought process chain is shown. Figure 6 A schematic diagram of data-generated prompt words is shown; Figure 7 An example image of quality assessment prompts is shown; Figure 8 An example graph illustrating the impact of the number of target sample data is shown. Figure 9 An example graph illustrating the impact of the target sample data quality is shown. Figure 10 The diagram shows a functional block diagram of an aspect-level emotion triplet information extraction device provided in an embodiment of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can be arranged and designed in various different configurations.

[0014] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0015] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0016] Aspect Sentiment Triplet Extraction (ASTE) is an important task in sentiment analysis and is widely used in scenarios such as content recommendation, user comment analysis, and public opinion monitoring.

[0017] The goal of ASTE is to identify multiple semantically complete triples from a text, including aspect words (i.e., the object being evaluated, such as "screen" or "service"), opinion words (i.e., the specific evaluation of that aspect, such as "clear" or "poor"), and sentiment polarity (indicating the emotional tendency of the evaluation, such as positive, negative, or neutral). Figure 1 For an example diagram of the ASTE task, please refer to [link / reference]. Figure 1 For a sentence containing m words The ASTE task requires extracting n representations of the form as follows: The triples in sentence T are given by the expression $n$, where $n$ represents the number of triples in sentence $T$. As an aspect term, it indicates the object or attribute being evaluated, such as "steak"; Opinion words, that is, specific opinions or modifiers describing this aspect, such as "unpalatable" or "not fresh"; The emotional polarity indicates whether the attitude toward this aspect is positive, negative, or neutral.

[0018] exist Figure 1 In the example, taking the sentence "It tastes bad and isn't fresh, especially the steak," we can extract two triples: "(steak, tastes bad, negative)" and "(steak, not fresh, negative)", indicating that "steak" received negative evaluations in terms of both taste and freshness. This task helps to gain a deeper understanding of fine-grained sentiment bias and has important application value in scenarios such as public opinion analysis and product review mining.

[0019] In related technologies, Large Language Models (LLMs) can be used to perform the ASTE task. LLMs are trained on massive amounts of text data and possess a large parameter scale, enabling them to deeply understand the complex structure and semantic relationships of language. These models have demonstrated powerful capabilities in various natural language processing tasks, showing great potential, especially in tasks like ASTE that require sophisticated semantic analysis. Advanced LLMs, such as LLaMA, can not only capture the surface connections between words but also understand the implicit logic and sentiment in the context, thus more accurately identifying aspect words, corresponding opinion words, and their sentiment polarity. Therefore, using LLMs to perform the ASTE task helps improve the overall performance of the model, particularly when handling complex sentence structures or obscure expressions.

[0020] However, existing aspect-level sentiment triple extraction methods often suffer from poor recognition accuracy. The root cause lies in the inherent flaws of traditional model architectures and training mechanisms. Specifically, traditional large language models employ a full-parameter fine-tuning strategy, requiring the simultaneous optimization of billions of parameters. This process demands extremely high computational resources, and gradient updates are susceptible to data fluctuations, causing oscillations in the optimization path and leading to instability in parameter training. Furthermore, overfitting can occur, resulting in decreased model performance and limited recognition accuracy.

[0021] Based on this, embodiments of this application provide a method and related apparatus for extracting aspect-level emotion triplet information to solve the above problems.

[0022] Figure 2 For a block diagram of the electronic device provided in the embodiments of this application, please refer to [link / reference]. Figure 2 The electronic device includes a memory, a processor, and a communication module. These components are electrically connected directly or indirectly to enable data transmission or interaction. For example, they can be electrically connected via one or more communication buses or signal lines.

[0023] Memory is used to store computer programs or data that can be executed by a processor. Memory can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc.

[0024] The processor is used to read / write data or computer programs stored in the memory and execute the computer program to implement the aspect-level sentiment triplet information extraction method provided in the embodiments of this application.

[0025] The communication module is used to establish communication connections between electronic devices and other communication terminals via a network, and to send and receive data via the network.

[0026] It should be understood that, Figure 2 The structure shown is only a schematic diagram of an electronic device; the electronic device may also include components that are larger than those shown. Figure 2 The more or fewer components shown, or having the same Figure 2 The different configurations shown. Figure 2 The components shown can be implemented using hardware, software, or a combination thereof.

[0027] The following is based on the above. Figure 2 The electronic device in this application serves as the execution entity. The method for extracting aspect-level emotion triplet information provided in this application is illustrated with a flowchart. Specifically, Figure 3 A flowchart illustrating the aspect-level sentiment triplet information extraction method provided in this application embodiment is available. Figure 3 The method includes: Step S20: Obtain the statement to be analyzed.

[0028] Step S21: Input the statement to be analyzed into the pre-trained information extraction model for processing to obtain the triple information of the statement to be analyzed; the triple information includes aspect words, opinion words and sentiment polarity of opinion words in the statement to be analyzed; the information extraction model includes multiple Transformer layers, and the information extraction model is obtained by setting low-rank matrices for each Transformer layer and updating the parameters of the low-rank matrices corresponding to multiple Transformer layers during training.

[0029] Optionally, the statement to be analyzed is natural language text containing one or more word sequences, which may take the form of user comments, product descriptions, or other sentences with emotional expression.

[0030] In this embodiment, the statement to be analyzed can be fed into a pre-trained information extraction model as model input for processing, thereby obtaining the triplet information of the statement to be analyzed.

[0031] Optionally, the triplet information includes aspect words, opinion words, and their corresponding sentiment polarities in the statement to be analyzed.

[0032] In one possible implementation, the information extraction model could be a Transformer-based LLM model, whose structure consists of multiple cascaded Transformer layers, each composed of a multi-head attention (MHA) mechanism and a feed-forward network (FFN) for learning deep semantic representations of the input statement.

[0033] In this embodiment, the information extraction model is obtained through adaptive low-rank fine-tuning. That is, during the model training process, the original pre-trained weight matrix is ​​not directly adjusted, but a low-rank matrix structure is introduced for the weight matrix in each Transformer layer, and the parameters are updated for the low-rank matrix.

[0034] It should be noted that the low-rank matrix refers to the incremental update matrix used for efficient parameter fine-tuning. Its function is to perform lightweight adjustments to the original weight matrix in the pre-trained language model. Specifically, during the training of the information extraction model, instead of directly updating the complete pre-trained weight matrix, an incremental matrix in the form of a low-rank decomposition is introduced, thereby significantly reducing the number of trainable parameters. The aspect-level sentiment triple information extraction method provided in this application's embodiments inputs the sentence to be analyzed into a pre-trained information extraction model for processing, thereby obtaining the triple information of the sentence to be analyzed. The information extraction model includes multiple Transformer layers, and this model obtains the information by setting a low-rank matrix for each Transformer layer and updating the parameters only on the low-rank matrix corresponding to each Transformer layer during training. Because the training process of the information extraction model only updates the parameters in the smaller low-rank matrix rather than the entire original weight matrix, the degrees of freedom involved in each update can be limited, avoiding optimization fluctuations caused by large-scale synchronous parameter changes. This reduces the risk of gradient oscillations during training, ensures the stability of model convergence, and effectively alleviates overfitting. The model can better learn common language patterns and semantic rules, improve recognition accuracy, and enable the information extraction model to achieve good generalization performance. In practical applications, this improves the accuracy and robustness of triple information extraction.

[0035] Next, we will provide a possible implementation method for generating the information extraction model. It should be noted that the aspect-level sentiment triple extraction task still faces the problems of scarce training data and high annotation costs in practical applications. Traditional methods rely on manual annotation of a large number of domain-specific statements and their corresponding triple information (including aspect words, opinion words, and sentiment polarity). This process is not only time-consuming and labor-intensive, but also requires a high level of professional knowledge from the annotators, resulting in the difficulty, long cycle, and high cost of obtaining high-quality training data.

[0036] In addition, existing models often employ complex structural designs or rule systems to model the relationships between triples, such as methods based on span analysis, syntactic dependency trees, or sequence labeling. These methods often require manual design of features or decoding logic for different task scenarios, resulting in complex model rule design and limited generalization ability.

[0037] The root cause of these problems lies in the lack of sufficiently large and semantically rich training samples to support effective learning by large language models. Simultaneously, the models lack interpretable and guided reasoning mechanisms to simplify the solution path for complex tasks. Since end-to-end training directly on raw text is susceptible to noise interference and struggles to capture implicit logical reasoning chains, model performance improvement is limited. Furthermore, increasing structural complexity is necessary to compensate for insufficient reasoning ability, further exacerbating the burden of rule design.

[0038] To address the aforementioned issues, the aspect-level sentiment triplet information extraction method provided in this application can generate target sample data through data augmentation methods and construct thought chains to regulate and guide the internal reasoning behavior of the model.

[0039] Specifically, in Figure 3 On this basis, Figure 4 For another flowchart illustrating the aspect-level sentiment triplet information extraction method provided in this application embodiment, please refer to [link to flowchart illustration]. Figure 4 The information extraction model can be obtained through the following steps: Step S10: Generate target sample data based on preset task prompts, and construct training samples using the target sample data, preset thought chains, and preset sentence sets.

[0040] Among them, the thought chain is used to train the reasoning path of the information extraction model when extracting information from triples.

[0041] In this embodiment, the electronic device can generate high-quality target sample data through task prompts, and combine this data with thought chains and a preset set of statements to construct a complete training sample for training the information extraction model. The preset task prompts refer to natural language instructions in a specific format, which perform the dual functions of generation and filtering during the target sample data generation process. Their purpose is to guide the large language model to generate high-quality sentiment triplet-related statements and their corresponding tags that conform to the semantic distribution of a specific domain.

[0042] Understandably, the target sample data refers to high-quality sample data generated and processed by a large language model based on task prompts. This sample data retains high authenticity, high lexical diversity, and good syntactic structure.

[0043] In this embodiment, the task prompt can be input into a large language model in Prompt form to perform the corresponding operation. In one possible implementation, the large language model can be GPT-4.

[0044] In this embodiment, the thought chain refers to a pre-designed structured reasoning guidance mechanism comprising three parts: instructions, principles, and examples. Its function is to explicitly model the logical reasoning process in the triple extraction task. The instructions explicitly inform the model of the task to be performed: extracting triples composed of aspect words, opinion words, and sentiment polarities from the input text. The principles systematically explain the reasoning logic required to complete the task, including how to identify implicit aspect words not explicitly mentioned, how to establish the association between opinion words and corresponding aspect words, and how to determine the sentiment category based on the context. The examples provide input and output samples with complete reasoning processes, enabling the model to learn the specific methods for progressively deriving the final triple set from the original statements.

[0045] In one possible implementation, the thought chain can be in the form of a Prompt. In one example, Figure 5 For a diagram of the thought process, please refer to [link / reference]. Figure 5 It consists of three parts: Instruction, Rationale, and Examples.

[0046] In practical applications, a pre-defined thought process chain, target sample data, and a pre-defined set of statements jointly contribute to the construction of training samples. The thought process chain provides the reasoning path specification, the target sample data provides high-quality supervision signals, and the set of statements provides the contextual input basis. These three elements are integrated into a unified input format through sequence concatenation. In one possible implementation, this set of statements can be a pre-defined ASTE task dataset.

[0047] In addition, the training sample may also include a set of statements and triplet labels corresponding to the target sample data.

[0048] Specifically, electronic devices can provide ASTE mind chain prompts. With augmented data The set of statements x is concatenated into a joint input sequence. Here, `concat` represents the sequence concatenation operation. This joint input sequence can be used as a training sample input into the initial information extraction model, enabling the model to internalize step-by-step reasoning behavior guided by thought chains while learning triplet annotations.

[0049] Understandably, the above steps generate target sample data through preset task prompts. This allows the large language model to automatically generate sentences that conform to specific domain and grammatical norms, along with their corresponding real triplet labels, without human intervention, thus significantly reducing the cost of data collection and annotation. Simultaneously, the introduction of a preset thought chain mechanism aims to construct an explicit, step-by-step reasoning path for the model. This enables it to mimic human thought processes when handling triplet extraction tasks, progressively completing sub-tasks such as aspect word recognition, opinion word matching, and sentiment polarity judgment, rather than relying on complex external rules or highly customized network structures, thereby reducing the overall model complexity.

[0050] Then, the training samples are input into the pre-built initial information extraction model for training, and the model is adaptively fine-tuned in low rank during the training process.

[0051] Step S11: Input the training samples into the pre-built initial information extraction model for training. This initial information extraction model includes the low-rank matrices corresponding to each Transformer layer.

[0052] Step S12: Based on the importance of each Transformer layer in the initial information extraction model, allocate computing resources to the low-rank matrix of each Transformer layer to update the parameters of the low-rank matrix of each Transformer layer and obtain the information extraction model.

[0053] Optionally, the initial information extraction model refers to the pre-trained large language model used before fine-tuning. This initial information extraction model consists of multiple stacked Transformer layers, each containing two core modules: a multi-head attention mechanism (MHA) and a feedforward neural network (FFN).

[0054] It should be noted that this model has not been specifically optimized for aspect-level sentiment triple extraction tasks; its parameters are mainly used to represent general language knowledge. In this embodiment, the initial information extraction model serves as the basic framework for efficient parameter fine-tuning. After receiving training samples generated from task prompts, it is incrementally adjusted by introducing low-rank matrices and other methods, thereby gradually evolving into a target information extraction model capable of accurately performing triple information extraction tasks.

[0055] During the above steps, the electronic device can input the training samples constructed from the target sample data, thought chain prompts, and sentence set as input sequences into the initial information extraction model, triggering the forward propagation and parameter optimization process. During training, a low-rank matrix is ​​introduced into each Transformer layer. It should be noted that the structure of this low-rank matrix can be defined beforehand when constructing the initial information extraction model, so that it can directly participate in subsequent training.

[0056] Understandably, when fine-tuning the initial information extraction model in low rank, the roles of each Transformer layer in capturing the semantic structure of aspect-level sentiment triples differ, and therefore the expressive power required by their corresponding low-rank matrices also differs.

[0057] During the above steps, the same number of parameters or computational resources are not allocated to the low-rank matrices of all Transformer layers. Instead, the allowed parameter size is dynamically adjusted according to the relative importance of each layer to the overall task performance.

[0058] Specifically, during training iterations, the electronic device can assess the importance of each Transformer layer by analyzing the impact of weight changes on the loss function or other quantifiable metrics. Based on this importance score, the system determines the rank (i.e., parameter capacity) of the corresponding low-rank matrix. For layers with higher importance, the system allocates more computational resources, allowing their low-rank matrices to retain more significant singular value directions to enhance expressive power; conversely, for layers with lower importance, resource allocation is reduced accordingly, limiting the magnitude of parameter updates. It should be noted that this resource allocation process must be conducted within a pre-defined total parameter budget constraint to ensure the efficiency and feasibility of the entire fine-tuning process.

[0059] Therefore, by implementing a differentiated resource allocation strategy for the low-rank matrices of each Transformer layer, it is possible to concentrate the limited trainable parameters on the network layers that contribute more to the task while controlling the overall computational cost, thereby improving the final convergence effect of the model and obtaining an information extraction model with stronger triple extraction capabilities.

[0060] The following provides a possible implementation method for generating target sample data based on preset task prompts. In this embodiment, the task prompts may include data generation prompts and quality assessment prompts. The electronic device can input the data generation prompts into a large language model for processing to obtain multiple sample data. Then, the quality assessment prompts and multiple sample data are input into the large language model for processing, each sample data is scored, and multiple target sample data are selected from the multiple sample data based on the scores of each sample data.

[0061] In this embodiment, the electronic device can utilize a large language model to automatically generate and filter high-quality aspect sentiment triple training data through task prompts, thereby improving the training effect of the subsequent information extraction model.

[0062] Specifically, electronic devices can first input data-generated prompts into a large language model for processing. These prompts are designed to include control instructions such as domain-specific parameters, sentence length constraints, triplet quantity requirements, and semantic category ranges for aspect words and opinion words. This guides the large language model to generate natural language comment statements that conform to the characteristics of the target domain and their corresponding aspect sentiment triplet annotation information, thereby obtaining multiple sample data.

[0063] In one example Figure 6 For an illustration of generating prompt words for the data, please refer to [link / reference]. Figure 6 ,<domain n> This indicates the domain for which the augmented dataset was generated; the specific domain settings are based on the target test dataset. <n1> 、 <n2> 、 <n3>These represent the number of words in the sentence, the number of triples, and the number of sentences generated, respectively.<aspect n> and<opinion n> These respectively represent the domain of the aspect term and the domain of the opinion term.

[0064] In this example, by adjusting the domain identifier (Domain n), sentence length (n1), number of triples (n2), and total number of generated sentences (n3) in the prompt words, as well as the specific semantic categories of the aspect words (Aspect n) and opinion words (opinion n), the theme focus and diversity control of the generated content can be achieved, ensuring that the generated sample data is both close to the real user comment scenario and has sufficient semantic coverage.

[0065] In this embodiment, the electronic device can also input the quality assessment prompts and the aforementioned multiple sample data into the same or another large language model for processing, so as to perform quality assessment and scoring on the sample data.

[0066] In one possible implementation method Figure 7 For an example image of quality assessment prompts, please see [link / image]. Figure 7 The quality assessment prompts embed clear scoring dimension definitions and scoring rules, specifically automating the scoring of each sample data from three dimensions: authenticity, lexical richness, and syntactic structure. Authenticity measures the reasonableness and credibility of the generated sentences in real-world scenarios; lexical richness reflects the diversity of vocabulary and the complexity of expression; and syntactic structure assesses the grammatical completeness and structural complexity of sentences. The score for each dimension is set to a two-decimal-point floating-point number between 1 and 10, with higher scores indicating higher data quality in that dimension.

[0067] Based on the above scoring mechanism, the electronic device can output three-dimensional scores for each sample data and calculate their arithmetic mean as the comprehensive quality score of the sample data. This comprehensive quality score represents a multi-dimensional quantitative evaluation of the sample data. On this basis, the electronic device can objectively identify high-quality samples that are semantically reasonable, linguistically natural, and structurally rich based on the comprehensive quality score of the sample data.

[0068] Specifically, the electronic device can sort and filter samples based on the overall quality score of each sample data, retaining the sample data with scores higher than the preset threshold or ranked higher as target sample data for subsequent construction of training sample sets.

[0069] It should be noted that this screening process aims to filter out low-quality, semantically contradictory, or structurally anomalous generated results, ensuring that the data entering the model training phase has a high signal-to-noise ratio and representativeness. This can be understood as achieving task-specific domain-adaptive data synthesis and automatic optimization by introducing a dual mechanism of data generation prompts and quality assessment prompts. This effectively alleviates the problems of high manual annotation costs and insufficient coverage of public datasets, providing a reliable data foundation for the efficient training of information extraction models.

[0070] Afterwards, the electronic device can concatenate the target sample data, thought chain, and preset set of sentences to obtain training samples, and input the training samples into a pre-built initial information extraction model for training.

[0071] The initial information extraction model includes several Transformer layers, each containing multiple pre-trained weight matrices. Based on this, the low-rank matrix corresponding to each Transformer layer can be constructed during model structure definition through the following steps: For each Transformer layer, the LoRA method is used to generate the corresponding increment matrix for each pre-trained weight matrix in the Transformer layer, and the singular value decomposition is performed on the increment matrix to obtain the low-rank matrix corresponding to each pre-trained weight matrix.

[0072] In this embodiment, the initial information extraction model includes multiple Transformer layers. Each Transformer layer consists of a multi-head attention (MHA) module and a feedforward neural network (FFN) module. The MHA module executes the attention function of h heads. The attention function of each head consists of a query projection matrix, a key projection matrix, a value projection matrix, and an output projection matrix, as shown in the following formula:

[0073]

[0074] in, Characterize the training samples, and , Characterize the output projection matrix. Characterizing the attention function, Representing the i-th head The query projection matrix, Representing the i-th head The key projection matrix, Representing the i-th head The value of the projection matrix, and , Let n represent the real number field, n represent the number of words in the training samples, and d represent the embedding dimension of each word. The dimension representing each attention head, and Set as .

[0075] Furthermore, the FFN module consists of two linear transformation matrices, as shown in the following formula:

[0076] in, Characterize the activation function, and It is a linear transformation matrix, and , ,in, The intermediate dimension representing the interior of FFN. and Characterize the bias vector.

[0077] In this embodiment, the query projection matrix, key projection matrix, value projection matrix, and output projection matrix in the MHA module, along with the two linear transformation matrices in the FFN module, together constitute multiple pre-trained weight matrices for this layer.

[0078] During the above steps, for each Transformer layer, the LoRA method is used to generate corresponding incremental matrices for each pre-trained weight matrix in the Transformer layer, that is, without changing the original pre-trained weight matrix. Under the premise of introducing a trainable low-rank increment matrix This enables efficient fine-tuning of parameters.

[0079] The increment matrix The low-rank decomposition form is represented as the product of two small-dimensional matrices, BA, where and , and rank ,Right now, ,in, It is the final actual weight (i.e., runtime equivalent weight) of a certain weight matrix in a certain layer of the model after low-rank fine-tuning. Since only the parameters in A and B need to be trained, the number of trainable parameters and memory usage are significantly reduced.

[0080] However, in the LoRA method, the incremental matrix of all pre-trained weight matrices All layers are pre-assigned the same rank r, ignoring the differences in the contribution of different layers or modules to the final performance. This uniform rank allocation method may result in important layers receiving insufficient representational power, while unimportant layers consume too many resources, thus causing fluctuations and instability in parameter updates during training, affecting model convergence and final performance.

[0081] Based on this, to improve the stability of model training, the incremental matrix can be further decomposed into singular value decompositions, and then further parameterized as follows: This allows us to obtain the low-rank matrix corresponding to the pre-trained weight matrix, in the form of:

[0082] Among them, the incremental matrix It is parameterized into three components: the left singular vector matrix. Right singular vector matrix and diagonal matrices containing singular values Its diagonal elements are And the rank r is much smaller than the dimension. and The minimum value. Each singular value and its corresponding left and right vectors form a triplet. , is used to represent parameter changes in a single semantic or functional direction.

[0083] It should be noted that this process extends LoRA's original fixed-rank incremental structure to an explicit low-rank representation based on singular value decomposition, enabling incremental updates to capture principal component features along the weight change direction. This allows the model to more finely control the learning dynamics in different directions during fine-tuning, thereby improving training stability.

[0084] In the initial state, P and Q are set to zero matrices and initialized with random Gaussian to ensure that the increment is zero in the initial training phase, thus avoiding the destruction of pre-trained knowledge. Furthermore, to maintain the orthogonality of P and Q (i.e., satisfying...),... The paper introduces a regularization term R(P,Q), defined as follows:

[0085] This regularizer can prevent numerical instability during the optimization process by constraining the orthogonality of singular vectors, thereby further enhancing the robustness and convergence of the fine-tuning process.

[0086] Therefore, by performing singular value decomposition on the incremental matrix, not only is the LoRA method's advantage in terms of computational resource consumption preserved, but the directionality and structure of parameter updates are also enhanced, laying the foundation for the subsequent dynamic allocation of low-rank dimensions based on the importance of each layer.

[0087] Next, we will discuss a possible implementation method for how to allocate computational resources to the low-rank matrices of each Transformer layer based on the importance of each Transformer layer in the initial information extraction model, so as to update the parameters of the low-rank matrices of each Transformer layer and obtain the information extraction model.

[0088] In this embodiment, the electronic device can use the current iteration training as the target iteration. For each Transformer layer, it calculates the target left singular value vector, target right singular value vector, and gradient update value of the target singular value of each target low-rank matrix in the Transformer layer under the target iteration. The target singular value of the target low-rank matrix is ​​obtained through the previous iteration training. Based on the trainable parameters of each target low-rank matrix in each direction and the preset exponential moving average coefficient, the importance score of each target low-rank matrix in each direction is calculated. For each target low-rank matrix, the target singular value is updated based on the importance score of all target low-rank matrices in each direction, the gradient update value of the target singular value, and the remaining resource budget, so as to allocate resources to the target low-rank matrix and obtain the new target singular value for the next iteration. The remaining resource budget is updated according to the preset resource budget update method, and the next iteration training is performed. The next iteration training is used as the target iteration, and the above steps are repeated until the remaining resource budget reaches the preset allocation completion condition. Finally, when the initial information extraction model reaches the preset model convergence condition, the trained information extraction model is obtained.

[0089] In this embodiment, the electronic device can perform dynamic resource allocation and iterative optimization for the target low-rank matrix in each Transformer layer during the model training process.

[0090] During the execution of the above steps, for the current iteration training phase, the electronic device can calculate the target value of the target low-rank matrix generated in each Transformer layer in the current iteration training phase, namely the target left singular value vector, the target right singular value vector, and the gradient update value of the target singular value, as the input basis for this iteration update.

[0091] It should be noted that the target singular values ​​of the target low-rank matrix in the current iteration of training are obtained after the previous iteration of training. This is understandable, as each Transformer layer has multiple pre-trained weight matrices, thus each Transformer layer includes multiple target low-rank matrices.

[0092] Based on this, the electronic device can further calculate the importance score in each direction according to the trainable parameters of each target low-rank matrix in different directions and the preset exponential moving average coefficient, so as to obtain the key indicators that measure the contribution of different singular value components to the model performance. Then, for each target low-rank matrix, the electronic device can use the importance score of each target low-rank matrix in each direction, the gradient update value of the target singular value, and the current remaining resource budget to perform the target singular value update operation in the target low-rank matrix, thereby obtaining the target singular value in the next iteration training phase.

[0093] This process achieves efficient allocation of resources with limited computing power by pruning unimportant singular values ​​and prioritizing the retention of highly important components.

[0094] Optionally, the remaining resource budget refers to the dynamic control variable used to constrain the total number of updatable parameters in the low-rank matrix of each Transformer layer during the training of the information extraction model. It should be noted that this remaining resource budget is not fixed but is gradually adjusted during the training iterations to achieve efficient allocation of computing resources and synergistic optimization of model performance.

[0095] During the above steps, the initial remaining resource budget (hereinafter referred to as the initial resource budget) is set slightly higher than the target budget to ensure sufficient parameter adjustment space in the early stage of training, thereby avoiding the model from getting trapped in a local optimum too early due to limited resources.

[0096] Specifically, the initial resource budget is initialized based on the total number of singular values ​​in the low-rank matrices. The initial rank of each low-rank matrix is ​​determined by dividing the initial resource budget by the total number of weight matrices, i.e., the initial rank r is... ,in, Characterizes the initial resource budget, The total number of weight matrices represents the total number of layers, ensuring that each layer receives a balanced resource allocation at the beginning of training.

[0097] As training progresses, the initial resource budget is gradually reduced according to a preset decreasing strategy, gradually converging from the initial budget to the target budget. This process is uniformly controlled by the global budget scheduler to ensure that the total number of parameter updates during the entire training process does not exceed the predetermined upper limit.

[0098] Understandably, the remaining resource budget can be updated by the global budget scheduler according to a preset resource budget update method. In one possible implementation, the resource budget update method can be set according to the actual application, such as linear scheduling (resource budget decreases linearly with the number of training steps) or exponential scheduling (resource budget decreases exponentially with the number of training steps, rapidly reducing the budget in the early stage and slowly approaching the target in the later stage).

[0099] Therefore, the aspect-level sentiment triplet information extraction method provided in this application can improve fine-tuning efficiency and the final model triplet extraction accuracy by dynamically adjusting the remaining resource budget, ensuring the model convergence stability, and prioritizing the allocation of limited computing resources to the Transformer layer and its corresponding low-rank matrix, which are more critical to the model performance.

[0100] Based on this, the entire training process can proceed to the next iteration cycle, resetting the next iteration as the target iteration and repeating the steps of gradient calculation, importance assessment, singular value update, and budget adjustment. This loop continues until the remaining resource budget reaches the preset allocation completion condition, at which point resource allocation can be stopped. When the initial information extraction model reaches the preset model convergence condition, training can be considered complete, thus obtaining a fully optimized information extraction model.

[0101] In this embodiment, the allocation completion condition can be that the remaining resource budget converges to the target budget, which can be set in advance according to the actual situation. The model convergence condition can be that the loss function is stable or the maximum number of iterations has been reached, etc.

[0102] In one possible implementation, for each Transformer layer under the target iteration, the electronic device can calculate the target left singular value vector and the target right singular value vector of each target low-rank matrix using the stochastic gradient method. The training cost of the Transformer layer is calculated based on each target low-rank matrix in the Transformer layer. Based on the training cost of the Transformer layer and the target left singular value vector and the target right singular value vector of each target low-rank matrix, the loss function is calculated. Based on the loss function, the preset learning rate, and the target singular values, the gradient update value of the target singular values ​​is calculated.

[0103] In this embodiment, the electronic device can first update the target left singular value vector and the target right singular value vector using a stochastic gradient method.

[0104] In one possible implementation, at each training iteration step, the pre-trained weight matrix (including query, key, value, and linear transformation matrices in the feedforward network) in each Transformer layer is first subjected to singular value decomposition to obtain a low-rank matrix, which is then represented as a left singular vector matrix. Singular value diagonal matrix and right singular vector matrix The product form, i.e. ,in, Let represent the low-rank matrix corresponding to the k-th pre-trained weight matrix, and Each triplet The importance of a combination of parameters on a singular direction i of a low-rank matrix is ​​determined by an importance score. Conduct a quantitative assessment.

[0105] The training cost of the model during the above steps The parameter set includes all left singular vectors. Singular values and right singular vectors This training cost reflects the overall cost to model performance under the current parameter configuration.

[0106] To prevent instability during parameter updates, a regularizer also needs to be introduced. And set the regularization coefficient. Based on this, the final loss function can be obtained using the following formula:

[0107] In the t-th iteration, the stochastic gradient method can be used to update the left and right singular vectors, thereby obtaining the target's left singular value vector under the current iteration. and the target right singular value vector .

[0108] Based on this, the gradient update value of the target singular value can be calculated according to the loss function, learning rate, and the target singular value obtained in the previous iteration.

[0109] Specifically, given a learning rate The current target singular value gradient update value It can be obtained through the following formula:

[0110] in, Characterizing the singular values ​​of the overall loss function L with respect to the low-rank matrix of the k-th target. The partial derivative of , which reflects the direction and strength of the influence of each singular value on the total loss of the model under the current parameter configuration.

[0111] The following provides a possible implementation method for calculating the importance score of each target low-rank matrix in each direction based on the trainable parameters of each target low-rank matrix in each direction and the preset exponential moving average coefficient.

[0112] In this embodiment, the electronic device can, for each target low-rank matrix, calculate the average smoothing sensitivity and uncertainty term corresponding to each trainable parameter of the target low-rank matrix in each direction based on each trainable parameter of the target low-rank matrix in each direction and a preset exponential moving average coefficient; calculate the sensitivity corresponding to each trainable parameter of the target low-rank matrix in each direction based on the average smoothing sensitivity and uncertainty term corresponding to each trainable parameter of the target low-rank matrix in each direction; the sensitivity includes the target left singular value vector sensitivity, the target right singular value vector sensitivity, and the target singular value sensitivity of the trainable parameters; and calculate the importance score of the target low-rank matrix in each direction based on the target left singular value vector sensitivity, the target right singular value vector sensitivity, and the target singular value sensitivity corresponding to each trainable parameter of the target low-rank matrix in each direction.

[0113] In this embodiment, in order to achieve dynamic perception and resource allocation of the importance of each parameter during the low-rank fine-tuning process, a sensitivity smoothing and uncertainty quantification mechanism is introduced.

[0114] Specifically, for each trainable parameter (Including the left singular vector, right singular vector, and elements in the singular values), its importance is assessed not based on the instantaneous gradient, but through two auxiliary variables. and Modeling is performed. Among them, This represents the sensitivity of the parameter after exponential moving average smoothing at the t-th iteration, i.e., the average smoothing sensitivity, which can be calculated using the following formula:

[0115] in, The coefficient is the preset exponential moving average, and , This characterizes the sensitivity of the parameter to the exponential moving average at the (t-1)th iteration.

[0116] also, The uncertainty term, defined as the local variation of the parameter at the t-th iteration, can be calculated using the following formula:

[0117] in, The coefficient is the preset exponential moving average, and , This characterizes the uncertainty term caused by local changes in the parameter during the (t-1)th iteration. Characterizes smoothness sensitivity.

[0118] In this embodiment, the average smoothing sensitivity and the uncertainty term can together constitute the final sensitivity. It can be calculated using the following formula:

[0119] Understandably, this product form integrates the long-term significance of the parameters with the uncertainty of their changes, forming a joint measure of the dynamic behavior of the parameters. The sensitivity here covers the evaluation results of three types of trainable parameters: the left singular value vector of the target, the right singular value vector of the target, and the target singular value. Each type of parameter independently undergoes the above-mentioned smoothing and uncertainty quantification process to ensure that the importance of different structural components can be characterized separately.

[0120] Based on this, the importance score of the target low-rank matrix in each direction is jointly determined by the sensitivity of all relevant trainable parameters in that direction. That is, the overall importance score of the i-th singular direction in the k-th low-rank matrix at the t-th iteration. It is obtained by weighted summation of three parts: first, the singular values ​​in that direction. sensitivity Second, left singular vectors The average sensitivity of all elements is denoted as . Thirdly, right singular vectors The average sensitivity of all elements is denoted as . . That is, It can be calculated using the following formula:

[0121] in, and Let represent the dimensions of the left and right singular vector matrices involved in the singular value decomposition of the k-th low-rank matrix, respectively. The left singular vector matrix representing the k-th weight matrix module the number of rows, The right singular vector matrix representing the k-th weight matrix module The number of columns.

[0122] Therefore, this calculation mechanism can comprehensively reflect the combined contribution of each singular direction in the low-rank decomposition space, avoiding reliance on a single parameter or static rules for resource allocation. In essence, this method, by introducing time-series smoothing and uncertainty modeling, achieves a refined and dynamic assessment of the importance of each direction during low-rank fine-tuning, providing a reliable basis for subsequent on-demand allocation of computational resources.

[0123] In this embodiment, at the t-th iteration, the electronic device can calculate the overall importance score of each direction in each low-rank matrix. Then, the importance scores of each singular direction of all weight matrices are formed. ,and ,in This represents the overall contribution of the i-th singular triplet in the k-th low-rank matrix. This score integrates the trainable parameter sensitivity of the corresponding singular value, the left singular vector column, and the right singular vector row. Simultaneously, the currently available remaining resource budget... It is used to constrain the total number of singular values ​​that can be retained in this instance.

[0124] Specifically, in the (t+1)th iteration, the target singular value in the k-th low-rank matrix It can be obtained through the following formula:

[0125] In the above process, all singular directions are first scored according to their importance. Perform a global descending sort and base it on the current remaining resource budget. Determine the retention threshold: Only allow top-ranked entries. The singular values ​​of the named matrix continue to participate in subsequent optimizations. For each low-rank matrix, its updated singular value matrix... Through threshold function The function generates the i-th singular value and operates as follows: if the i-th singular value corresponds to an importance score... Ranked first in the global sort The name is then updated to its gradient value. Retained in If the singular value is zero, then it is discarded; otherwise, the singular value is set to zero.

[0126] Therefore, by guiding computational resources towards the most influential parameters through importance scoring, high-contribution, low-rank matrices acquire more effective rank (i.e., the number of non-zero singular values), while low-contribution directions are compressed to free up budget. Thus, this process achieves adaptive reallocation of fine-tuning resources under limited total parameters, improving model performance while ensuring the stability and efficiency of the training process.

[0127] Next, in order to further illustrate the effectiveness of the aspect-level sentiment triplet information extraction method provided in this application embodiment, the above information extraction model was evaluated through experiments in this embodiment.

[0128] In this embodiment, to comprehensively evaluate the performance of the information extraction model in the aspect-level sentiment triple extraction task, the ASTE-Data-V2 dataset is selected as the benchmark test platform, and the statistical information of the ASTE-Data-V2 dataset is shown in Table 1.

[0129] Table 1

[0130] This dataset is a classic publicly available dataset widely used in the field of sentiment analysis, possessing both good representativeness and challenge. The dataset consists of four subsets: 14R, 15R, 16R, and 14L. 14R, 15R, and 16R belong to the restaurant review domain, covering user evaluations of dining services, food quality, etc.; 14L belongs to the laptop product domain, containing user feedback on hardware performance, design, and user experience, thus ensuring that the model's generalization ability under different semantic scenarios is fully validated.

[0131] The statistical information of the aforementioned dataset is characterized by the number of sentences (S) and the distribution of various sentiment polarity labels, where POS represents positive sentiment, NEG represents negative sentiment, and NEU represents neutral sentiment, corresponding to the sentiment tendency expressed by opinion words and their frequency of occurrence in the dataset, respectively. Experiments conducted on these finely labeled subsets of data allow for precise measurement of the model's ability to recognize complex language structures and diverse sentiment expressions.

[0132] In this embodiment, to improve the domain coverage and semantic representativeness of the generated augmented data, the original labeled data was integrated before model training. Specifically, the training and validation sets of multiple subsets (including 14R, 15R, and 16R) belonging to the restaurant domain in the ASTE-Data-V2 dataset were merged to obtain high-quality restaurant domain data containing 2945 labeled aspect sentiment triples. At the same time, the training and validation sets of the 14L subset of the laptop domain were merged to construct a laptop domain dataset containing 1576 triples.

[0133] After the above two parts of real labeled data are standardized in format, they can be used as a reliable source of knowledge (i.e., a pre-set set of sentences) to supplement the augmented dataset, and then used to form a mixed training sample together with the target sample data and thought chain generated by the large language model.

[0134] For the evaluation metrics, to assess the performance of the information extraction model, precision (P), recall (R), and F1 score (F_1) are used. The calculation methods for these metrics are as follows:

[0135] Where TP represents the number of correctly labeled sentiment triples, FP represents the number of incorrectly labeled triples, and FN represents the number of unidentified triples. A triple is considered correct (TP) only when the output triple label exactly matches the original label.

[0136] To ensure the stability and efficiency of the model training process, the experimental environment was built based on high-performance computing hardware. Specifically, the operating system used was Ubuntu 22.04 LTS, the deep learning framework was PyTorch 2.0.1, and the computing device was a single NVIDIA RTX 4090 graphics card equipped with 128GB of system memory, meeting the requirements of large-scale language model fine-tuning for GPU and system memory resources.

[0137] On this hardware platform, based on a large language model with a Transformer decoder architecture, the LLaMA3 model with 32.5 billion parameters (32.5B) from the LLaMA series was selected as the base model for fine-tuning and training. This model maintains powerful language understanding capabilities while also considering feasibility on consumer-grade graphics cards, enabling efficient parameter updates and inference execution under limited hardware conditions.

[0138] During the training process described above, the initial LoRA rank was set to 32, the target compression rank was set to 16, and the regularization coefficient was set to 0.1; the exponential moving average coefficient was also set to... and All values ​​are set to 0.85 to smooth out the historical accumulation of sensitivity and uncertainty terms, thereby improving the stability of importance assessment. The initial learning rate is set to 5e-5, which, combined with an adaptive scheduling mechanism, ensures convergence speed while avoiding training oscillations.

[0139] For attribute design, in order to effectively generate high-quality domain-specific target sample data and improve the model's performance in aspect-level sentiment triple extraction tasks, based on the above dataset, two core domains were set for the target application scenarios: laptops and restaurants.

[0140] Specifically, in the laptop computer sector, 21 aspects (such as hardware, memory, keyboard, battery, operating system, etc.) were systematically defined, taking into account the technical attributes of this type of product and user concerns, covering the main functional modules of the product; at the same time, 7 opinion areas (such as quality, performance, design, price, support, etc.) were set to describe users' subjective evaluation tendencies on each aspect, as shown in Table 2.

[0141] Table 2

[0142] In the restaurant sector, six areas were defined based on the characteristics of the dining consumption scenario: food, service, price, atmosphere, cleanliness, and location. Correspondingly, six opinion areas were set, such as taste, quality, attitude, value for money, comfort, and style, to cover common customer feedback dimensions, as shown in Table 3.

[0143] Table 3

[0144] The aforementioned attribute system, serving as a semantic constraint framework for data generation, is embedded in task prompts, guiding the large language model to focus on real-world product dimensions and their typical evaluation expressions when generating comment statements.

[0145] To comprehensively validate the effectiveness and advancement of the proposed information extraction model in the aspect-level sentiment triple extraction task, its performance is compared with five representative state-of-the-art baseline models in this field in recent years, covering DS (Dual-Channel Span), DSP (Dual Sequential Prompting), STD (Span-based Segment Tagging and Dual Extractors), TAG (Tagging-Assisted Generation model with Encoder and Decoder Supervision), and DTF (Dual Table Filling with Knowledge Complementation). These baseline models implement triple extraction based on different technical paths, reflecting the performance boundaries of current mainstream methods from multiple dimensions.

[0146] Specifically, the DS model identifies the association between aspect terms and opinion terms by parsing the dependency syntax tree of the input text and utilizing the grammatical dependencies between words; the DSP model introduces a dual learning mechanism to improve the accuracy of collaborative extraction by jointly modeling the feedback relationship between sentiment polarity and opinion expression; the STD model adopts a segmentation labeling strategy and designs a dual extractor structure, which is used to locate aspect terms and extract the corresponding sentiment words respectively; the TAG model integrates sequence labeling information during the encoding process and assists in triple generation by enhancing the representation of the encoder's hidden state; and the DTF model uses graph neural networks to transform syntactic structures into graph representations and achieves effective fusion of syntactic information through message passing mechanisms, thereby enhancing semantic understanding capabilities.

[0147] The following section presents the comparison results between the information extraction model provided in this application and the aforementioned baseline model.

[0148] As shown in Table 4, in the comparative experiments with the baseline model, the information extraction model provided in this application outperformed other models on the four datasets 14R, 15R, 16R, and 14L. Furthermore, the F1 score of the DCA model improved by 9.6, 9.2, 5.9, and 13.4 percentage points compared to the suboptimal model, respectively. In particular, on the 14L dataset, DCA performance was significantly improved, with precision 9.2 points higher than the suboptimal model and recall increased by 13.9 points.

[0149] Table 4

[0150] The impact on the target sample data will be evaluated next.

[0151] Figure 8 This is an example graph illustrating the impact of the quantity of target sample data, reflecting the influence of different target sample data sizes on the overall model. Please see... Figure 8 As the amount of augmented data gradually increases from 0 to 6000, the F1 score on the 14L and 14R datasets shows a trend of first increasing and then stabilizing. The F1 score reaches its highest value at 5000 data points, and further increasing the amount does not improve the F1 score. Therefore, 5000 augmented data points are taken as the optimal data size to achieve the best model performance.

[0152] Figure 9 This is an example graph illustrating the impact of target sample data quality, showing how different qualities of target sample data affect the overall model. Please see below. Figure 9 On the 14L and 14R datasets, the F1 score gradually increased as the augmented data quality improved from below 4 points to between 9 and 10 points, confirming a positive correlation between data quality and model performance. The dataset with the highest data quality was selected as the optimal result. Based on the data quality calculation scheme, the average quality score was 9.46 points, indicating that training with high-quality data can effectively improve the model's recognition performance.

[0153] Furthermore, as shown in Table 5, ablation experiments revealed the impact of different components of the information extraction model provided in this application on performance. LLM exhibited the worst F1 score. LA applied adaptive low-rank fine-tuning to LLM, resulting in an F1 score improvement of over 5 compared to LLM, indicating that fine-tuning can improve performance. CA added a thought chain to LA, improving the F1 score by nearly 18, demonstrating that the thought chain effectively improves performance. DA added data augmentation to AL, improving the F1 score by nearly 17, indicating that data augmentation can also improve performance.

[0154] The information extraction model provided in this application combines thought chain and data augmentation. Compared with CA and DA, the F1 score is improved by nearly 11, indicating that the combination of data augmentation and thought chain has a significant effect on performance improvement.

[0155] Table 5

[0156] Next, to intuitively verify the information extraction model's ability to extract triples in real-world contexts, the test examples in Table 6 are selected for analysis. This case contains multiple user comment sentences with complex structures, covering positive, negative, and neutral sentiment polarities, and involves the co-occurrence and association of multiple aspect words and opinion words.

[0157] Table 6

[0158] As can be seen from the table, the information extraction model in this application can accurately extract aspect-level sentiment triples, fully identify the three sentiment polarities of positive, negative and neutral in the sentence, and also accurately identify the aspect words corresponding to the sentiment words.

[0159] To perform the corresponding steps in the above embodiments and various possible methods, an implementation of an aspect-level emotion triplet information extraction device is given below. Optionally, this aspect-level emotion triplet information extraction device can adopt the above-described... Figure 2 The device structure of the electronic device is shown. Further, please refer to... Figure 10 , Figure 10 This is a functional block diagram of an aspect-level emotion triplet information extraction device provided in this application embodiment. It should be noted that the basic principle and technical effects of the aspect-level emotion triplet information extraction device provided in this embodiment are the same as those in the above embodiments. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above embodiments. The aspect-level emotion triplet information extraction device includes: an acquisition module and a processing module.

[0160] This acquisition module is used to acquire the statement to be analyzed.

[0161] Understandably, this acquisition module can also be used to perform the above step S20.

[0162] This processing module is used to input the statement to be analyzed into a pre-trained information extraction model for processing, and to obtain the triple information of the statement to be analyzed. The triple information includes aspect words, opinion words, and sentiment polarity of opinion words in the statement to be analyzed. The information extraction model includes multiple Transformer layers, and the information extraction model is obtained by setting low-rank matrices for each Transformer layer and updating the parameters of the low-rank matrices corresponding to multiple Transformer layers during training.

[0163] Understandably, this processing module can also be used to perform the above step S21.

[0164] This aspect-level emotion triplet information extraction device may also include a model training module.

[0165] The model training module generates target sample data based on preset task prompts. It then constructs training samples using the target sample data, preset thought chains, and preset sentence sets. The thought chains are used to train the inference path of the information extraction model during triplet information extraction. The training samples are input into a pre-constructed initial information extraction model for training, and low-rank matrices corresponding to each Transformer layer are generated during the training process. Based on the importance of each Transformer layer in the initial information extraction model, computational resources are allocated to the low-rank matrices of each Transformer layer to update their parameters, thus obtaining the information extraction model.

[0166] Understandably, the model training module can also be used to perform the above steps S10 to S12.

[0167] Optionally, the above modules can be stored in the form of software or firmware. Figure 2 The memory shown is either stored in or embedded in the operating system (OS) of the electronic device, and can be used by... Figure 2 The processor executes the commands. Meanwhile, the data and program code required to execute these modules can be stored in memory.

[0168] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, can implement the aspect-level sentiment triplet information extraction method provided in this application.

[0169] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application. < / n2> < / n1>

Claims

1. A method for extracting aspect-level emotion triplet information, characterized in that, The method includes: Obtain the statement to be analyzed; The statement to be analyzed is input into a pre-trained information extraction model for processing to obtain the triple information of the statement to be analyzed. The triple information includes aspect words, opinion words, and the sentiment polarity of the opinion words in the statement to be analyzed. The information extraction model includes multiple Transformer layers, and the information extraction model is obtained by setting a low-rank matrix for each Transformer layer and updating the parameters of the low-rank matrices corresponding to the multiple Transformer layers during training.

2. The method according to claim 1, characterized in that, The information extraction model is trained through the following steps: Target sample data is generated based on preset task prompts, and training samples are constructed using the target sample data, preset thought chains, and preset sentence sets; the thought chains are used to train the reasoning path of the information extraction model when extracting triple information. The training samples are input into a pre-constructed initial information extraction model for training; the initial information extraction model includes low-rank matrices corresponding to each Transformer layer; Based on the importance of each Transformer layer in the initial information extraction model, computational resources are allocated to the low-rank matrix of each Transformer layer to update the parameters of the low-rank matrix of each Transformer layer, thereby obtaining the information extraction model.

3. The method according to claim 2, characterized in that, The task prompts include data generation prompts and quality assessment prompts. Generating target sample data based on the preset task prompts includes: The data is used to generate prompt words, which are then input into a large language model for processing to obtain multiple sample data. The quality assessment prompts and multiple sample data are input into the large language model for processing. Each sample data is scored, and multiple target sample data are selected from the multiple sample data based on the scores of each sample data.

4. The method according to claim 2, characterized in that, Each Transformer layer includes multiple pre-trained weight matrices; the low-rank matrices are constructed through the following steps: For each Transformer layer, the LoRA method is used to generate a corresponding increment matrix for each pre-trained weight matrix in the Transformer layer, and the increment matrix is ​​subjected to singular value decomposition to obtain the low-rank matrix corresponding to each pre-trained weight matrix.

5. The method according to claim 2, characterized in that, The low-rank matrix includes a left singular value vector, a right singular value vector, and singular values; The step of allocating computational resources to the low-rank matrices of each Transformer layer based on their importance in the initial information extraction model, and updating the parameters of the low-rank matrices of each Transformer layer to obtain the information extraction model, includes: Taking the current iteration as the target iteration, for each Transformer layer, calculate the gradient update values ​​of the target left singular value vector, target right singular value vector, and target singular value of each target low-rank matrix in the Transformer layer under the target iteration; wherein, the target singular value of the target low-rank matrix is ​​obtained through the previous iteration of training; Based on the trainable parameters of each target low-rank matrix in each direction and the preset exponential moving average coefficient, calculate the importance score of each target low-rank matrix in each direction. For each of the target low-rank matrices, the target singular values ​​are updated based on the importance scores of all the target low-rank matrices in each direction, the gradient update values ​​of the target singular values, and the remaining resource budget, so as to allocate resources to the target low-rank matrices and obtain new target singular values ​​for the next iteration. The remaining resource budget is updated according to the preset resource budget update method, and the next iteration training is carried out. The next iteration training is used as the target iteration, and the above steps are repeated until the remaining resource budget reaches the preset allocation completion condition. When the initial information extraction model reaches the preset model convergence condition, the information extraction model is obtained.

6. The method according to claim 5, characterized in that, The calculation of the target left singular value vector, target right singular value vector, and gradient update values ​​of the target singular values ​​of each target low-rank matrix in the Transformer layer under the target iteration includes: The left singular value vector and the right singular value vector of each of the target low-rank matrices are calculated using the stochastic gradient method. Calculate the training cost of the Transformer layer based on each of the target low-rank matrices in the Transformer layer; The loss function is calculated based on the training cost of the Transformer layer and the target left singular value vector and the target right singular value vector of each target low-rank matrix; The gradient update value of the target singular value is calculated based on the loss function, the preset learning rate, and the target singular value.

7. The method according to claim 5, characterized in that, The step of calculating the importance score of each target low-rank matrix in each direction based on each trainable parameter of each target low-rank matrix in each direction and a preset exponential moving average coefficient includes: For each of the target low-rank matrices, the average smoothing sensitivity and uncertainty term corresponding to each trainable parameter of the target low-rank matrix in each direction are calculated based on each trainable parameter of the target low-rank matrix in each direction and the preset exponential moving average coefficient. Based on the average smoothness sensitivity and uncertainty term corresponding to each trainable parameter of the target low-rank matrix in each direction, calculate the sensitivity corresponding to each trainable parameter of the target low-rank matrix in each direction; the sensitivity includes the target left singular value vector sensitivity, the target right singular value vector sensitivity, and the target singular value sensitivity of the trainable parameters; Based on the target left singular value vector sensitivity, target right singular value vector sensitivity, and target singular value sensitivity corresponding to each trainable parameter of the target low-rank matrix in each direction, calculate the importance score of the target low-rank matrix in each direction.

8. A device for extracting aspect-level emotion triplet information, characterized in that, The device includes: The acquisition module is used to acquire the statements to be analyzed. The processing module is used to input the statement to be analyzed into a pre-trained information extraction model for processing to obtain the triple information of the statement to be analyzed; the triple information includes aspect words, opinion words, and the sentiment polarity of the opinion words in the statement to be analyzed; the information extraction model includes multiple Transformer layers, and the information extraction model is obtained by setting a low-rank matrix for each Transformer layer and updating the parameters of the low-rank matrices corresponding to the multiple Transformer layers during training.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a computer program executable by the processor, the processor being able to execute the computer program to implement the method of any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-7.

Images