A context learning-based emotion prediction method, device and terminal

Abstract

The application discloses a kind of based on context learning sentiment prediction method, device and terminal, method includes: obtaining training sample and corresponding label in sentiment prediction training set, generate sample example set;Query test sentence is obtained, and query test sentence is spliced with each sample example respectively, and target example set is obtained;Target example set and query test sentence are respectively input into large language model, and the meta-gradient of target example set is calculated, and large language model is trained based on the update of meta-gradient, and label prediction model is obtained;Obtain to be predicted example;To be predicted example is input into label prediction model, and the prediction label of to-be-processed text is obtained;According to prediction label, obtain sentiment prediction result.The application embodiment is updated by meta-gradient, so that model better understands context information;Parallel method is used to learn each example independently, and the influence of example order on ICL is reduced to the greatest extent, so that the robustness and reliability of model are improved.

Description

Technical Field

[0001] This invention relates to the field of financial technology, and in particular to a sentiment prediction method, device, and terminal based on context learning. Background Technology

[0002] Currently, numerous large-scale language models (LLMs), such as Chat-GPT, PaLM, and LLMa, are driving the widespread development of Natural Language Processing (NLP) applications. As the application areas of NLP continue to expand, LLMs participate in various ways, including tasks such as text generation, language understanding, and logical reasoning. The large number of parameters in these models has given rise to a new learning paradigm: In-Context Learning (ICL), which differs from traditional gradient-based backpropagation. In ICL, LLMs predict test prompts using contextual examples and a set of fixed templates. ICL has become a new trend in evaluating and inferring the capabilities of LLMs, demonstrating their powerful potential in handling complex tasks, such as solving mathematical reasoning problems. These capabilities have been widely validated and have become one of the new features of large-scale language models.

[0003] For example, in the financial sector, insurance institutions can predict customer attitudes towards insurance purchases by analyzing the sentiment during telephone sales conversations. A significant challenge in applying ICL (Improved Contextualization) is ensuring that LLMs (Layered Management Models) can adequately learn knowledge to solve downstream tasks with limited examples. Existing ICL methods typically rely on prior set examples used to construct contextual templates and guide the model's inference process. However, ICL methods are susceptible to subtle variations in input examples; the use of different template examples or the different order of examples can significantly impact the final output. There is no guarantee that LLMs have adequately learned from the contextual examples, leading to lower prediction accuracy in sentiment prediction.

[0004] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the present invention provides a sentiment prediction method, device and terminal based on context learning, which aims to solve the problem that the ICL method in the prior art is easily affected by subtle changes in the input examples. The use of different template examples or different order of example arrangement may have a significant impact on the final output results. There is no way to guarantee that LLMs have fully learned the context examples, resulting in low prediction accuracy in the sentiment prediction process.

[0006] The technical solution of the present invention is as follows:

[0007] The first embodiment of the present invention provides a sentiment prediction method based on context learning, the method comprising:

[0008] Obtain the training samples and corresponding labels for sentiment classification in the sentiment prediction training set, and generate a sample example set based on the training samples and corresponding labels;

[0009] Obtain the query test statement, and concatenate the query test statement with each sample example in the sample example set to obtain the target example set;

[0010] The target example set and the query test statement are input into the large language model, and the meta-gradient of the target example set is calculated. The large language model is trained based on the meta-gradient update to obtain the label prediction model.

[0011] Obtain the call data to be processed, preprocess the call data to be processed, and obtain the example to be predicted;

[0012] The example to be predicted is input into the label prediction model, and the predicted label of the call data to be processed is obtained according to the label prediction model.

[0013] Based on a preset label mapping relationship, the sentiment label corresponding to the predicted label is obtained as the sentiment prediction result of the call data to be processed.

[0014] Further, the step of obtaining training samples and corresponding labels for sentiment classification in the sentiment prediction training set, and generating a sample example set based on the training samples and corresponding labels, includes:

[0015] Obtain the training samples and corresponding labels for sentiment classification in the sentiment prediction training set, and use a training sample and its corresponding label as a sample example;

[0016] Each sample instance is stored in a preset sample model to generate a sample instance set.

[0017] Further, the step of obtaining the query test statement involves concatenating the query test statement with each sample example in the sample example set to obtain the target example set, including:

[0018] Obtain the query test statement and store it in a preset sample template;

[0019] Each example is then appended to the query statement in a pre-defined format to obtain the target example set.

[0020] Further, the step of inputting the target example set and the query test statement into the large language model, calculating the meta-gradient of the target example set, and training the large language model based on the meta-gradient update to obtain the label prediction model includes:

[0021] Each target example in the target example set is input into the large language model, and the meta-gradient key-value matrix of the target example is calculated based on the large language model;

[0022] Input the query test statement into the large language model to obtain the key-value matrix of the query test statement;

[0023] The meta-gradient key-value matrix is ​​concatenated with the key-value matrix of the query test statement to obtain the concatenated target key-value matrix.

[0024] Based on the calculation results of the target key value matrix, the large language model is updated with meta-gradient key values ​​until the calculation results meet the preset conditions, thus obtaining the label prediction model.

[0025] Furthermore, the large language model comprises several layers of attention modules, and the meta-gradient key-value matrix of the target example is calculated based on the large language model, including:

[0026] Obtain the attention key-value matrix and query matrix corresponding to each position of each attention module in each layer;

[0027] The current key-value matrix of each layer is concatenated with the historical key-value matrix, and the concatenated matrix is ​​used as the meta-gradient key-value matrix of the updated target example.

[0028] Furthermore, before obtaining the sentiment label corresponding to the predicted label as the sentiment prediction result of the call data to be processed based on the preset label mapping relationship, the method further includes:

[0029] The mapping relationship between sentiment labels and sentiment prediction results is set in advance.

[0030] Further, the step of acquiring the call data to be processed and preprocessing the call data to obtain the example to be predicted includes:

[0031] Acquire the call data to be processed, perform speech recognition on the call data to be processed, and obtain the text data to be processed;

[0032] After text cleaning of the text data to be processed, the example to be predicted is obtained.

[0033] Another embodiment of the present invention provides a sentiment prediction device based on context learning, the device comprising:

[0034] The sample example set generation module is used to obtain the training samples and corresponding labels of sentiment classification in the sentiment prediction training set, and generate a sample example set based on the training samples and corresponding labels.

[0035] The target example set generation module is used to obtain the query test statement and concatenate the query test statement with each sample example in the sample example set to obtain the target example set.

[0036] The model training module is used to input the target example set and the query test statement into the large language model, calculate the meta-gradient of the target example set, and train the large language model based on the meta-gradient update to obtain the label prediction model.

[0037] The data preprocessing module is used to acquire call data to be processed, preprocess the call data to be processed, and obtain the example to be predicted.

[0038] The label prediction module is used to input the example to be predicted into the label prediction model and obtain the predicted label of the call data to be processed according to the label prediction model.

[0039] The sentiment prediction module is used to obtain the sentiment label corresponding to the predicted label as the sentiment prediction result of the call data to be processed, based on a preset label mapping relationship.

[0040] Another embodiment of the present invention provides a terminal, the terminal including at least one processor; and,

[0041] A memory communicatively connected to the at least one processor; wherein,

[0042] The memory stores instructions that can be executed by the at least one processor, which enables the at least one processor to perform the aforementioned context-based sentiment prediction method.

[0043] Another embodiment of the present invention provides a non-volatile computer-readable storage medium storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform the above-described context-based sentiment prediction method.

[0044] Beneficial Effects: The context-based sentiment prediction method in this invention explicitly teaches LLMs to learn contextual example content (in-context) through meta-gradient updates, enabling them to better understand contextual information in practical applications. It employs a parallel approach to learn each example independently, minimizing the impact of example order on ICL, thereby improving the model's robustness and reliability. This approach helps ensure that the model can fully utilize its learning of different examples without being limited by the example order. Attached Figure Description

[0045] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0046] Figure 1 This is a flowchart of a preferred embodiment of a sentiment prediction method based on context learning according to the present invention;

[0047] Figure 2 This is a schematic diagram of the functional modules of a preferred embodiment of a context-based emotion prediction device of the present invention;

[0048] Figure 3 This is a schematic diagram of the hardware structure of a preferred embodiment of a terminal according to the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention is further described in detail below. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0050] In financial applications, sentiment analysis of users' insurance sales call records can better classify target groups, create more accurate user profiles, and improve sales success rates. While the widespread adoption of LLMs has solved the problems of model training, how to better utilize LLMs and maximize their performance depends on the model's input and its optimization. Current business models often use only a few examples concatenated with the query as input when using LLMs. This approach is highly dependent on example selection, making it difficult to improve prediction accuracy, leading to data distortion and failing to obtain accurate user profiles, thus hindering further improvements in life insurance sales.

[0051] To address the aforementioned problems, this invention proposes a sentiment prediction method based on context learning. By structurally decomposing the prompt examples into an in-context learning approach, and based on the Transformer structure of LLMs, the method cleverly designs a grouped connection between the example prompt and the query, which greatly increases the number of input examples, thus enabling more accurate sentiment prediction and providing greater assistance to life insurance sales.

[0052] The embodiments of the present invention will be described below with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0053] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Here, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0054] The method provided in this application can be applied to artificial intelligence (AI) scenarios. AI is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. In other words, artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine capable of reacting in a way similar to human intelligence. Artificial intelligence studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making functions. Research in the field of artificial intelligence includes robotics, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, and fundamental AI theories.

[0055] To address the above problems, embodiments of the present invention provide a sentiment prediction method based on context learning. Please refer to [link to relevant documentation]. Figure 1 , Figure 1 This is a flowchart illustrating a preferred embodiment of a sentiment prediction method based on context learning according to the present invention. Figure 1 As shown, it includes:

[0056] Step S100: Obtain the training samples and corresponding labels for sentiment classification in the sentiment prediction training set, and generate a sample example set based on the training samples and corresponding labels;

[0057] Step S200: Obtain the query test statement, and concatenate the query test statement with each sample example in the sample example set to obtain the target example set;

[0058] Step S300: Input the target example set and the query test statement into the large language model respectively, calculate the meta-gradient of the target example set, and train the large language model based on the meta-gradient update to obtain the label prediction model.

[0059] Step S400: Obtain the call data to be processed, preprocess the call data to be processed, and obtain the example to be predicted;

[0060] Step S500: Input the example to be predicted into the label prediction model, and obtain the predicted label of the call data to be processed according to the label prediction model;

[0061] Step S600: Based on the preset label mapping relationship, obtain the sentiment label corresponding to the predicted label as the sentiment prediction result of the text to be processed.

[0062] In practical implementation, the context-based emotion prediction method of this invention is used to analyze the emotions of users during a call, thereby classifying the target group and forming a more accurate user profile.

[0063] This invention obtains a sample example set by randomly selecting one training sample and one corresponding label for each class of sentiment classification from the training set, and placing each training sample and its corresponding label into a preset example template. Sample examples are an important component in building a high-quality prompt. These examples are typically used to demonstrate to the model how to perform a specific task, especially in Few-Shot Learning scenarios.

[0064] Sample examples refer to a set of labeled or solved examples used to show the model how to generate the expected output for a given task. These examples can take various forms, such as text, images, and audio, depending on the task type. Sample examples can: improve model performance: By providing concrete examples, the model can learn to perform tasks faster and achieve better performance even without a large amount of training data. Enhance model interpretability: Sample examples make the model's decision-making process more transparent, helping to understand how the model generates output based on input. Reduce data requirements: In Few-Shot Learning scenarios, only a small number of sample examples are needed to train the model, reducing the need for large amounts of data.

[0065] Obtain the query test statement, put the query test statement into the preset example template, and concatenate the query test statement with each example to obtain the target example set;

[0066] Each target example in the target example set is input into a large language model, and the meta-gradient of the target example set is calculated. A query test statement is then input into the large language model, and the meta-gradient of the target example set is updated based on the query test statement. The large language model is then trained based on the updated meta-gradient, resulting in a label prediction model. The output of the label prediction model is the label corresponding to the sentiment classification. Meta-gradient is an important concept in Reinforcement Learning (RL), involving the optimization of meta-parameters in reinforcement learning algorithms. These meta-parameters typically do not directly participate in the decision-making process when the agent interacts with the environment, but they significantly affect the overall performance of the algorithm. In reinforcement learning, meta-gradient refers to updating the meta-parameters in the algorithm using optimization algorithms such as gradient descent to maximize the agent's cumulative reward or optimize other performance metrics. These meta-parameters may include discount factors, learning rates, exploration rates, etc., which have a significant impact on the learning process and final performance of the algorithm. Meta-gradients allow algorithms to adaptively adjust meta-parameters during training to adapt to different learning tasks and environmental changes. This adaptability helps algorithms maintain efficient and stable learning performance in complex and dynamic environments. By optimizing meta-parameters, meta-gradients can significantly improve the performance of reinforcement learning algorithms. For example, in DeepMind's research, optimizing meta-parameters such as discount factors and learning rates through meta-gradients significantly improved the performance of large-scale deep reinforcement learning applications. Traditional reinforcement learning algorithms often require manual setting and tuning of a large number of hyperparameters, which is both time-consuming and error-prone. Meta-gradients, by automatically optimizing meta-parameters, reduce the need for manual intervention and lower the barrier to entry for using the algorithm.

[0067] Obtain the call data to be processed, preprocess the call data according to the input type of the label prediction model, and record the preprocessed call data as the prediction example.

[0068] The example to be predicted is input into the label prediction model to obtain the predicted label of the label classification prediction model. Based on the predicted label, the preset label mapping relationship is queried to obtain the sentiment prediction result of the call data to be processed.

[0069] To address the input example sensitivity issue in Inference Language Processing (ICL) and ensure that LLMs can fully utilize the useful knowledge in examples, this invention addresses a crucial problem in applying ICL methods to real-world production environments. ICL achieves a parameter learning process similar to gradient backpropagation by concatenating the content of example teaching with the example demonstration before the query. Based on this idea, this invention improves the learning performance of LLMs on examples and strengthens the robustness of ICL by updating the meta-gradient for different examples, building upon existing meta-gradient methods. During the inference phase, the updated meta-gradient is concatenated with the example before the test query sequence to ensure the model can better understand and handle various input scenarios. This innovative method provides a more powerful and reliable solution for ICL applications, enabling LLMs to better handle complex natural language processing tasks.

[0070] In one embodiment, training samples and corresponding labels for sentiment classification in a sentiment prediction training set are obtained, and a sample example set is generated based on the training samples and corresponding labels, including:

[0071] Obtain the training samples and corresponding labels for sentiment classification in the sentiment prediction training set, and use a training sample and its corresponding label as a sample example;

[0072] Each sample instance is stored in a preset sample model to generate a sample instance set.

[0073] In practical implementation, taking n-class sentiment analysis of real-world call data as an example, the query test input is x. test We need to obtain the predicted labels generated by the model.

[0074] Randomly select one training sample and one corresponding label for each class from the training set, denoted as . Then, each pair is placed into the example template used to obtain the sample example set: τ(x i ,y i =input:x i label:y i .

[0075] In one embodiment, a query test statement is obtained, and the query test statement is concatenated with each sample example in the sample example set to obtain a target example set, including:

[0076] Obtain the query test statement and store it in a preset sample template;

[0077] Each example is then appended to the query statement in a pre-defined format to obtain the target example set.

[0078] In practice, the query test input is placed in the template: τ(x test =input:x i The label is then concatenated with each example in the following manner to obtain the target example set: τ(x test )\nτ(x i ,y i )=τ i .

[0079] In one embodiment, the target example set and the query test statement are input into a large language model, and the meta-gradient of the target example set is calculated. The large language model is then trained based on the meta-gradient update to obtain a label prediction model, including:

[0080] Each target example in the target example set is input into the large language model, and the meta-gradient key-value matrix of the target example is calculated based on the large language model;

[0081] Input the query test statement into the large language model to obtain the key-value matrix of the query test statement;

[0082] The meta-gradient key-value matrix is ​​concatenated with the key-value matrix of the query test statement to obtain the concatenated target key-value matrix.

[0083] Based on the calculation results of the target key value matrix, the large language model is updated with meta-gradient key values ​​until the calculation results meet the preset conditions, thus obtaining the label prediction model.

[0084] In specific implementation, τ will be respectively i ={υ1,υ2} (υ1 represents the test part sequence, υ2 represents the example part sequence), input it separately into the LLM and extract its meta-gradient.

[0085] x alone test Inputting into the LLM, the meta-gradient key-value matrix of each layer in each example above is compared with x. test The corresponding key-value matrix is ​​concatenated as follows: Perform calculations and output the results of the target key-value matrix calculation.

[0086] The large language model is updated with meta-gradient key values ​​based on the calculation results of the target key value matrix until the calculation results meet the preset conditions, thus obtaining the label prediction model.

[0087] In one embodiment, if the large language model consists of several layers of attention modules, calculating the meta-gradient key-value matrix of the target example based on the large language model includes:

[0088] Obtain the attention key-value matrix and query matrix corresponding to each position of each attention module in each layer;

[0089] The current key-value matrix of each layer is concatenated with the historical key-value matrix, and the concatenated matrix is ​​used as the meta-gradient key-value matrix of the updated target example.

[0090] In practice, the calculation of the meta-gradient will be described in detail below, where the meta-gradient exists in the form of a key-value matrix:

[0091] Suppose that the LLM has an L-layer Transformer-type attention module. The Transformer-type attention module is the core component in the Transformer model. It is based on the self-attention mechanism and can effectively capture the dependencies in the input sequence, especially long-distance dependencies.

[0092] The Transformer model mainly consists of two parts: an encoder and a decoder, each containing multiple stacked layers. Among these layers, the attention module plays a crucial role. The attention module captures dependencies in the sequence by calculating the correlation score (i.e., attention score) between any two positions in the input sequence.

[0093] The core of the Transformer attention mechanism lies in three matrices: Query, Key, and Value. These three matrices are obtained through embedding sequences and linear transformations, and are used to calculate the attention score. Query: Represents the element or position currently requiring attention. Key: Used to compare with the Query to calculate relevance. Value: Contains the actual information; its weighted sum is the output of the attention mechanism.

[0094] The Transformer calculation process is as follows:

[0095] Dot Product Attention: First, calculate the dot product of the Query and all Keys to obtain the relevance score of each position to the current position.

[0096] Scaling Dot Product Attention: To prevent the dot product result from becoming too large and causing the gradient of the softmax function to vanish, the dot product result is usually scaled.

[0097] Softmax normalization: The scaled dot product result is normalized using the softmax function to obtain the attention weights.

[0098] Weighted summation: Finally, the attention weights are summed with the corresponding values ​​to obtain the output representation at that position.

[0099] in This represents the hidden state from the output of layer (l-1) to the input of layer l. The attention key-value matrix and query matrix for each position in each layer are as follows:

[0100]

[0101] Next, we will compare the current key value matrix with the historical key value matrix. Perform row concatenation, i.e.: Use the concatenated matrix as the new key-value matrix to calculate the hidden state of the j-th part from the output and input of layer l to layer l+1. Update the historical key-value matrix. The calculation method is as follows:

[0102]

[0103] in Similarly.

[0104] Next, we will discuss x separately. test Inputting into the LLM, the meta-gradient key-value matrix of each layer in each example above is compared with x. t The corresponding key-value matrix is ​​concatenated as follows: Perform calculations and output the final target key value matrix calculation results.

[0105] After performing the above operations on the key-value matrix of each layer, the final meta-gradient key-value matrix is ​​obtained:

[0106]

[0107] In one embodiment, before obtaining the sentiment label corresponding to the predicted label as the sentiment prediction result of the call data to be processed based on a preset label mapping relationship, the method further includes:

[0108] The mapping relationship between sentiment labels and sentiment prediction results is set in advance.

[0109] In practice, sentiment prediction results include, but are not limited to, positive or negative purchase intentions. Sentiment labels can be set to positive or negative. A positive sentiment label indicates that the user has a positive purchase intention. A negative sentiment label indicates that the user does not have a purchase intention.

[0110] In one embodiment, acquiring call data to be processed and preprocessing the call data to obtain a prediction example includes:

[0111] Acquire the call data to be processed, perform speech recognition on the call data to be processed, and obtain the text data to be processed;

[0112] After text cleaning of the text data to be processed, the example to be predicted is obtained.

[0113] In practice, after obtaining user authorization, the call data between the user and the salesperson is acquired as call data to be processed. Speech recognition is then performed on this call data to obtain the corresponding text data to be processed. The speech recognition algorithm mainly includes the following steps:

[0114] Voice signal acquisition: Voice signals are acquired through devices such as microphones, the sound waves are converted into voltage signals, and then sampled and quantized to convert them into digital signals that can be processed by computers.

[0115] Preprocessing: The acquired digital signals are preprocessed, including filtering, noise reduction, and segmentation, to improve the accuracy and efficiency of subsequent processing.

[0116] Feature extraction: Extracting characteristic parameters that represent the characteristics of the speech signal from the preprocessed speech signal, such as Mel-frequency cepstral coefficients (MFCC) and linear predictive cepstral coefficients (LPCC). These characteristic parameters can reflect the acoustic and linguistic characteristics of the speech signal.

[0117] Acoustic modeling: Using the extracted feature parameters, an acoustic model is used to calculate the score of each feature in the acoustic feature space, thereby determining the acoustic features of the speech signal. Acoustic models generally employ machine learning models such as Hidden Markov Models (HMMs) and Deep Neural Networks (DNNs).

[0118] Language modeling: Based on linguistic knowledge, language models are used to calculate the probability of possible word sequences corresponding to speech signals, thereby determining the linguistic features of the speech signals. Language models can reflect the co-occurrence relationships between words and grammatical rules.

[0119] Decoding: The outputs of the acoustic model and the language model are combined, and the decoding algorithm finds the most likely word sequence as the recognition result. Decoding algorithms generally employ dynamic programming algorithms such as the Viterbi algorithm.

[0120] Commonly used speech recognition algorithms include: Hidden Markov Model (HMM): Principle: This algorithm is based on a statistical model, treating the time series as a series of state transitions and using a probabilistic model to describe these transitions. Advantages: It can better preserve the structure and semantic information of the speech signal and adapt to different signal characteristics and noise environments. Disadvantages: Model design and training require high computational and training resources; for large-scale speech datasets, the training time and computational cost are high.

[0121] Deep Neural Networks (DNNs): Principle: DNNs use multi-layered neural networks to perform nonlinear transformations on speech signals, extracting high-level feature representations for subsequent acoustic and language modeling. Advantages: They can automatically learn features from data, improving prediction and classification accuracy; they adapt to complex model structures and tasks, improving model robustness and adaptability. Disadvantages: They have high model complexity, requiring large amounts of training data and computational resources; specific tasks and datasets may require specific deep learning algorithms and models.

[0122] Support Vector Machine (SVM): Principle: A commonly used machine learning algorithm for binary and multi-class classification problems. It separates data into different categories by finding the optimal hyperplane. Advantages: Easy to implement; performs well on large datasets; can handle non-linear data classification problems. Disadvantages: Difficult to handle large numbers of training samples due to the need for significant storage space; sensitive to the choice of parameters and kernel function.

[0123] Template matching method: Principle: During the training phase, the pronunciation features of each word in the vocabulary are recorded and stored as templates; during the recognition phase, the test speech is matched with the templates, and the most similar template is used as the recognition result. Advantages: Simple and intuitive, suitable for specific speakers, small vocabulary, and isolated word recognition systems. Disadvantages: High computational cost, less efficient for large vocabulary and continuous speech recognition systems.

[0124] Text cleaning is performed on the text data to be processed to remove non-text content such as special characters, HTML tags, and URL links, keeping only the plain text. Traditional and simplified Chinese characters are then converted to obtain the prediction examples corresponding to the text data to be processed.

[0125] Compared to existing technologies, the sentiment prediction method based on context learning in this invention, building upon existing meta-gradient methods, updates the meta-gradients of different examples separately. This ensures that the LLM learns fully from the examples, and updating each example separately prevents performance degradation caused by example order, thus enhancing the robustness of ICL. During the inference phase, the updated meta-gradients are concatenated with the examples preceding the test query sequence to ensure the model can better understand and handle various input scenarios. This innovative method provides a more powerful and reliable solution for ICL applications, enabling LLMs to better handle complex natural language processing tasks. Unlike methods that concatenate examples for learning, this method first inputs the query sequence and concatenates each example, allowing the LLM to learn each example separately. This avoids mutual interference between different examples during learning. After obtaining the meta-gradients, the meta-gradients of each example are concatenated with the query sequence and input into the LLM for output prediction.

[0126] The embodiments of the present invention have the following technical advantages:

[0127] By leveraging the meta-gradient update mechanism, LLMs are guided in an explicit way to actively learn the in-context of contextual examples, thereby enhancing the model's ability to understand contextual information in practical applications.

[0128] By employing a parallel learning approach and processing each example independently, the potential impact of example order on ICL is minimized, thus improving the model's robustness and reliability. This strategy helps ensure that the model fully utilizes each example, unaffected by the example arrangement.

[0129] By placing the query sentence at the forefront of the learning examples, meta-gradient learning is performed. This allows the model to first capture the key features of the query and then selectively learn relevant information from the examples. This approach enhances the model's understanding of the relationship between the context and the query, providing more favorable conditions for successful task execution.

[0130] It should be noted that there is no necessary order between the above steps. Those skilled in the art will understand from the description of the embodiments of the present invention that the above steps may have different execution orders in different embodiments, that is, they may be executed in parallel or in turn, etc.

[0131] Another embodiment of the present invention provides a sentiment prediction device based on context learning, such as... Figure 2 As shown, device 1 includes:

[0132] The sample example set generation module 11 is used to obtain the training samples and corresponding labels of sentiment classification in the sentiment prediction training set, and generate a sample example set based on the training samples and corresponding labels.

[0133] The target example set generation module 12 is used to obtain the query test statement and concatenate the query test statement with each sample example in the sample example set to obtain the target example set.

[0134] The model training module 13 is used to input the target example set and the query test statement into the large language model, calculate the meta-gradient of the target example set, and train the large language model based on the meta-gradient update to obtain the label prediction model.

[0135] Data preprocessing module 14 is used to acquire call data to be processed, preprocess the call data to be processed, and obtain the example to be predicted;

[0136] The label prediction module 15 is used to input the example to be predicted into the label prediction model and obtain the predicted label of the call data to be processed according to the label prediction model.

[0137] The sentiment prediction module 16 is used to obtain the sentiment label corresponding to the predicted label as the sentiment prediction result of the call data to be processed based on the preset label mapping relationship.

[0138] For specific implementation details, please refer to the method embodiment; they will not be repeated here.

[0139] In one embodiment, the sample example set generation module 11 is specifically used for:

[0140] Obtain the training samples and corresponding labels for sentiment classification in the sentiment prediction training set, and use a training sample and its corresponding label as a sample example;

[0141] Each sample instance is stored in a preset sample model to generate a sample instance set.

[0142] For specific implementation details, please refer to the method embodiment; they will not be repeated here.

[0143] In one embodiment, the target example set generation module is specifically used for:

[0144] Obtain the query test statement and store it in a preset sample template;

[0145] Each example is then appended to the query statement in a pre-defined format to obtain the target example set.

[0146] For specific implementation details, please refer to the method embodiment; they will not be repeated here.

[0147] In one embodiment, the model training module 13 is specifically used for:

[0148] Each target example in the target example set is input into the large language model, and the meta-gradient key-value matrix of the target example is calculated based on the large language model;

[0149] Input the query test statement into the large language model to obtain the key-value matrix of the query test statement;

[0150] The meta-gradient key-value matrix is ​​concatenated with the key-value matrix of the query test statement to obtain the concatenated target key-value matrix.

[0151] Based on the calculation results of the target key value matrix, the large language model is updated with meta-gradient key values ​​until the calculation results meet the preset conditions, thus obtaining the label prediction model.

[0152] For specific implementation details, please refer to the method embodiment; they will not be repeated here.

[0153] In one embodiment, if the large language model consists of several layers of attention modules, the model training module 13 is further used for:

[0154] Obtain the attention key-value matrix and query matrix corresponding to each position of each attention module in each layer;

[0155] The current key-value matrix of each layer is concatenated with the historical key-value matrix, and the concatenated matrix is ​​used as the meta-gradient key-value matrix of the updated target example.

[0156] For specific implementation details, please refer to the method embodiment; they will not be repeated here.

[0157] In one embodiment, the apparatus further includes a setting module, the setting module being used for:

[0158] The mapping relationship between sentiment labels and sentiment prediction results is set in advance.

[0159] For specific implementation details, please refer to the method embodiment; they will not be repeated here.

[0160] In one embodiment, the data preprocessing module 14 is specifically used for:

[0161] Acquire the call data to be processed, perform speech recognition on the call data to be processed, and obtain the text data to be processed;

[0162] After text cleaning of the text data to be processed, the example to be predicted is obtained.

[0163] For specific implementation details, please refer to the method embodiment; they will not be repeated here.

[0164] Another embodiment of the present invention provides a terminal. The terminal in this application can be a mobile phone, tablet computer, wearable device, vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc. The embodiments of this application do not impose any limitations on this.

[0165] like Figure 3 As shown, terminal 10 includes:

[0166] One or more processors 110 and memory 120, Figure 3 The following description uses a processor 110 as an example. The processor 110 and the memory 120 can be connected via a bus or other means. Figure 3 Taking the example of a connection between China and Israel via a bus.

[0167] Processor 110 is used to implement various control logics of terminal 10. It can be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), microcontroller, ARM (Acorn RISC Machine) or other programmable logic device, discrete gate or transistor logic, discrete hardware controls, or any combination of these components. Furthermore, processor 110 can also be any conventional processor, microprocessor, or state machine. Processor 110 can also be implemented as a combination of computing devices, such as a combination of DSP and microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration.

[0168] The memory 120, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions corresponding to the context-based sentiment prediction method in the embodiments of the present invention. The processor 110 executes various functional applications and data processing of the device 10 by running the non-volatile software programs, instructions, and units stored in the memory 120, thereby implementing the context-based sentiment prediction method in the above method embodiments.

[0169] The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store applications required for operating the device and at least one function; and the data storage area may store data created according to the use of the device 10. Furthermore, the memory 120 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 120 may optionally include memory remotely located relative to the processor 110, and these remote memories may be connected to the device 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0170] One or more units are stored in memory 120, and when executed by one or more processors 110, they perform the context-learning-based sentiment prediction method in any of the above method embodiments, for example, performing the above-described... Figure 1 The method steps S100 to S600.

[0171] This invention provides a non-volatile computer-readable storage medium storing computer-executable instructions that are executed by one or more processors, for example, to perform the operations described above. Figure 1 The method steps S100 to S600.

[0172] As an example, non-volatile storage media can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) as an external cache memory. By way of illustration and not limitation, RAM can be obtained in many forms such as synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct Rambus RAM (DRRAM). The memory controls or memories disclosed in the operating environment described herein are intended to include one or more of these and / or any other suitable types of memory.

[0173] Another embodiment of the present invention provides a computer program product comprising a computer program stored on a non-volatile computer-readable storage medium. The computer program includes program instructions that, when executed by a processor, cause the processor to perform the context-based sentiment prediction method of the above-described method embodiments. For example, performing the above-described... Figure 1 The method steps S100 to S600.

[0174] The embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0175] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general-purpose hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can exist in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0176] Among other things, conditional language such as “can,” “may,” “may,” or “may,” unless otherwise specifically stated or otherwise understood as in the context in which they are used, is generally intended to convey that a particular implementation may include (but not others) certain features, elements, and / or operations. Therefore, such conditional language is also generally intended to imply that features, elements, and / or operations are necessary for one or more implementations in any way, or that one or more implementations must include logic for determining, with or without input or prompting, whether such features, elements, and / or operations are included or will be performed in any particular implementation.

[0177] The contents already described herein in this specification and accompanying drawings include examples of methods and apparatuses capable of providing context-based emotion prediction. It is certainly not possible to describe every conceivable combination of elements and / or methods for the purpose of describing the various features of this disclosure, but it will be appreciated that many other combinations and substitutions of the disclosed features are possible. Therefore, it will be apparent that various modifications can be made to this disclosure without departing from the scope or spirit of this disclosure. Furthermore, or in alternatives, other embodiments of this disclosure may become apparent from consideration of this specification and accompanying drawings and from practice of this disclosure as presented herein. It is intended that the examples presented in this specification and accompanying drawings be considered illustrative rather than restrictive in all respects. Although specific terminology is used herein, it is used in a general and descriptive sense and is not intended for limiting purposes.

Claims

1. A sentiment prediction method based on context learning, characterized in that... The method includes: Obtain the training samples and corresponding labels for sentiment classification in the sentiment prediction training set, and generate a sample example set based on the training samples and corresponding labels; Obtain the query test statement, and concatenate the query test statement with each sample example in the sample example set to obtain the target example set; The target example set and the query test statement are input into the large language model, and the meta-gradient of the target example set is calculated. The large language model is trained based on the meta-gradient update to obtain the label prediction model. Obtain the call data to be processed, preprocess the call data to be processed, and obtain the example to be predicted; The example to be predicted is input into the label prediction model, and the predicted label of the call data to be processed is obtained according to the label prediction model. Based on a preset label mapping relationship, the sentiment label corresponding to the predicted label is obtained as the sentiment prediction result of the call data to be processed. The process of inputting the target example set and the query test statement into the large language model, calculating the meta-gradient of the target example set, and training the large language model based on the meta-gradient update to obtain the label prediction model includes: Each target example in the target example set is input into the large language model, and the meta-gradient key-value matrix of the target example is calculated based on the large language model; Input the query test statement into the large language model to obtain the key-value matrix of the query test statement; The meta-gradient key-value matrix is ​​concatenated with the key-value matrix of the query test statement to obtain the concatenated target key-value matrix. Based on the calculation results of the target key value matrix, the large language model is updated with meta-gradient key values ​​until the calculation results meet the preset conditions, and then the label prediction model is obtained. The large language model consists of several layers of attention modules. Based on the large language model, the meta-gradient key-value matrix of the target example is calculated, including: Obtain the attention key-value matrix and query matrix corresponding to each position of each attention module in each layer; The current key-value matrix of each layer is concatenated with the historical key-value matrix, and the concatenated matrix is ​​used as the meta-gradient key-value matrix of the updated target example.

2. The method according to claim 1, characterized in that, The step of obtaining training samples and corresponding labels for sentiment classification in the sentiment prediction training set, and generating a sample example set based on the training samples and corresponding labels, includes: Obtain the training samples and corresponding labels for sentiment classification in the sentiment prediction training set, and use a training sample and its corresponding label as a sample example; Each sample instance is stored in a preset sample model to generate a sample instance set.

3. The method according to claim 2, characterized in that, The step of obtaining the query test statement involves concatenating the query test statement with each sample example in the sample example set to obtain the target example set, including: Obtain the query test statement and store it in a preset sample template; Each example is then appended to the query test statement in a pre-defined format to obtain the target example set.

4. The method according to claim 3, characterized in that, Before obtaining the sentiment label corresponding to the predicted label as the sentiment prediction result of the call data to be processed based on the preset label mapping relationship, the method further includes: The mapping relationship between sentiment labels and sentiment prediction results is set in advance.

5. The method according to claim 4, characterized in that, The process of acquiring call data to be processed and preprocessing the call data to obtain a prediction example includes: Acquire the call data to be processed, perform speech recognition on the call data to be processed, and obtain the text data to be processed; After text cleaning of the text data to be processed, the example to be predicted is obtained.

6. A sentiment prediction device based on context learning, characterized in that, The device includes: The sample example set generation module is used to obtain the training samples and corresponding labels of sentiment classification in the sentiment prediction training set, and generate a sample example set based on the training samples and corresponding labels. The target example set generation module is used to obtain the query test statement and concatenate the query test statement with each sample example in the sample example set to obtain the target example set. The model training module is used to input the target example set and the query test statement into the large language model, calculate the meta-gradient of the target example set, and train the large language model based on the meta-gradient update to obtain the label prediction model. Specifically, the model training module is used for: Each target example in the target example set is input into the large language model, and the meta-gradient key-value matrix of the target example is calculated based on the large language model; Input the query test statement into the large language model to obtain the key-value matrix of the query test statement; The meta-gradient key-value matrix is ​​concatenated with the key-value matrix of the query test statement to obtain the concatenated target key-value matrix. Based on the calculation results of the target key value matrix, the large language model is updated with meta-gradient key values ​​until the calculation results meet the preset conditions, and then the label prediction model is obtained. The large language model comprises several layers of attention modules. Based on the large language model, the meta-gradient key-value matrix of the target example is calculated, including: Obtain the attention key-value matrix and query matrix corresponding to each position of each attention module in each layer; The current key-value matrix of each layer is concatenated with the historical key-value matrix, and the concatenated matrix is ​​used as the meta-gradient key-value matrix of the updated target example. The data preprocessing module is used to acquire call data to be processed, preprocess the call data to be processed, and obtain the example to be predicted. The label prediction module is used to input the example to be predicted into the label prediction model and obtain the predicted label of the call data to be processed according to the label prediction model. The sentiment prediction module is used to obtain the sentiment label corresponding to the predicted label as the sentiment prediction result of the call data to be processed, based on a preset label mapping relationship.

7. A terminal, characterized in that, The terminal includes at least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the context-based sentiment prediction method according to any one of claims 1-5.

8. A non-volatile computer-readable storage medium, characterized in that, The non-volatile computer-readable storage medium stores computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform the context-based sentiment prediction method according to any one of claims 1-5.

Images