Emotion recognition method and computer readable storage medium
By introducing a memory database and a cross-attention mechanism into the emotion recognition model, the problem of insufficient accuracy in emotion recognition in existing technologies is solved, and the accuracy of emotion recognition is improved by utilizing historical event information.
Patent Information
- Application Number
- CN202511776698.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-28
- Publication Date
- 2026-02-17
AI Technical Summary
Existing deep learning-based emotion recognition models are insufficient in accuracy and struggle to effectively utilize historical event information to improve the accuracy of emotion recognition.
By introducing a memory database and a cross-attention mechanism, emotion recognition is achieved by fusing historical event vectors with high similarity obtained through similarity retrieval with the current event vector, thereby improving accuracy by utilizing historical event information.
By introducing a memory database and a cross-attention mechanism, historical event information can be effectively utilized to improve the accuracy of emotion recognition.
Smart Images
Figure CN121542853A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, specifically to an emotion recognition method and a computer-readable storage medium. Background Technology
[0002] With the development of human-computer interaction technology, emotion recognition is playing an increasingly important role in scenarios such as intelligent assistants, in-vehicle systems, and mental health monitoring. Among related technologies, deep learning-based emotion recognition models typically use recurrent neural networks (RNNs), long short-term memory networks (LSTMs), or Transformer architectures to extract features from user-input text, speech, or multimodal signals, and then output the emotion category through a classifier.
[0003] While the above-mentioned emotion recognition methods can improve the accuracy of emotion recognition to some extent, there are still many shortcomings, resulting in poor accuracy in emotion recognition. Summary of the Invention
[0004] In view of the above problems, this application provides an emotion recognition method and a computer-readable storage medium. By introducing a memory database and a cross-attention mechanism, it is possible to obtain historical event vectors with high similarity, which can be fused with the current event vector for emotion recognition. This can effectively utilize historical event information to improve the accuracy of emotion recognition.
[0005] The first aspect of this application provides an emotion recognition method, comprising: acquiring a current event; encoding the current event into a current event vector; extracting key information from the current event vector to obtain a current backbone vector; and calculating the residual between the current event vector and the current backbone vector as a current detail vector; performing a similarity search in a memory database based on the current backbone vector to obtain a target number of historical event vectors with similarity from high to low; wherein the historical event vectors include historical backbone vectors and historical detail vectors; and fusing the target number of historical event vectors with the current event vector based on a cross-attention mechanism to obtain a fused event vector, and inputting the fused event vector into a classifier for emotion recognition.
[0006] In some specific embodiments, after the steps of obtaining the current event, encoding the current event into a current event vector, extracting key information from the current event vector to obtain the current backbone vector, and calculating the residual between the current event vector and the current backbone vector as the current detail vector, the method further includes: storing the current backbone vector and the current detail vector in a memory database to serve as the historical backbone vector and historical detail vector of the memory database; updating the historical backbone vector through a first forgetting mechanism, and updating the historical detail vector through a second forgetting mechanism.
[0007] In some specific embodiments, the step of updating the historical backbone vector through a first forgetting mechanism includes: if the storage duration of the historical backbone vector in the memory database reaches a first duration, then the historical backbone vector is compressed for the first time by means of information distillation; if the storage duration of the historical backbone vector after the first information compression reaches a second duration, then the historical backbone vector is compressed for the second time by means of information distillation; wherein the second duration is longer than the first duration.
[0008] In some specific embodiments, the step of updating the historical detail vector through a second forgetting mechanism includes: if the storage duration of the historical detail vector in the memory database reaches a third duration, then the historical detail vector is subjected to initial dimensional compression according to a first retention ratio; if the storage duration of the historical detail vector after the initial dimensional compression reaches a fourth duration, then the historical detail vector is subjected to secondary dimensional compression according to a second retention ratio; wherein the fourth duration is longer than the third duration, and the second retention ratio is less than the first retention ratio.
[0009] In some specific embodiments, the step of updating the historical detail vector through a second forgetting mechanism includes: after the historical detail vector is stored in the memory database, the historical backbone vector is zeroed out at preset intervals according to a preset zero ratio.
[0010] In some specific embodiments, the step of performing a similarity search in the memory database based on the current backbone vector to obtain a target number of historical event vectors with similarity from high to low includes: performing a similarity search in the memory database based on the current backbone vector to obtain a first number of historical backbone vectors and corresponding first number of historical detail vectors with similarity from high to low; wherein, a vector combination consisting of a historical backbone vector and a historical detail vector corresponds to a historical event vector; reconstructing the memory of the historical backbone vectors and historical detail vectors under each vector combination to obtain the first number of historical event vectors; and performing a similarity search on the first number of historical time vectors based on the current backbone vector and the current detail vector to obtain a target number of historical event vectors with similarity from high to low; wherein, the target number is less than the first number.
[0011] In some specific embodiments, the step of reconstructing the historical backbone vectors and historical detail vectors under each vector combination to obtain a first number of historical event vectors includes: if the dimension of the historical detail vector of the first vector combination is not compressed, then the historical backbone vector and the historical detail vector under the first vector combination are fused to obtain the corresponding historical event vector; if the dimension of the historical detail vector of the second vector combination is compressed, then the historical detail vector of the second vector combination is reconstructed to obtain the reconstructed historical detail vector, and the reconstructed historical detail vector is fused with the historical backbone vector of the second vector combination to obtain the corresponding historical event vector; wherein, the historical backbone vector and the reconstructed detail vector under the second vector combination have the same dimension.
[0012] In some specific embodiments, the step of performing a similarity search in the memory database based on the current backbone vector to obtain a first number of historical backbone vectors and a first number of historical detail vectors with similarity from high to low includes: performing an initial similarity calculation in the memory database based on the current backbone vector using a preset calculation method to obtain a second number of historical backbone vectors and corresponding second number of historical detail vectors with initial similarity from high to low; wherein the second number is greater than the first number; and performing a weighted calculation on the initial similarity of the second number of historical backbone vectors to obtain a first number of historical backbone vectors and corresponding first number of historical detail vectors with target similarity from high to low after initial similarity weighting.
[0013] In some specific embodiments, the step of weighting the initial similarity of the second number of historical backbone vectors includes: obtaining the intensity of user emotions corresponding to the historical event vectors of each historical backbone vector and the storage duration in the memory database; determining the weight value of each historical backbone vector based on the intensity of user emotions and the storage duration, and weighting the initial similarity of each historical backbone vector using the weight value; wherein, when the intensity of user emotions remains unchanged, the longer the storage duration, the lower the weight value; and when the storage duration remains unchanged, the stronger the intensity of user emotions, the higher the weight value.
[0014] A second aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the emotion recognition method as described above.
[0015] The beneficial technical effects of this application are as follows: Based on the emotion recognition method and computer-readable storage medium provided in this application, the method includes: acquiring the current event; encoding the current event into a current event vector; extracting key information from the current event vector to obtain a current backbone vector; and calculating the residual between the current event vector and the current backbone vector as the current detail vector; performing a similarity search in a memory database based on the current backbone vector to obtain a target number of historical event vectors with similarity from high to low; wherein the historical event vectors include historical backbone vectors and historical detail vectors; and fusing the target number of historical event vectors with the current event vector based on a cross-attention mechanism to obtain a fused event vector, and inputting the fused event vector into a classifier for emotion recognition. Therefore, by introducing a memory database and a cross-attention mechanism, historical event vectors with high similarity can be obtained and fused with the current event vector for emotion recognition, effectively utilizing historical event information to improve the accuracy of emotion recognition.
[0016] The above description is merely an overview of the technical solutions of the embodiments of this application. In order to better understand the technical means of the embodiments of this application and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of this application more obvious and understandable, specific implementation methods of this application are described below. Attached Figure Description
[0017] The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a flowchart illustrating an embodiment of the emotion recognition method provided in this application; Figure 2 This is a flowchart illustrating another embodiment of the emotion recognition method provided in this application; Figure 3 This is a flowchart illustrating another embodiment of the emotion recognition method provided in this application; Figure 4 This is a flowchart illustrating another embodiment of the emotion recognition method provided in this application; Figure 5 This is a flowchart illustrating another embodiment of the emotion recognition method provided in this application; Figure 6 This is a flowchart illustrating another embodiment of the emotion recognition method provided in this application; Figure 7 This is a flowchart illustrating another embodiment of the emotion recognition method provided in this application; Figure 8 This is a flowchart illustrating another embodiment of the emotion recognition method provided in this application; Figure 9This is a schematic diagram of the structural framework of an embodiment of the computer-readable storage medium provided in this application. Detailed Implementation
[0018] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited to the embodiments set forth herein. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without inventive effort are within the scope of protection of the present application.
[0019] If the embodiments of this application involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Furthermore, if the word "and / or" appears throughout the text, it means including three parallel solutions. For example, "A and / or B" includes solution A, solution B, or a solution that simultaneously satisfies A and B. Additionally, the technical solutions of various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.
[0020] The first aspect of this application provides an emotion recognition method. Figure 1 This is a flowchart illustrating an embodiment of the emotion recognition method provided in this application. Figure 1 This method includes the following steps: S101: Obtain the current event, encode the current event into a current event vector, extract key information from the current event vector to obtain the current backbone vector, and calculate the residual between the current event vector and the current backbone vector as the current detail vector.
[0021] The current event refers to input information reflecting the user's emotional state, which can be in the form of text statements, voice clips, or behavioral logs. For example, the current event could be a text statement entered by the user in an intelligent customer service dialogue, such as "I've been under a lot of work pressure lately, and I feel like I can't hold on much longer," or it could be a voice clip of the user expressing anger captured by the in-vehicle system. After acquiring the current event, a pre-trained encoding model can be used to encode it into a fixed-dimensional current event vector. This vector can comprehensively and compactly represent the feature information of the current event. For example, the current event vector can be a d-dimensional numerical vector, and the dimension d of the current event vector can be set according to actual needs, such as 128 dimensions, 256 dimensions, etc., without specific limitations.
[0022] After obtaining the current event vector, key information is extracted from it to obtain the current backbone vector. For example, key information extraction algorithms or neural network structures can be used to separate the core semantic components from the current event vector as the current backbone vector. In some application scenarios, an information summarization encoder can be used to extract key information from the current event vector to obtain the current backbone vector. An information summarization encoder is a neural network module used to extract key information from event vectors. It can identify and retain the most representative features in the event vector while removing redundant and noisy information.
[0023] After obtaining the backbone vector, the residual between the current event vector and the current backbone vector is further calculated as the current detail vector. The current detail vector captures subtle features in the current event vector beyond the core semantics, and these features are also valuable for emotion recognition. For example, in text emotion recognition, the current backbone vector may primarily reflect the main sentiment of the text, while the current detail vector may include factors such as tone and rhetorical devices that subtly influence emotion. In some application scenarios, let the current event vector be F_e and the backbone vector be F_c, then the current detail vector F_r = F_e - F_c.
[0024] It should be understood that the dimensions of the current event vector, the current backbone vector, and the current detail vector in this embodiment can be the same, for example, all three are d-dimensional numerical vectors.
[0025] S102: Based on the current backbone vector, perform a similarity search in the memory database to obtain a target number of historical event vectors with similarity from high to low; wherein, the historical event vectors include historical backbone vectors and historical detail vectors.
[0026] The memory database is pre-built. The current backbone vector and current detail vector from the above steps are also stored in the memory database after completing emotion recognition-related operations, serving as historical reference information for subsequent emotion recognition. Therefore, the memory database is a pre-built and continuously updated dataset. Similarity retrieval is performed in the memory database based on the current backbone vector to obtain a target number of historical event vectors with decreasing similarity. Specifically, this involves searching the memory database for a target number of historical event vectors with decreasing similarity to the current backbone vector, and then finding the corresponding historical detail vectors to obtain the historical event vectors. It should be understood that the target number of historical event vectors are semantically or feature-wise closest to the current backbone vector. The target number can be set according to actual needs, such as 10, 15, etc.
[0027] In some application scenarios, during the retrieval process, common similarity calculation methods such as cosine similarity and Euclidean distance can be used to calculate the similarity between the current backbone vector and each historical backbone vector in the memory database.
[0028] S103: Based on the cross-attention mechanism, the target number of historical event vectors are fused with the current event vector to obtain a fused event vector, and the fused event vector is input into the classifier for emotion recognition.
[0029] It should be understood that cross-attention is a deep learning mechanism used to capture the dependencies between two sets of vectors. It automatically learns and assigns different weights to each element in both the historical and current event vectors, thus more effectively fusing their information. Specifically, cross-attention calculates the attention weights between the current event vector and each historical event vector separately. These weights reflect the importance of different historical event vectors for the emotion recognition of the current event vector. Then, based on these attention weights, the historical event vectors are weighted and summed to obtain a fused event vector that is integrated with the current event vector. The fused event vector combines the core features of the current event and the relevant features of the historical events, providing a more comprehensive reflection of the emotional state of the current event. For example, when the current event is "I am happy," the cross-attention mechanism can fuse historical event vectors related to the "happy" emotion, thereby enhancing the accuracy of identifying the current happy emotion.
[0030] After obtaining the fused event vector, it is input into a pre-trained classifier. The classifier can output the emotion category of the current event, such as happy, sad, or angry, based on the features of the fused event vector. The classifier can employ various machine learning or deep learning models, such as support vector machines or neural networks. To improve the accuracy of emotion recognition, the classifier can be trained on a large amount of training data, and its model parameters can be continuously optimized.
[0031] In some application scenarios, based on the cross-attention mechanism, the target number of historical event vectors are fused with the current event vector to obtain a fused event vector, which can be expressed by the following formula: F_e_enhanced = F_e + MLP(Concat(Attn_em1, Attn_em2, Attn_em3)), where F_e_enhanced is the fused event vector, F_e is the current event vector, MLP is a multilayer perceptron, Concat represents the concatenation operation, and Attn_em1, Attn_em2, and Attn_em3 are the attention vectors calculated by the cross-attention mechanism between the current event vector and each historical event vector (assuming the target number is 3).
[0032] For content where a fused event vector is input into a classifier for emotion recognition, assuming there are four emotion categories: happy, sad, angry, and surprised, the classifier might output a four-dimensional vector, such as [0.7, 0.1, 0.1, 0.1]. This indicates that the current event has a 70% probability of belonging to the happy emotion category, while the probability of belonging to other emotion categories is relatively small. In practical implementation, the above emotion recognition method can be flexibly adjusted and optimized for different application scenarios and needs.
[0033] Therefore, based on the emotion recognition method provided in the above embodiments, by introducing a memory database and a cross-attention mechanism, it is possible to obtain historical event vectors with high similarity, which can be fused with the current event vector for emotion recognition, thereby effectively utilizing historical event information to improve the accuracy of emotion recognition.
[0034] Figure 2 This is a flowchart illustrating another embodiment of the emotion recognition method provided in this application.
[0035] Combination Figure 2 In some specific embodiments, after the steps of obtaining the current event, encoding the current event into a current event vector, extracting key information from the current event vector to obtain the current backbone vector, and calculating the residual between the current event vector and the current backbone vector as the current detail vector, i.e., after the above-mentioned step S101, the method includes: S201: Store the current backbone vector and the current detail vector in the memory database to serve as the historical backbone vector and historical detail vector of the memory database.
[0036] After obtaining the current backbone vector and the current event vector, the current backbone vector and the current detail vector are stored in the memory database. This operation continuously enriches the content of the memory database. As the memory database is continuously updated, the number and variety of historical backbone vectors and historical detail vectors stored in it increase, providing more comprehensive and representative historical reference information for subsequent emotion recognition.
[0037] S202: Update the historical backbone vector using the first forgetting mechanism, and update the historical detail vector using the second forgetting mechanism.
[0038] The current backbone vector stored in the memory database is dynamically updated through a first forgetting mechanism. This first forgetting mechanism is designed to preserve the core features of historical backbone vectors while gradually weakening information that becomes less important over time, and this process does not change the vector's dimension. For example, if the historical backbone vector has 100 dimensions, its dimension remains 100 after the first forgetting mechanism update. It should be understood that the first forgetting mechanism is designed based on simulating the tendency of human memory to retain the "key points / core" of events.
[0039] Simultaneously, the historical detail vector is updated using a second forgetting mechanism. Unlike the first forgetting mechanism, the second forgetting mechanism changes the vector's dimensions during the update process. For example, if the initial dimension of the historical detail vector is 100, after the second forgetting mechanism update, its dimension may become 80. The second forgetting mechanism is designed to simulate the tendency of human memory to easily lose specific "details." Historical detail vectors typically contain more subtle and specific feature information; therefore, the second forgetting mechanism simulates the loss of "details" by changing the vector's dimensions.
[0040] Therefore, by updating the historical backbone vector and the historical detail vector through the first forgetting mechanism and the second forgetting mechanism respectively, the information in the memory database can be made more in line with the laws of human memory. It retains the core features while gradually weakening the less important details, thus providing a more effective historical reference in subsequent emotion recognition.
[0041] Figure 3 This is a flowchart illustrating another embodiment of the emotion recognition method provided in this application.
[0042] Combination Figure 3 In some specific embodiments, the step of updating the historical backbone vector through a first forgetting mechanism includes: S301: If the storage time of the historical backbone vector in the memory database reaches the first duration, the historical backbone vector is compressed for the first time through information distillation.
[0043] If the historical backbone vector has been stored in the memory database for the first time, it means that the historical backbone vector has existed in the memory database for some time, and some of the information it contains may become less important over time (analogous to the human forgetting mechanism). At this point, information distillation is used for the first information compression. Information distillation is a technique that reduces the data size by retaining key information and removing redundant information. It can simplify the historical backbone vector without changing its vector dimension, retaining its core features. For example, the most representative features can be extracted from the historical backbone vector to form a new historical backbone vector with the same dimension but more compact information.
[0044] It should be understood that the first duration can be set according to the actual situation. For example, the first duration can be set to 1 day, but this is not a limitation.
[0045] S302: If the storage duration of the historical backbone vector after the first information compression reaches the second duration, then the historical backbone vector is subjected to a second information compression through information distillation; wherein the second duration is longer than the first duration.
[0046] If the storage duration of the historical backbone vector after the initial information compression reaches the second duration, it indicates that the historical backbone vector has existed in the memory database for a period of time after the initial information compression, and some of its information may have become less important. In this case, information distillation is used again to perform a second information compression on the historical backbone vector, further simplifying it without changing the vector dimension, making its core features more prominent. It should be understood that the second duration can be set according to actual conditions, and the second duration is longer than the first duration. For example, if the first duration is set to one week, the second duration can be set to one month, but this is not a limitation.
[0047] Based on this embodiment, through such multiple information compressions, the historical backbone vectors in the memory database can always remain in a relatively refined state, preserving core features while reducing redundant information, thus providing a more effective reference for subsequent emotion recognition.
[0048] Figure 4 This is a flowchart illustrating another embodiment of the emotion recognition method provided in this application.
[0049] Combination Figure 4 In some specific embodiments, the step of updating the historical detail vector through a second forgetting mechanism includes: S401: If the storage duration of the historical detail vector in the memory database reaches the third duration, then the historical detail vector is subjected to the first dimensional compression according to the first retention ratio.
[0050] If the historical detail vector has been stored in the memory database for three consecutive periods, it means that the vector has existed for some time, and some subtle, specific features may have become less critical over time. At this point, it undergoes initial dimensional compression according to a first retention ratio to simulate the fading of human memory. This first retention ratio can be set according to actual needs, for example, to 90%, meaning that 90% of the original historical detail vector's dimensional information is retained, while 10% of the relatively less important dimensional information is removed, thus simulating the process of "detail" loss in human memory. For example, an original historical detail vector of 100 dimensions becomes 90 dimensions after compression with a 90% retention ratio.
[0051] S402: If the storage duration of the historical detail vector after the first dimensional compression reaches the fourth duration, then the historical detail vector is subjected to a second dimensional compression according to the second retention ratio; wherein the fourth duration is longer than the third duration, and the second retention ratio is less than the first retention ratio.
[0052] If the storage duration of the historical detail vector after the initial dimensional compression reaches the fourth duration, it indicates that the historical detail vector has been stored for a period of time, and some remaining feature information may have further lost its criticality. At this point, a second dimensional compression is performed according to the second retention ratio. The second retention ratio is less than the first retention ratio; for example, if the first retention ratio is set to 90%, the second retention ratio can be set to 80%. Again, taking the original 100-dimensional historical detail vector as an example, if a second dimensional compression is performed with a second retention ratio of 80%, it becomes 80-dimensional. It should be understood that both the first and second retention ratios are compression ratios relative to the historical detail vector before compression. The fourth duration is longer than the third duration; the third duration could be 1 day, and the fourth duration could be 3 days. Furthermore, the third duration is shorter than the first duration, and the fourth duration is correspondingly shorter than the second duration. This setting is to more meticulously simulate the process of human memory gradually forgetting details over time.
[0053] In conjunction with the above, in some specific embodiments, the step of updating the historical detail vector through a second forgetting mechanism includes: after the historical detail vector is stored in the memory database, the historical backbone vector is zeroed out at preset time intervals according to a preset zero-ratio. Here, zeroing out the historical backbone vector at preset time intervals is a special dimensional compression method, different from the compression based on the retention ratio mentioned earlier. It directly sets some dimensions of the historical detail vector to zero to simulate human memory forgetting.
[0054] The preset zero-ratio can be set according to actual needs, such as 10%, meaning that 10% of the dimensions of the historical detail vector will be set to zero every preset time interval. The preset time interval can also be set according to actual needs, such as 10 hours, meaning that the dimension zeroing operation will be performed every 10 hours. This processing method can more directly simulate the gradual loss of "details" in human memory, because as time goes by, some subtle and specific feature information may be completely forgotten, which corresponds to the value of some dimensions of the vector becoming zero.
[0055] Figure 5 This is a flowchart illustrating another embodiment of the emotion recognition method provided in this application.
[0056] Combination Figure 5In some specific embodiments, the step of performing a similarity search in the memory database based on the current backbone vector to obtain a target number of historical event vectors with similarity from high to low, i.e., the above step S102, includes: S501: Based on the current backbone vector, perform a similarity search in the memory database to obtain the first number of historical backbone vectors and the corresponding first number of historical detail vectors with similarity from high to low; wherein, a vector combination consisting of a historical backbone vector and a historical detail vector corresponds to a historical event vector.
[0057] This embodiment specifically defines the steps for retrieving a target number of historical event vectors. In the memory database, a similarity retrieval operation is performed based on the current backbone vector. Using specific similarity calculation methods, such as the cosine similarity and Euclidean distance mentioned earlier, the first number of historical backbone vectors, ranked from highest to lowest similarity to the current backbone vector, are accurately identified. Each historical backbone vector has a corresponding historical detail vector in the memory database. These vectors together form a vector combination, which corresponds to a historical event vector. For example, if five historical backbone vectors are retrieved, there will be five corresponding historical detail vectors, thus forming five historical event vectors. These historical event vectors are arranged from highest to lowest similarity to the current backbone vector.
[0058] S502: Reconstruct the historical backbone vector and historical detail vector under each vector combination to obtain the first number of historical event vectors.
[0059] After obtaining the first set of historical backbone vectors and their corresponding first set of historical detail vectors, a memory reconstruction operation is required. The memory reconstruction process involves combining the historical backbone vectors and their corresponding historical detail vectors according to certain rules or models to reconstruct a complete historical event vector. It should be understood that this reconstruction method can fully utilize the core feature information in the historical backbone vectors and the subtle, specific feature information in the historical detail vectors, thereby obtaining a more accurate and comprehensive historical event vector. For example, specific matrix operations or neural network models can be used to fuse the historical backbone vectors and historical detail vectors to obtain the final historical event vector.
[0060] S503: Based on the current backbone vector and the current detail vector, perform similarity retrieval on a first number of historical time vectors to obtain a target number of historical event vectors with similarity from high to low; wherein the target number is less than the first number.
[0061] After obtaining the first number of historical event vectors, a secondary similarity search is needed, combining the current backbone vector and the current detail vector. This search is conducted within the scope of the first number of historical event vectors obtained previously. By applying similarity calculation methods again, a target number of historical event vectors are selected from these first number of vectors, ranked from highest to lowest similarity to the current backbone vector and current detail vector. Unlike the previous step, which only retrieved the first number of historical event vectors from the current backbone vector, this step combines the current backbone vector and current detail vector for the search. Here, the target number is less than the first number. This setting is to further narrow the scope and more accurately find the historical event vectors most similar to the current event, providing strong support for more accurate emotion recognition in the future. For example, if the first number is 10, the target number can be set to 3, that is, finding the 3 most similar ones from the 10 historical event vectors.
[0062] Figure 6 This is a flowchart illustrating another embodiment of the emotion recognition method provided in this application.
[0063] Combination Figure 6 In some specific embodiments, the step of reconstructing the historical backbone vector and historical detail vector under each vector combination to obtain a first number of historical event vectors, i.e., the above-mentioned step S502, includes: S601: If the dimension of the historical detail vector of the first vector combination is not compressed, then the historical backbone vector and the historical detail vector under the first vector combination are fused to obtain the corresponding historical event vector.
[0064] If the historical detail vector of the first vector combination is not compressed, it means that the historical detail vector retains relatively complete information. In this case, the historical backbone vector and the historical detail vector under the first vector combination are directly fused. This fusion can be achieved through a simple concatenation operation, such as concatenating the historical backbone vector and the historical detail vector in sequence into a new vector. For example, if the historical backbone vector is 5-dimensional and the historical detail vector is 5-dimensional and uncompressed, fusion through concatenation will yield a 5-dimensional historical event vector.
[0065] S602: If the dimension of the historical detail vector of the second vector combination is compressed, the historical detail vector of the second vector combination is reconstructed to obtain the reconstructed historical detail vector. The reconstructed historical detail vector is then fused with the historical backbone vector of the second vector combination to obtain the corresponding historical event vector. The historical backbone vector under the second vector combination has the same dimension as the reconstructed detail vector.
[0066] If the dimension of the historical detail vector of the second vector combination is compressed, it means that some subtle and specific feature information is lost during storage. To more accurately reconstruct the historical event vector, it is necessary to reconstruct the historical detail vector of the second vector combination. This reconstruction can be achieved using various methods, such as interpolation, fitting, or machine learning models like autoencoders, to recover the compressed historical detail vector. In some applications, reconstruction can be performed using an MLP network, a multi-layer feedforward neural network with powerful nonlinear mapping capabilities.
[0067] It should be understood that the reconstructed detail vector is close to the original historical detail vector. After obtaining the reconstructed historical detail vector, it is fused with the historical backbone vector of the second vector combination. Since the historical backbone vector under the second vector combination has the same dimension as the reconstructed detail vector, the fusion process can be smoother. For example, the same splicing operation can be used to connect them into a new vector, that is, the corresponding historical event vector.
[0068] Based on the above, combinations of historical detail vectors whose dimensions are not compressed are considered the first vector combination, while combinations of historical detail vectors whose dimensions are compressed are considered the second vector combination.
[0069] Figure 7 This is a flowchart illustrating another embodiment of the emotion recognition method provided in this application.
[0070] Combination Figure 7 In some specific embodiments, the step of performing a similarity search in the memory database based on the current backbone vector to obtain a first number of historical backbone vectors and a first number of historical detail vectors with similarity from high to low, i.e., the above-mentioned step S501, includes: S701: Based on a preset calculation method, perform initial similarity calculation in the memory database according to the current backbone vector to obtain a second number of historical backbone vectors with initial similarity from high to low and a corresponding second number of historical detail vectors; wherein, the second number is greater than the first number.
[0071] The preset calculation method can be the similarity calculation method described in the above embodiments, such as cosine similarity or Euclidean distance. Through calculation, a series of similarity values can be obtained. Based on these values in descending order, a second number of historical backbone vectors are selected. Since each historical backbone vector has a corresponding historical detail vector in the memory database, a corresponding second number of historical detail vectors can also be obtained. Setting the second number to be greater than the first number is to cover as many potentially related historical events as possible in the initial retrieval stage, providing a sufficient data foundation for more accurate subsequent filtering. For example, the second number can be set to 20. If there are many historical backbone vectors in the memory database, the top 20 historical backbone vectors with the highest similarity to the current backbone vector are found through calculation.
[0072] S702: The initial similarity of the second number of historical backbone vectors is weighted to obtain the first number of historical backbone vectors and the corresponding first number of historical detail vectors with target similarity from high to low after weighting the initial similarity.
[0073] At this point, weighting the initial similarity of the second set of historical backbone vectors is to more accurately measure the similarity between these historical backbone vectors and the current backbone vector. The weighting calculation can consider various factors to set weights, and no specific restrictions are imposed here.
[0074] By weighting and adjusting the initial similarity, the target similarity is obtained. Then, based on the target similarity from high to low, a first set of historical backbone vectors and their corresponding first set of historical detail vectors are selected. The historical event vectors selected in this way better reflect the core information related to the current event, providing a more valuable reference for subsequent emotion recognition. For example, if 20 historical backbone vectors are initially selected, after weighted calculation, the 10 historical backbone vectors with the highest similarity and their corresponding 10 historical detail vectors are selected.
[0075] Figure 8 This is a flowchart illustrating another embodiment of the emotion recognition method provided in this application.
[0076] Combination Figure 8 In some specific embodiments, the step of weighting the initial similarity of the second number of historical backbone vectors includes: S801: Obtain the intensity of user emotions corresponding to the historical event vectors of each historical backbone vector in the second number of historical backbone vectors, as well as the storage duration in the memory database.
[0077] This embodiment obtains the intensity of user emotions related to historical event vectors corresponding to each historical backbone vector in the second set of historical backbone vectors, as well as the storage duration in the memory database. The corresponding data information can be directly extracted from the memory database. The intensity of user emotions refers to the intensity of user emotions corresponding to the historical event vector. It can be measured in various ways, such as user emotion ratings or intensity levels of emotion descriptions.
[0078] S802: Determine the weight value of each historical backbone vector based on the intensity of user emotions and storage duration, and calculate the initial similarity of each historical backbone vector by weighting the weight values; where the intensity of user emotions remains unchanged, the longer the storage duration, the lower the weight value, and the longer the storage duration, the stronger the intensity of user emotions, the higher the weight value.
[0079] This step further determines the weight values of each historical backbone vector based on the intensity of the user's emotion and the storage duration. The principle followed in determining the weight values is: when the intensity of the user's emotion remains constant, a longer storage duration means that the historical event may have a smaller impact on the current emotion recognition, and therefore a lower weight value. Conversely, when the storage duration remains constant, a stronger user's emotion indicates a more significant impact of the historical event on the user's emotion, and therefore a higher weight value.
[0080] In some application scenarios, the weight value = Where I represents the intensity of the user's emotion (e.g., range 0-5), and ΔT (≥1) represents the time interval between the event and the present. (Range from 1 to e≈2.78) Increases with increasing I, and decreases with increasing ΔT. λ is responsible for controlling the adjustment range according to the actual situation; for example, if λ=0.1, then... The range is approximately 1.1–1.278.
[0081] A second aspect of this application provides a computer-readable storage medium 40, Figure 9 This is a schematic diagram of the structural framework of an embodiment of the computer-readable storage medium 40 provided in this application.
[0082] Combination Figure 9 The computer-readable storage medium 40 stores a computer program 41, which, when executed by a processor, implements the emotion-based recognition method as described in any of the above embodiments.
[0083] It should be noted that the computer-readable medium 40 shown in the embodiments of this application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.
[0084] In summary, the emotion recognition method and computer-readable storage medium provided in this application include: acquiring the current event; encoding the current event into a current event vector; extracting key information from the current event vector to obtain a current backbone vector; and calculating the residual between the current event vector and the current backbone vector as the current detail vector; performing a similarity search in a memory database based on the current backbone vector to obtain a target number of historical event vectors with varying similarity from high to low; wherein the historical event vectors include historical backbone vectors and historical detail vectors; and fusing the target number of historical event vectors with the current event vector based on a cross-attention mechanism to obtain a fused event vector, and inputting the fused event vector into a classifier for emotion recognition. Therefore, by introducing a memory database and a cross-attention mechanism, historical event vectors with high similarity can be obtained and fused with the current event vector for emotion recognition, effectively utilizing historical event information to improve the accuracy of emotion recognition.
[0085] The above description is merely a preferred exemplary embodiment of this application and is not intended to limit the implementation of this application. Those skilled in the art can easily make corresponding modifications or alterations based on the main concept and spirit of this application. Therefore, the scope of protection of this application should be determined by the scope of protection claimed in the claims.
Claims
1. A method of emotion recognition, characterized by, The method comprises the following steps: obtaining a current event, encoding the current event into a current event vector, extracting key information from the current event vector to obtain a current backbone vector, and calculating the residual of the current event vector and the current backbone vector as a current detail vector; performing similarity retrieval in a memory database based on the current backbone vector to obtain a target number of historical event vectors in descending order of similarity; wherein the historical event vector comprises a historical backbone vector and a historical detail vector; based on the cross-attention mechanism, the target number of historical event vectors and the current event vector are fused to obtain a fusion event vector, and the fusion event vector is input into a classifier for emotion recognition.
2. The emotion recognition method of claim 1, wherein, after the steps of obtaining a current event, encoding the current event into a current event vector, extracting key information from the current event vector to obtain a current backbone vector, and calculating the residual of the current event vector and the current backbone vector as a current detail vector, the method comprises the following steps: storing the current backbone vector and the current detail vector in the memory database as a historical backbone vector and a historical detail vector of the memory database; updating the historical backbone vector through a first forgetting mechanism, and updating the historical detail vector through a second forgetting mechanism.
3. The emotion recognition method of claim 2, wherein, the step of updating the historical backbone vector through a first forgetting mechanism comprises: if the storage duration of the historical backbone vector in the memory database reaches a first duration, performing first information compression on the historical backbone vector through information distillation; if the storage duration of the historical backbone vector after the first information compression reaches a second duration, performing second information compression on the historical backbone vector through information distillation; wherein the second duration is longer than the first duration.
4. The emotion recognition method of claim 2, wherein, the step of updating the historical detail vector through a second forgetting mechanism comprises: if the storage duration of the historical detail vector in the memory database reaches a third duration, performing first dimension compression on the historical detail vector according to a first retention ratio; if the storage duration of the historical detail vector after the first dimension compression reaches a fourth duration, performing second dimension compression on the historical detail vector according to a second retention ratio; wherein the fourth duration is longer than the third duration, and the second retention ratio is less than the first retention ratio.
5. The emotion recognition method of claim 2, wherein, the step of updating the historical detail vector through a second forgetting mechanism comprises: after the historical detail vector is stored in the memory database, performing dimension zero processing on the historical backbone vector according to a preset zero ratio every preset duration.
6. The emotion recognition method of claim 1, wherein, The step of performing similarity retrieval in the memory database based on the current backbone vector to obtain target number of historical event vectors in descending order of similarity comprises: performing similarity retrieval in the memory database based on the current backbone vector to obtain first number of historical backbone vectors and corresponding first number of historical detail vectors in descending order of similarity; wherein one historical backbone vector and one historical detail vector form a vector combination corresponding to one historical event vector; reconstructing the historical backbone vector and the historical detail vector under each vector combination to obtain first number of historical event vectors; performing similarity retrieval in the first number of historical event vectors based on the current backbone vector and the current detail vector to obtain target number of historical event vectors in descending order of similarity; wherein the target number is less than the first number.
7. The emotion recognition method of claim 6, wherein the step of reconstructing the historical backbone vector and the historical detail vector under each vector combination to obtain first number of historical event vectors comprises: if the dimension of the historical detail vector of the first vector combination is not compressed, fusing the historical backbone vector and the historical detail vector under the first vector combination to obtain a corresponding historical event vector; if the dimension of the historical detail vector of the second vector combination is compressed, reconstructing the historical detail vector of the second vector combination to obtain a reconstructed historical detail vector, and fusing the reconstructed historical detail vector and the historical backbone vector of the second vector combination to obtain a corresponding historical event vector; wherein the dimension of the historical backbone vector and the reconstructed detail vector under the second vector combination is the same.
8. The emotion recognition method of claim 6, wherein the step of performing similarity retrieval in the memory database based on the current backbone vector to obtain first number of historical backbone vectors and first number of historical detail vectors in descending order of similarity comprises: based on a pre-designed calculation method, performing initial similarity calculation in the memory database based on the current backbone vector to obtain second number of historical backbone vectors and corresponding second number of historical detail vectors in descending order of initial similarity; wherein the second number is greater than the first number; performing weighted calculation on the initial similarity of the second number of historical backbone vectors to obtain first number of historical backbone vectors and corresponding first number of historical detail vectors in descending order of target similarity after initial similarity weighting.
9. The emotion recognition method of claim 8, wherein the step of performing weighted calculation on the initial similarity of the second number of historical backbone vectors comprises: obtaining the user emotional intensity corresponding to each historical event vector corresponding to each historical backbone vector in the second number of historical backbone vectors and the storage duration stored in the memory database. The weight value of each historical backbone vector is determined according to the user emotion intensity and the storage duration, and the initial similarity of each historical backbone vector is calculated by weighting the weight value; wherein, when the user emotion intensity is unchanged, the longer the storage duration is, the lower the weight value is, and when the storage duration is unchanged, the stronger the user emotion intensity is, the higher the weight value is.
10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is executed by the processor to implement the emotion recognition method in any one of claims 1-9.