User state identification method and device, computer equipment and storage medium

Abstract

The invention discloses a user state recognition method and device, computer equipment and a storage medium, relates to the technical field of artificial intelligence, and is used for solving the problem that user interaction experience and service accuracy are affected due to one-sided and unstable state recognition results in related technologies. The method comprises the following steps: acquiring multi-modal data of a user, and fusing the multi-modal data to obtain a target feature; performing similarity retrieval in a historical database based on the target features to obtain at least one target historical data record; and performing state recognition based on the target features and the target historical data records to obtain target state data of the user.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a user status recognition method, apparatus, computer device, and storage medium. Background Technology

[0002] In the field of artificial intelligence, especially in human-computer interaction, affective computing, and personalized services, real-time identification of a user's current state is a fundamental and crucial task.

[0003] In related technologies, methods for user state recognition mainly include: First, extracting features from the raw data input by the user and transforming it into a machine-processable feature representation. Then, based on this feature representation, using a classification model to determine the user's state and outputting the corresponding state label. Although this method can achieve fast, instantaneous state judgment, it only performs isolated analysis on a "snapshot" of user data at the current moment, which can easily lead to biased and unstable state recognition results, thus affecting the user interaction experience and service accuracy. Summary of the Invention

[0004] Based on this, a user status recognition method, device, computer equipment, and storage medium are provided to solve the problem that the status recognition results in related technologies are one-sided and unstable, thereby affecting the user interaction experience and service accuracy.

[0005] In a first aspect, the present invention provides a user state recognition method, the method comprising: Acquire the user's multimodal data and fuse the multimodal data to obtain target features; Based on the target features, a similarity search is performed in the historical database to obtain at least one target historical data record; Based on the target features and the target historical data records, state identification is performed to obtain the user's target state data.

[0006] In one embodiment, the step of performing a similarity search in a historical database based on the target features to obtain at least one target historical data record includes: Based on the target features, a similarity search is performed in the historical database to obtain M historical data records, and based on the target features, the user's initial state data is generated; Based on the initial state data, N target historical data records are selected from the M historical data records, where N is an integer greater than 0.

[0007] In one embodiment, the step of selecting N target historical data records from the M historical data records based on the initial state data includes: The user's personality traits are obtained. Based on the personality traits, the target traits, and the initial state data, the scores of the M historical data records are rearranged and re-scored to obtain the scores corresponding to the M historical data records. Based on the scores corresponding to the M historical data records, the N target historical data records with the highest scores are selected from the M historical data records.

[0008] In one embodiment, the step of performing state identification based on the target features and the target historical data records to obtain the user's target state data includes: Obtain the user's initial state data and personality traits, wherein the initial state data includes the user's basic emotion category; By using a pre-set large model, state recognition is performed based on the initial state data, the personality traits, the target traits, and the target historical data records to obtain the target state data; The target state data includes the current context information defined by the target feature, the personalized performance of the basic emotion category under the current context information after being affected by the personality feature, and the historical descriptive information related to the basic emotion category in the target historical data record.

[0009] In one embodiment, after obtaining the user's target state data, the method further includes: Obtain the target score corresponding to the target historical data record; If the target score is less than or equal to a preset score threshold, a new data record is generated based on the target state data and the target features, and the new data record is stored in the historical database. If the target score is greater than the preset score threshold, the target historical data record is updated based on the target state data and the target features.

[0010] In one embodiment, the historical database includes multiple historical data records, each of which includes at least historical features and a memory strength value characterizing the importance of the historical features. The updating steps of the historical database include: Based on a preset attenuation factor, the memory strength value of each historical data record in the historical database is attenuated to obtain the attenuated memory strength value. Based on the decayed memory strength value, historical data records to be cleared are determined from the historical database, and a forgetting operation is performed on the historical data records to be cleared.

[0011] In one embodiment, the step of updating the personality trait includes: Retrieve new data records from the historical database and generate the user's current personality traits based on the new data records; The fusion weight is determined based on the total number of data records in the historical database; Based on the fusion weight, the personality traits before the update are weighted and fused with the current personality traits to obtain the updated personality traits.

[0012] In a second aspect, the present invention provides a user status identification device, the device comprising: The fusion module is used to acquire the user's multimodal data and fuse the multimodal data to obtain target features; The retrieval module is used to perform a similarity search in the historical database based on the target features to obtain at least one target historical data record; The status recognition module is used to perform status recognition based on the target features and the target historical data records to obtain the user's target status data.

[0013] Thirdly, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the user state recognition method of the first aspect described above.

[0014] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the user state recognition method of the first aspect described above.

[0015] The aforementioned user state recognition method, device, computer equipment, and storage medium integrate multimodal data to form comprehensive and robust user target features. Subsequently, these target features are used to perform efficient similarity searches in historical databases to quickly locate similar historical records. Finally, the target features and similar historical records are combined for comprehensive analysis to achieve accurate and reliable recognition of the user's current state, thereby providing users with more accurate and consistent personalized services and interactive experiences. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a user state recognition method in one embodiment; Figure 2 This is a structural block diagram of a user state recognition device in one embodiment; Figure 3 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. The specific operational methods in the method embodiments can also be applied to the device embodiments or system embodiments. It should be noted that in the description of this invention, "multiple" is understood as "at least two". "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing together, or B existing alone. A connected to B can represent: A and B directly connected, or A and B connected through C. Furthermore, in the description of this invention, terms such as "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or order.

[0018] Before introducing the user status recognition method provided by this invention, the technical background of this invention will be described in detail below for ease of understanding.

[0019] With the rapid development of artificial intelligence technology, personalized services have become a core pursuit for improving the quality of human-computer interaction and user experience. To achieve this goal, personalized user profile modeling is essential. Existing personalized technology solutions mainly focus on the following aspects: Content-based recommendation and interaction: The core of this approach lies in driving system feedback based on the direct, explicit content of the user's current input. For example, in recommendation systems, similar items are recommended based on the products the user is currently browsing or their historical click history; in dialogue systems, relevant information is returned based on query keywords. This method is simple to implement and responds quickly, but it lacks an understanding of the user's deep psychological state, resulting in superficial interactions and difficulty in achieving deep resonance.

[0020] Personalization based on static profiles or personality traits: This approach attempts to construct a relatively fixed user profile or personality model (such as introverted / extroverted, rational / emotional) by analyzing user registration information or long-term historical behavioral data. Once this user profile or personality model is established, it serves as the benchmark for personalization in subsequent interactions. However, a significant drawback of this method is that it is difficult to dynamically evolve with continuous user interaction with the system, leading to a gradual rigidity in the personalization effect and a deviation from the user's true state.

[0021] Interaction based on instantaneous emotions: Advances in affective computing have enabled systems to identify a user's current emotional state (e.g., happiness, sadness, anger) in real time using multimodal information (such as the sentiment of text, tone of voice, and facial expressions), and react accordingly, such as playing soothing music when sadness is detected. This technology achieves a preliminary "empathic" response, but it only performs isolated analysis of a "snapshot" of user data at the current moment, ignoring the continuity and contextual relationship of the user's state over time. This isolated analysis paradigm easily leads to one-sided and unstable state recognition results, thus affecting the overall interactive experience and service accuracy.

[0022] Application of memory mechanisms: To overcome the "stateless" problem of interaction, some advanced dialogue systems have introduced memory mechanisms, which can store and recall specific factual information mentioned by users in the conversation (such as "I like coffee" or "My profession is engineer"). However, existing memories are mostly flat, factual records, lacking emotional dimensions and contextual relevance.

[0023] The technical solution provided by the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0024] Figure 1 This is a flowchart illustrating a user state recognition method in one embodiment. This process can be executed by a user state recognition device, which can be implemented in software, hardware, or a combination of both. Figure 1 As shown, the process includes the following steps: S101, acquire the user's multimodal data, and fuse the multimodal data to obtain the target features; S102, perform a similarity search in the historical database based on the target features to obtain at least one historical data record of the target; S103, based on target features and historical data records of the target, performs state recognition to obtain the user's target state data.

[0025] The user's multimodal data includes, but is not limited to: visual image data (such as facial expressions, body movements, etc., which can be obtained through cameras), text data (such as written text entered by the user, chat logs, etc.), auditory data (such as voice content, which can be obtained through recording devices), grip strength signal data (which can be obtained through grip strength sensors), and electrocardiogram signal data (which can be collected through non-invasive physiological wristbands).

[0026] Status data includes, but is not limited to: emotional status data (such as happiness, sadness, anger, irritability, etc.) and physiological status data (such as fatigue, excitement, etc.).

[0027] By using the above methods, multimodal data is integrated to form comprehensive and robust user target features. Then, these target features are used to perform efficient similarity retrieval in historical databases to quickly locate similar historical records. Finally, the target features and similar historical records are combined for comprehensive analysis to achieve accurate and reliable identification of the user's current state, thereby providing users with more accurate and consistent personalized services and interactive experiences.

[0028] In one embodiment, it is exemplarily illustrated that in S101, multimodal data is fused to obtain target features, including but not limited to: First, feature extraction is performed on the data of each modality to obtain the feature data of each modality.

[0029] For example, preprocessing is performed on the modal data to obtain preprocessed modal feature data, thus eliminating data scale differences and individual environmental interference. Then, feature extraction is performed on each preprocessed modal data to obtain the modal feature data. Specifically: Face detection and alignment are performed on the visual image data, followed by pixel value normalization and standardization based on channel mean and standard deviation to obtain preprocessed visual image data. Image features are then extracted from the preprocessed visual image data using a pre-defined neural network model (such as a convolutional neural network model, a recurrent neural network model, or a neural network model based on a self-attention mechanism, etc., the specific model depending on the situation and not limited here). The aforementioned methods for extracting image feature vectors are well-known and will not be elaborated upon here.

[0030] The text data undergoes standardization and cleaning, including lowercase conversion, noise removal, word segmentation, and lexical reconstruction, to obtain preprocessed text data. A pre-defined large model is then used to extract text features from the preprocessed text data, resulting in text feature vectors. Similarly, after converting auditory data into text data using an automatic speech recognition model, the corresponding text feature vectors are extracted using the same method described above. The methods for extracting these text feature vectors are well-known and will not be elaborated upon here.

[0031] Adaptive sliding window differential normalization was performed on the grip strength signal data and electrocardiogram (ECG) signal data respectively. This involved normalizing the signal data using the mean and standard deviation within a preset sliding window to obtain preprocessed grip strength and ECG signal data. Then, max pooling was performed along the time axis on the preprocessed grip strength and ECG signal data to capture key fluctuation events, resulting in grip strength and ECG signal feature vectors.

[0032] Then, based on the attention mechanism, cross-modal fusion is performed on the feature data of each modality to obtain the target feature.

[0033] For example, feature data from at least one modality is used as the query vector, and feature data from other modalities are used as key and value vectors. A first similarity (e.g., dot product similarity, cosine similarity, etc.) is calculated between the query vector and the key vectors, and this first similarity is normalized using a normalized exponential function to obtain attention weights. These weights characterize the relative importance of each value vector in the context of the current query. The value vectors are then weighted and summed based on the attention weights to obtain a fused feature vector, which is used as the user's target feature.

[0034] Alternatively, the fused feature vector can be concatenated with the query vector, and then dimensionality reduced or integrated through a fully connected layer. The final output is the target feature, which can preserve the information of the original modality and avoid information loss during the attention process.

[0035] By employing the above method, based on the cross-modal attention mechanism, multimodal data are effectively fused, achieving information complementarity and synergistic enhancement between modalities. This results in the generation of more discriminative and robust target features, significantly improving the accuracy and reliability of subsequent state recognition tasks.

[0036] In one embodiment, it is exemplarily illustrated that in S102, a similarity search is performed in the historical database based on the target features to obtain at least one historical data record of the target, including but not limited to: First, a similarity search is performed in the historical database based on the target features to obtain M historical data records. The historical database includes multiple historical data records. Each historical data record includes at least the historical feature, the occurrence time of the historical feature (in this invention, the time when the historical feature is stored in the historical database is taken as its occurrence time), and the memory strength value that characterizes the importance of the historical feature. That is, each historical data record is a triple of {"historical feature", "occurrence time", "memory strength value"}.

[0037] For example, the second similarity between the target feature and the historical features of each historical data record in the historical database is calculated. Based on the second similarity between the target feature and the historical features, the M historical data records with the highest second similarity are retrieved from the historical database. The second similarity is calculated based on at least one of the following methods: distance metric, nearest neighbor algorithm, or approximate nearest neighbor algorithm. The specific retrieval method depends on the situation and is not limited here.

[0038] Simultaneously, based on the target features, the system outputs the user's initial state data through a preset recognition model. The preset recognition model can be a multilayer perceptron, regression model, preset large model, recurrent neural network, etc. The specific model depends on the situation and is not limited here. The initial state data includes the user's basic emotional categories, such as the emotional tendency (e.g., pleasant-unpleasant) representing whether the emotion is positive or negative, and the arousal level (e.g., high-low) representing the intensity of the emotion. The emotional tendency and arousal level can be represented numerically. For example, 0 represents the lowest level of pleasure or the lowest level of arousal, 50% represents neutral emotion (neither sad nor happy) or calm arousal, and 100% represents the highest level of pleasure or the highest level of arousal.

[0039] Then, based on the initial state data, N target historical data records are selected from M historical data records, where N is less than M and N is an integer greater than 0.

[0040] Using the above method, preliminary similarity screening is performed based on target features to narrow down the search scope. Then, a second round of refined filtering is performed by combining the semantic understanding of the initial state data. Finally, the most relevant target historical data records are locked from massive historical data, providing a high-quality and highly relevant information foundation for subsequent in-depth state analysis.

[0041] Alternatively, similarity searches can be performed directly in the historical database based on the target features to obtain N historical data records of the target (the search method is consistent with the above method and will not be described again here).

[0042] In one embodiment, exemplarily illustrated, N target historical data records are selected from M historical data records based on initial state data, including but not limited to: First, acquire the user's personality traits. For example, acquire historical data records within a preset time period (such as the past week, past month, etc., the specific time period depends on the situation and is not limited here), and infer the user's personality traits based on this historical data record using a preset large model. Specifically: Personality traits can be the degree of a user's tendency across multiple psychological dimensions, such as the four psychological dimensions of extroversion (E) and introversion (I), sensing (S) and intuition (N), thinking (T) and feeling (F), judging (J) and perceiving (P). The specific dimensions depend on the situation and are not limited here.

[0043] For example, if a user's personality traits are {"EI": 0.4, "SN": 0.5, "TF": 0.6, "JP": 0.7}, it means that the user is 40% extroverted and 60% introverted; when receiving information, the user's tendency to rely on concrete facts and abstract concepts is 50% each; when making decisions, the user is 60% inclined to think and 40% inclined to feel; and in life, the user is 70% inclined to judge and 30% inclined to perceive.

[0044] Then, based on personality traits, target features, and initial state data, a preset re-ranking model is used to re-rank and score the M historical data records, resulting in scores for each of the M historical data records. The preset re-ranking model can be a gradient boosting decision tree, a multilayer perceptron, a learning ranking model, etc., depending on the situation; no specific model is limited here.

[0045] Finally, based on the scores corresponding to the M historical data records, the N target historical data records with the highest scores are selected from the M historical data records.

[0046] By incorporating user personality traits into the above methods, historical data is re-ranked and scored in multiple dimensions, achieving personalized and accurate filtering and effectively improving the matching degree between target historical data records and users' actual status.

[0047] In one embodiment, exemplarily illustrated, in S103, state recognition is performed based on target features and target historical data records to obtain the user's target state data, including but not limited to: The system acquires the user's initial state data and personality traits. The initial state data includes the user's basic emotion categories. Based on the initial state data, the user's personality traits, target traits, and target historical data records, a pre-set large model is used to identify the user's target state data. The target state data includes the current context information defined by the target traits, the personalized performance of the basic emotion categories under the current context information after being influenced by the personality traits, and the historical descriptive information related to the basic emotion categories in the target historical data records.

[0048] For example, by using a pre-defined decoder to convert target features and historical data records into natural language descriptions, we obtain first text data and second text data. Based on the initial state data, the user's personality traits, the first text data, and the second text data, we construct a prompt: "You are a sentiment analysis model. Please generate a text describing your current inner emotions based on the following information: Current event: [first text data], Current emotional feeling: [initial state data], Personality traits: [personality traits], Possible past experiences: [second text data]. Please comprehensively analyze all the above information to generate a detailed description of your emotional state." Therefore, by pre-setting a large model, current situation information is extracted from the first text data, and descriptive information related to the basic emotion category is extracted from the second text data. Based on personality traits, the personalized performance of the basic emotion category under the current situation information is inferred. Finally, the descriptive information and the personalized performance are merged to obtain the user's target state data.

[0049] By using the above methods, precise prompt words are constructed by integrating user personality, target characteristics, real-time status, and historical records. The reasoning ability of the pre-set large model is used to generate target status data that is highly consistent with the user's personality and the current situation, which significantly improves the accuracy and robustness of status recognition.

[0050] Optionally, the target features, initial state data, personality traits, and historical features from the target's historical data records can be mapped to a unified semantic space through independent projection layers to obtain mapped feature vectors. These mapped feature vectors are then concatenated to form a comprehensive feature vector. This comprehensive feature vector is then input into a multilayer perceptron to output the user's target state data.

[0051] In the above scheme, a large model is used to identify the state of multi-source data, including but not limited to: initial state data, user personality traits, target characteristics, and historical data records of the target. These data sources are optional, not mandatory, in the state identification process. In practical applications, any single data source can be used, or any combination of multiple data sources can be used. Those skilled in the art should understand that the selection of data sources depends on the specific application scenario, data availability, and the required identification accuracy.

[0052] The scheme described herein, which utilizes all four data sources simultaneously for identification, is merely a preferred embodiment. This preferred embodiment can provide the most comprehensive and accurate identification results in most scenarios. However, this is not a limitation on the scope of protection of this scheme; any state identification method based on one or a combination of the above data sources should be considered within the scope of this scheme's conception.

[0053] In one embodiment, exemplarily illustrated, after obtaining the user's target state data in S103, the method further includes: Obtain the target score corresponding to the target historical data record. If the target score is less than or equal to a preset score threshold (e.g., 0.8; the specific threshold depends on the situation and is not limited here), then generate a new data record based on the target state data and target features, and store the new data record in the historical database.

[0054] For example, target state data and target features are encoded using a preset encoder to obtain new features, and the memory strength value of the new features is set to an initial value (e.g., 1, the specific value depends on the situation and is not limited here). Based on the new features and their memory strength values, new data records are constructed and stored in the historical database. The occurrence time in the new data record is the storage time of the new feature, and the tag for this new data record is "new".

[0055] If the target score is greater than the preset score threshold, the target historical data record will be updated based on the target status data and target features.

[0056] For example, by encoding the target state data, target features, and historical features in the target historical data records using a preset encoder, updated historical features are obtained. Then, the memory strength value in the target historical data records is increased to obtain an updated memory strength value, where the updated memory strength value = the original memory strength value + the target score (representing the greater the number of times the target historical data record has been retrieved). Finally, the occurrence time in the target historical data records is updated to the current time to obtain an updated occurrence time.

[0057] By using the above method, based on the target score corresponding to the target historical data record, new data records or updated target historical data records are automatically generated, realizing dynamic, efficient and accurate optimization of the historical database, thereby ensuring that the data can always reflect the latest user status.

[0058] In one embodiment, the steps for updating the historical database include, but are not limited to: First, based on a preset decay factor, the memory strength value of each historical data record in the historical database is decayed to obtain the decayed memory strength value. For example, the decayed memory strength value = the memory strength value before decay / (1 + α × n), where α is the preset decay factor, which can be a time decay factor or an interaction count decay factor, depending on the situation and is not limited here. n is the time interval. When α is a time decay factor, n is the preset decay period (such as weekly, monthly, etc., the specific period depends on the situation and is not limited here); when α is an interaction count decay factor, n is the number of times or days that the historical data record interacted with the user.

[0059] Then, based on the decayed memory strength value, the historical data records to be cleaned are identified from the historical database.

[0060] For example, historical data records whose decayed memory strength value is less than a preset strength threshold (the specific threshold depends on the situation and is not limited here) in the historical database are identified as historical data records to be cleaned up.

[0061] Optionally, if the total number of historical data records in the historical database is greater than a preset threshold (the specific threshold depends on the situation and is not limited here), each historical data record can be sorted based on the decayed memory strength value, and the P historical data records with the lowest memory strength values ​​can be identified as historical data records to be cleaned, so that the total number of remaining historical data records in the historical database other than the historical data records to be cleaned is less than or equal to the preset threshold, where P is an integer greater than 0.

[0062] For example, if the preset threshold for the number of records is 3, and there are 4 historical data records in the historical database, and their decayed memory strength values ​​are arranged in descending order as [historical data record 1, historical data record 2, historical data record 3, historical data record 4], then historical data record 3 and historical data record 4 have the same decayed memory strength value, and historical data record 3 and historical data record 4 are identified as historical data records to be cleaned up.

[0063] Finally, a forgetting operation is performed on the historical data records to be cleared. This forgetting operation includes, but is not limited to: Calculate the third similarity between the historical data record to be cleaned and the remaining historical data records in the historical database (the method for calculating the second similarity is the same as described above, and will not be repeated here). If there are related historical data records in the remaining historical data records whose third similarity with the historical data record to be cleaned is greater than the target score threshold (e.g., 0.7, where the target score threshold is less than the preset score threshold, and the specific threshold depends on the situation, and is not limited here), then the historical features in the historical data record to be cleaned will be merged into the historical features in the related historical data records, and the historical data record to be cleaned will be deleted. If there are no related historical data records, then the historical data record to be cleaned will be deleted directly.

[0064] By introducing the above methods and memory strength values ​​and decay mechanisms, the natural forgetting process of human memory is simulated, enabling dynamic optimization and storage management of the historical database. This retains high-value information while automatically cleaning up low-importance data, thereby improving database efficiency and resource utilization.

[0065] In one embodiment, the steps for updating personality traits include, but are not limited to: Retrieve new data records from the historical database, and generate the user's current personality traits based on the new data records using a pre-set large model.

[0066] Among these methods, new data records in the historical database can be retrieved by adding new tags, indicating that the new data record has not participated in personality modeling. After the new data record participates in personality modeling, its new tags are converted into historical tags.

[0067] By using the above methods, user personality traits are dynamically updated using newly added data records in the historical database, thereby ensuring that personality profiles continue to evolve over time and with the accumulation of data, maintaining their accuracy and timeliness.

[0068] In one embodiment, exemplarily illustrated, before retrieving newly added data records from the historical database, the method further includes: Retrieve the personality traits before the update. If the personality traits before the update meet the preset personality update conditions, retrieve the new data records in the historical database and update and generate the user's current personality traits based on the new data records.

[0069] Among them, meeting the preset personality update conditions includes, but is not limited to: reaching the preset update cycle (such as updating once a month, once every six months, etc., the specific cycle depends on the situation and is not limited here), and the total number of newly added data records in the historical database being greater than or equal to the target number threshold (the specific threshold depends on the situation and is not limited here).

[0070] By introducing preset personality update conditions using the above method, the update of personality traits is set as an event-driven, triggered update. This not only ensures that the personality model remains timely as user behavior evolves, but also effectively avoids redundant overhead caused by high-frequency calculations and prevents fluctuations in personality profiles caused by small amounts of occasional new data.

[0071] In one embodiment, an example is provided, illustrating the generation of a user's current personality traits based on newly added data records, including but not limited to: Use the current personality trait as the updated personality trait. Or, Based on the total number of data records in the historical database, a fusion weight β is determined. Based on β, the personality traits before the update are weighted and fused with the current personality traits to obtain the updated personality traits. For example, β is the first weight corresponding to the current personality trait, and the second weight corresponding to the personality traits before the update can be obtained from the first weight as (1-β). In this case, the updated personality trait = β × current personality trait + (1-β) × personality traits before the update.

[0072] Where β = 1 / (1 + γ × t), t is the number of times or days the personality trait is updated. γ is the decay coefficient, used to control the update magnitude, and is related to the total number of historical data records in the historical database. That is, when the total number of data records in the historical database is less than the target number threshold, the first weight is greater than the second weight; when the total number of data records in the historical database is greater than or equal to the target number threshold, the first weight is less than or equal to the second weight. Specifically: When the total number of records in the historical database is less than the target threshold (the specific threshold depends on the situation and is not limited here), γ is set to a small value. This means that when updating personality traits, the weight of the previous personality trait will be significantly reduced, while the weight of the current personality trait will be significantly increased, aiming for rapid learning. When the total number of records in the historical database is greater than or equal to the target threshold, the stability of personality traits needs to be maintained. A user's occasional abnormal behavior should not cause a drastic change in their entire personality trait. In this case, γ is set to a large value. This means that when updating personality traits, most of the influence of the previous personality trait will be retained, while the current personality trait will only undergo minor adjustments. The specific value of γ depends on the situation and is not limited here.

[0073] By using the above method, the personality traits generated from newly added data records are weighted and fused with the original traits, achieving a smooth and stable update of user personality traits. This not only absorbs new behavioral information but also preserves the continuity of historical personality, effectively avoiding abrupt changes in user profiles.

[0074] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order in which these steps are executed, and they can be performed in other orders. Furthermore, Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but may be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0075] In one embodiment, such as Figure 2 As shown, a user status recognition device is provided, including: a fusion module 201, a retrieval module 202, and a status recognition module 203, wherein: The fusion module 201 is used to acquire the user's multimodal data and fuse the multimodal data to obtain target features; The retrieval module 202 is used to perform similarity retrieval in the historical database based on target features and obtain at least one target historical data record; The status recognition module 203 is used to perform status recognition based on target features and historical data records of the target to obtain the user's target status data.

[0076] In one embodiment, the retrieval module 202 is further configured to: Based on the target features, a similarity search is performed in the historical database to obtain M historical data records, and the user's initial state data is generated based on the target features. Based on the initial state data, select N target historical data records from M historical data records, where N is an integer greater than 0.

[0077] In one embodiment, the retrieval module 202 is further configured to: Obtain the user's personality traits. Based on the personality traits, target traits, and initial state data, rearrange and score the M historical data records respectively to obtain the scores corresponding to the M historical data records. Based on the scores corresponding to M historical data records, select the N target historical data records with the highest scores from the M historical data records.

[0078] In one embodiment, the state recognition module 203 is further configured to: Obtain the user's initial state data and personality traits, where the initial state data includes the user's basic emotional categories; By using a pre-set large model, the target state data is obtained by identifying the state based on initial state data, personality traits, target characteristics, and target historical data records. The target state data includes current contextual information defining the target features, personalized performance of basic emotion categories under the current contextual information after being influenced by personality traits, and historical descriptive information related to basic emotion categories in the target historical data records.

[0079] In one embodiment, the state recognition module 203 is further configured to: Obtain the target score corresponding to the target's historical data records; If the target score is less than or equal to the preset score threshold, a new data record is generated based on the target status data and target features, and the new data record is stored in the historical database. If the target score is greater than the preset score threshold, the target historical data record will be updated based on the target status data and target features.

[0080] In one embodiment, the historical database includes multiple historical data records, each of which includes at least historical features and a memory strength value characterizing the importance of the historical features. The retrieval module 202 is further configured to: Based on a preset attenuation factor, the memory strength value of each historical data record in the historical database is attenuated to obtain the attenuated memory strength value. Based on the decayed memory strength value, the historical data records to be cleared are identified from the historical database, and a forgetting operation is performed on the historical data records to be cleared.

[0081] In one embodiment, the retrieval module 202 is further configured to: Retrieve new data records from the historical database and generate the user's current personality traits based on the new data records; The fusion weights are determined based on the total number of data records in the historical database. Based on the fusion weight, the personality traits before the update are fused with the current personality traits to obtain the updated personality traits.

[0082] Specific limitations regarding the user state identification device can be found in the limitations of the user state identification method described above, and will not be repeated here. Each module in the aforementioned user state identification device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0083] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 3 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores user status identification data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a user status identification method. The display screen can be an LCD screen or an e-ink display screen. The input device can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device casing, or an external keyboard, touchpad, or mouse.

[0084] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device to which the present invention is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0085] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: Acquire users' multimodal data and fuse the multimodal data to obtain target features; Based on the target features, perform a similarity search in the historical database to obtain at least one historical data record of the target. State recognition is performed based on target features and historical data records to obtain the user's target state data.

[0086] In one embodiment, the processor, when executing a computer program, also performs the following steps: Based on the target features, a similarity search is performed in the historical database to obtain M historical data records, and the user's initial state data is generated based on the target features. Based on the initial state data, select N target historical data records from M historical data records, where N is an integer greater than 0.

[0087] In one embodiment, the processor, when executing a computer program, also performs the following steps: Obtain the user's personality traits. Based on the personality traits, target traits, and initial state data, rearrange and score the M historical data records respectively to obtain the scores corresponding to the M historical data records. Based on the scores corresponding to M historical data records, select the N target historical data records with the highest scores from the M historical data records.

[0088] In one embodiment, the processor, when executing a computer program, also performs the following steps: Obtain the user's initial state data and personality traits, where the initial state data includes the user's basic emotional categories; By using a pre-set large model, the target state data is obtained by identifying the state based on initial state data, personality traits, target characteristics, and target historical data records. The target state data includes current contextual information defining the target features, personalized performance of basic emotion categories under the current contextual information after being influenced by personality traits, and historical descriptive information related to basic emotion categories in the target historical data records.

[0089] In one embodiment, the processor, when executing a computer program, also performs the following steps: Obtain the target score corresponding to the target's historical data records; If the target score is less than or equal to the preset score threshold, a new data record is generated based on the target status data and target features, and the new data record is stored in the historical database. If the target score is greater than the preset score threshold, the target historical data record will be updated based on the target status data and target features.

[0090] In one embodiment, the processor, when executing a computer program, also performs the following steps: Based on a preset attenuation factor, the memory strength value of each historical data record in the historical database is attenuated to obtain the attenuated memory strength value. Based on the decayed memory strength value, the historical data records to be cleared are identified from the historical database, and a forgetting operation is performed on the historical data records to be cleared.

[0091] In one embodiment, the processor, when executing a computer program, also performs the following steps: Retrieve new data records from the historical database and generate the user's current personality traits based on the new data records; The fusion weights are determined based on the total number of data records in the historical database. Based on the fusion weight, the personality traits before the update are fused with the current personality traits to obtain the updated personality traits.

[0092] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: Acquire users' multimodal data and fuse the multimodal data to obtain target features; Based on the target features, perform a similarity search in the historical database to obtain at least one historical data record of the target. State recognition is performed based on target features and historical data records to obtain the user's target state data.

[0093] In one embodiment, when the computer program is executed by a processor, it also performs the following steps: Based on the target features, a similarity search is performed in the historical database to obtain M historical data records, and the user's initial state data is generated based on the target features. Based on the initial state data, select N target historical data records from M historical data records, where N is an integer greater than 0.

[0094] In one embodiment, when the computer program is executed by a processor, it also performs the following steps: Obtain the user's personality traits. Based on the personality traits, target traits, and initial state data, rearrange and score the M historical data records respectively to obtain the scores corresponding to the M historical data records. Based on the scores corresponding to M historical data records, select the N target historical data records with the highest scores from the M historical data records.

[0095] In one embodiment, when the computer program is executed by a processor, it also performs the following steps: Obtain the user's initial state data and personality traits, where the initial state data includes the user's basic emotional categories; By using a pre-set large model, the target state data is obtained by identifying the state based on initial state data, personality traits, target characteristics, and target historical data records. The target state data includes current contextual information defining the target features, personalized performance of basic emotion categories under the current contextual information after being influenced by personality traits, and historical descriptive information related to basic emotion categories in the target historical data records.

[0096] In one embodiment, when the computer program is executed by a processor, it also performs the following steps: Obtain the target score corresponding to the target's historical data records; If the target score is less than or equal to the preset score threshold, a new data record is generated based on the target status data and target features, and the new data record is stored in the historical database. If the target score is greater than the preset score threshold, the target historical data record will be updated based on the target status data and target features.

[0097] In one embodiment, when the computer program is executed by a processor, it also performs the following steps: Based on a preset attenuation factor, the memory strength value of each historical data record in the historical database is attenuated to obtain the attenuated memory strength value. Based on the decayed memory strength value, the historical data records to be cleared are identified from the historical database, and a forgetting operation is performed on the historical data records to be cleared.

[0098] In one embodiment, when the computer program is executed by a processor, it also performs the following steps: Retrieve new data records from the historical database and generate the user's current personality traits based on the new data records; The fusion weights are determined based on the total number of data records in the historical database. Based on the fusion weight, the personality traits before the update are fused with the current personality traits to obtain the updated personality traits.

[0099] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0100] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0101] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A user status recognition method, characterized in that, The method includes: Acquire the user's multimodal data and fuse the multimodal data to obtain target features; Based on the target features, a similarity search is performed in the historical database to obtain at least one target historical data record; Based on the target features and the target historical data records, state identification is performed to obtain the user's target state data.

2. The method according to claim 1, characterized in that, The step of performing a similarity search in the historical database based on the target features to obtain at least one target historical data record includes: Based on the target features, a similarity search is performed in the historical database to obtain M historical data records, and based on the target features, the user's initial state data is generated; Based on the initial state data, N target historical data records are selected from the M historical data records, where N is an integer greater than 0.

3. The method according to claim 2, characterized in that, The step of selecting N target historical data records from the M historical data records based on the initial state data includes: The user's personality traits are obtained. Based on the personality traits, the target traits, and the initial state data, the scores of the M historical data records are rearranged and re-scored to obtain the scores corresponding to the M historical data records. Based on the scores corresponding to the M historical data records, the N target historical data records with the highest scores are selected from the M historical data records.

4. The method according to claim 1, characterized in that, The process of identifying the user's target state data based on the target features and the target's historical data records includes: Obtain the user's initial state data and personality traits, wherein the initial state data includes the user's basic emotion category; By using a pre-set large model, state recognition is performed based on the initial state data, the personality traits, the target traits, and the target historical data records to obtain the target state data; The target state data includes the current context information defined by the target feature, the personalized performance of the basic emotion category under the current context information after being affected by the personality feature, and the historical descriptive information related to the basic emotion category in the target historical data record.

5. The method according to claim 1, characterized in that, After obtaining the user's target state data, the process further includes: Obtain the target score corresponding to the target historical data record; If the target score is less than or equal to a preset score threshold, a new data record is generated based on the target state data and the target features, and the new data record is stored in the historical database. If the target score is greater than the preset score threshold, the target historical data record is updated based on the target state data and the target features.

6. The method according to claim 1, characterized in that, The historical database includes multiple historical data records, each of which includes at least historical features and a memory strength value representing the importance of those features. The updating steps of the historical database include: Based on a preset attenuation factor, the memory strength value of each historical data record in the historical database is attenuated to obtain the attenuated memory strength value. Based on the decayed memory strength value, historical data records to be cleared are determined from the historical database, and a forgetting operation is performed on the historical data records to be cleared.

7. The method according to claim 3 or 4, characterized in that, The steps for updating the personality traits include: Retrieve new data records from the historical database and generate the user's current personality traits based on the new data records; The fusion weight is determined based on the total number of data records in the historical database; Based on the fusion weight, the personality traits before the update are weighted and fused with the current personality traits to obtain the updated personality traits.

8. A user status identification device, characterized in that, The device includes: The fusion module is used to acquire the user's multimodal data and fuse the multimodal data to obtain target features; The retrieval module is used to perform a similarity search in the historical database based on the target features to obtain at least one target historical data record; The status recognition module is used to perform status recognition based on the target features and the target historical data records to obtain the user's target status data.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images