Dialogue reply method and device for elderly users
By acquiring the dialogue context and psychological information of elderly users, and using reinforcement learning and multi-objective reward models to generate candidate responses aligned with preference vectors, the problem that general dialogue systems cannot adapt to elderly users is solved, and personalized and safe dialogue responses are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- TSINGHUA UNIVERSITY
- Filing Date
- 2026-03-04
- Publication Date
- 2026-05-15
AI Technical Summary
Existing general dialogue systems cannot effectively adapt to the special needs of elderly users, including their psychological state, psychological needs, and psychological safety information, resulting in insufficient adaptability to dialogue scenarios.
By acquiring the dialogue context of elderly users, a preference vector is determined based on psychological state information, psychological needs information, and psychological safety information. This vector is then input into a pre-trained dialogue preference model to generate candidate responses aligned with the preference vector. Reinforcement learning and a multi-objective reward model are then used to optimize the dialogue preference model.
The generated candidate responses are better suited to the needs of elderly users, providing personalized psychological support, emotional guidance, and health education, reducing anxiety, and ensuring safety and reliability.
Smart Images
Figure CN122045375A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of natural language processing technology, and in particular to a dialogue response method and apparatus for elderly users. Background Technology
[0002] Currently, dialogue systems based on large-scale language models have demonstrated strong interactive capabilities in general domains, effectively understanding user intent and generating fluent responses, providing users with basic dialogue functionality support.
[0003] However, the relevant general dialogue systems are mainly aimed at ordinary adult users, focusing on indicators such as fluency and information content, without considering the special needs of elderly users. Therefore, they cannot be well adapted to dialogue scenarios for elderly users. Summary of the Invention
[0004] This application provides a dialogue response method and apparatus for elderly users, which can effectively solve the shortcomings of existing general dialogue systems that cannot be well adapted to dialogue scenarios for elderly users.
[0005] This application provides a dialogue response method for elderly users, including: Obtain the dialogue context of the target elderly user; Based on the psychological state information, psychological needs information, and psychological safety information of the target elderly users, determine the preference vector corresponding to the target elderly users; The dialogue context and the preference vector are input into a pre-trained dialogue preference model aligned with the preference vector, and the dialogue preference model outputs candidate responses aligned with the preference vector. The dialogue preference model is trained based on dialogue context samples of elderly users, candidate response samples corresponding to the dialogue context samples, and preference vector samples corresponding to the elderly users.
[0006] According to the dialogue response method for elderly users provided in this application, the step of determining the preference vector corresponding to the target elderly user based on the target elderly user's psychological state information, psychological needs information, and psychological safety information includes: Based on the aforementioned psychological state information, a psychological state vector is constructed; Based on the aforementioned psychological needs information, a psychological needs vector is constructed; Based on the aforementioned psychological security information, a psychological security vector is constructed; The preference vector is generated based on the psychological state vector, the psychological need vector, and the psychological safety vector.
[0007] According to the dialogue response method for elderly users provided in this application, the step of generating the preference vector based on the psychological state vector, the psychological needs vector, and the psychological safety vector includes: based on Generate the preference vector; in, Represents the preference vector. This represents the psychological state vector. This represents the psychological need vector. This represents the psychological safety vector. This represents the mapping matrix corresponding to the mental state vector. This represents the mapping matrix corresponding to the psychological need vector. The mapping matrix represents the psychological safety vector.
[0008] According to the dialogue response method for elderly users provided in this application, the dialogue preference model is trained based on dialogue context samples of elderly user samples, and candidate response samples corresponding to the dialogue context samples and preference vector samples corresponding to the elderly user samples, including: The dialogue context sample is input into the base dialogue preference model, and the candidate response sample is output through the base dialogue preference model. The dialogue preference model is trained based on the dialogue context samples, the candidate response samples, and the preference vector samples to obtain the dialogue preference model.
[0009] According to the dialogue response method for elderly users provided in this application, the step of training the base dialogue preference model based on the dialogue context sample, the candidate response sample, and the preference vector sample to obtain the dialogue preference model includes: The dialogue context sample, the candidate response sample, and the preference vector sample are input into the evaluation and discrimination model, and the evaluation and discrimination model outputs a multi-dimensional score of the candidate response sample on multiple target dimensions. Based on the dialogue context sample, the candidate response sample, and the multi-dimensional score, a multi-objective reward model is trained. The multi-objective reward model is used to output the reward value corresponding to the candidate response sample based on the dialogue context sample, the candidate response sample, and the multi-dimensional score. The reward value is used to characterize the matching degree of the candidate response sample with the psychological state preference, psychological need preference, and psychological safety preference of the elderly user sample indicated by the preference vector sample. The dialogue preference model is obtained by performing reinforcement learning training on the base dialogue model based on the reward value.
[0010] According to a dialogue response method for elderly users provided in this application, the multi-objective reward model is used to output the reward value corresponding to the candidate response sample based on the dialogue context sample, the candidate response sample, and the multi-dimensional score, including: based on Output the reward value corresponding to the candidate response sample; in, This represents the reward value. This represents the candidate response sample. This represents the dialogue context sample. Representing the multiple target dimensions The m-th target dimension in This represents the weight corresponding to the m-th target dimension. This represents the score on the m-th target dimension. This represents the safety penalty coefficient. The indicator function is defined as follows: if the candidate response sample y belongs to the aversion response type, the indicator function takes the value of 1; otherwise, the indicator function takes the value of 0.
[0011] According to the dialogue response method for elderly users provided in this application, the step of training the base dialogue model through reinforcement learning based on the reward value to obtain the dialogue preference model includes: The reward value is used as a reinforcement learning signal, and the dialogue context sample and the preference vector sample are input into the base dialogue model to generate new candidate response samples; The dialogue context sample, the new candidate response sample, and the preference vector sample are input into the evaluation and discrimination model, and the evaluation and discrimination model outputs a new multi-dimensional score corresponding to the new candidate response sample. The dialogue context sample, the new candidate response sample, and the new multi-dimensional score are input into the multi-objective reward model, and the multi-objective reward model outputs the target reward value corresponding to the new candidate response sample. With the goal of maximizing the target reward value, the model parameters of the base dialogue model are continuously updated through reinforcement learning algorithms until the dialogue preference model is trained.
[0012] According to the dialogue response method for elderly users provided in this application, the method further includes: Perform a security score on the dialogue context; If the security score is less than a preset score threshold, a prompt message is output, which prompts the user to respond to the dialogue context manually.
[0013] According to the dialogue response method for elderly users provided in this application, the method further includes: Under the condition that the preset conditions are met, the preference vector is redefined, and / or an iterative update of the dialogue preference model is performed; The preset conditions include at least one of the following: The score for the candidate response is less than a preset score threshold; The dialogue metric is below the metric threshold; An anomaly alarm has been detected.
[0014] This application also provides a dialogue response device for elderly users, including: The acquisition unit is used to acquire the dialogue context of the target elderly user. The determining unit is used to determine the preference vector corresponding to the target elderly user based on the target elderly user's psychological state information, psychological needs information, and psychological safety information; The processing unit is configured to input the dialogue context and the preference vector into a pre-trained dialogue preference model aligned with the preference vector, and output candidate responses aligned with the preference vector through the dialogue preference model; The dialogue preference model is trained based on dialogue context samples of elderly users, candidate response samples corresponding to the dialogue context samples, and preference vector samples corresponding to the elderly users.
[0015] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the dialogue response method for elderly users as described in any of the preceding claims.
[0016] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the dialogue response method for elderly users as described in any of the preceding claims.
[0017] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the dialogue response method for elderly users as described in any of the preceding claims.
[0018] The dialogue response method and apparatus for elderly users provided in this application obtain the dialogue context of the target elderly user; determine the corresponding preference vector based on the target elderly user's psychological state information, psychological needs information, and psychological safety information; and input the dialogue context and preference vector into a pre-trained dialogue preference model aligned with the preference vector, through which the dialogue preference model outputs candidate responses aligned with the preference vector. In this way, by jointly determining the preference vector based on psychological state information, psychological needs information, and psychological safety information, the special needs of elderly users are fully considered. Furthermore, by jointly inputting the target elderly user's dialogue context and preference vector into the pre-trained dialogue preference model aligned with the preference vector, the dialogue preference model can combine the target elderly user's psychological state information, psychological needs information, and psychological safety information to generate targeted candidate responses aligned with the preference vector. This effectively addresses the shortcomings of existing general-purpose dialogue systems that cannot adequately adapt to dialogue scenarios for elderly users. Attached Figure Description
[0019] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0020] Figure 1 This is a flowchart illustrating a dialogue response method for elderly users, provided as an embodiment of this application.
[0021] Figure 2 This is a flowchart illustrating a process for determining the preference vector corresponding to a target elderly user, as provided in an embodiment of this application.
[0022] Figure 3 This is a schematic diagram of a process for training a dialogue preference model, provided in an embodiment of this application.
[0023] Figure 4 This is a schematic diagram of a dialogue response device for elderly users provided in an embodiment of this application.
[0024] Figure 5 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0025] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0026] In the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone, where A and B can be singular or plural. In the textual description of this application, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.
[0027] The relevant general dialogue systems are mainly aimed at ordinary adult users, focusing on indicators such as fluency and information content, but do not consider the special needs of elderly users, such as information on psychological state, psychological needs, and psychological safety. Therefore, they cannot be well adapted to dialogue scenarios for elderly users.
[0028] Based on this, this application provides a dialogue response method for elderly users, which involves obtaining the dialogue context of the target elderly user; determining the preference vector corresponding to the target elderly user based on the target elderly user's psychological state information, psychological needs information, and psychological safety information; and inputting the dialogue context and preference vector into a pre-trained dialogue preference model aligned with the preference vector, and outputting candidate responses aligned with the preference vector through the dialogue preference model.
[0029] The dialogue preference model is trained based on dialogue context samples from elderly user samples, candidate response samples corresponding to the dialogue context samples, and preference vector samples corresponding to elderly user samples.
[0030] The above method, which uses psychological state information, psychological needs information, and psychological safety information to jointly determine the preference vector corresponding to the target elderly user, fully considers the special needs of elderly users. The dialogue context of the target elderly user and the preference vector are input into a pre-trained dialogue preference model aligned with the preference vector. This allows the dialogue preference model to combine the psychological state information, psychological needs information, and psychological safety information of the target elderly user to generate candidate responses that are aligned with the preference vector in a targeted manner. This can effectively solve the shortcomings of existing general dialogue systems that cannot adapt well to dialogue scenarios for elderly users.
[0031] For example, the dialogue response method for elderly users provided in this application embodiment can be applied to scenarios such as community psychological stations, family care assistants, online psychological counseling queuing systems, or health and wellness content push platforms.
[0032] In the community-based mental health service scenario, precise preference responses can provide personalized psychological support to elderly users and automatically alert social workers to high-risk conversations. In the family care assistant scenario, communication suggestions and emotional guidance scripts can be generated based on the elderly user's cognition and needs preferences, alleviating intergenerational communication conflicts. In the online psychological counseling queuing system scenario, preference collection and emotional reassurance can reduce anxiety during the waiting period before an expert connects. In the health and wellness content push platform scenario, content such as health science and cultural stories can be automatically filtered based on a preference system, and the expression style can be adjusted to match cognitive abilities.
[0033] It is understood that the entity executing the dialogue response method for elderly users provided in this application can be a computer, a server, or a specially designed dialogue response device for elderly users, such as an intelligent robot or other electronic device. It can also be a dialogue response device for elderly users installed in such electronic device. The dialogue response device for elderly users can be implemented by software, hardware, or a combination of both, and can be set according to actual needs.
[0034] The dialogue response method for elderly users provided in this application will be described in detail below through several specific embodiments. It is understood that these specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.
[0035] Figure 1 A flowchart illustrating a dialogue response method for elderly users provided in this application embodiment is shown below. For example, please refer to... Figure 1 As shown, this dialogue response method for elderly users may include: S101. Obtain the dialogue context of the target elderly user.
[0036] Dialogue context refers to a set of structured information that can fully represent the current interaction state, user intent, and historical dialogue process.
[0037] For example, in the embodiments of this application, the dialogue context of the target elderly user can be collected in real time through the front end; the dialogue context of the target elderly user can be retrieved back through the session storage system; or, the dialogue context of the target elderly user can be dynamically generated through real-time dialogue summaries, etc., which can be set according to actual needs.
[0038] S102. Based on the psychological state information, psychological needs information, and psychological safety information of the target elderly users, determine the preference vector corresponding to the target elderly users.
[0039] For example, psychological state information may include cognitive function, emotional state, and social adaptability, and can be specifically set according to actual needs.
[0040] Psychological needs information can include autonomy needs, competence needs, or belonging needs, and can be set according to actual needs.
[0041] Psychological safety information can include medical compliance, privacy protection, and value guidance, and can be set according to actual needs. Its purpose is to avoid diagnostic statements and leakage of sensitive information.
[0042] In this embodiment, a preference vector corresponding to the target elderly user is determined based on the target elderly user's psychological state information, psychological needs information, and psychological safety information. This fully considers the special needs of elderly users and maps the psychological state information, psychological needs information, and psychological safety information into a preference vector. This allows for the generation of candidate responses aligned with the preference vector in a targeted manner, based on the target elderly user's psychological state information, psychological needs information, and psychological safety information. This effectively solves the problem that existing general dialogue systems cannot adapt well to dialogue scenarios for elderly users.
[0043] For example, when generating preference vectors for target elderly users, additional layers can be added based on the regional culture or dialect characteristics of the target elderly users, such as religious beliefs and lifestyle habits, to achieve a more refined preference profile.
[0044] S103. Input the dialogue context and preference vector into a pre-trained dialogue preference model aligned with the preference vector, and output candidate responses aligned with the preference vector through the dialogue preference model.
[0045] The dialogue preference model is trained based on dialogue context samples from elderly user samples, candidate response samples corresponding to the dialogue context samples, and preference vector samples corresponding to elderly user samples.
[0046] For example, the dialogue preference model can be an optimization model based on reinforcement learning alignment, such as Proximal Policy Optimization (PPO) and Direct Preference Optimization (DPO), or a generative language model based on conditional decoding, such as the Generative Pre-trained Transformer (GPT) series of models, LLaMA, ChatGLM, etc. The specific model can be configured according to actual needs.
[0047] As can be seen, in this embodiment, the dialogue context of the target elderly user is obtained; based on the target elderly user's psychological state information, psychological needs information, and psychological safety information, the corresponding preference vector of the target elderly user is determined; and the dialogue context and preference vector are input into a pre-trained dialogue preference model aligned with the preference vector, through which the dialogue preference model outputs candidate responses aligned with the preference vector. In this way, the preference vector corresponding to the target elderly user is determined based on psychological state information, psychological needs information, and psychological safety information, fully considering the special needs of elderly users. Furthermore, by inputting the target elderly user's dialogue context and preference vector into the pre-trained dialogue preference model aligned with the preference vector, the dialogue preference model can combine the target elderly user's psychological state information, psychological needs information, and psychological safety information to generate targeted candidate responses aligned with the preference vector. This effectively solves the deficiency of existing general dialogue systems in adapting well to dialogue scenarios for elderly users.
[0048] Based on the above Figure 1 As illustrated in the embodiments shown, in this application embodiment, after obtaining the dialogue context, a security score can be performed on the dialogue context; if the security score is less than a preset score threshold, a prompt message is output, which prompts the user to respond to the dialogue context manually. In this way, dialogues with security risks are transferred to human responses, which can effectively intercept high-risk dialogues, prevent the dialogue preference model from generating inappropriate responses, and thus ensure the safety of elderly users and the reliability of services.
[0049] The preset scoring threshold can be set according to actual needs, and this application embodiment does not impose specific restrictions.
[0050] For example, when scoring the safety of a dialogue context, one can do so based on rule matching using a sensitive word database; or one can build a binary or multi-class classification model based on a lightweight discriminative model for text classification, such as Bidirectional Encoder Representations from Transformers (BERT) or RoBERTa, etc., to input the dialogue context and output whether it belongs to a high-risk topic or a specific risk category, thus scoring the safety of the dialogue context. The specific settings can be configured according to actual needs.
[0051] Based on the above Figure 1 In the illustrated embodiment, for example, the specific implementation of determining the preference vector corresponding to the target elderly user based on the target elderly user's psychological state information, psychological needs information, and psychological safety information in S102 above can be found in the following... Figure 2 The example shown.
[0052] Figure 2 This application provides a flowchart illustrating a method for determining the preference vector corresponding to a target elderly user. For example, the method may include: S201. Construct a psychological state vector based on psychological state information.
[0053] Taking psychological state information, including cognitive function, emotional state, and social adaptability, as an example, the psychological state vector can be denoted as: It can instruct the dialogue preference model to control speech rate, information density, and emotional intensity when responding to dialogue.
[0054] Taking psychological state information, including cognitive function, emotional state, and social adaptability, as an example, when constructing a psychological state vector based on psychological state information, the cognitive function assessment score can be mapped to a speech rate coefficient and an information density coefficient. The lower the score, the slower the speech rate and the lower the information density. The emotional state score can be mapped to an emotional intensity coefficient. The higher the negative emotion, the more gentle and soothing the response. The social adaptability score can be mapped to a social initiative coefficient. The three coefficients are concatenated to form a three-dimensional psychological state vector.
[0055] S202. Construct a psychological needs vector based on psychological needs information.
[0056] Taking psychological needs information, including autonomy needs, competence needs, or belonging needs, as an example, the psychological needs vector can be denoted as: Different content goals can be set according to specific application scenarios, such as providing choices, affirming the value of actions, and strengthening social connections.
[0057] Taking psychological needs information, including autonomy needs, competence needs, or belonging needs, as an example, when constructing a psychological needs vector based on psychological needs information, taking the companionship scenario as an example, the autonomy needs score can be mapped to a coefficient for providing choices, driving the dialogue preference model to add option-based expressions in the response; the competence needs score can be mapped to a coefficient for affirmation and encouragement, guiding the dialogue preference model to strengthen positive feedback on the value of user actions; and the belonging needs score can be mapped to a coefficient for social connection, prompting the dialogue preference model to actively mention family, friends, or community resources. The combination of these three coefficients constitutes a three-dimensional psychological needs vector.
[0058] S203. Construct a psychological security vector based on psychological security information.
[0059] Taking psychological safety information as an example, which may include medical compliance, privacy protection, and value guidance, the psychological safety vector can be denoted as follows: .
[0060] Taking psychological safety information, which can include medical compliance, privacy protection, and value guidance, as an example, when constructing a psychological safety vector based on psychological safety information, medical compliance requirements can be mapped to a diagnosis prohibition coefficient, controlling the dialogue preference model to output a disclaimer rather than a judgmental conclusion when the topic involves illness; privacy protection requirements can be mapped to a sensitive information blocking coefficient, triggering the detection and generalization of patterns such as name and address; and value guidance requirements can be mapped to a positive guidance intensity coefficient, proactively injecting a sense of hope and suggestions for help when negative remarks are detected. The three coefficients are spliced together to form a three-dimensional psychological safety vector.
[0061] S204. Generate a preference vector based on the psychological state vector, psychological need vector, and psychological safety vector.
[0062] For example, in an embodiment of this application, generating a preference vector based on a psychological state vector, a psychological need vector, and a psychological safety vector may include: based on Generate a preference vector; in, Represents a preference vector. Represents a vector of mental states. Represents the vector of psychological needs. Represents the psychological safety vector. This represents the mapping matrix corresponding to the mental state vector. This represents the mapping matrix corresponding to the psychological needs vector. This represents the mapping matrix corresponding to the psychological safety vector.
[0063] This approach, based on the psychological state, psychological needs, and psychological safety information of the target elderly users, determines the corresponding preference vector for them. It fully considers the special needs of elderly users, mapping the psychological state, psychological needs, and psychological safety information into preference vectors. This allows for the targeted generation of candidate responses aligned with the preference vectors, effectively addressing the shortcomings of existing general dialogue systems that cannot adequately adapt to dialogue scenarios for elderly users.
[0064] Based on any of the above embodiments, for example, in this application embodiment, the specific implementation of the dialogue preference model trained based on the dialogue context samples of elderly user samples, the candidate response samples corresponding to the dialogue context samples, and the preference vector samples corresponding to the elderly user samples can be found below. Figure 3 The example shown.
[0065] Figure 3 This application provides a flowchart illustrating a method for training a dialogue preference model. For example, the method may include: S301. Input the dialogue context sample into the base dialogue preference model, and output the candidate response sample through the base dialogue preference model.
[0066] For example, dialogue context samples can cover scenarios such as companionship chat, health education, emotional crisis, and family communication to ensure that the dialogue preference model generated during training can maintain alignment in multiple scenario tasks.
[0067] The base dialogue preference model can be understood as the foundational framework model for the dialogue preference model, used for training to generate the dialogue preference model. For example, the base dialogue preference model can be an optimization model based on reinforcement learning alignment, such as PPO or DPO, or a generative language model based on conditional decoding, such as the GPT series models, LLaMA, ChatGLM, etc., depending on actual needs.
[0068] S302. Based on dialogue context samples, candidate response samples, and preference vector samples, the base dialogue preference model is trained to obtain the dialogue preference model.
[0069] The method for determining the preference vector sample is similar to the method for determining the preference vector in S102 above. It can be based on the psychological state information, psychological needs information and psychological safety information of the elderly user sample to determine the preference vector sample corresponding to the elderly user sample. For details, please refer to the relevant description above on determining the preference vector corresponding to the target elderly user based on the psychological state information, psychological needs information and psychological safety information of the target elderly user. Here, the embodiments of this application will not be repeated.
[0070] For example, to improve the accuracy of preference vector samples, in this embodiment, expert calibration can be combined to continuously produce high-quality preference vector samples, providing a stable data source for subsequent multi-objective reward models. This maps psychological state vectors, psychological need vectors, and psychological safety vectors into preference vector samples, including triggering conditions, suggested tone, prohibited words, priorities, and weights, to drive subsequent evaluation and reward calculations.
[0071] For example, in the embodiments of this application, training the base dialogue preference model based on dialogue context samples, candidate response samples, and preference vector samples may include: The dialogue context samples, candidate response samples, and preference vector samples are input into the evaluation and discrimination model, which outputs multi-dimensional scores of the candidate response samples across multiple objective dimensions. Based on the dialogue context samples, candidate response samples, and multi-dimensional scores, a multi-objective reward model is trained. This model outputs reward values corresponding to candidate response samples based on the dialogue context samples, candidate response samples, and multi-dimensional scores. The reward values characterize the matching degree of the candidate response samples with the psychological state preferences, psychological need preferences, and psychological safety preferences of elderly user samples indicated by the preference vector samples. The base dialogue model is then trained using reinforcement learning based on the reward values to obtain the dialogue preference model.
[0072] For example, multiple target dimensions may include emotional fit, cognitive friendliness, social support, and safety compliance. The emotional fit dimension can be denoted as... The cognitive friendliness dimension can be denoted as: The social support dimension can be denoted as: The security compliance dimension can be denoted as: .
[0073] For example, the evaluation discriminant model can be implemented using few-sample hints or PEFT fine-tuning, making it a lightweight discriminant model. When the evaluation discriminant model outputs multi-dimensional scores of candidate response samples across multiple target dimensions, see the following formula: in, This indicates a multi-dimensional rating. This indicates the evaluation and discrimination model. This represents a sample of the dialogue context. Indicates candidate response samples, This represents a preference vector sample.
[0074] For example, in the embodiments of this application, the evaluation discrimination model can be a lightweight discrimination model or a multimodal discrimination model, which can introduce voice emotion or facial expression features to improve the judgment accuracy.
[0075] For example, a multi-objective reward model can use a language model of the same or reduced scale as the base dialogue preference model, take dialogue context samples, candidate response samples and multi-dimensional scores as input, and output the reward value corresponding to the candidate response sample.
[0076] For example, in an embodiment of this application, a multi-objective reward model is used to output the reward value corresponding to the candidate response sample based on the dialogue context sample, candidate response sample, and multi-dimensional scoring, including: Based on multi-objective reward model The reward value corresponding to the candidate response sample is output. By setting emotional relevance, cognitive friendliness, social support and safety compliance, and using a combination of weighted summation and penalty terms to construct the overall reward, comprehensive preference optimization is achieved, which can improve the accuracy of the reward value to a certain extent.
[0077] in, Indicates the reward value. Indicates candidate response samples, This represents a sample of the dialogue context. Representing multiple target dimensions The m-th target dimension in This represents the weight corresponding to the m-th target dimension. This represents the score on the m-th target dimension. This represents the safety penalty coefficient. The indicator function is defined as follows: if the candidate response sample y belongs to the aversion response type, the indicator function takes the value of 1; otherwise, the indicator function takes the value of 0.
[0078] For example, in the embodiments of this application, when training the base dialogue model with reinforcement learning based on the reward value to obtain the dialogue preference model, a reward method based on human feedback reinforcement learning (RLHF) can be used, or a conditional preference optimization (CPO) or self-attention reward can be used to adapt to multimodal inputs or more complex preference structures, thereby training the base dialogue model with reinforcement learning. The specific settings can be configured according to actual needs.
[0079] It is understood that, in the embodiments of this application, after outputting multi-dimensional scores of candidate response samples on multiple target dimensions through the evaluation and discrimination model, the response type corresponding to the candidate response sample can also be determined based on the multi-dimensional scores. For example, the response type may include a preferred response type, a suboptimal response type, and an averse response type. A preferred response type indicates that the response fully conforms to the preferred principle; a suboptimal response type indicates that the response meets the main requirements but can be optimized; and an averse response type indicates that the response may cause negative impacts.
[0080] For example, when determining the response type corresponding to candidate response samples based on multi-dimensional scoring, expert review samples can be introduced as a small number of annotations, i.e., real user feedback as incremental data, which can improve the accuracy of the response type corresponding to the candidate response samples. In this way, combined with the response type corresponding to the candidate response samples, the multi-objective reward model can be supervised and learned, and the multi-dimensional objectives of emotional fit, cognitive friendliness, social support and safety compliance can be uniformly incorporated into the reward modeling and reinforcement learning framework. At the same time, a safety constraint and real-time feedback mechanism that runs through the entire life cycle can be introduced, thereby achieving accurate adaptation and sustainable optimization for elderly mental health service scenarios.
[0081] For example, in an embodiment of this application, training a base dialogue model using reinforcement learning based on reward values to obtain a dialogue preference model may include: The reward value is used as a reinforcement learning signal, and the dialogue context samples and preference vector samples are input into the base dialogue model to generate new candidate response samples. The dialogue context samples, new candidate response samples, and preference vector samples are input into the evaluation and discriminant model, which outputs a new multi-dimensional score corresponding to the new candidate response sample. The dialogue context samples, new candidate response samples, and new multi-dimensional scores are input into the multi-objective reward model, which outputs the target reward value corresponding to the new candidate response sample. With the goal of maximizing the target reward value, the model parameters of the base dialogue model are iteratively updated through the reinforcement learning algorithm until a dialogue preference model is trained.
[0082] Taking the multi-objective reward model RLHF as an example, the specific implementation of iteratively updating the model parameters of the base dialogue model through reinforcement learning algorithm with the goal of maximizing the objective reward value can be seen in the following formula: in, This means that the model parameters of the base dialogue model are iteratively updated using a reinforcement learning algorithm to maximize the target reward value. This represents a sample of the dialogue context. This represents a new candidate response sample. Indicates the target reward value. KL divergence is used to measure the difference between two probability distributions. This represents the weighting coefficient of the KL divergence penalty term. This represents the updated model parameters of the base dialogue model. This represents the model parameters before the base dialogue model was updated.
[0083] The aforementioned approach updates the model using policy gradient or contrastive loss, reinforcing preferred responses and suppressing aversive responses. It achieves precise alignment through a multi-objective reward model and reinforcement learning, maintaining a stable and reliable service experience supported by security policies and real-time feedback. Based on the base dialogue model, by determining multi-level preference vector samples, automatically outputting candidate response samples, automatically evaluating the multi-dimensional scores and reward values of candidate response samples across multiple objective dimensions, and performing reinforcement learning training and iterative alignment on the base dialogue model, a closed loop of "preference modeling—quality assessment—reward feedback—alignment training—security review" is formed. This allows the trained dialogue preference model to combine the psychological state, psychological needs, and psychological safety information of the target elderly user to generate targeted candidate responses aligned with the preference vector. This effectively addresses the shortcomings of existing general-purpose dialogue systems that cannot adequately adapt to dialogue scenarios targeting elderly users.
[0084] For example, in the embodiments of this application, the trained dialogue preference model can be deployed collaboratively with edge devices and share preference signals through federated learning, thereby achieving cross-agency collaboration while protecting privacy.
[0085] Based on any of the above embodiments, in this application embodiment, under the condition of satisfying preset conditions, the preference vector is re-determined, and / or the iterative update of the dialogue preference model is performed. This can ensure that the dialogue preference model continues to adapt to the dynamic psychological needs and safety standards of elderly users in long-term service.
[0086] The preset conditions include at least one of the following: The score for the candidate response is less than the preset score threshold; The dialogue metric is below the metric threshold; An anomaly alarm has been detected.
[0087] The preset scoring threshold and indicator threshold can be set according to actual needs, and this application embodiment does not impose further restrictions.
[0088] For example, dialogue metrics can be described from four dimensions: interaction fluency, emotional experience, cognitive adaptation, and security compliance. For example, they can include quantifiable features such as user-initiated extension rate, clarification request rate, frequency of negative emotional expression, and security barrier trigger rate. The specific features can be set according to actual needs.
[0089] For example, anomaly alerts may include explicit user complaints / reports, a surge in the frequency of manual intervention for high-risk topics, incorrect model output format, or timeouts in inference service responses. These are usually accompanied by impaired service availability or significant security risks. Upon detecting anomaly alerts, the preference vector is redefined, and / or an iterative update of the dialogue preference model is performed.
[0090] The dialogue response device for elderly users provided in this application is described below. The dialogue response device for elderly users described below can be referred to in correspondence with the dialogue response method for elderly users described above.
[0091] Figure 4 This application provides a schematic diagram of a dialogue response device for elderly users, as illustrated in the embodiments of this application. For example, please refer to... Figure 4 As shown, the dialogue response device 40 for elderly users may include: Acquisition unit 401 is used to acquire the dialogue context of the target elderly user; The determining unit 402 is used to determine the preference vector corresponding to the target elderly user based on the psychological state information, psychological needs information and psychological safety information of the target elderly user; Processing unit 403 is used to input the dialogue context and the preference vector into a pre-trained dialogue preference model aligned with the preference vector, and output candidate responses aligned with the preference vector through the dialogue preference model; The dialogue preference model is trained based on dialogue context samples of elderly users, candidate response samples corresponding to the dialogue context samples, and preference vector samples corresponding to the elderly users.
[0092] For example, in this embodiment of the application, the determining unit 402 is used to determine the preference vector corresponding to the target elderly user based on the target elderly user's psychological state information, psychological needs information, and psychological safety information, including: Based on the aforementioned psychological state information, a psychological state vector is constructed; Based on the aforementioned psychological needs information, a psychological needs vector is constructed; Based on the aforementioned psychological security information, a psychological security vector is constructed; The preference vector is generated based on the psychological state vector, the psychological need vector, and the psychological safety vector.
[0093] For example, in an embodiment of this application, the determining unit 402 is configured to generate the preference vector based on the psychological state vector, the psychological need vector, and the psychological safety vector, including: based on Generate the preference vector; in, Represents the preference vector. This represents the psychological state vector. This represents the psychological need vector. This represents the psychological safety vector. This represents the mapping matrix corresponding to the mental state vector. This represents the mapping matrix corresponding to the psychological need vector. The mapping matrix represents the psychological safety vector.
[0094] For example, in an embodiment of this application, the dialogue preference model is trained based on dialogue context samples of elderly user samples, candidate response samples corresponding to the dialogue context samples, and preference vector samples corresponding to the elderly user samples, including: The dialogue context sample is input into the base dialogue preference model, and the candidate response sample is output through the base dialogue preference model. The dialogue preference model is trained based on the dialogue context samples, the candidate response samples, and the preference vector samples to obtain the dialogue preference model.
[0095] For example, in an embodiment of this application, training the base dialogue preference model based on the dialogue context sample, the candidate response sample, and the preference vector sample to obtain the dialogue preference model includes: The dialogue context sample, the candidate response sample, and the preference vector sample are input into the evaluation and discrimination model, and the evaluation and discrimination model outputs a multi-dimensional score of the candidate response sample on multiple target dimensions. Based on the dialogue context sample, the candidate response sample, and the multi-dimensional score, a multi-objective reward model is trained. The multi-objective reward model is used to output the reward value corresponding to the candidate response sample based on the dialogue context sample, the candidate response sample, and the multi-dimensional score. The reward value is used to characterize the matching degree of the candidate response sample with the psychological state preference, psychological need preference, and psychological safety preference of the elderly user sample indicated by the preference vector sample. The dialogue preference model is obtained by performing reinforcement learning training on the base dialogue model based on the reward value.
[0096] For example, in an embodiment of this application, the multi-objective reward model is used to output a reward value corresponding to the candidate response sample based on the dialogue context sample, the candidate response sample, and the multi-dimensional score, including: based on Output the reward value corresponding to the candidate response sample; in, This represents the reward value. This represents the candidate response sample. This represents the dialogue context sample. Representing the multiple target dimensions The m-th target dimension in This represents the weight corresponding to the m-th target dimension. This represents the score on the m-th target dimension. This represents the safety penalty coefficient. The indicator function is defined as follows: if the candidate response sample y belongs to the aversion response type, the indicator function takes the value of 1; otherwise, the indicator function takes the value of 0.
[0097] For example, in an embodiment of this application, the step of training the base dialogue model using reinforcement learning based on the reward value to obtain the dialogue preference model includes: The reward value is used as a reinforcement learning signal, and the dialogue context sample and the preference vector sample are input into the base dialogue model to generate new candidate response samples; The dialogue context sample, the new candidate response sample, and the preference vector sample are input into the evaluation and discrimination model, and the evaluation and discrimination model outputs a new multi-dimensional score corresponding to the new candidate response sample. The dialogue context sample, the new candidate response sample, and the new multi-dimensional score are input into the multi-objective reward model, and the multi-objective reward model outputs the target reward value corresponding to the new candidate response sample. With the goal of maximizing the target reward value, the model parameters of the base dialogue model are continuously updated through reinforcement learning algorithms until the dialogue preference model is trained.
[0098] For example, in an embodiment of this application, the dialogue response device 40 for elderly users further includes: A scoring unit is used to perform a security score on the dialogue context; The output unit is used to output a prompt message when the security score is less than a preset score threshold. The prompt message is used to suggest that the user respond to the dialogue context manually.
[0099] For example, in this embodiment of the application, the processing unit 403 is further configured to redetermine the preference vector and / or perform iterative updates to the dialogue preference model when preset conditions are met; The preset conditions include at least one of the following: The score for the candidate response is less than a preset score threshold; The dialogue metric is below the metric threshold; An anomaly alarm has been detected.
[0100] The dialogue response device 40 for elderly users provided in this application embodiment can execute the technical solution of the dialogue response method for elderly users in any of the above embodiments. Its implementation principle and beneficial effects are similar to those of the dialogue response method for elderly users. Please refer to the implementation principle and beneficial effects of the dialogue response method for elderly users. It will not be repeated here.
[0101] Figure 5 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of this application, such as... Figure 5 As shown, the electronic device may include: a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communications interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a dialogue response method for elderly users. This method includes: acquiring the dialogue context of the target elderly user; determining the preference vector corresponding to the target elderly user based on the target elderly user's psychological state information, psychological needs information, and psychological safety information; inputting the dialogue context and the preference vector into a pre-trained dialogue preference model aligned with the preference vector; and outputting candidate responses aligned with the preference vector through the dialogue preference model. The dialogue preference model is trained based on dialogue context samples of elderly user samples, candidate response samples corresponding to the dialogue context samples, and preference vector samples corresponding to the elderly user samples.
[0102] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0103] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the dialogue response method for elderly users provided by the above methods. The method includes: obtaining the dialogue context of a target elderly user; determining a preference vector corresponding to the target elderly user based on the target elderly user's psychological state information, psychological needs information, and psychological safety information; inputting the dialogue context and the preference vector into a pre-trained dialogue preference model aligned with the preference vector; and outputting candidate responses aligned with the preference vector through the dialogue preference model. The dialogue preference model is trained based on dialogue context samples of elderly user samples, candidate response samples corresponding to the dialogue context samples, and preference vector samples corresponding to the elderly user samples.
[0104] In another aspect, this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a dialogue response method for elderly users provided by the above methods. This method includes: acquiring a dialogue context of a target elderly user; determining a preference vector corresponding to the target elderly user based on the target elderly user's psychological state information, psychological needs information, and psychological safety information; inputting the dialogue context and the preference vector into a pre-trained dialogue preference model aligned with the preference vector; and outputting candidate responses aligned with the preference vector through the dialogue preference model; wherein the dialogue preference model is trained based on dialogue context samples of elderly user samples, and candidate response samples corresponding to the dialogue context samples and preference vector samples corresponding to the elderly user samples.
[0105] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0106] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A dialogue response method for elderly users, characterized in that, include: Obtain the dialogue context of the target elderly user; Based on the psychological state information, psychological needs information, and psychological safety information of the target elderly users, determine the preference vector corresponding to the target elderly users; The dialogue context and the preference vector are input into a pre-trained dialogue preference model aligned with the preference vector, and the dialogue preference model outputs candidate responses aligned with the preference vector. The dialogue preference model is trained based on dialogue context samples of elderly users, candidate response samples corresponding to the dialogue context samples, and preference vector samples corresponding to the elderly users.
2. The method according to claim 1, characterized in that, The step of determining the preference vector corresponding to the target elderly user based on the target elderly user's psychological state information, psychological needs information, and psychological safety information includes: Based on the aforementioned psychological state information, a psychological state vector is constructed; Based on the aforementioned psychological needs information, a psychological needs vector is constructed; Based on the aforementioned psychological security information, a psychological security vector is constructed; The preference vector is generated based on the psychological state vector, the psychological need vector, and the psychological safety vector.
3. The method according to claim 2, characterized in that, The step of generating the preference vector based on the psychological state vector, the psychological need vector, and the psychological safety vector includes: based on Generate the preference vector; in, Represents the preference vector. This represents the psychological state vector. This represents the psychological need vector. This represents the psychological safety vector. This represents the mapping matrix corresponding to the mental state vector. This represents the mapping matrix corresponding to the psychological need vector. The mapping matrix represents the psychological safety vector.
4. The method according to any one of claims 1-3, characterized in that, The dialogue preference model is trained based on dialogue context samples of elderly user samples, and is obtained by training candidate response samples corresponding to the dialogue context samples and preference vector samples corresponding to the elderly user samples, including: The dialogue context sample is input into the base dialogue preference model, and the candidate response sample is output through the base dialogue preference model. The dialogue preference model is trained based on the dialogue context samples, the candidate response samples, and the preference vector samples to obtain the dialogue preference model.
5. The method according to claim 4, characterized in that, The step of training the base dialogue preference model based on the dialogue context samples, the candidate response samples, and the preference vector samples to obtain the dialogue preference model includes: The dialogue context sample, the candidate response sample, and the preference vector sample are input into the evaluation and discrimination model, and the evaluation and discrimination model outputs a multi-dimensional score of the candidate response sample on multiple target dimensions. Based on the dialogue context sample, the candidate response sample, and the multi-dimensional score, a multi-objective reward model is trained. The multi-objective reward model is used to output the reward value corresponding to the candidate response sample based on the dialogue context sample, the candidate response sample, and the multi-dimensional score. The reward value is used to characterize the matching degree of the candidate response sample with the psychological state preference, psychological need preference, and psychological safety preference of the elderly user sample indicated by the preference vector sample. The dialogue preference model is obtained by performing reinforcement learning training on the base dialogue model based on the reward value.
6. The method according to claim 5, characterized in that, The multi-objective reward model is used to output the reward value corresponding to the candidate response sample based on the dialogue context sample, the candidate response sample, and the multi-dimensional score, including: based on Output the reward value corresponding to the candidate response sample; in, This represents the reward value. This represents the candidate response sample. This represents the dialogue context sample. Representing the multiple target dimensions The m-th target dimension in This represents the weight corresponding to the m-th target dimension. This represents the score on the m-th target dimension. This represents the safety penalty coefficient. The indicator function is defined as follows: if the candidate response sample y belongs to the aversion response type, the indicator function takes the value of 1; otherwise, the indicator function takes the value of 0.
7. The method according to claim 5, characterized in that, The step of training the base dialogue model using reinforcement learning based on the reward value to obtain the dialogue preference model includes: The reward value is used as a reinforcement learning signal, and the dialogue context sample and the preference vector sample are input into the base dialogue model to generate new candidate response samples; The dialogue context sample, the new candidate response sample, and the preference vector sample are input into the evaluation and discrimination model, and the evaluation and discrimination model outputs a new multi-dimensional score corresponding to the new candidate response sample. The dialogue context sample, the new candidate response sample, and the new multi-dimensional score are input into the multi-objective reward model, and the multi-objective reward model outputs the target reward value corresponding to the new candidate response sample. With the goal of maximizing the target reward value, the model parameters of the base dialogue model are continuously updated through reinforcement learning algorithms until the dialogue preference model is trained.
8. The method according to any one of claims 1-3, characterized in that, The method further includes: Perform a security score on the dialogue context; If the security score is less than a preset score threshold, a prompt message is output, which prompts the user to respond to the dialogue context manually.
9. The method according to any one of claims 1-3, characterized in that, The method further includes: Under the condition that the preset conditions are met, the preference vector is redefined, and / or an iterative update of the dialogue preference model is performed; The preset conditions include at least one of the following: The score for the candidate response is less than a preset score threshold; The dialogue metric is below the metric threshold; An anomaly alarm has been detected.
10. A dialogue response device for elderly users, characterized in that, include: The acquisition unit is used to acquire the dialogue context of the target elderly user. The determining unit is used to determine the preference vector corresponding to the target elderly user based on the target elderly user's psychological state information, psychological needs information, and psychological safety information; The processing unit is configured to input the dialogue context and the preference vector into a pre-trained dialogue preference model aligned with the preference vector, and output candidate responses aligned with the preference vector through the dialogue preference model; The dialogue preference model is trained based on dialogue context samples of elderly users, candidate response samples corresponding to the dialogue context samples, and preference vector samples corresponding to the elderly users.
11. The apparatus according to claim 10, characterized in that, The determining unit is configured to determine the preference vector corresponding to the target elderly user based on the target elderly user's psychological state information, psychological needs information, and psychological safety information, including: Based on the aforementioned psychological state information, a psychological state vector is constructed; Based on the aforementioned psychological needs information, a psychological needs vector is constructed; Based on the aforementioned psychological security information, a psychological security vector is constructed; The preference vector is generated based on the psychological state vector, the psychological need vector, and the psychological safety vector.
12. The apparatus according to claim 11, characterized in that, The determining unit is configured to generate the preference vector based on the psychological state vector, the psychological need vector, and the psychological safety vector, including: based on Generate the preference vector; in, Represents the preference vector. This represents the psychological state vector. This represents the psychological need vector. This represents the psychological safety vector. This represents the mapping matrix corresponding to the mental state vector. This represents the mapping matrix corresponding to the psychological need vector. The mapping matrix represents the psychological safety vector.