Self-adaptive psychological counseling large model dialogue method and system based on user state perception

Abstract

The invention discloses a self-adaptive psychological counseling large model dialogue method and system based on user state perception, and belongs to the cross field of artificial intelligence and psychological counseling, and the method comprises the following steps: S1, a multi-modal user data real-time collection step; s2, a multi-level user state sensing step; s3, a self-adaptive consultation strategy decision-making step; s4, a controlled dialogue generation and execution step; and S5, a dynamic optimization and report generation step. According to the method, non-language information such as voices and mood and facial expressions is incorporated into an emotion analysis system, so that the limitation of traditional text analysis in the aspect of capturing real emotions of customers is made up, and the psychological counseling model can more comprehensively and accurately understand the psychological states of the customers; therefore, a multi-dimensional basis is provided for subsequent generation of more targeted and effective responses, the professional and personalized level of psychological counseling services is improved, and the requirements of the public for high-quality psychological counseling are better met.

Description

TECHNICAL FIELD

[0001] The present application belongs to the cross field of artificial intelligence and psychological counseling, specifically a psychological counseling dialogue method and system based on a psychological counseling large model. BACKGROUND

[0002] The learning, work and life pressure in today's society is relatively large, resulting in the incidence of psychological diseases increasing year by year, and psychological health problems are getting more and more attention. Traditional psychological counseling methods mostly rely on face-to-face communication or telephone counseling, which has the problems of geographical restriction, high time cost, insufficient privacy protection, etc., and it is difficult to meet the growing psychological counseling needs of the public. With the rapid development of artificial intelligence technology, intelligent dialogue systems based on large models are widely used in various fields. Combining it with psychological counseling can effectively break through the limitations of traditional counseling mode and provide more convenient, efficient and personalized psychological counseling services for users. However, existing psychological counseling models often only focus on the analysis of text information, ignoring the rich emotional clues contained in non-verbal information such as voice tone and facial expressions, resulting in an incomplete and inaccurate understanding of the user's psychological state, which affects the counseling effect. SUMMARY

[0003] The present application provides a psychological counseling dialogue method and system based on a psychological counseling large model to solve the defects in the prior art.

[0004] The present application is implemented by the following technical solutions: A self-adaptive psychological counseling large model dialogue method based on user state perception, comprising the following steps: S1. Multi-modal user data real-time acquisition step: through a microphone array and a camera, synchronously acquire the user's voice stream and facial video stream; S2. Multi-level user state perception step: parallel processing of the collected multi-modal data, constructing a comprehensive and quantifiable user state vector, which includes: semantic understanding vector based on speech recognition and natural language processing, paralanguage emotion vector based on speech signal analysis, and facial micro-expression vector based on computer vision analysis; S3. Adaptive counseling strategy decision step: input the user state vector into the psychological counseling large model, the large model dynamically selects the optimal counseling strategy from the pre-defined counseling strategy library based on the state vector and the current dialogue context; the counseling strategy includes empathy support strategy, cognitive restructuring strategy, behavior activation strategy; S4. Controlled dialogue generation and execution step: based on the selected counseling strategy, generate and output the answer content that meets the strategy framework and professional specifications, while continuously collecting real-time feedback data of the user during the output process, if the user has no subsequent feedback, the dialogue ends, if the user continues to provide feedback, repeat the operation of S1-S4 until the user has no subsequent feedback and the dialogue ends; S5. Dynamic optimization and report generation step: during the entire dialogue process, dynamically update the user state vector according to the real-time feedback data of the user, and trigger the re-decision of the counseling strategy based on the updated state vector; After the dialogue ends, based on the multi-modal state vector sequence of the whole period, generate a psychological counseling report containing emotion change graph and personalized suggestions.

[0005] The user state vector construction process in the above-mentioned adaptive psychological counseling large model dialogue method based on user state perception includes the following steps: S2.1, extraction of semantic understanding vector: recognizing and transcribing the speech stream, and performing deep semantic analysis on the text to extract semantic features including emotional tendency words, core appeal words and cognitive distortion patterns, forming a semantic understanding vector; S2.2, extraction of paralinguistic emotion vector: extracting fundamental frequency, energy, speech rate and their time-varying features from the speech stream, and generating a paralinguistic emotion vector independent of the text content by matching with a pre-set acoustic emotion model; S2.3, extraction of facial micro-expression vector: performing continuous key point detection and motion unit analysis on the facial video stream to capture instantaneous and subtle expression changes, and generating a facial micro-expression vector; S2.4, multi-modal vector fusion: weighting and fusing the semantic understanding vector, paralinguistic emotion vector and facial micro-expression vector, when the analysis results of different modalities conflict, giving the paralinguistic emotion vector and facial micro-expression vector higher weights to generate the final comprehensive user state vector.

[0006] The counseling strategy decision in the above-mentioned adaptive psychological counseling large model dialogue method based on user state perception is a multi-factor decision-making process based on context and state. The psychological counseling large model maintains a counseling stage state machine, which includes the stages of "trust establishment", "problem exploration", "intervention implementation" and "consolidation summary". The decision-making process considers the current counseling stage, the dominant emotion and core appeal reflected by the user state vector, and the effectiveness feedback of historical strategies, to select the most suitable counseling strategy.

[0007] The controlled dialogue generation in S4 is achieved by injecting the selected counseling strategy as a generation constraint condition into the large model, and the constraint condition is defined in the form of a structured prompt template, which specifies the elements that must be included in the response, the language to be avoided, and the expected dialogue behavior; the large model creatively fills in the template to generate professional and natural response content.

[0008] The dynamic optimization in S5 specifically manifests as: After outputting the response, the instantaneous changes of the user's facial micro-expression vector and paralinguistic emotional vector are monitored in real time; If a significant increase in negative emotions is monitored, a strategy re-evaluation process is triggered immediately, and a soothing intervention sub-dialogue is prepared to start.

[0009] The adaptive psychological counseling large model dialogue method based on user state perception as described above, before S1, a personalized history archive loading step needs to be performed: the specific operation of the personalized history archive loading step is: before the dialogue starts, based on the user identifier, load the user's long-term state portrait formed by the history counseling records.

[0010] The adaptive psychological counseling large model dialogue method based on user state perception as described above, after S5 is completed, the following S6 also needs to be performed: S6. Archive updating step: update the psychological counseling report generated this time and the user state evolution pattern extracted to the user's long-term state portrait.

[0011] In S2 of the adaptive psychological counseling large model dialogue method based on user state perception as described above, the user state vector is specifically assessed, and high-risk keywords, extreme tones or painful expressions of self-harm and harm to others are identified; upon identifying high risk, the regular counseling process is interrupted, and the pre-set safety intervention protocol is triggered, which includes outputting standardized crisis response dialogues, providing emergency help channels and starting the manual alarm process.

[0012] An adaptive psychological counseling large model dialogue system based on user state perception, characterized by comprising a multi-modal data acquisition module, a user state perception engine, a counseling strategy decision maker, a controlled dialogue generator, a dialogue session management module, and an evaluation report generation module.

[0013] The adaptive psychological counseling large model dialogue system based on user state perception as described above, characterized by, The multi-modal data acquisition module is configured to acquire voice and visual data of the user. The user state perception engine is used for user state vector construction and risk assessment. The consultation strategy decision maker is used for a strategy decision process. The controlled dialogue generator is used for a controlled generation process. The dialogue session management module is used for maintaining a dialogue context, a consultation stage state machine and a user long-term state portrait. The evaluation report generation module is used for generating a visual psychological consultation report.

[0014] The advantage of the present application is that the present application makes up for the limitations of traditional text analysis in capturing the real emotions of customers by incorporating non-verbal information such as voice tone and facial expressions into the emotional analysis system, so that the psychological counseling model can more comprehensively and accurately understand the psychological state of the customer, thereby providing multi-dimensional basis for subsequent generation of more targeted and effective responses, improving the professionalism and individualization level of psychological counseling services, and better meeting the needs of the public for high-quality psychological counseling. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0016] Figure 1 is a flowchart of the present application; Figure 2 is a system structure diagram of the present application; Figure 3 is a dialogue interface schematic diagram of the present application. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical scheme and advantages of the embodiments of the present application more clear, the technical scheme in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0018] As Figure 1 shown, a self-adaptive psychological counseling large model dialogue method based on user state perception includes the following steps: S1. Multimodal user data real-time acquisition step: through the microphone array and the camera, the user's voice stream and facial video stream are synchronously acquired; S2. Multilevel user state perception step: the acquired multimodal data are processed in parallel to construct a comprehensive and quantifiable user state vector, which includes a semantic understanding vector obtained based on speech recognition and natural language processing, a paralinguistic emotion vector obtained based on speech signal analysis, and a facial micro-expression vector obtained based on computer vision analysis; S3. Adaptive counseling strategy decision step: the user state vector is input into a psychological counseling large model, the large model dynamically selects the optimal counseling strategy from a predefined counseling strategy library based on the state vector and the current dialogue context; the counseling strategy includes empathy support strategy, cognitive restructuring strategy, and behavior activation strategy; S4. Controlled dialogue generation and execution step: based on the selected counseling strategy, the response content conforming to the strategy framework and professional specification is generated and output, and in the output process, the real-time feedback data of the user are continuously acquired, if the user has no subsequent feedback, the dialogue ends, if the user continues to provide feedback, the operations of S1-S4 are repeated until the user has no subsequent feedback and the dialogue ends; S5. Dynamic optimization and report generation step: in the whole dialogue process (the dialogue interface is as shown in Figure 3 ), the user state vector is dynamically updated according to the real-time feedback data of the user, and the re-decision of the counseling strategy is triggered based on the updated state vector; after the dialogue ends, the psychological counseling report containing the emotion change graph and the personalized suggestion is generated based on the whole period multimodal state vector sequence.

[0019] Specifically, the process of constructing the user state vector in S2 of the embodiment includes the following steps: S2.1, extraction of semantic understanding vector: the voice stream is recognized and transcribed, and the text is deeply analyzed to extract semantic features including emotional tendency words, core appeal words and cognitive distortion patterns, to form a semantic understanding vector; S2.2, extraction of paralinguistic emotion vector: the fundamental frequency, energy, speech rate and their time-varying characteristics are extracted from the voice stream, and through matching with the pre-set acoustic emotion model, a paralinguistic emotion vector independent of the text content is generated; S2.3, extraction of facial micro-expression vector: continuous key point detection and motion unit analysis are performed on the facial video stream to capture instantaneous and subtle expression changes, and a facial micro-expression vector is generated; S2.4, multi-modal vector fusion: the semantic understanding vector, paralinguistic emotion vector and facial micro-expression vector are weighted and fused, when the analysis results of different modalities conflict, the paralinguistic emotion vector and facial micro-expression vector are given higher weights to generate the final comprehensive user state vector.

[0020] Specifically, the counseling strategy decision in S3 described in the present example is a multi-factor decision-making process based on context and state. The counseling model maintains a counseling stage state machine, which includes stages of "trust establishment", "problem exploration", "intervention implementation" and "consolidation summary". The decision-making process considers the current counseling stage, the dominant emotion and core appeal reflected by the user state vector, and the effectiveness feedback of historical strategies, to select the most suitable counseling strategy.

[0021] More specifically, the controlled dialogue generation in S4 described in the present example is achieved by injecting the selected counseling strategy as a generation constraint condition into the large model. The constraint condition is defined in the form of a structured prompt template, which specifies the elements that must be included in the response, the language that should be avoided, and the expected dialogue behavior. The large model creatively fills in the template under the constraints to generate professional and natural response content.

[0022] Further specifically, the dynamic optimization in S5 described in the present example specifically manifests as: After outputting the response, the instantaneous changes of the user's facial micro-expression vector and paralinguistic emotion vector are monitored in real time; If a significant increase in negative emotions is detected, a strategy re-evaluation process is triggered immediately, and a soothing intervention sub-dialogue is prepared to start.

[0023] Further, the present example S1 needs to perform a personalized historical archive loading step before: the specific operation of the personalized historical archive loading step is: before the dialogue starts, based on the user identifier, load the user's long-term state portrait formed by the historical counseling records.

[0024] Further, after the completion of S5 described in the present example, the following S6 is also needed: S6. Archive updating step: the psychological counseling report generated by this dialogue and the user state evolution pattern extracted are updated to the user's long-term state portrait.

[0025] Further, in S2 described in the present example, the user state vector is specifically evaluated for risk, and high-risk keywords, extreme tone or painful expressions of self-injury and injury tendency are identified; upon identifying high risk, the regular counseling process is interrupted, and a preset safety intervention protocol is triggered, including outputting standardized crisis response dialogues, providing emergency help channels and starting an artificial alarm process.

[0026] As shown in Figure 2 A user state perception-based adaptive psychological counseling large model dialogue system, comprising a multi-modal data acquisition module, a user state perception engine, a counseling strategy decision maker, a controlled dialogue generator, a dialogue session management module and an evaluation report generation module.

[0027] Specifically, the multi-modal data acquisition module described in the present embodiment is used to acquire voice and visual data of the user.

[0028] Specifically, the user state perception engine described in the present example is used for user state vector construction and risk assessment.

[0029] More specifically, the counseling strategy decision maker described in the present example is used for strategy decision process.

[0030] Further, the controlled dialogue generator described in the present example is used for controlled generation process.

[0031] Further, the dialogue session management module described in the present example is used to maintain dialogue context, counseling stage state machine and user long-term state portrait.

[0032] Further, the evaluation report generation module described in the present example is used to generate a visual psychological counseling report.

[0033] Compared with the traditional analysis method through text only, the judgment accuracy of the present application is 37.6% higher, especially in identifying hidden emotions and complex emotional states of customers. Through the fusion analysis of multi-modal information, the psychological counseling model can be closer to the comprehensive perception process of the counselor to the customer in the real counseling scene, reduce the misjudgment caused by the deviation of single text information interpretation; in practical application, the present application effectively improves the customer satisfaction with psychological counseling services, so that customers feel deeply understood and empathized in the dialogue, the average dialogue time is prolonged by 28.3%, the proportion of customers voluntarily disclosing deep psychological problems is increased by 42.1%, and the present application provides reliable technical support for the intelligent and precise development of psychological counseling.

[0034] It should be pointed out finally that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit the same; and although the present application has been described in detail with reference to the foregoing embodiments, it should be appreciated by those skilled in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features thereof can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An adaptive large-scale psychological counseling dialogue method based on user state awareness, characterized in that, Includes the following steps: S1. Real-time acquisition steps for multimodal user data: Simultaneously acquire the user's voice stream and facial video stream through a microphone array and camera; S2. Multi-level user state perception steps: Parallel processing of the collected multimodal data to construct a comprehensive and quantifiable user state vector, which includes: a semantic understanding vector based on speech recognition and natural language processing, a paralinguistic emotion vector based on speech signal analysis, and a facial micro-expression vector based on computer vision analysis. S3. Adaptive Counseling Strategy Decision-Making Steps: The user state vector is input into the psychological counseling big model. Based on the state vector and the current dialogue context, the big model dynamically selects the optimal counseling strategy from a predefined counseling strategy library. The counseling strategies include empathy support strategies, cognitive restructuring strategies, and behavioral activation strategies. S4. Controlled Dialogue Generation and Execution Steps: Based on the selected consultation strategy, generate and output response content that conforms to the strategy framework and professional standards. At the same time, during the output process, continuously collect real-time feedback data from the user. If the user has no further feedback, the dialogue ends. If the user continues to provide feedback, repeat the operations of S1-S4 until the user has no further feedback and the dialogue ends. S5. Dynamic Optimization and Report Generation Steps: Throughout the dialogue, the user's state vector is dynamically updated based on the user's real-time feedback data, and the consultation strategy is re-determined based on the updated state vector; after the dialogue ends, a psychological consultation report containing an emotion change map and personalized suggestions is generated based on the multimodal state vector sequence throughout the entire time period.

2. The adaptive psychological counseling large-scale model dialogue method based on user state awareness according to claim 1, characterized in that, The process of constructing the user state vector in S2 includes the following steps: S2.1, Extraction of semantic understanding vector: The speech stream is identified and transcribed, and the text is subjected to deep semantic analysis to extract semantic features including sentiment words, core appeal words and cognitive distortion patterns, forming a semantic understanding vector; S2.2, Extraction of paralinguistic emotion vector: Extract fundamental frequency, energy, speech rate and their time-varying features from the speech stream, and generate a paralinguistic emotion vector independent of the text content by matching it with a pre-set acoustic emotion model. S2.3, Extraction of facial micro-expression vectors: Continuous keypoint detection and motion unit analysis are performed on the facial video stream to capture instantaneous and subtle expression changes and generate facial micro-expression vectors; S2.4, Multimodal vector fusion: The semantic understanding vector, paralinguistic emotion vector, and facial micro-expression vector are weighted and fused. When the analysis results of different modalities conflict, the paralinguistic emotion vector and facial micro-expression vector are given higher weights to generate the final comprehensive user state vector.

3. The adaptive psychological counseling large-scale model dialogue method based on user state awareness according to claim 1, characterized in that, The consultation strategy decision in S3 is a multi-factor decision-making process based on context and state. The psychological counseling big model maintains a consultation stage state machine, which includes the stages of "trust building", "problem exploration", "intervention implementation" and "consolidation and summary". The decision-making process takes into account the current consultation stage, the dominant emotions and core demands reflected in the user's state vector, and the effectiveness feedback of historical strategies, in order to select the most suitable consultation strategy.

4. The adaptive psychological counseling large-scale model dialogue method based on user state awareness according to claim 1, characterized in that, The controlled dialogue generation in S4 is achieved by injecting the selected consultation strategy as a generation constraint into the large model. The constraint is defined in the form of a structured prompt template, which specifies the elements that the response must include in this round, the language to be avoided, and the expected dialogue behavior. The large model is creatively filled in under the constraints of the template to generate response content that is both professional and natural.

5. The adaptive psychological counseling large-scale model dialogue method based on user state awareness according to claim 1, characterized in that, The dynamic optimization in S5 is specifically manifested as follows: After the response is output, the instantaneous changes of the user's facial micro-expression vector and paralinguistic emotion vector are monitored in real time; If a significant increase in negative emotions is detected, the strategy reassessment process will be triggered immediately, and preparations will be made to initiate a reassuring intervention sub-dialogue.

6. The adaptive psychological counseling large-scale model dialogue method based on user state awareness according to claim 1, characterized in that, Before proceeding with S1, a personalized history file loading step is required: The specific operation of the personalized history file loading step is as follows: Before the dialogue begins, based on the user identifier, load the user's long-term status profile formed by their historical consultation records.

7. The adaptive psychological counseling large-scale model dialogue method based on user state awareness according to claim 1, characterized in that, After S5 is completed, the following S6 needs to be performed: S6. File update steps: Update the user long-term status profile with the psychological counseling report generated in this dialogue and the extracted user status evolution pattern.

8. The adaptive psychological counseling large-scale model dialogue method based on user state awareness according to claim 1, characterized in that, In S2, a risk assessment is specifically performed on the user's state vector to identify high-risk keywords, extreme tones, or painful expressions indicating self-harm or harm to others. Upon identification of a high risk, the regular consultation process is immediately interrupted, triggering a preset safety intervention protocol. The safety intervention protocol includes outputting standardized crisis response dialogue, providing emergency assistance channels, and initiating a manual alarm process.

9. A user-state-aware adaptive large-scale psychological counseling dialogue system for implementing the method of any one of claims 1-8, characterized in that, It includes a multimodal data acquisition module, a user state awareness engine, a consultation strategy decision-maker, a controlled dialogue generator, a dialogue session management module, and an evaluation report generation module.

10. The adaptive psychological counseling large-scale dialogue system based on user state awareness according to claim 1, characterized in that, The multimodal data acquisition module is used to collect the user's voice and visual data; The user state awareness engine is used for user state vector construction and risk assessment. The consultation strategy decision-making device is used in the strategy decision-making process; The controlled dialogue generator is used for the controlled generation process; The dialogue session management module is used to maintain the dialogue context, the consultation phase state machine, and the user's long-term state profile. The assessment report generation module is used to generate visualized psychological counseling reports.

Images