Agent-based psychological counseling strategy generation method, device, equipment and medium

Abstract

The present application discloses a method, device, equipment and medium for generating a psychological counseling strategy based on an intelligent agent, and relates to the field of computer technology, including: collecting audio and video signals of a target user who is undergoing psychological counseling; using computer vision technology and speech recognition technology to perform facial emotion recognition and speech recognition on the audio and video signals to obtain a multimodal signal; inputting the multimodal signal into a decision-making model created based on a decision tree and a multimodal large model to generate a psychological counseling strategy; obtaining a user portrait model of the target user generated based on the historical counseling record of the target user; adjusting the psychological counseling strategy according to the user portrait model and professional psychology knowledge to obtain a personalized counseling strategy for the target user. The present application provides a deep understanding of the user's emotional state and personal needs, meets the user's personalized psychological counseling strategy generation needs, and improves the accuracy of the psychological counseling strategy generation.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, equipment and medium for generating a psychological counseling strategy based on an intelligent agent. Background Art

[0002] With the advent of the digital age, artificial intelligence (AI) technology has developed rapidly and has been widely used in multiple application fields, such as medical care, education, finance, etc., especially in the field of psychological counseling, where it has gradually shown great potential and value, especially in imitating human communication, understanding and generating natural language.

[0003] At present, in order to solve the problems of insufficient resources, limited service scope, and susceptibility to time and location restrictions in traditional offline diagnosis and treatment, an artificial intelligence-based psychological counseling solution has been proposed, which can provide timely and efficient mental health support and mental health services by analyzing the user's non-verbal behavior and language content. However, although the current AI psychological counseling solutions have achieved certain results in improving service efficiency and accessibility, most of them rely on simple keyword triggers or limited decision trees, as well as simple text interactions, and only stay at the level of simple questions and answers and preliminary analysis. They lack a deep understanding and personalized feedback of the user's emotional state and personal needs, and ignore individual differences.

[0004] In addition, current solutions also have limitations in the construction and utilization of knowledge bases. They often rely on static, preset rule bases, lack effective knowledge accumulation mechanisms, lack the ability to learn dynamically, and find it difficult to accumulate and optimize experience to adapt to users' personalized needs and provide flexible communication experiences. This results in limited perception capabilities, inability to deeply understand users' emotions and needs, and difficulty in building trust with users. Summary of the invention

[0005] In view of this, the purpose of this application is to provide a method, device, equipment and medium for generating psychological counseling strategies based on an intelligent agent, which can deeply understand the user's emotional state and personal needs, meet the user's personalized psychological counseling strategy generation needs, and improve the accuracy of psychological counseling strategy generation, realize deeply personalized psychological counseling, and thus enhance user experience and satisfaction. The specific scheme is as follows:

[0006] In a first aspect, the present application discloses a method for generating a psychological counseling strategy based on an intelligent agent, wherein the intelligent agent is an intelligent agent based on a multimodal large model; wherein the method comprises:

[0007] Collect audio and video signals of target users who are undergoing psychological consultation;

[0008] Using computer vision technology and speech recognition technology to perform facial emotion recognition and speech recognition on the audio and video signals to obtain a multimodal signal;

[0009] Inputting the multimodal signal into a decision-making model created based on a decision tree and a multimodal model to generate a psychological counseling strategy for the target user; the decision-making model is composed of an input layer, a feature extraction layer, a fusion layer, a decision tree layer and an output layer;

[0010] Obtaining a user portrait model of the target user generated based on the historical consultation records of the target user;

[0011] The psychological counseling strategy is adjusted according to the user portrait model and professional psychology knowledge to obtain a personalized counseling strategy for the target user.

[0012] Optionally, the step of inputting the multimodal signal into a decision-making big model created based on a decision tree and a multimodal big model comprises:

[0013] Performing format conversion on the multimodal signal to obtain a converted modal signal;

[0014] The converted modal signal is input into a decision-making big model created based on a decision tree and a multi-modal big model.

[0015] Optionally, the agent-based psychological counseling strategy generation method further includes:

[0016] Use the pre-trained language model to initialize the multimodal large model to obtain the initialized model;

[0017] The historical consultation records of the target user are obtained, and the initialized model is fine-tuned using the historical consultation records to obtain the large decision model.

[0018] Optionally, the step of inputting the multimodal signal into a decision-making macro model created based on a decision tree and a multimodal macro model to generate a psychological counseling strategy for the target user includes:

[0019] Inputting the multimodal signal into a decision-making macro model created based on a decision tree and a multimodal macro model to generate a psychological counseling strategy and a corresponding psychological counseling service method for the target user;

[0020] Accordingly, after the psychological counseling strategy is adjusted according to the user portrait model and professional psychology knowledge to obtain a personalized counseling strategy for the target user, the method further includes:

[0021] The personalized counseling strategy is output according to the psychological counseling service mode.

[0022] Optionally, generating a psychological counseling strategy for the target user includes:

[0023] Receiving the multimodal signal through an input layer, and extracting features of the multimodal signal in the input layer through a feature extraction layer to obtain multimodal features;

[0024] Performing feature fusion on all the extracted multimodal features through a fusion layer to obtain fused modal features;

[0025] Classifying or regressing the fused modal features through a decision tree layer to generate a decision tree for the target user;

[0026] The decision tree is output through an output layer to generate a psychological counseling strategy for the target user.

[0027] Optionally, the step of fusing all the extracted multimodal features through a fusion layer to obtain fused modal features includes:

[0028] All the extracted multimodal features are spliced ​​through a fusion layer to obtain spliced ​​modal features, and each feature in the spliced ​​modal features is weighted according to a preset feature weight to obtain a fused modal feature.

[0029] Optionally, the agent-based psychological counseling strategy generation method further includes:

[0030] Acquire the historical consultation records of the target user from the temporary memory knowledge base;

[0031] Generate a user portrait model of the target user based on the historical consultation records and corresponding user feedback information;

[0032] Accordingly, after the psychological counseling strategy is adjusted according to the user portrait model and professional psychology knowledge to obtain a personalized counseling strategy for the target user, the method further includes:

[0033] The audio and video signals, the personalized consulting strategy and the feedback information generated by the target user for the personalized consulting strategy are saved in the temporary memory knowledge base, and the decision-making model is fine-tuned based on the information in the temporary memory knowledge base.

[0034] In a second aspect, the present application discloses a psychological counseling strategy generation device based on an intelligent agent, wherein the intelligent agent is an intelligent agent based on a multimodal large model; wherein the device comprises:

[0035] A signal acquisition module, used to collect audio and video signals of target users who are undergoing psychological counseling;

[0036] A recognition module, used to perform facial emotion recognition and speech recognition on the audio and video signals using computer vision technology and speech recognition technology to obtain a multimodal signal;

[0037] A strategy generation module, used for inputting the multimodal signal into a decision-making model created based on a decision tree and a multimodal big model, so as to generate a psychological counseling strategy for the target user; the decision-making big model is composed of an input layer, a feature extraction layer, a fusion layer, a decision tree layer and an output layer;

[0038] A portrait model acquisition module, used to acquire a user portrait model of the target user generated based on the historical consultation records of the target user;

[0039] The strategy adjustment module is used to adjust the psychological counseling strategy according to the user portrait model and professional psychology knowledge to obtain a personalized counseling strategy for the target user.

[0040] In a third aspect, the present application discloses an electronic device, comprising a processor and a memory; wherein, when the processor executes a computer program stored in the memory, the aforementioned agent-based psychological counseling strategy generation method is implemented.

[0041] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned agent-based psychological counseling strategy generation method is implemented.

[0042] It can be seen that the present application first collects the audio and video signals of the target user who is undergoing psychological counseling, and then uses computer vision technology and speech recognition technology to perform facial emotion recognition and speech recognition on the audio and video signals to obtain a multimodal signal, and inputs the multimodal signal into a decision-making model created based on a decision tree and a multimodal large model to generate a psychological counseling strategy for the target user; wherein the decision-making large model is composed of an input layer, a feature extraction layer, a fusion layer, a decision tree layer and an output layer; then, a user portrait model of the target user generated based on the historical consultation record of the target user is obtained, and then the psychological counseling strategy is adjusted according to the user portrait model and professional psychology knowledge to obtain a personalized counseling strategy for the target user. This application pre-creates an intelligent agent based on a multimodal large model, first obtains the audio and video signals of the target user who is undergoing psychological counseling by using computer vision technology and speech recognition technology to perform facial emotion recognition and speech recognition to obtain a multimodal signal, and then inputs the multimodal signal into the decision-making large model created based on the decision tree and the multimodal large model, thereby generating a psychological counseling strategy for the target user, and then adjusting the psychological counseling strategy according to the user portrait model of the target user and professional psychology knowledge, thereby obtaining a personalized counseling strategy for the target user. It can be seen that when generating the user's psychological counseling strategy, this application not only collects the user's audio information, but also collects video information. In this way, the user's emotional state and personal needs can be deeply understood, and the multimodal large model is used to generate the psychological counseling strategy. Further, the above-mentioned psychological counseling strategy is adjusted based on the user portrait model of the current user and professional psychology knowledge. In this way, the individual differences of users are taken into account, thereby meeting the user's personalized psychological counseling strategy generation needs, and improving the accuracy of the psychological counseling strategy generation, realizing deeply personalized psychological counseling, and thus enhancing user experience and satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0044] Figure 1 A flow chart of a method for generating a psychological counseling strategy based on an intelligent agent disclosed in this application;

[0045] Figure 2 Generate a flow chart for a specific psychological counseling strategy disclosed in this application;

[0046] Figure 3A flowchart of a specific agent-based psychological counseling strategy generation method disclosed in this application;

[0047] Figure 4 This is a schematic diagram of the structure of a psychological counseling strategy generation device based on an intelligent agent disclosed in this application;

[0048] Figure 5 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0050] The embodiment of the present application discloses a method for generating a psychological consultation strategy based on an intelligent agent, wherein the intelligent agent is an intelligent agent based on a multimodal large model, see Figure 1 As shown, the method includes:

[0051] Step S11: collecting audio and video signals of the target user who is undergoing psychological consultation.

[0052] It should be pointed out that the psychological counseling strategy generation scheme proposed in this application is applied to an AI agent based on multimodal large language models (MLLM). For details, see Figure 2As shown in the figure, the intelligent agent includes a perception layer, a planning layer, a memory layer and an action layer; wherein the perception layer has a deep perception capability, and is used to perceive and obtain audio and video signals of the target user who is undergoing psychological counseling in real time, which may specifically include video signals, voice signals and image signals, and analyze the obtained audio and video signals (such as user expression analysis, emotion analysis, tone analysis, psychological needs analysis, etc.) to accurately capture the user's emotional information and expression characteristics, and accurately obtain the user's deep-level needs; in the planning layer, a decision-making large model is pre-created based on the decision tree and the multimodal large model to simulate the psychological counselor's processing and decision-making of the information input by the perception layer, so as to formulate a personalized psychological counseling plan based on the user information; in the memory layer, a decision-making large model is pre-created based on the decision tree and the multimodal large model to simulate the psychological counselor's processing and decision-making of the information input by the perception layer, so as to formulate a personalized psychological counseling plan according to ... It includes two knowledge bases, namely the psychological knowledge base and the memory retrieval knowledge base. The memory retrieval knowledge base has a real-time memory function, which can store the consultation records of each user, and can input the saved data into the decision-making model to optimize and adjust the decision-making model. The two knowledge bases integrate professional psychology knowledge and users' historical consultation data, providing a more comprehensive and in-depth knowledge reserve, and can achieve in-depth mining and precise matching of users' historical consultation records, providing strong support for the generation of personalized consultation strategies, and providing comprehensive and accurate psychological consultation services; the action layer is used to adjust the psychological consultation strategy output by the decision-making model according to the current user's portrait model, so as to obtain a personalized psychological consultation strategy.

[0053] In this embodiment, the audio signal and video signal of the target user who is undergoing psychological counseling can be collected through the perception layer in the intelligent body and with the help of corresponding tools, such as cameras and microphones, wherein the specific collection points of the video signal are the points that the human eye is interested in. Specifically, face recognition technology can be used to collect video signals containing facial features for analysis of facial emotions, feelings, etc.

[0054] Step S12: Using computer vision technology and speech recognition technology to perform facial emotion recognition and speech recognition on the audio and video signals to obtain a multimodal signal.

[0055] In this embodiment, after collecting the audio and video signals of the target user who is undergoing psychological counseling, the perception layer can further use computer vision (CV) technology and speech recognition technology to perform facial emotion recognition (including expression recognition, emotion recognition, etc.) and speech recognition (including recognition of speech content, intonation recognition (i.e., speech emotion recognition), etc.) on the audio signals and video signals in the above audio and video signals, thereby obtaining a multimodal signal for psychological counseling.

[0056] Step S13: Input the multimodal signal into a decision model created based on a decision tree and a multimodal big model to generate a psychological counseling strategy for the target user; the decision model consists of an input layer, a feature extraction layer, a fusion layer, a decision tree layer and an output layer.

[0057] In this embodiment, the perception layer uses computer vision technology and speech recognition technology to perform facial emotion recognition and speech recognition on the audio and video signals to obtain multimodal signals, and then inputs the multimodal signals into the decision-making model created based on the decision tree and the multimodal model at the planning layer; when the decision-making model receives the multimodal signal, it passes through the input layer, feature extraction layer, fusion layer, decision tree layer and output layer in sequence, thereby generating a psychological counseling strategy for the current target user. It should be pointed out that in order to improve the accuracy and personalization of decision-making, this application combines a multimodal model on the basis of the decision tree.

[0058] Specifically, the creation process of the decision-making large model includes: initializing the multimodal large model using a pre-trained language model to obtain an initialized model; obtaining the historical consultation records of the target user, and fine-tuning the initialized model using the historical consultation records to obtain the decision-making large model. In this embodiment, a pre-trained language model (PLM), such as BERT, RoBERTa, GPT-3, ALBERT, etc., can be used to initialize the multimodal large model to obtain an initialized model, then obtain the historical consultation records of the current target user, and then use the above historical consultation records to fine-tune the initialized model to obtain a decision-making large model for generating psychological consultation strategies.

[0059] In a specific implementation, the generating of the psychological counseling strategy for the target user may specifically include: receiving the multimodal signal through an input layer, and performing feature extraction on the multimodal signal in the input layer through a feature extraction layer to obtain multimodal features; performing feature fusion on all the extracted multimodal features through a fusion layer to obtain fused modal features; classifying or regressing the fused modal features through a decision tree layer to generate a decision tree for the target user; and outputting the decision tree through an output layer to generate a psychological counseling strategy for the target user. In this embodiment, the decision-making model is composed of an input layer, a feature extraction layer, a fusion layer, a decision tree layer and an output layer. When the language generation ability of the decision-making model is used to generate a psychological counseling strategy, the multimodal signal output by the perception layer is first received through the input layer, and then input into the feature extraction layer to extract features of multimodal signals of different types (such as image type, text type and time series type, etc.) output by the input layer, thereby obtaining multimodal features. Then, all the extracted multimodal features are subjected to feature fusion through the fusion layer to obtain corresponding fused modal features. Further, the fused modal features are classified or regressed through the decision tree layer to generate a decision tree for the current target user. Specifically, suitable multimodal features can be selected according to the importance of the features, and then the selected multimodal features can be node-split using information gain or Gini index and other standards, and then the decision tree obtained after the split is pre-pruned and post-pruned to prevent overfitting. Finally, the decision tree is output through the output layer to generate a psychological counseling strategy for the current target user, and then the generated psychological counseling strategy is transmitted to the action layer.

[0060] Specifically, the feature fusion of all the multimodal features extracted through the fusion layer to obtain the fused modal features may include: feature splicing of all the multimodal features extracted through the fusion layer to obtain the spliced ​​modal features, and weighting each feature in the spliced ​​modal features according to preset feature weights to obtain the fused modal features. It should be pointed out that the present application sets different weights for different multimodal features in advance according to the importance of the features. When the fusion layer receives the multimodal features, it first performs feature splicing on all the extracted multimodal features to obtain the corresponding spliced ​​modal features, and then performs weighted averaging on each feature in the spliced ​​modal features according to the preset feature weights to obtain the fused modal features.

[0061] Step S14: Obtain a user portrait model of the target user generated based on the historical consultation records of the target user.

[0062] In this embodiment, after generating the psychological counseling strategy for the target user, a user portrait model for the current target user generated in advance based on the historical counseling records of the current target user is obtained. The user portrait model refers to a user feature model formed by analyzing and describing multi-dimensional data such as the user's personal information, interests, hobbies, and behavioral habits.

[0063] In this embodiment, see Figure 2 As shown, it may also include: obtaining the historical consultation records of the target user from a temporary memory knowledge base; generating a user portrait model of the target user based on the historical consultation records and the corresponding user feedback information. It should be pointed out that the present application includes two knowledge bases in the memory layer of the intelligent body, namely a psychological knowledge base and a memory retrieval knowledge base (i.e., a temporary memory knowledge base); wherein the psychological knowledge base is used to provide comprehensive professional psychological knowledge, and the memory retrieval knowledge base (i.e., a temporary memory knowledge base) is used to store information such as the user's historical consultation records. Through the above-mentioned types of knowledge bases, the user's specific needs, preferences, and historical feedback can be retained and memorized.

[0064] Specifically, the psychological knowledge base may include psychological multimedia and textual professional knowledge bases in the era of big data, such as specific psychological treatment methods, relevant psychological education materials, audio and video data related to emotional topics. This knowledge base has the characteristics of large data volume, multiple data types, and long storage time; the memory retrieval knowledge base (i.e., temporary memory knowledge base) is a short-term memory knowledge base, which may specifically include patient consultation records, patient information, treatment records and other short-term recorded data. This knowledge base facilitates understanding of patient information recorded data. Through the data in the above two types of knowledge bases, the decision-making model can generate professional and personalized psychological counseling strategies.

[0065] Correspondingly, after adjusting the psychological counseling strategy according to the user portrait model and professional psychology knowledge to obtain a personalized counseling strategy for the target user, it may also include: saving the audio and video signals, the personalized counseling strategy and the feedback information generated by the target user for the personalized counseling strategy into the temporary memory knowledge base, and fine-tuning the decision-making model based on the information in the temporary memory knowledge base. In this embodiment, after obtaining the personalized consultation strategy for the single target user, the current psychological consultation process can also be recorded, and the decision-making model can be optimized and adjusted according to the user's feedback. Specifically, the audio and video signals generated in the current consultation process, the personalized consultation strategy and the feedback information generated by the target user for the personalized consultation strategy can be saved in a temporary memory knowledge base, and then the decision-making model can be optimized and adjusted based on all historical information in the temporary memory knowledge base. In this way, the memory retrieval knowledge base can have a real-time memory function, and is continuously updated and expanded according to the interaction history with the user, thereby continuously learning and accumulating the user's personal interaction history. In addition, through the user's consultation history and feedback, a more personalized user portrait model can be formed, thereby providing a reference for the recommendation and precise matching of personalized psychological consultation strategies, providing strong support for the design of consultation strategies, and ensuring the continuous improvement of consultation services.

[0066] Step S15: adjusting the psychological counseling strategy according to the user portrait model and professional psychology knowledge to obtain a personalized counseling strategy for the target user.

[0067] In this embodiment, after obtaining the user portrait model of the target user generated based on the historical consultation records of the target user, the user's personal information, interests, hobbies, and behavioral habits are understood according to the above user portrait model, and the above psychological consultation strategy is adjusted accordingly in combination with the professional psychology knowledge in the psychological knowledge base, thereby generating a personalized consultation strategy for the current target user.

[0068] It can be seen that the embodiment of the present application first collects the audio and video signals of the target user who is undergoing psychological counseling, and then uses computer vision technology and speech recognition technology to perform facial emotion recognition and speech recognition on the audio and video signals to obtain a multimodal signal, and inputs the multimodal signal into a decision-making model created based on a decision tree and a multimodal large model to generate a psychological counseling strategy for the target user; wherein the decision-making large model is composed of an input layer, a feature extraction layer, a fusion layer, a decision tree layer and an output layer; then, a user portrait model of the target user generated based on the historical consultation record of the target user is obtained, and then the psychological counseling strategy is adjusted according to the user portrait model and professional psychology knowledge to obtain a personalized counseling strategy for the target user. The embodiment of the present application pre-creates an intelligent agent based on a multimodal large model, first obtains the audio and video signals of the target user who is undergoing psychological counseling by using computer vision technology and speech recognition technology to perform facial emotion recognition and speech recognition to obtain a multimodal signal, and then inputs the multimodal signal into the decision large model created based on the decision tree and the multimodal large model, thereby generating a psychological counseling strategy for the target user, and then adjusting the psychological counseling strategy according to the user portrait model of the target user and professional psychology knowledge, thereby obtaining a personalized counseling strategy for the target user. It can be seen that when the embodiment of the present application generates the psychological counseling strategy of the user, not only the audio information of the user is collected, but also the video information is collected. In this way, the emotional state and personal needs of the user can be deeply understood, and the multimodal large model is used to generate the psychological counseling strategy. Further, the above-mentioned psychological counseling strategy is adjusted based on the user portrait model of the current user and professional psychology knowledge. In this way, the individual differences of users are taken into account, thereby meeting the personalized psychological counseling strategy generation needs of users, and improving the accuracy of the psychological counseling strategy generation, realizing deeply personalized psychological counseling, and thus enhancing user experience and satisfaction.

[0069] The embodiment of the present application discloses a specific method for generating a psychological counseling strategy based on an intelligent agent, wherein the intelligent agent is an intelligent agent based on a multimodal large model, see Figure 3 As shown, the method includes:

[0070] Step S21: collecting audio and video signals of a target user who is undergoing psychological consultation.

[0071] Step S22: Using computer vision technology and speech recognition technology to perform facial emotion recognition and speech recognition on the audio and video signals to obtain a multimodal signal.

[0072] Step S23: formatting and converting the multimodal signal to obtain a converted modal signal.

[0073] In this embodiment, after performing facial emotion recognition and speech recognition on the audio and video signals to obtain a multimodal signal, in order to facilitate the subsequent processing of the decision-making large model, the multimodal signal can be formatted and converted according to the input requirements of the decision-making large model to obtain a converted modal signal.

[0074] Step S24: Input the converted modal signal into a decision model created based on a decision tree and a multimodal big model to generate a psychological counseling strategy and a corresponding psychological counseling service method for the target user; the decision model is composed of an input layer, a feature extraction layer, a fusion layer, a decision tree layer and an output layer.

[0075] In this embodiment, after obtaining the converted modal signal, it is input into a decision model created based on a decision tree and a multimodal model, thereby generating a psychological counseling strategy and a corresponding psychological counseling service method for the current target user; wherein the psychological counseling service method includes but is not limited to text, voice and other methods.

[0076] Step S25: Obtain a user portrait model of the target user generated based on the historical consultation records of the target user.

[0077] Step S26: adjusting the psychological counseling strategy according to the user portrait model and professional psychology knowledge to obtain a personalized counseling strategy for the target user, and outputting the personalized counseling strategy according to the psychological counseling service mode.

[0078] In this embodiment, the psychological counseling strategy generated by the above-mentioned decision-making model can be adjusted according to the user portrait model and based on professional psychology knowledge, and the personalized counseling strategy finally generated can be output according to the above-mentioned psychological counseling service mode. For example, the personalized counseling strategy finally generated can be outputted by voice in a natural, smooth and friendly way through the psychological robot, thereby realizing interactive psychological counseling services. In addition, the psychological robot can also further communicate based on the user's feedback information, such as raising new questions based on the user's feedback, or giving new personalized counseling strategies, thereby realizing personalized psychological counseling services and improving the diversity of interaction and user experience.

[0079] In another specific implementation, the psychological counseling strategy can be adjusted based on the user portrait model and professional psychology knowledge in combination with the user recommendation model and the text generation model, so as to obtain a text containing personalized counseling strategies for target users.

[0080] For more specific processing procedures of the above steps S21, S22, and S25, reference may be made to the corresponding contents disclosed in the aforementioned embodiments, which will not be described in detail here.

[0081] It can be seen that the embodiment of the present application first formats and converts the multimodal signal, and then inputs the converted modal signal into the decision-making model created based on the decision tree and the multimodal large model to generate a psychological counseling strategy for the target user and the corresponding psychological counseling service mode, and then adjusts the psychological counseling strategy according to the user portrait model and professional psychology knowledge to obtain a personalized counseling strategy for the target user, and outputs the personalized counseling strategy according to the psychological counseling service mode. The present application can provide accurate information for subsequent model processing by formatting the multimodal signal, and can effectively solve the problems of limited perception ability, inability to deeply understand user emotions and needs, lack of personalized counseling solution design and building trust, lack of intelligent memory and single form of expression in traditional psychological counseling through the continuous learning and optimization function of the intelligent body based on the multimodal large model. In addition, by accurately obtaining the user's emotional information, it can provide more timely, accurate and personalized psychological counseling services, and can improve the intelligence level and user experience of psychological counseling through different psychological counseling service modes. At the same time, through the dynamic learning mechanism of the intelligent body, it can continuously optimize the counseling strategy, so as to continuously adapt to the changes in user needs.

[0082] Correspondingly, the embodiment of the present application also discloses a psychological counseling strategy generation device based on an intelligent agent, wherein the intelligent agent is an intelligent agent based on a multimodal large model, see Figure 4 As shown, the device comprises:

[0083] The signal acquisition module 11 is used to collect audio and video signals of the target user who is undergoing psychological consultation;

[0084] A recognition module 12, used to perform facial emotion recognition and speech recognition on the audio and video signals using computer vision technology and speech recognition technology to obtain a multimodal signal;

[0085] A strategy generation module 13 is used to input the multimodal signal into a decision-making model created based on a decision tree and a multimodal big model to generate a psychological counseling strategy for the target user; the decision-making big model is composed of an input layer, a feature extraction layer, a fusion layer, a decision tree layer and an output layer;

[0086] A portrait model acquisition module 14, used to acquire a user portrait model of the target user generated based on the historical consultation records of the target user;

[0087] The strategy adjustment module 15 is used to adjust the psychological counseling strategy according to the user portrait model and professional psychology knowledge to obtain a personalized counseling strategy for the target user.

[0088] Among them, the specific working processes of the above-mentioned modules can refer to the corresponding contents disclosed in the aforementioned embodiments, which will not be repeated here.

[0089] It can be seen that in the embodiment of the present application, the audio and video signals of the target user who is undergoing psychological counseling are first collected, and then the audio and video signals are subjected to facial emotion recognition and speech recognition using computer vision technology and speech recognition technology to obtain a multimodal signal, and the multimodal signal is input into a decision-making model created based on a decision tree and a multimodal large model to generate a psychological counseling strategy for the target user; wherein the decision-making large model is composed of an input layer, a feature extraction layer, a fusion layer, a decision tree layer, and an output layer; then, a user portrait model of the target user generated based on the historical counseling record of the target user is obtained, and then the psychological counseling strategy is adjusted according to the user portrait model and professional psychology knowledge to obtain a personalized counseling strategy for the target user. The embodiment of the present application pre-creates an intelligent agent based on a multimodal large model, first obtains the audio and video signals of the target user who is undergoing psychological counseling by using computer vision technology and speech recognition technology to perform facial emotion recognition and speech recognition to obtain a multimodal signal, and then inputs the multimodal signal into the decision large model created based on the decision tree and the multimodal large model, thereby generating a psychological counseling strategy for the target user, and then adjusting the psychological counseling strategy according to the user portrait model of the target user and professional psychology knowledge, thereby obtaining a personalized counseling strategy for the target user. It can be seen that when the embodiment of the present application generates the psychological counseling strategy of the user, not only the audio information of the user is collected, but also the video information is collected. In this way, the emotional state and personal needs of the user can be deeply understood, and the multimodal large model is used to generate the psychological counseling strategy. Further, the above-mentioned psychological counseling strategy is adjusted based on the user portrait model of the current user and professional psychology knowledge. In this way, the individual differences of users are taken into account, thereby meeting the personalized psychological counseling strategy generation needs of users, and improving the accuracy of the psychological counseling strategy generation, realizing deeply personalized psychological counseling, and thus enhancing user experience and satisfaction.

[0090] In some specific embodiments, the strategy generation module 13 may specifically include:

[0091] A format conversion unit, used for performing format conversion on the multimodal signal to obtain a converted modal signal;

[0092] A signal input unit is used to input the converted modal signal into a decision model created based on a decision tree and a multimodal large model.

[0093] In some specific embodiments, the agent-based psychological counseling strategy generation device may further include:

[0094] An initialization unit, used to initialize the multimodal large model using a pre-trained language model to obtain an initialized model;

[0095] A first record acquisition unit, used to acquire the historical consultation records of the target user;

[0096] The first model fine-tuning unit is used to fine-tune the initialized model using the historical consultation records to obtain the large decision model.

[0097] In some specific embodiments, the strategy generation module 13 may specifically include:

[0098] A strategy generating unit, used for inputting the multimodal signal into a decision-making macro model created based on a decision tree and a multimodal macro model, so as to generate a psychological counseling strategy and a corresponding psychological counseling service mode for the target user;

[0099] Correspondingly, after the strategy adjustment module 15, the following may also be included:

[0100] The first output unit is used to output the personalized consulting strategy according to the psychological consulting service mode.

[0101] In some specific embodiments, the strategy generation module 13 may specifically include:

[0102] A signal receiving unit, configured to receive the multimodal signal through an input layer;

[0103] A feature extraction unit, configured to extract features of the multimodal signal in the input layer through a feature extraction layer to obtain multimodal features;

[0104] A feature fusion unit, used for fusing all the extracted multimodal features through a fusion layer to obtain fused modal features;

[0105] A decision tree generating unit, configured to classify or regress the fused modal features through a decision tree layer to generate a decision tree for the target user;

[0106] The second output unit is used to output the decision tree through an output layer to generate a psychological counseling strategy for the target user.

[0107] In some specific embodiments, the feature fusion unit may specifically include:

[0108] A feature splicing unit, used for performing feature splicing on all the multimodal features extracted through a fusion layer to obtain a spliced ​​modal feature;

[0109] The weighting unit is used to weight each feature in the spliced ​​modal features according to a preset feature weight to obtain a fused modal feature.

[0110] In some specific embodiments, the agent-based psychological counseling strategy generation device may further include:

[0111] A second record acquisition unit, used to acquire the historical consultation records of the target user from a temporary memory knowledge base;

[0112] A portrait model generating unit, configured to generate a user portrait model of the target user based on the historical consultation records and corresponding user feedback information;

[0113] Correspondingly, after the strategy adjustment module 15, the following may also be included:

[0114] An information storage unit, used to store the audio and video signals, the personalized consulting strategy, and the feedback information generated by the target user for the personalized consulting strategy in the temporary memory knowledge base;

[0115] The second model fine-tuning unit is used to fine-tune the decision-making large model based on the information in the temporary memory knowledge base.

[0116] Furthermore, the present application also discloses an electronic device. Figure 5 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.

[0117] Figure 5 The present invention provides a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the agent-based psychological counseling strategy generation method disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0118] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0119] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0120] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, which can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program that can be used to complete the agent-based psychological counseling strategy generation method executed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include computer programs that can be used to complete other specific tasks.

[0121] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned agent-based psychological counseling strategy generation method disclosed above is implemented. The specific steps of the method can be referred to the corresponding contents disclosed in the aforementioned embodiments, and will not be repeated here.

[0122] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0123] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0124] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0125] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0126] The above is a detailed introduction to the agent-based psychological counseling strategy generation method, device, equipment and medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for generating psychological counseling strategies based on an intelligent agent, characterized in that: The intelligent agent is an intelligent agent based on a multimodal large model; wherein the method comprises: Collect audio and video signals of target users who are undergoing psychological consultation; Using computer vision technology and speech recognition technology to perform facial emotion recognition and speech recognition on the audio and video signals to obtain a multimodal signal; Inputting the multimodal signal into a decision-making model created based on a decision tree and a multimodal model to generate a psychological counseling strategy for the target user; the decision-making model is composed of an input layer, a feature extraction layer, a fusion layer, a decision tree layer and an output layer; Obtaining a user portrait model of the target user generated based on the historical consultation records of the target user; The psychological counseling strategy is adjusted according to the user portrait model and professional psychology knowledge to obtain a personalized counseling strategy for the target user; Wherein, the generating of the psychological counseling strategy for the target user comprises: receiving the multimodal signal through an input layer, and extracting features of the multimodal signal in the input layer through a feature extraction layer to obtain multimodal features; fusing all the extracted multimodal features through a fusion layer to obtain fused modal features; classifying or regressing the fused modal features through a decision tree layer to generate a decision tree for the target user; and outputting the decision tree through an output layer to generate the psychological counseling strategy for the target user; The acquisition process of the decision tree also includes: selecting a target multimodal feature from the multiple multimodal features according to the importance of the feature, and using information gain or Gini coefficient to split the nodes of the selected target multimodal feature, and then pre-pruning and post-pruning the decision tree obtained after the split to generate a decision tree for the target user.

2. The agent-based psychological counseling strategy generation method according to claim 1, characterized in that: The step of inputting the multimodal signal into a decision-making large model created based on a decision tree and a multimodal large model comprises: Performing format conversion on the multimodal signal to obtain a converted modal signal; The converted modal signal is input into a decision-making big model created based on a decision tree and a multi-modal big model.

3. The agent-based psychological counseling strategy generation method according to claim 1, characterized in that: Also includes: Use the pre-trained language model to initialize the multimodal large model to obtain the initialized model; The historical consultation records of the target user are obtained, and the initialized model is fine-tuned using the historical consultation records to obtain the large decision model.

4. The agent-based psychological counseling strategy generation method according to claim 1, characterized in that: The step of inputting the multimodal signal into a decision-making macro model created based on a decision tree and a multimodal macro model to generate a psychological counseling strategy for the target user comprises: Inputting the multimodal signal into a decision-making macro model created based on a decision tree and a multimodal macro model to generate a psychological counseling strategy and a corresponding psychological counseling service method for the target user; Accordingly, after the psychological counseling strategy is adjusted according to the user portrait model and professional psychology knowledge to obtain a personalized counseling strategy for the target user, the method further includes: The personalized counseling strategy is output according to the psychological counseling service mode.

5. The agent-based psychological counseling strategy generation method according to claim 1, characterized in that: The step of fusing all the multimodal features extracted through the fusion layer to obtain fused modal features includes: All the extracted multimodal features are spliced ​​through a fusion layer to obtain spliced ​​modal features, and each feature in the spliced ​​modal features is weighted according to a preset feature weight to obtain a fused modal feature.

6. The agent-based psychological counseling strategy generation method according to any one of claims 1 to 5, characterized in that: Also includes: Acquire the historical consultation records of the target user from the temporary memory knowledge base; Generate a user portrait model of the target user based on the historical consultation records and corresponding user feedback information; Accordingly, after the psychological counseling strategy is adjusted according to the user portrait model and professional psychology knowledge to obtain a personalized counseling strategy for the target user, the method further includes: The audio and video signals, the personalized consulting strategy and the feedback information generated by the target user for the personalized consulting strategy are saved in the temporary memory knowledge base, and the decision-making model is fine-tuned based on the information in the temporary memory knowledge base.

7. A psychological counseling strategy generation device based on an intelligent agent, characterized in that: The intelligent agent is an intelligent agent based on a multimodal large model; wherein the device comprises: A signal acquisition module, used to collect audio and video signals of target users who are undergoing psychological counseling; A recognition module, used to perform facial emotion recognition and speech recognition on the audio and video signals using computer vision technology and speech recognition technology to obtain a multimodal signal; A strategy generation module, used for inputting the multimodal signal into a decision-making model created based on a decision tree and a multimodal big model, so as to generate a psychological counseling strategy for the target user; the decision-making big model is composed of an input layer, a feature extraction layer, a fusion layer, a decision tree layer and an output layer; A portrait model acquisition module, used to acquire a user portrait model of the target user generated based on the historical consultation records of the target user; A strategy adjustment module, used to adjust the psychological counseling strategy according to the user portrait model and professional psychology knowledge to obtain a personalized counseling strategy for the target user; The strategy generation module is specifically used to receive the multimodal signal through the input layer, and extract features from the multimodal signal in the input layer through the feature extraction layer to obtain multimodal features; perform feature fusion on all the extracted multimodal features through the fusion layer to obtain fused modal features; classify or regress the fused modal features through the decision tree layer to generate a decision tree for the target user; and output the decision tree through the output layer to generate a psychological counseling strategy for the target user. The device is also used to select a target multimodal feature from the multiple multimodal features according to the importance of the feature, and use information gain or Gini coefficient to split the nodes of the selected target multimodal feature, and then pre-prune and post-prune the decision tree obtained after the split to generate a decision tree for the target user.

8. An electronic device, characterized in that: It comprises a processor and a memory; wherein, when the processor executes the computer program stored in the memory, it implements the agent-based psychological counseling strategy generation method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: Used to store computer programs; wherein, when the computer program is executed by a processor, it implements the agent-based psychological counseling strategy generation method as described in any one of claims 1 to 6.

Images