Artificial intelligence remote psychological counseling platform and method based on emotion cognition

Abstract

The invention provides an artificial intelligence remote psychological counseling platform and method based on emotion cognition. The platform comprises a multi-mode emotion recognition module, an emotion map construction module, a personalized counseling file generation module, a long-term emotion state prediction intervention module and a psychological counseling platform optimization module. The invention provides an artificial intelligence remote psychological counseling platform and method based on emotion cognition, and the method comprises the steps: integrating multi-mode emotion recognition, emotion map tracking, emotion historical records and personalized counseling archives, long-term emotion state prediction and intervention, and a community emotion support network; a psychological counseling solution which is more intelligent, personalized and interactive is provided.

Description

Technical Field

[0001] The present invention belongs to the field of psychological counseling, and in particular relates to an artificial intelligence remote psychological counseling platform and method based on emotion cognition. Background Art

[0002] In modern society, with the accelerated pace of life and the increase of social pressure, mental health issues have received increasing attention. Traditional psychological counseling methods mainly rely on face-to-face communication, which has many limitations. First, face-to-face psychological counseling requires users and counselors to communicate at the same place and time, which is very inconvenient for users who are time-constrained or geographically remote. In addition, face-to-face communication may make some users feel uncomfortable, especially those who are introverted or not good at expressing emotions. They may find it difficult to fully express their true emotions and inner thoughts during the consultation process. This not only affects the effectiveness of the consultation, but may also cause users to reject psychological counseling.

[0003] With the rapid development of artificial intelligence technology, remote psychological counseling platforms based on artificial intelligence have gradually emerged. These platforms use advanced technologies such as natural language processing, emotion recognition and big data analysis to try to solve the shortcomings of traditional psychological counseling. For example, through voice recognition and text analysis technology, the system can automatically generate psychological counseling suggestions, improving the efficiency and coverage of counseling. However, there are still many problems with current technologies in practical applications.

[0004] First, existing emotion recognition technologies mostly rely on single-modal data (such as voice or text) and lack comprehensive analysis of multimodal data (voice, expression, and text), resulting in low accuracy of emotion recognition.

[0005] Secondly, the existing remote psychological counseling platforms lack long-term tracking and analysis of users' emotional changes and are unable to provide users with personalized counseling files and long-term emotional management solutions.

[0006] In addition, existing platforms mostly feature one-way information transmission and lack interactivity and user community support mechanisms. When users encounter problems, they can only passively accept the advice of consultants and lack proactive emotional support and interaction. Summary of the invention

[0007] The purpose of the present invention is to propose an artificial intelligence remote psychological counseling platform and method based on emotion cognition, which can improve the accuracy and comprehensiveness of emotion recognition by comprehensively analyzing various data of users through a multimodal emotion recognition system integrating speech, facial expression and text analysis.

[0008] In order to achieve the above-mentioned object, a first aspect of the present invention provides an artificial intelligence remote psychological consultation platform based on emotion cognition, characterized in that the platform comprises:

[0009] A multimodal emotion recognition module is used to collect the user's voice data, facial expression data and text data and pre-process the data, perform multimodal feature extraction on the pre-processed data to obtain a voice feature vector, an expression feature vector and a text feature vector, then perform multimodal feature fusion on the voice feature vector, the expression feature vector and the text feature vector to obtain a comprehensive feature vector, use a deep neural network to perform emotion classification based on the comprehensive feature vector, and transmit the voice feature vector, the expression feature vector and the text feature vector to the emotion map construction module;

[0010] An emotion map construction module is used to construct an emotion database according to the input voice feature vector, expression feature vector and text feature vector, and perform time series processing on the emotion database to establish an emotion map. The user emotion map includes nodes and edges, where nodes represent emotion state nodes and edges represent transition edges between emotion states, wherein each emotion feature vector corresponds to an emotion state node, and each emotion feature vector includes a voice feature vector of the current node, an expression feature vector of the current node and a text feature vector of the current node. The transition edges between the emotion states of the emotion map are created according to timestamp information, and then the emotion map is regularly updated according to the real-time collected data, and the emotion map is transmitted to the personalized consultation archive generation module;

[0011] The personalized consultation file generation module is used to construct an emotional history database, analyze it, extract the user's emotional change trend, and construct an emotional fluctuation detection algorithm to identify key emotional events in the emotional history data, and generate a personalized consultation file for the user based on the emotional change trend and key emotional events; wherein the emotional fluctuation detection algorithm specifically includes:

[0012] Assume that the emotional fluctuation threshold is δ. When the emotional feature vector change of two consecutive time points exceeds the threshold, it is marked as a key emotional event. Assume that t i and t i+1 The emotional feature vectors at each moment are F i and F i+1 , then the identification condition of key emotional events is:

[0013] ∥F i+1 -F i ∥>δ

[0014] Among them, δ represents the threshold of emotional state change, and the emotional feature vector pairs that meet the conditions are recorded as key emotional events;

[0015] The long-term emotional state prediction and intervention module is used to predict the emotional state by constructing an emotional state prediction model based on the user's emotional feature vector sequence extracted from the emotional history record and personalized consultation archive, and at the same time design an emotional prediction threshold to detect changes in the emotional feature vector sequence, and use reinforcement learning to optimize the emotional state prediction model and generate a personalized emotional regulation plan; wherein, the emotional prediction threshold is designed to detect changes in the emotional feature vector sequence, and when the change of the emotional feature vector at two consecutive time points exceeds the threshold, the emotional intervention mechanism is triggered, and t i+1 and t i The emotional feature vectors at each moment are F i+1 and F i , then the emotion change detection condition is:

[0016] ∥F i+1 -F i ∥>δ

[0017] Wherein, δ represents the threshold of emotional state change. For the significant emotional changes detected, personalized emotional intervention suggestions are generated, and the emotional intervention suggestions include emotional regulation strategies, psychological intervention measures and behavioral suggestions;

[0018] The psychological counseling platform optimization module is used to build a deep reinforcement learning optimization model. It takes the current emotional state as input and passes it to the deep reinforcement learning optimization model for dynamic path adjustment. It selects the optimal counseling strategy and dynamic counseling path based on the output of the deep reinforcement learning optimization model. At the same time, it detects the user's emotional fluctuations after each consultation and collects the user's feedback score. It collects user feedback and updates the emotional database. It then uses the collected user feedback and emotional data to update the Q value function of the deep reinforcement learning optimization model. Finally, it generates a personalized counseling strategy based on the user's historical emotional data and predicted emotional state.

[0019] Preferably, the platform further comprises:

[0020] The community emotion support module is used to obtain the user's current emotion feature vector and emotion classification label from the multimodal emotion recognition module and the emotion map construction module, and publish the user's own emotion status on the community platform based on the real-time collection. The emotion status includes the emotion feature vector, emotion classification label and text description, and then calculates the emotion similarity between each user to match users with similar emotions, and recommends other users with similar emotions to the user for interaction based on the user matching degree, and then stores the interaction records and emotion support scores between users in the community emotion map. At the same time, the community emotion map is updated regularly, new user nodes and interaction edges are added, and the edge weights are updated.

[0021] Preferably, the preprocessing includes: performing noise reduction processing on the speech data, converting the speech signal into a spectrogram representation using short-time Fourier transform; graying and normalizing the expression image data; performing word segmentation and stop word removal processing on the text data and converting the text into a word vector matrix using word embedding technology;

[0022] Among them, convolutional neural networks are used to extract spectrum features; convolutional neural networks are used to extract image features; and bidirectional Transformer models are used to extract text features.

[0023] Preferably, the transition edge between the emotional states of the emotional graph is created according to the timestamp information, specifically:

[0024] Suppose two nodes V i and V j The corresponding timestamp is t i and t j , if t i <t j If the distance between the two nodes is less than the sentiment threshold δ, a path from V i To V j The directed edge E ij , which is expressed as follows:

[0025] E ij =(V i →V j )if t i <t j and||z i -z j ||<δ;

[0026] The specific steps of regularly updating the emotion map include:

[0027] Let the new sentiment data be F at each update new =[F v,new ; F f,new ; F t,new ], where F v,new represents the new speech feature vector, F f,new Represents the new expression feature vector and F t,new Represents the new text feature vector, then:

[0028] The new data F new =[F v,new ; F f,new ; F t,new ] is mapped to two-dimensional space and new node coordinates are generated, which are expressed as follows:

[0029] z new =t-SNE(F new )

[0030] Among them, t-SNE represents -SNE dimensionality reduction technology;

[0031] Add new node V new To the node set V:

[0032] V=V∪{V new}

[0033] Update the edge set E according to the timestamp and position of the new node:

[0034]

[0035] Among them, z i represents the coordinate of the i-th node in two-dimensional space, z new represents the coordinates in the new two-dimensional space, δ ratio represents the emotion threshold, t i represents the past time t, t new Indicates a new moment.

[0036] Preferably, in the personalized consultation archive generation module, analyzing the emotion history database to extract the user's emotion change trend includes:

[0037] Assume that the user's emotional feature vector sequence is {F1, F2, ..., F n}, the sentiment classification label sequence is {y1,y2,…,y n}, use the weighted moving average method to calculate the user's emotional change trend; set the window size to w, the weighted moving average of the emotional feature vector It is expressed as follows:

[0038]

[0039] Among them, w j represents the weight within the window, w represents the number of windows, and F j Represents the sentiment feature vector of the jth user.

[0040] Preferably, the emotional state prediction model is a long short-term memory network, specifically comprising:

[0041] Let the parameter of the long short-term memory network be θ lstm , then the output of the emotional state prediction model is expressed as follows:

[0042] {F′ n+1 , FF' n+2 , ..., F′ n+m}=LSTM({F′1,F′2,...,F′ n};θ lstm )

[0043] Among them, {F′1,F′2,…,F′ n} represents the normalized emotional feature vector sequence; {F′1,F′2,…,F′ n} represents the predicted sentiment feature vector output;

[0044] Add an emotion fluctuation regularization term to the emotion state prediction model to ensure that the model can capture subtle fluctuations in emotion changes. Set the emotion fluctuation coefficient to λ w , the sentiment fluctuation regularization term is expressed as follows:

[0045]

[0046] in, represents the sentiment fluctuation regularization term, λ w represents the weight of the regularization term of sentiment fluctuation, and n represents the total number of sentiment types.

[0047] Preferably, in the optimization module of the psychological counseling platform, a deep reinforcement learning optimization model is used to perform path optimization, and the state s of the deep reinforcement learning optimization model is t Including the current sentiment feature vector F t and sentiment classification label y t , action a t For the next step of consulting strategy selection, the reward function R(s t ,a t ) is designed in combination with user feedback and consulting effects to maximize the improvement of user emotions, and the user emotion fluctuation term λ is introduced into the reward function e and the behavioral bias term λ b , capturing subtle changes in user emotions and the degree of deviation from normal behavior, expressed as follows:

[0048] R(s t ,a t )=αΔy t +βF b -λ e ||F t -F t-1 ||-λ b B t

[0049] Among them, Δy t Indicates the degree of improvement of sentiment classification labels, F b represents the user feedback score, α and β are weight coefficients, and λ e is the sentiment fluctuation coefficient, λ b is the behavioral deviation coefficient, B t is the behavioral deviation item;

[0050] According to the output of the deep reinforcement learning optimization model, select the optimal consulting strategy a t , and dynamically adjust the consultation path, assuming that the Q value function of the optimization model is Q(s t ,a t ), select action a t To maximize the Q value, it is expressed as follows

[0051]

[0052] The consulting strategy that will be selected is a t Applied to current emotional state t , generate new consultation paths.

[0053] Preferably, after each consultation, the user's emotional fluctuations are detected and the user's feedback score F is collected. b , let the emotional feature vector at the current time t be F t , the emotional feature vector after the last consultation is F t-1 , where the emotion fluctuation detection formula is as follows:

[0054] W t =||F t -F t-1 ||

[0055] If W t Exceeding the preset emotional fluctuation threshold δ w , it is recorded as a significant emotional fluctuation, and a feedback mechanism is generated, and the user's feedback score F b And the emotional feature vector F after consultation t+1 and sentiment classification label y t+1 Update to sentiment database;

[0056] Use the collected user feedback and sentiment data to update the Q-value function of the deep reinforcement learning optimization model, where the Q-value function update formula is expressed as follows:

[0057]

[0058] Among them, γ represents the discount factor and α represents the learning rate;

[0059] According to the user's historical emotional data and predicted emotional state, a personalized consulting strategy is generated, wherein the personalized consulting strategy includes psychological intervention measures, and the selection of the consulting strategy is set based on the emotional feature vector and the emotional classification label, and the strategy that best suits the user's current emotional state is selected. The selection formula of the personalized consulting strategy is expressed as follows:

[0060]

[0061] Among them, St represents the consulting strategy selected at time t, P(s|F t ,y t ) represents the current sentiment feature vector F t and sentiment classification label y t Next, the probability of strategy s.

[0062] Preferably, in the community emotional support module, users publish their emotional states on the community platform, including the emotional feature vector F t , sentiment classification label y t and text description, where the emotional status publishing format is:

[0063] P t ={F t ,y t , T t}

[0064] Among them, T t Indicates the user's current emotional description text;

[0065] Calculate the sentiment similarity between each user:

[0066] Assume that the emotional feature vectors of two users at time t are and The sentiment classification labels are and The similarity The calculation formula is as follows:

[0067]

[0068] Among them, · represents the vector inner product, ∥·∥ represents the Euclidean norm of the vector, and δ represents the Kroneckerdelta function. hour otherwise λ c is the classification label similarity weight;

[0069] According to the calculated emotional similarity, other users with similar emotions are recommended for interaction. Let the set of emotionally similar users of user i be M i , then the matching algorithm selects the first k users with the highest similarity, which is expressed as follows:

[0070] M i ={j|j∈Top-k(S i，j ), j≠i}

[0071] Among them, Top-k(S i,j ) represents the similarity S i,j The top k users

[0072] At the same time, there is an emotional support scoring mechanism, assuming that user i’s support score for user j is R i,j , users rate each other's emotional support after interaction, and the rating mechanism formula is as follows:

[0073] R i，j =Feedback i，j ·S i，j

[0074] Among them, Feedback i,j represents the subjective feedback rating of user i to user j;

[0075] Update the community sentiment graph regularly, add new user nodes and interaction edges according to the updated community sentiment graph, and update the edge weights.

[0076] In a second aspect of the present invention, an artificial intelligence remote psychological consultation method based on emotion cognition is provided, the method comprising:

[0077] S1. Collect the user's voice data, facial expression data and text data and pre-process the data, perform multimodal feature extraction on the pre-processed data to obtain a voice feature vector, an expression feature vector and a text feature vector, then perform multimodal feature fusion on the voice feature vector, the expression feature vector and the text feature vector to obtain a comprehensive feature vector, and use a deep neural network to perform emotion classification based on the comprehensive feature vector;

[0078] S2. Construct an emotion database according to the input speech feature vector, expression feature vector and text feature vector, perform time series processing on the emotion database, and establish an emotion map. The user emotion map includes nodes and edges. The nodes represent emotion state nodes, and the edges represent transition edges between emotion states. Each emotion feature vector corresponds to an emotion state node. Each emotion feature vector includes a speech feature vector of the current node, an expression feature vector of the current node and a text feature vector of the current node. The transition edges between the emotion states of the emotion map are created according to the timestamp information, and then the emotion map is regularly updated according to the real-time collected data.

[0079] S3, construct an emotion history database, analyze it, extract the emotion change trend of the user, and construct an emotion fluctuation detection algorithm to identify key emotion events in the emotion history data, and generate a personalized consultation file for the user based on the emotion change trend and key emotion events; wherein the emotion fluctuation detection algorithm specifically includes:

[0080] Assume that the emotional fluctuation threshold is δ. When the emotional feature vector change of two consecutive time points exceeds the threshold, it is marked as a key emotional event. Assume that t i and t i+1The emotional feature vectors at each moment are F i and F i+1 , then the identification condition of key emotional events is:

[0081] ∥F i+1 -F i ∥>δ

[0082] Among them, δ represents the threshold of emotional state change, and the emotional feature vector pairs that meet the conditions are recorded as key emotional events;

[0083] S4, predicting the emotional state of the user by constructing an emotional state prediction model based on the emotional feature vector sequence extracted from the emotional history record and personalized consultation archive, and designing an emotional prediction threshold to detect changes in the emotional feature vector sequence, and using reinforcement learning to optimize the emotional state prediction model and generate a personalized emotional regulation plan; wherein, the emotional prediction threshold is designed to detect changes in the emotional feature vector sequence, and when the change of the emotional feature vector at two consecutive time points exceeds the threshold, the emotional intervention mechanism is triggered, and t is set i+1 and t i The emotional feature vectors at each moment are F i+1 and F i , then the emotion change detection condition is:

[0084] ∥F i+1 -F i ∥>δ

[0085] Wherein, δ represents the threshold of emotional state change. For the significant emotional changes detected, personalized emotional intervention suggestions are generated, and the emotional intervention suggestions include emotional regulation strategies, psychological intervention measures and behavioral suggestions;

[0086] S5. Construct a deep reinforcement learning optimization model, take the current emotional state as input, pass it to the deep reinforcement learning optimization model for dynamic path adjustment, select the optimal consultation strategy and dynamic consultation path according to the output of the deep reinforcement learning optimization model, and detect the user's emotional fluctuations after each consultation and collect the user's feedback score. Collect user feedback and update the emotional database, and then use the collected user feedback and emotional data to update the Q value function of the deep reinforcement learning optimization model. Finally, generate a personalized consultation strategy based on the user's historical emotional data and predicted emotional state.

[0087] The beneficial technical effects of the present invention are at least as follows:

[0088] (1) The present invention uses a multimodal emotion recognition system that integrates speech, facial expression and text analysis to comprehensively analyze multiple user data to improve the accuracy and comprehensiveness of emotion recognition. This solves the problem in the prior art that emotion recognition relies on single modality data and has low accuracy.

[0089] (2) The present invention establishes a user's emotional map, records and analyzes the user's emotional changes and psychological state in different time periods, and uses knowledge map technology combined with emotional cognition models for analysis. This solves the problem of the lack of long-term tracking and analysis of user emotional changes in the prior art.

[0090] (3) The present invention automatically records the user's emotional change history, and generates a personalized consultation file based on this data, saves the user's emotional data and consultation content for each consultation, and generates a personalized consultation file based on historical data, including emotional change trends and important event records. This solves the problem of the lack of personalized consultation solutions in the prior art.

[0091] (4) Based on the user's historical emotional data and current emotional state, the present invention uses the time series prediction model in deep learning combined with emotional analysis data to predict the user's long-term emotional change trend and conduct psychological intervention in advance. This solves the problem of the lack of long-term emotional management solutions in the prior art.

[0092] (5) The present invention establishes a community support network based on sentiment analysis, where users can share their emotional status on the platform, obtain emotional support and suggestions from other users, analyze community content through sentiment classification models, and match users with similar sentiments for interaction. This solves the problem of lack of interactivity and user community support mechanism in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] The present invention is further described using the accompanying drawings, but the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative work.

[0094] Figure 1 This is a framework diagram of an artificial intelligence remote psychological consultation platform based on emotion cognition in the present invention. DETAILED DESCRIPTION

[0095] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0096] like Figure 1As shown, the multimodal emotion recognition module provided by the embodiment of the present invention is used to collect the user's voice data, facial expression data and text data and preprocess the data, perform multimodal feature extraction on the preprocessed data to obtain a voice feature vector, an expression feature vector and a text feature vector, then perform multimodal feature fusion on the voice feature vector, the expression feature vector and the text feature vector to obtain a comprehensive feature vector, use a deep neural network to perform emotion classification based on the comprehensive feature vector, and transmit the voice feature vector, the expression feature vector and the text feature vector to the emotion map construction module.

[0097] Specifically, in the current module, the user's voice, facial expression and text data are collected. Voice data is collected through a microphone, expression data is captured through a camera, and text data is recorded through the text input by the user. These data are stored in different databases, named voice database D and voice database D respectively. v , Expression Database D f , text database D t .

[0098] Furthermore, data preprocessing includes:

[0099] The speech data is denoised and the short-time Fourier transform (STFT) is used to convert the speech signal into a spectrogram. Assume that the original speech signal is x(t), and its spectrogram is represented as:

[0100]

[0101] Among them, x(t) represents the change of speech signal with time t, and S(f,t) represents the change of spectrum with frequency f and time t.

[0102] Mel frequency cepstral coefficients (MFCC) are extracted from the spectrogram to capture the key features of speech. Let the spectrogram be S(f,t) and the MFCC feature vector be:

[0103]

[0104] Among them, MFCC n represents the nth MFCC feature, and N represents the total number of frequencies.

[0105] Grayscale and normalize the expression image. Let the original expression image be I and the grayscale image be I g , the normalized image is:

[0106]

[0107] Among them, I′ is the normalized image, min(I g ) and max(I g) are the minimum and maximum values ​​of the grayscale image, respectively.

[0108] Use the Dlib library to detect facial key points and generate a coordinate matrix of 68 facial key points:

[0109] P={(x1,y1),(x2,y2),...,(x 68 ,y 68 )} (4)

[0110] Among them, P is the key point coordinate set, (x i ,y i ) is the coordinate of the i-th key point.

[0111] Perform word segmentation and stop word removal on the text data. Let the original text be T, and the processed word vector sequence be:

[0112] W=tokenize(remove_stopwords(T)) (5)

[0113] Among them, W is the word vector sequence, tokenize is the word segmentation function, and remove_stopwords is the stop word removal function.

[0114] Use word embedding technology (such as Word2Vec or GloVe) to convert text into word vector matrix. Suppose the word embedding matrix is:

[0115] E={e1,e2,...,e n} (6)

[0116] Among them, E is the word embedding matrix, e i is the word vector of the i-th word, and n is the total number of words.

[0117] Furthermore, speech feature extraction uses a convolutional neural network (CNN) to extract spectrogram features. Let the spectrogram input be S and the CNN model parameter be θ v , the output feature vector is:

[0118] F v =CNN(S;θ v ) (7)

[0119] Among them, F v is the speech feature vector, θ v are the parameters of the CNN model.

[0120] Assume that CNN includes multiple convolutional layers and pooling layers, and the feature map output by each convolutional layer is:

[0121] F v，i =ReLU(Conv(F v,i-1 ,Wi )+b i ) (8)

[0122] Among them, F v,i is the feature map after the i-th convolution, Conv is the convolution operation, W i and b i are the convolution kernel and bias of the i-th layer respectively, and ReLU is the activation function.

[0123] Furthermore, expression feature extraction uses convolutional neural network to extract image features. Let the preprocessed expression image be I′ and the CNN model parameter be θ f , the output feature vector is:

[0124] F f =CNN(I′;θ f ) (9)

[0125] Among them, F f is the facial expression feature vector, θ f are the parameters of the CNN model.

[0126] Assume that CNN includes multiple convolutional layers and pooling layers, and the feature map output by each convolutional layer is:

[0127] F f，i =ReLU(Conv(F f,i-1 ,W i )+b i ) (10)

[0128] Among them, F f,i is the feature map after the i-th convolution, Conv is the convolution operation, W i and b i are the convolution kernel and bias of the i-th layer respectively, and ReLU is the activation function.

[0129] Furthermore, text feature extraction uses a bidirectional Transformer model to extract text features. Let the processed word vector sequence be W and the Transformer model parameter be θ t , the output feature vector is:

[0130] F t = Transformer(W;θ t ) (11)

[0131] Among them, F t is the text feature vector, θ t are the parameters of the Transformer model.

[0132] Assume that Transformer includes a multi-head self-attention mechanism and a feed-forward neural network layer. The output of each self-attention mechanism is:

[0133]

[0134] Where Q, K, and V are query, key, and value matrices, respectively, and d k The dimension of the key.

[0135] Furthermore, the feature vectors of speech, expression and text are concatenated to form a comprehensive feature vector. Let the speech feature vector be F v , the facial expression feature vector is F f , the text feature vector is F t , the comprehensive feature vector is:

[0136] F concat =[F v ; F f ; F t ] (13)

[0137] Among them, F concat is the comprehensive feature vector.

[0138] Furthermore, principal component analysis (PCA) is used to reduce the dimension of the comprehensive feature vector. Let the comprehensive feature vector be F concat , the eigenvector after dimensionality reduction is:

[0139] F pca = PCA(F concat ) (14)

[0140] Among them, F pca is the feature vector after dimensionality reduction.

[0141] Let the PCA dimension reduction matrix be W pca , the eigenvector after dimensionality reduction is:

[0142] F pca =F concat W pca (15)

[0143] Among them, W pca is the dimension reduction matrix of PCA.

[0144] Furthermore, the classifier training uses a deep neural network (DNN) for sentiment classification. Let the feature vector after dimensionality reduction be F pca , the DNN model parameters are θ c , the sentiment classification label is y, and the model output is:

[0145] y=DNN(F pca θ c) (16)

[0146] Among them, y is the sentiment classification label, θ c are the parameters of the DNN model.

[0147] The output of each layer of DNN is:

[0148] h i =ReLU(W i h i-1 +b i ) (17)

[0149] Among them, h i is the output of the i-th layer, W i and b i are the weight matrix and bias vector of the i-th layer, h i-1 is the output of the previous layer.

[0150] Furthermore, an L2 regularization term is added to the loss function to prevent the model from overfitting. Let the true sentiment label be y true , the predicted sentiment label is y, and the loss function is:

[0151]

[0152] in, is the loss function, y true,i is the i-th true sentiment label, y i is the i-th predicted sentiment label, λ is the regularization coefficient, W j is the weight matrix of the model.

[0153] Furthermore, the Adam optimizer is used to optimize the parameters and update the DNN model parameters θ c The parameter update formula of the Adam optimizer is:

[0154]

[0155] Among them, θ c is the parameter of the DNN model, α is the learning rate, m t and v t are the first-order and second-order momentum estimates, respectively, and ∈ is a small constant to prevent division by zero errors.

[0156] The emotion map construction module is used to build an emotion database based on the input voice feature vector, expression feature vector and text feature vector, and perform time series processing on the emotion database to establish an emotion map. The user emotion map contains nodes and edges. The nodes represent emotion state nodes, and the edges represent transition edges between emotion states. Each emotion feature vector corresponds to an emotion state node. Each emotion feature vector includes the voice feature vector of the current node, the expression feature vector of the current node and the text feature vector of the current node. The transition edges between the emotion states of the emotion map are created according to the timestamp information. Then, the emotion map is regularly updated according to the data collected in real time, and the emotion map is transmitted to the personalized consultation archive generation module.

[0157] Specifically, the emotion data storage obtains the emotion feature vector F from the multimodal emotion recognition step v 、F f 、F t . Store these feature vectors and corresponding timestamps in the sentiment database. The structure of the sentiment database is as follows:

[0158] D e ={(t1,F v1 ,F f1 ,F t1 ),(t2,F v2 ,F f2 , F t2 ),...,(t n ,F vn ,F fn ,F tn )} (20)

[0159] Among them, t9 represents the timestamp of the i-th emotional data collection, F vi 、F fi 、F ti Respectively represent the speech, expression and text feature vectors collected for the i-th time.

[0160] Furthermore, the emotional data is processed in time series to build the user emotional graph G. The emotional graph G is a knowledge graph with a time dimension, which contains nodes and edges. Nodes represent emotional states, and edges represent transitions between emotional states. Initialize the emotional graph node set V and edge set E:

[0161] V={} (21)

[0162] E={} (22)

[0163] Furthermore, an emotional state node is created, where each emotional feature vector F i =[F vi ; F fi ; Ft9 ] corresponds to an emotional state node. Each emotional feature vector is mapped to the emotional state space, and the high-dimensional feature vector is projected into the two-dimensional space through dimensionality reduction technology (such as t-SNE) to generate node coordinates.

[0164] z9=t-SNE(F9) (23)

[0165] Among them, z i Represents the coordinates of the i-th node in two-dimensional space.

[0166] Furthermore, the transition edges between emotional states are created based on the timestamp information. i and V j The corresponding timestamp is t i and t j , if t i <t j If the distance between the two nodes is less than the threshold δ, a path from V i To V j The directed edge E ij .

[0167] E ij =(V i →V j )if t i <t j and||z i -z j ||<δ (24)

[0168] Furthermore, the emotion map is updated regularly to reflect the latest changes in the user's emotional state. Let the new emotion data at each update be F new =[F v,new ; F f,new ; F t,new ],but:

[0169] Map the new data into two-dimensional space and generate new node coordinates:

[0170] z new =t-SNE(F new ) (25)

[0171] Add new node V new To the node set V:

[0172] V=V∪{V new} (26)

[0173] Update the edge set E according to the timestamp and position of the new node:

[0174]

[0175] Furthermore, the emotional map is displayed to users and psychological counselors using visualization tools. The color and size of the nodes represent the category and intensity of the emotional state, and the color and thickness of the edges represent the frequency and intensity of the emotional state transition. Through graph analysis, the key turning points and emotional fluctuation patterns of the user's emotional state can be identified, helping psychological counselors better understand the user's emotional change trends and develop personalized psychological counseling plans.

[0176] The personalized consultation file generation module is used to build an emotional history database, analyze it, extract the user's emotional change trends, and build an emotional fluctuation detection algorithm to identify key emotional events in the emotional history data. Based on the emotional change trends and key emotional events, the user's personalized consultation file is generated.

[0177] Specifically, the emotional data record is obtained from the psychological state tracking step based on the emotional graph to obtain the information of the emotional state node V and the edge E. The information of each emotional state node, including the timestamp t i , sentiment feature vector F i and sentiment classification label y i , stored in the emotion history database. The structure of the emotion history database is as follows:

[0178] D h ={(t1,F1,y1),(t2,F2,y2),…,(t n , F n ,y n )} (28)

[0179] Furthermore, the data in the emotion history database is analyzed to extract the user's emotion change trend and key emotion events. Assume that the user's emotion feature vector sequence is {F1, F2, ..., F n}, the sentiment classification label sequence is {y1,y2,…,y n}. Use the weighted moving average method to calculate the user's emotional change trend. Assume that the window size is w, and the weighted moving average of the emotional feature vector is:

[0180]

[0181] Among them, w j is the weight within the window, usually set to decreasing weights so that the most recent data points have greater weights.

[0182] Furthermore, a custom emotion fluctuation detection algorithm is used to identify key emotion events in emotion history data. Let the emotion fluctuation threshold be δ. When the emotion feature vector change of two consecutive time points exceeds the threshold, it is marked as a key emotion event. Let t i and t i+1The emotional feature vectors at each moment are F i and F i+1 , then the identification condition of key emotional events is:

[0183] ||F i+1 -F i ||>δ (30)

[0184] The pairs of emotional feature vectors that meet the conditions are recorded as key emotional events.

[0185] Furthermore, based on the emotion change trend and key emotion events, a personalized consultation profile of the user is generated. The personalized consultation profile includes the user's emotion change trend graph, key emotion event list and personalized consultation suggestions. The emotion change trend graph is a weighted moving average of the user's emotion feature vector displayed using a line graph. Changes over time. The key emotional event list records the time of each key emotional event, the change of emotional feature vectors, and the corresponding emotional classification labels. Personalized counseling suggestions generate targeted psychological counseling suggestions based on the user's emotional change trends and key emotional events. The suggestions include emotional regulation strategies, behavioral suggestions, and psychological intervention measures.

[0186] The long-term emotional state prediction intervention module is used to predict the emotional state by constructing an emotional state prediction model based on the user's emotional feature vector sequence extracted from emotional history records and personalized consultation files. At the same time, the emotional prediction threshold is designed to detect changes in the emotional feature vector sequence, and reinforcement learning is used to optimize the emotional state prediction model and generate personalized emotional regulation solutions.

[0187] Specifically, the emotional data sequence is constructed by extracting the user's emotional feature vector sequence {F1, F2, ..., F n}, and the corresponding timestamp sequence {t1,t2,...,t n}.

[0188] In order to build a time series model, the sentiment feature vector and timestamp need to be normalized. Let the time interval Δt = t i+1 -t9, the normalized timestamp is:

[0189]

[0190] The normalized emotional feature vector sequence is {F′1, F′2, ..., F′ n}.

[0191] Furthermore, the emotion state prediction model uses a long short-term memory (LSTM) network to predict the emotion state. Assume that the input of the LSTM network is the normalized emotion feature vector sequence {F′1, F′2, ..., F′ n}, the output is the predicted sentiment feature vector {F′ n+1 ,F′ n+2 ,...,F′ n+m}. Let the parameters of the LSTM network be θ lstm , the output of the prediction model is:

[0192] {F′ n+1 , F′ n+2 , ... F′ n+m}=LSTM({F′1, F′2,...,F′ n};θ lstm ) (32)

[0193] In order to improve the accuracy of prediction, a sentiment fluctuation regularization term is added to ensure that the model can capture the subtle fluctuations of sentiment changes. Let the sentiment fluctuation coefficient be λ w , the emotional fluctuation regularization term is:

[0194]

[0195] Furthermore, the predicted normalized sentiment feature vector {F′ n+1 ,F′ n+2 ,...,F′ n+m}Inverse normalization to obtain the actual predicted sentiment feature vector {F n+1 ,F n+2 ,...,F n+m}.

[0196] F i =F′ i ·(max(F)-min(F))+min(F),i∈{n+1,n+2,...,n+m} (34)

[0197] Furthermore, we set the emotional state change threshold δ to detect significant changes in the predicted emotional feature vector sequence. When the emotional feature vector change at two consecutive time points exceeds the threshold, the emotional intervention mechanism is triggered. i+1 and t i The emotional feature vectors at each moment are F i+1 and F i , then the emotion change detection condition is:

[0198] ||F i+1 -F i ||>δ (35)

[0199] For significant emotion changes detected, personalized emotion intervention suggestions are generated, including emotion regulation strategies, psychological intervention measures, and behavioral suggestions. The intervention suggestions are stored in the user's personalized counseling profile and notified to the user and counselor through the user interface.

[0200] Furthermore, the sentiment prediction model optimization uses reinforcement learning to optimize the sentiment prediction model to ensure that the model can be continuously improved based on user feedback. Set the reward function R and calculate the reward value based on the prediction accuracy and user feedback. Use the Q-learning algorithm to update the model parameters:

[0201]

[0202] Among them, s and s′ represent the current and next emotional states, respectively, a and a′ represent the current and next actions, respectively, α is the learning rate, and γ is the discount factor.

[0203] Furthermore, a personalized emotion regulation program is generated based on the user's historical emotion data and predicted emotion state. The emotion regulation program includes relaxation training, cognitive behavioral therapy, social activities, etc. The user's emotion feature vector and classification label are used to select the most suitable regulation strategy for the user and record it in the personalized consultation file.

[0204] The psychological counseling platform optimization module is used to build a deep reinforcement learning optimization model. It takes the current emotional state as input and passes it to the deep reinforcement learning optimization model for dynamic path adjustment. It selects the optimal counseling strategy and dynamic counseling path based on the output of the deep reinforcement learning optimization model. At the same time, it detects the user's emotional fluctuations after each consultation and collects the user's feedback score. It collects user feedback and updates the emotional database. It then uses the collected user feedback and emotional data to update the Q value function of the deep reinforcement learning optimization model. Finally, it generates a personalized counseling strategy based on the user's historical emotional data and predicted emotional state.

[0205] Specifically, the user's emotional feature vector F is obtained in real time from multimodal emotion recognition and emotion map t , and combined with historical emotional data, generate the current emotional state. Let the user's current emotional feature vector be F t , the sentiment classification label is y t , update the latest data in the emotion database in real time.

[0206] The current emotional state is used as input and passed to the optimization model for dynamic path adjustment. A deep reinforcement learning model (such as DQN) is used for path optimization. The state of the model s t Including the current sentiment feature vector F t and sentiment classification label y t , action at The reward function R(s) is the choice of the next consulting strategy. t ,a t ) is designed in combination with user feedback and consultation effects to maximize the improvement of user emotions. The user emotion fluctuation term λ is introduced into the reward function e and the behavioral bias term λ b , to capture subtle changes in user emotions and deviations from normal behavior.

[0207] R(s t ,a t )=αΔy t +βF b -λ e ||F t -F t-1 ||-λ b B t (37)

[0208] Among them, Δy t Indicates the degree of improvement of sentiment classification labels, F b represents the user feedback score, α and β are weight coefficients, and λ e is the sentiment fluctuation coefficient, λ b is the behavioral deviation coefficient, B t Behavioral deviation item.

[0209] Furthermore, according to the output of the optimization model, the optimal consulting strategy a is selected. t , and dynamically adjust the consultation path. Suppose the Q value function of the optimization model is Q(s t ,a t ), select action a t To maximize the Q value:

[0210]

[0211] The consulting strategy that will be selected is a t Applied to current emotional state t , generate new consultation paths.

[0212] Furthermore, after each consultation, the user's emotional fluctuations are detected and the user's feedback score F is collected. b . Let the emotional feature vector at the current time t be F t , the emotional feature vector after the last consultation is F t-1 , the emotion fluctuation detection formula is:

[0213] W t =||F t -F t-1 || (39)

[0214] If Wt Exceeding the preset emotional fluctuation threshold δ w , it is recorded as a significant emotional fluctuation and a feedback mechanism is generated. User feedback score F b And the emotional feature vector F after consultation t+1 and sentiment classification label y t+1 Updates to the sentiment database.

[0215] Furthermore, the collected user feedback and sentiment data are used to update the Q-value function of the reinforcement learning model. The Q-value function update formula is:

[0216]

[0217] Among them, γ is the discount factor and α is the learning rate. During the model update process, the user sentiment fluctuation term λ is introduced e and the behavioral bias term λ b , ensuring that the model can adjust to subtle changes in user emotions and behavioral deviations.

[0218] Furthermore, a personalized consultation strategy is generated based on the user's historical emotional data and predicted emotional state. The consultation strategy includes different psychological interventions, such as cognitive behavioral therapy (CBT), relaxation training, and social activities. The selection of the consultation strategy is set based on the emotional feature vector and the emotional classification label, and the strategy that best suits the user's current emotional state is selected. The selection formula for the personalized consultation strategy is:

[0219]

[0220] Among them, S t represents the consulting strategy selected at time t, P(s|F t ,y t ) represents the current sentiment feature vector F t and sentiment classification label y t Next, the probability of strategy s.

[0221] Through the above steps, the dynamic consultation path optimization scheme uses real-time emotional state monitoring, deep reinforcement learning models and user feedback to achieve dynamic adjustment of the consultation path. This scheme ensures the flexibility and personalization of the consultation strategy, provides users with the best psychological consultation path, and improves the consultation effect and user satisfaction. By introducing emotional fluctuation detection and behavioral deviation items, the dynamic optimization model can continuously improve the accuracy and effectiveness of consultation path selection.

[0222] Preferably, the platform further comprises:

[0223] The community emotion support module is used to obtain the user's current emotion feature vector and emotion classification label from the multimodal emotion recognition module and the emotion map construction module, and publish the user's own emotion status on the community platform based on the real-time collection. The emotion status includes the emotion feature vector, emotion classification label and text description, and then calculates the emotion similarity between each user to match users with similar emotions, and recommends other users with similar emotions to the user for interaction based on the user matching degree, and then stores the interaction records and emotion support scores between users in the community emotion map. At the same time, the community emotion map is updated regularly, new user nodes and interaction edges are added, and the edge weights are updated.

[0224] Specifically, the user's current emotion feature vector F is obtained from multimodal emotion recognition and emotion map t and sentiment classification label y t Users publish their emotional status on social platforms, including the emotional feature vector F t , sentiment classification label y t and text description. The format for publishing emotional status is:

[0225] P t ={F t ,y t ,T t} (42)

[0226] Among them, T t Represents the user's current emotion description text.

[0227] Furthermore, in order to match users with similar emotions, the emotional similarity between each user is calculated. Suppose the emotional feature vectors of two users at time t are and The sentiment classification labels are and The similarity The calculation formula is:

[0228]

[0229] Where · represents the vector inner product, ∥·∥ represents the Euclidean norm of the vector, and δ is the Kroneckerdelta function. hour otherwise λ c is the classification label similarity weight.

[0230] Furthermore, based on the calculated emotion similarity, other users with similar emotions are recommended for interaction. Suppose the set of emotion-similar users of user i is M i , the matching algorithm selects the first k users with the highest similarity:

[0231] M i ={j|j∈Top-k(S i,j ),j≠i} (44)

[0232] Among them, Top-k(S i,j ) represents the similarity S i,j The top k users.

[0233] Furthermore, users can interact with recommended users with similar emotions to share emotional experiences and support suggestions. The interactive content between users includes text messages, voice calls and video calls.

[0234] In order to promote emotional support among users, an emotional support scoring mechanism is designed. Suppose user i’s support score for user j is R i,j , users rate each other’s emotional support after interaction. The scoring mechanism formula is:

[0235] R i，j =Feedback i，j ·S i，j (45)

[0236] Among them, Feedback i,j represents the subjective feedback rating of user i to user j.

[0237] Furthermore, the interaction records and emotional support scores between users are stored in the community emotional map. s The nodes represent users, the edges represent the interactions and emotional support between users, and the weight of the edge is the emotional support score R i,j .

[0238] Update the community sentiment graph regularly, add new user nodes and interaction edges, and update the edge weights. Suppose the node set of the community sentiment graph is V s , the edge set is E s , then the graph update process is as follows:

[0239] V s =V s ∪{i,j}for all new users i,j (46)

[0240] E s =E s ∪{(i,j,R i，j )}for all new interactions(i,j) (47)

[0241] Through the above steps, the community emotional support network solution uses multimodal emotion recognition, emotion maps and user feedback to achieve emotional similarity matching and interactive support between users. This solution ensures the personalization and effectiveness of emotional support between users and provides users with a mutual emotional support network. By introducing emotional similarity calculation, user matching recommendation, emotional support scoring and community emotional map update, the interaction path between users is dynamically optimized, which improves the overall effect of the community emotional support network.

[0242] The embodiment of the present invention also provides an artificial intelligence remote psychological consultation method based on emotion cognition, the method comprising:

[0243] S1. Collect the user's voice data, facial expression data and text data and pre-process the data, perform multimodal feature extraction on the pre-processed data to obtain a voice feature vector, an expression feature vector and a text feature vector, then perform multimodal feature fusion on the voice feature vector, the expression feature vector and the text feature vector to obtain a comprehensive feature vector, and use a deep neural network to perform emotion classification based on the comprehensive feature vector;

[0244] S2. Construct an emotion database according to the input speech feature vector, expression feature vector and text feature vector, perform time series processing on the emotion database, and establish an emotion map. The user emotion map includes nodes and edges. The nodes represent emotion state nodes, and the edges represent transition edges between emotion states. Each emotion feature vector corresponds to an emotion state node. Each emotion feature vector includes a speech feature vector of the current node, an expression feature vector of the current node and a text feature vector of the current node. The transition edges between the emotion states of the emotion map are created according to the timestamp information, and then the emotion map is regularly updated according to the real-time collected data.

[0245] S3, construct an emotion history database, analyze it, extract the emotion change trend of the user, and construct an emotion fluctuation detection algorithm to identify key emotion events in the emotion history data, and generate a personalized consultation file for the user based on the emotion change trend and key emotion events; wherein the emotion fluctuation detection algorithm specifically includes:

[0246] Assume that the emotional fluctuation threshold is δ. When the emotional feature vector change of two consecutive time points exceeds the threshold, it is marked as a key emotional event. Assume that t i and t i+1 The emotional feature vectors at each moment are F i and F i+1 , then the identification condition of key emotional events is:

[0247] ||F i+1 -F i ||>δ

[0248] Among them, δ represents the threshold of emotional state change, and the emotional feature vector pairs that meet the conditions are recorded as key emotional events;

[0249] S4, predicting the emotional state of the user by constructing an emotional state prediction model based on the emotional feature vector sequence extracted from the emotional history record and personalized consultation archive, and designing an emotional prediction threshold to detect changes in the emotional feature vector sequence, and using reinforcement learning to optimize the emotional state prediction model and generate a personalized emotional regulation plan; wherein, the emotional prediction threshold is designed to detect changes in the emotional feature vector sequence, and when the change of the emotional feature vector at two consecutive time points exceeds the threshold, the emotional intervention mechanism is triggered, and t is set i+1 and t i The emotional feature vectors at each moment are F i+1 and F i , then the emotion change detection condition is:

[0250] ||F i+1 -F i ||>δ

[0251] Wherein, δ represents the threshold of emotional state change. For the significant emotional changes detected, personalized emotional intervention suggestions are generated, and the emotional intervention suggestions include emotional regulation strategies, psychological intervention measures and behavioral suggestions;

[0252] S5. Construct a deep reinforcement learning optimization model, take the current emotional state as input, pass it to the deep reinforcement learning optimization model for dynamic path adjustment, select the optimal consultation strategy and dynamic consultation path according to the output of the deep reinforcement learning optimization model, and detect the user's emotional fluctuations after each consultation and collect the user's feedback score. Collect user feedback and update the emotional database, and then use the collected user feedback and emotional data to update the Q value function of the deep reinforcement learning optimization model. Finally, generate a personalized consultation strategy based on the user's historical emotional data and predicted emotional state.

[0253] Unless otherwise specifically stated, the relative steps, numerical expressions and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present application.

[0254] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0255] In the description of the present application, it should be noted that the terms "upper", "lower", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, or the orientations or positional relationships in which the inventive product is usually placed when in use. These are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0256] In the description of this application, it should also be noted that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0257] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An artificial intelligence remote psychological consultation platform based on emotional cognition, characterized in that: The platform includes: A multimodal emotion recognition module is used to collect the user's voice data, facial expression data and text data and pre-process the data, perform multimodal feature extraction on the pre-processed data to obtain a voice feature vector, an expression feature vector and a text feature vector, then perform multimodal feature fusion on the voice feature vector, the expression feature vector and the text feature vector to obtain a comprehensive feature vector, use a deep neural network to perform emotion classification based on the comprehensive feature vector, and transmit the voice feature vector, the expression feature vector and the text feature vector to the emotion map construction module; An emotion map construction module is used to construct an emotion database according to the input voice feature vector, expression feature vector and text feature vector, and perform time series processing on the emotion database to establish an emotion map. The user emotion map includes nodes and edges, where nodes represent emotion state nodes and edges represent transition edges between emotion states, wherein each emotion feature vector corresponds to an emotion state node, and each emotion feature vector includes a voice feature vector of the current node, an expression feature vector of the current node and a text feature vector of the current node. The transition edges between the emotion states of the emotion map are created according to timestamp information, and then the emotion map is regularly updated according to the real-time collected data, and the emotion map is transmitted to the personalized consultation archive generation module; The personalized consultation file generation module is used to construct an emotional history database, analyze it, extract the user's emotional change trend, and construct an emotional fluctuation detection algorithm to identify key emotional events in the emotional history data, and generate a personalized consultation file for the user based on the emotional change trend and key emotional events; wherein the emotional fluctuation detection algorithm specifically includes: Assume that the emotional fluctuation threshold is 6. When the emotional feature vector change of two consecutive time points exceeds the threshold, it is marked as a key emotional event. Assume that t i and t i+1 The emotional feature vectors at each moment are F i and F i+1 , then the identification condition of key emotional events is: ||F i+1 -F i ||>d Among them, 6 represents the threshold of emotional state change, and the emotional feature vector pairs that meet the conditions are recorded as key emotional events; The long-term emotional state prediction and intervention module is used to predict the emotional state by constructing an emotional state prediction model based on the user's emotional feature vector sequence extracted from the emotional history record and personalized consultation archive, and at the same time design an emotional prediction threshold to detect changes in the emotional feature vector sequence, and use reinforcement learning to optimize the emotional state prediction model and generate a personalized emotional regulation plan; wherein, the emotional prediction threshold is designed to detect changes in the emotional feature vector sequence, and when the change of the emotional feature vector at two consecutive time points exceeds the threshold, the emotional intervention mechanism is triggered, and t i+1 and t i The emotional feature vectors at each moment are F i+1 and F i , then the emotion change detection condition is: ||F i+1 -F i ||>d Wherein, δ represents the threshold of emotional state change. For the significant emotional changes detected, personalized emotional intervention suggestions are generated, and the emotional intervention suggestions include emotional regulation strategies, psychological intervention measures and behavioral suggestions; The psychological counseling platform optimization module is used to build a deep reinforcement learning optimization model. It takes the current emotional state as input and passes it to the deep reinforcement learning optimization model for dynamic path adjustment. It selects the optimal counseling strategy and dynamic counseling path based on the output of the deep reinforcement learning optimization model. At the same time, it detects the user's emotional fluctuations after each consultation and collects the user's feedback score. It collects user feedback and updates the emotional database. It then uses the collected user feedback and emotional data to update the Q value function of the deep reinforcement learning optimization model. Finally, it generates a personalized counseling strategy based on the user's historical emotional data and predicted emotional state.

2. According to claim 1, an artificial intelligence remote psychological consultation platform based on emotional cognition is characterized in that: The platform also includes: The community emotion support module is used to obtain the user's current emotion feature vector and emotion classification label from the multimodal emotion recognition module and the emotion map construction module, and publish the user's own emotion status on the community platform based on the real-time collection. The emotion status includes the emotion feature vector, emotion classification label and text description, and then calculates the emotion similarity between each user to match users with similar emotions, and recommends other users with similar emotions to the user for interaction based on the user matching degree, and then stores the interaction records and emotion support scores between users in the community emotion map. At the same time, the community emotion map is updated regularly, new user nodes and interaction edges are added, and the edge weights are updated.

3. The artificial intelligence remote psychological consultation platform based on emotional cognition according to claim 1 is characterized in that: The preprocessing includes: performing noise reduction on the speech data, converting the speech signal into a spectrogram representation using short-time Fourier transform; graying and normalizing the expression image data; performing word segmentation and stop word removal on the text data, and converting the text into a word vector matrix using word embedding technology; Among them, convolutional neural networks are used to extract spectrum features; convolutional neural networks are used to extract image features; and bidirectional Transformer models are used to extract text features.

4. According to claim 1, an artificial intelligence remote psychological consultation platform based on emotional cognition is characterized in that: The transition edge between the emotional states of the emotional graph is created according to the timestamp information, specifically: Suppose two nodes V i and V j The corresponding timestamp is t i and t j , if t i <t j If the distance between the two nodes is less than the sentiment threshold δ, a path from V i To V j The directed edge E ij , which is expressed as follows: E ij =(V i →V j ) if t i <t j and ||z i -z j ||<δ; The specific steps of regularly updating the emotion map include: Let the new sentiment data be F at each update new =[F v，new ; F f，new ; F t，new ], where F v，new represents the new speech feature vector, F f，new Represents the new expression feature vector and F t，new Represents the new text feature vector, then: The new data F new =[F v，new ; F f，new ; F t，new ] is mapped to two-dimensional space and new node coordinates are generated, which are expressed as follows: from new =t-SNE(F new ) Among them, t-SNE represents -SNE dimensionality reduction technology; Add new node V new To the node set V: V=V∪{V new } Update the edge set E according to the timestamp and position of the new node: Among them, z i represents the coordinate of the i-th node in two-dimensional space, z new represents the coordinates in the new two-dimensional space, δ ratio represents the emotion threshold, t i represents the past time t, t new Indicates a new moment.

5. According to claim 1, an artificial intelligence remote psychological consultation platform based on emotional cognition is characterized in that: In the personalized consultation archive generation module, the emotion history database is analyzed to extract the emotion change trend of the user, including: Assume that the user's emotional feature vector sequence is {F1, F2, ..., F n }, the sentiment classification label sequence is {y1, y2, ..., y n }, use the weighted moving average method to calculate the user's emotional change trend; set the window size to w, the weighted moving average of the emotional feature vector It is expressed as follows: Among them, w i represents the weight within the window, w represents the number of windows, and F i Represents the sentiment feature vector of the jth user.

6. The artificial intelligence remote psychological consultation platform based on emotional cognition according to claim 1 is characterized in that: The emotional state prediction model is a long short-term memory network, which specifically includes: Let the parameter of the long short-term memory network be θ lstm , then the output of the emotional state prediction model is expressed as follows: {F′ n+1， F′ n+2， ...,F′ n+m }=LSTM({F′1,F′2,...,F′ n };θ lstm ) Among them, {F′1, F′2, ..., F′ n } represents the normalized emotional feature vector sequence; {F′1, F′2, ..., F′ n } represents the predicted sentiment feature vector output; Add an emotion fluctuation regularization term to the emotion state prediction model to ensure that the model can capture subtle fluctuations in emotion changes. Set the emotion fluctuation coefficient to λ w , the sentiment fluctuation regularization term is expressed as follows: in, represents the sentiment fluctuation regularization term, λ w represents the weight of the regularization term of sentiment fluctuation, and n represents the total number of sentiment types.

7. The artificial intelligence remote psychological consultation platform based on emotional cognition according to claim 1 is characterized in that: In the optimization module of the psychological counseling platform, a deep reinforcement learning optimization model is used for path optimization. The state s of the deep reinforcement learning optimization model t Including the current sentiment feature vector F t and sentiment classification label y t , action a t For the next step of consulting strategy selection, the reward function R(s t , a t ) is designed in combination with user feedback and consulting effects to maximize the improvement of user emotions, and the user emotion fluctuation term λ is introduced into the reward function e and the behavioral bias term λ b , capturing subtle changes in user emotions and the degree of deviation from normal behavior, expressed as follows: R(s t ,a t )=α△yt+β F b -l e ||F t -F t-1 ||-l b B t Among them, △y t Indicates the degree of improvement of sentiment classification labels, F b represents the user feedback score, α and β are weight coefficients, and λ e is the sentiment fluctuation coefficient, λ b is the behavioral deviation coefficient, B t is the behavioral deviation item; According to the output of the deep reinforcement learning optimization model, select the optimal consulting strategy a t , and dynamically adjust the consultation path, assuming that the Q value function of the optimization model is Q(s t , a t ), select action a t To maximize the Q value, it is expressed as follows The consulting strategy that will be selected is a t Applied to current emotional state t , generate new consultation paths.

8. The artificial intelligence remote psychological consultation platform based on emotional cognition according to claim 7 is characterized in that: After each consultation, the user's emotional fluctuations are detected and the user's feedback score Fb is collected. Let the emotional feature vector at the current time t be F t , the emotional feature vector after the last consultation is F t-1 , where the emotion fluctuation detection formula is as follows: W t =||F t -F t-1 || If W t Exceeding the preset emotional fluctuation threshold δ w , it is recorded as a significant emotional fluctuation, and a feedback mechanism is generated. At the same time, the user's feedback score Fb and the emotional feature vector F after consultation are recorded. t+1 and sentiment classification label y t+1 Update to sentiment database; Use the collected user feedback and sentiment data to update the Q-value function of the deep reinforcement learning optimization model, where the Q-value function update formula is expressed as follows: Among them, γ represents the discount factor and α represents the learning rate; According to the user's historical emotional data and predicted emotional state, a personalized consulting strategy is generated, wherein the personalized consulting strategy includes psychological intervention measures, and the selection of the consulting strategy is set based on the emotional feature vector and the emotional classification label, and the strategy that best suits the user's current emotional state is selected. The selection formula of the personalized consulting strategy is expressed as follows: Among them, S t represents the consulting strategy selected at time t, P(s|F t ,y t ) represents the current sentiment feature vector F t and sentiment classification label y t Next, the probability of strategy s.

9. The artificial intelligence remote psychological consultation platform based on emotional cognition according to claim 2 is characterized in that: In the community emotional support module, users publish their emotional states on the community platform, including the emotional feature vector F t , sentiment classification label y t and text description, where the emotional status publishing format is: P t ={F t ,y t ,T t } Among them, T t Represents the user's current emotional description text; Calculate the sentiment similarity between each user: Assume that the emotional feature vectors of two users at time t are and The sentiment classification labels are and The similarity The calculation formula is as follows: Among them, · represents the vector inner product, ||·|| represents the Euclidean norm of the vector, and 6 represents the Kroneckerdelta function. hour otherwise λ c is the classification label similarity weight; According to the calculated emotional similarity, other users with similar emotions are recommended for interaction. Let the set of emotionally similar users of user i be M i , then the matching algorithm selects the first k users with the highest similarity, which is expressed as follows: M i ={j|j∈Top-k(S i，j ),j≠i} Among them, Top-k(S i，j ) represents the similarity S i，j The top k users At the same time, there is an emotional support scoring mechanism, assuming that user i’s support score for user j is R i,j , users rate each other's emotional support after interaction, and the rating mechanism formula is as follows: R i,j =Feedback i,j ·S i，j Among them, Feedback i,j represents the subjective feedback rating of user i to user j; Update the community sentiment graph regularly, add new user nodes and interaction edges according to the updated community sentiment graph, and update the edge weights.

10. An artificial intelligence remote psychological consultation method based on emotional cognition, characterized in that: The method comprises: S1. Collect the user's voice data, facial expression data and text data and pre-process the data, perform multimodal feature extraction on the pre-processed data to obtain a voice feature vector, an expression feature vector and a text feature vector, then perform multimodal feature fusion on the voice feature vector, the expression feature vector and the text feature vector to obtain a comprehensive feature vector, and use a deep neural network to perform emotion classification based on the comprehensive feature vector; S2. Construct an emotion database according to the input speech feature vector, expression feature vector and text feature vector, perform time series processing on the emotion database, and establish an emotion map. The user emotion map includes nodes and edges. The nodes represent emotion state nodes, and the edges represent transition edges between emotion states. Each emotion feature vector corresponds to an emotion state node. Each emotion feature vector includes a speech feature vector of the current node, an expression feature vector of the current node and a text feature vector of the current node. The transition edges between the emotion states of the emotion map are created according to the timestamp information, and then the emotion map is regularly updated according to the real-time collected data. S3, construct an emotion history database, analyze it, extract the emotion change trend of the user, and construct an emotion fluctuation detection algorithm to identify key emotion events in the emotion history data, and generate a personalized consultation file for the user based on the emotion change trend and key emotion events; wherein the emotion fluctuation detection algorithm specifically includes: Assume that the emotional fluctuation threshold is δ. When the emotional feature vector change of two consecutive time points exceeds the threshold, it is marked as a key emotional event. Assume that t i and t i+1 The emotional feature vectors at each moment are F i and F i+1 , then the identification condition of key emotional events is: ||F i+1 -F i ||>d Among them, 6 represents the threshold of emotional state change, and the emotional feature vector pairs that meet the conditions are recorded as key emotional events; S4, predicting the emotional state of the user by constructing an emotional state prediction model based on the emotional feature vector sequence extracted from the emotional history record and personalized consultation archive, and designing an emotional prediction threshold to detect changes in the emotional feature vector sequence, and using reinforcement learning to optimize the emotional state prediction model and generate a personalized emotional regulation plan; wherein, the emotional prediction threshold is designed to detect changes in the emotional feature vector sequence, and when the change of the emotional feature vector at two consecutive time points exceeds the threshold, the emotional intervention mechanism is triggered, and t is set i+1 and t i The emotional feature vectors at each moment are F i+1 and F i , then the emotion change detection condition is: ||F i+1 -F i ||>d Wherein, δ represents the threshold of emotional state change. For the significant emotional changes detected, personalized emotional intervention suggestions are generated, and the emotional intervention suggestions include emotional regulation strategies, psychological intervention measures and behavioral suggestions; S5. Construct a deep reinforcement learning optimization model, take the current emotional state as input, pass it to the deep reinforcement learning optimization model for dynamic path adjustment, select the optimal consultation strategy and dynamic consultation path according to the output of the deep reinforcement learning optimization model, and detect the user's emotional fluctuations after each consultation and collect the user's feedback score. Collect user feedback and update the emotional database, then use the collected user feedback and emotional data to update the ω value function of the deep reinforcement learning optimization model, and finally generate a personalized consultation strategy based on the user's historical emotional data and predicted emotional state.

Images