Psychological state prediction system and method based on large language model

Abstract

The invention relates to the technical field of psychological state prediction, and discloses a psychological state prediction system and method based on a large language model, and the method comprises the steps: inputting multi-modal data into a trained emotion analysis model, outputting an emotion analysis result of a subject, and judging whether the emotion analysis results of the subject at different moments are consistent or not; if not, the emotion of the subject fluctuates; according to the change amplitude of the emotion intensity score, determining an emotion fluctuation amplitude; when the emotion fluctuation amplitude exceeds a set threshold value, it is considered that the psychological defense mechanism of the subject is broken through; selecting a new normal form set, and updating the new normal form set into the large language model to obtain an updated large language model; the emotion of the subject does not fluctuate any more; all dialogue records of the subject in the process of using the large language model are obtained, and a psychological diagnosis report is generated based on all the dialogue records. The psychological defense mechanism of a subject can be broken through, and more accurate psychological measurement and personality side writing are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of psychological state prediction, and in particular to a psychological state prediction system and prediction method based on a large language model. Background Art

[0002] Psychological defense is a physiological defense mechanism that aims to protect individuals from psychological pain and stress through unconscious psychological activities. However, this defense mechanism often becomes a major obstacle in the process of psychological measurement and psychological diagnosis.

[0003] China's invention patent publication number is CN113035232B, and the patent name is a psychological state prediction system, method and device based on speech recognition. It uses big data and deep technology to directly derive a psychological prediction model in combination with psychologists' diagnosis cases, and conducts psychological treatment. At the same time, sampling is used to correct the samples used to establish the model in order to make the model more accurate.

[0004] Chinese invention patent publication number CN111667926B, a psychological counseling conversation system and method based on artificial intelligence, the input module collects user conversation input; the user conversation input is analyzed for intent by using the psychological knowledge graph module and the multi-round dialogue management module; the input module and the language analysis module iterate in human-computer interaction, and the relationship between multi-round dialogues is managed by the multi-round dialogue management module; the language analysis module organizes and guides the user to conduct the logical direction of multi-round dialogues under the management of the multi-round dialogue management module according to the logical flow of the intelligent psychological counseling logic tree and the intention analysis of the user's dialogue; combined with the conversation information between the robot and the user, the information related to the user's psychological and emotional problems during the dialogue is fed back and output to formulate a psychological intervention plan.

[0005] In practice, it is difficult for existing scales to bypass psychological defenses, and some groups of people have a higher proportion of people with strong psychological defenses, which makes it impossible to accurately diagnose mental illnesses, conduct personality profiling, and other tasks. Summary of the invention

[0006] In order to solve the problem of low reliability and validity caused by psychological defense in the traditional psychological measurement process, the present invention provides a psychological state prediction system and prediction method based on a large language model; through the collaborative work of the large language model and the multimodal emotion classification large model, it is possible to break through the psychological defense mechanism of the subject and achieve more accurate psychological measurement and personality profiling.

[0007] On the one hand, a psychological state prediction system based on a large language model is provided, including: an initialization module, which is configured to: obtain a conversation record generated during a conversation between a subject and a trained large language model, and confirm a preliminary personality analysis result of the subject based on the conversation record; select a corresponding paradigm set based on the preliminary personality analysis result of the subject; a real-time acquisition module, which is configured to: obtain multimodal data generated by the subject when using the corresponding paradigm set, and pre-process the multimodal data; a sentiment analysis module, which is configured to: input the pre-processed multimodal data into the trained sentiment analysis model, and output the sentiment analysis result of the subject at the tth moment, where t is a positive integer; a model fine-tuning module, which is configured to: select a new paradigm set based on the amplitude of sentiment fluctuations; a repetition module, which is configured to: repeat the work of the real-time acquisition module, the sentiment analysis module and the model fine-tuning module until the subject's sentiment no longer fluctuates; obtain all conversation records of the subject in the process of using the large language model, and generate a psychological diagnosis report based on all conversation records.

[0008] On the other hand, a psychological state prediction method based on a large language model is provided, including: (201) obtaining a conversation record generated during a conversation between a subject and a trained large language model, and confirming a preliminary personality analysis result of the subject based on the conversation record; selecting a corresponding paradigm set based on the preliminary personality analysis result of the subject; (202) obtaining multimodal data generated when the subject uses the corresponding paradigm set, and preprocessing the multimodal data; (203) inputting the preprocessed multimodal data into a trained emotion analysis model, and outputting the emotion analysis result of the subject at the tth moment, where t is a positive integer; (204) selecting a new paradigm set based on the amplitude of emotion fluctuations; (205) repeating steps (202) to (204) until the subject's emotions no longer fluctuate; obtaining all conversation records of the subject during the process of using the large language model, and generating a psychological diagnosis report based on all conversation records.

[0009] The above technical solution has the following advantages or beneficial effects: The present invention proposes an active induction method based on a large language model, combined with a multimodal emotion classification model, through virtual reality, video stimulation and other paradigms, it can break through psychological defenses and conduct more accurate psychological diagnosis and personality profiling. Specifically, the present invention conducts a full range of psychological measurements on subjects through personalized interviews with a large language model, combined with physiological data perception and emotion analysis models, and ultimately achieves more accurate psychological diagnosis and personalized intervention. Through this method, not only can the accuracy of psychological measurements be improved, but also without infringing on the privacy of the subjects, their psychological state and personality traits can be deeply understood, and personalized mental health services can be provided, which has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0011] Figure 1 This is a system function module diagram of embodiment 1. DETAILED DESCRIPTION

[0012] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0013] As an advanced natural language processing technology, the large language model can understand and generate natural language text through learning large-scale text data. The active induction method based on the large language model can interact with the subject through dialogue, using the understanding and generation capabilities of the large language model to actively induce the subject's personality traits, thereby bypassing their psychological defense mechanisms, thereby obtaining more real and comprehensive personality information.

[0014] Embodiment 1: Figure 1 As shown, this embodiment provides a psychological state prediction system based on a large language model, including: an initialization module, which is configured to: obtain a conversation record generated during a conversation between a subject and a trained large language model, and confirm a preliminary personality analysis result of the subject based on the conversation record; and select a corresponding paradigm set based on the preliminary personality analysis result of the subject.

[0015] The real-time acquisition module is configured to: obtain the multimodal data generated by the subject when using the corresponding paradigm set, and pre-process the multimodal data.

[0016] The emotion analysis module is configured to: input the preprocessed multimodal data into the trained emotion analysis model, and output the emotion analysis result of the subject at the tth moment, where t is a positive integer.

[0017] The model fine-tuning module is configured to select a new set of paradigms according to the amplitude of emotion fluctuations.

[0018] The repetition module is configured to: repeat the work of the real-time acquisition module, the emotion analysis module and the model fine-tuning module until the subject's emotions no longer fluctuate; obtain all conversation records of the subject in the process of using the large language model, and generate a psychological diagnosis report based on all conversation records.

[0019] Furthermore, the conversation record generated during the conversation between the subject and the trained large language model is obtained, and the trained large language model is implemented using ChatGLM.

[0020] The training process of the trained large language model includes: constructing a first training set, which is known conversation context data; taking the previous conversation as the input value of the large language model, taking the following conversation as the output value of the large language model, training the large language model, and stopping the training when the first loss function value of the large language model no longer decreases, or the number of iterations exceeds a set number, to obtain the trained large language model.

[0021] It should be understood that for the large language model data sorting task, multimodal data integration is carried out, and natural conversations, emotion-labeled conversations, and online public psychological counseling cases are sorted into the form of data pairs.

[0022] The interview model obtains the basic information, emotional state, preference characteristics and preliminary psychological state of the subject through interaction with the subject. It uses the reasoning and analysis ability of the large language model to analyze the subject's language expression and decision-making method (such as the logic of answering questions, emotional words, decision-making style, etc.), break through the subject's psychological defense mechanism, and can identify their potential psychological problems or personality traits. Based on the content of the conversation and the results of emotion detection, it selects the most suitable paradigm for the current user's emotion induction and classification from the preset paradigm set. ChatGLM is used as the infrastructure.

[0023] ChatGLM is a powerful natural language processing model that can understand and generate complex natural language texts. It is suitable for interactive interviews with subjects to obtain their emotions and feature information. First, the dataset is preprocessed to remove noise data, such as non-text characters, and the text is segmented and the segmentation results are encoded into a format acceptable to the model. Secondly, the model is pre-trained using large-scale text data. Through training the model, the model can learn conversation habits and semantic information, understand and generate personalized conversations, and then fine-tune the pre-trained model by adding a linear layer for paradigm selection. The fine-tuning process will use emotionally labeled conversations and question-and-answer data from public mental health forums or websites to enable the model to understand the emotions of the human world and have a certain degree of professionalism in the fields of psychological counseling and personality profiling. It analyzes the potential psychological problems or personality traits of the subjects and autonomously selects the most appropriate stimulation paradigm based on the content of the conversation.

[0024] Furthermore, the acquisition of the conversation record generated during the conversation between the subject and the trained large language model, wherein the trained large language model acquires the conversation context input by the subject, and the trained large language model outputs the conversation context.

[0025] Furthermore, confirming the preliminary personality analysis result of the subject based on the conversation record specifically includes: inputting the conversation record into a trained personality classification model to obtain a personality classification result; the trained personality classification model is implemented using a support vector machine model; the training process of the trained personality classification model includes: constructing a second training set, the second training set is a conversation record with a known personality classification result; the personality classification result includes scores of five dimensions, namely, extroversion, openness to experience, conscientiousness, agreeableness, and neuroticism.

[0026] Extraversion refers to the degree of activity and enthusiasm an individual exhibits in social situations. People with strong extraversion are often sociable, lively, outgoing, like to communicate with others, and get energy from social activities; people with weak extraversion are often less sociable, quiet, introverted, prefer to be alone or in a small circle, and get energy from being alone.

[0027] Openness refers to an individual's open attitude towards new things and new experiences. People with strong openness are often characterized by strong curiosity and creativity, and a willingness to try different activities and ideas.

[0028] Conscientiousness refers to the sense of responsibility, organization, and self-discipline displayed by an individual. People with strong conscientiousness tend to be organized, pay attention to details, and have strong self-discipline.

[0029] Agreeableness refers to an individual's tendency to be friendly, sympathetic, helpful and cooperative. Individuals with high agreeableness can easily establish harmonious interpersonal relationships with others.

[0030] Neuroticism refers to the degree of an individual's emotional stability, which involves anxiety, depression, impulsivity, self-awareness and the ability to regulate emotions. Individuals with high levels of neuroticism tend to be emotionally unstable and are prone to experiencing negative emotions such as anxiety, depression, anger, etc., and have a low tolerance for stress.

[0031] The second training set is input into the personality classification model to train the model. When the second loss function value of the model no longer decreases or the number of iterations exceeds the set number, the training is stopped to obtain the trained personality classification model.

[0032] Furthermore, according to the preliminary personality analysis results of the subjects, the corresponding paradigm set is selected, specifically including: for subjects with strong extroversion in the preliminary personality analysis results, a paradigm set that promotes social interaction and active participation is selected to further activate their traits; the paradigm set that promotes social interaction and active participation includes: interactive paradigms (such as role-playing social interaction tasks) and social virtual reality, etc. By observing the performance of extroverted individuals in virtual social interactions, their extroverted traits can be truly reflected; for subjects with weak extroversion, a paradigm set that focuses on individual reflection and quiet environment is selected, including scale measurement and simulation of low-stimulation social scenes, or playing more soothing videos (such as videos containing natural landscapes). These paradigms can help individuals with weak extroversion express emotions in a comfortable environment. In addition, for the virtual reality paradigm, the behavior and reaction of individuals with weak extroversion in non-social situations can be evaluated by setting independent tasks in it.

[0033] For subjects with high openness, select a set of novel and creative paradigms, including unique creative games (such as fluid intelligence tasks). These paradigms can satisfy the desire for new experiences of open individuals.

[0034] For conscientious subjects, a set of structured and ordered paradigms, strategic planning and task management interactive games (such as the Conscientiousness Game) were selected to observe the planning and execution abilities of conscientious individuals. These paradigms are consistent with the organization and self-discipline of conscientious individuals.

[0035] For agreeable subjects, a set of paradigms emphasizing cooperation and empathy were selected, including cooperative interactive games and touching videos, etc. These paradigms can promote social behavior and emotional expression in agreeable individuals.

[0036] For neurotic types, choose the Depression Anxiety Stress Scale for assessment, or choose to play videos of different emotional types, etc., or choose stress games (such as cold stress task, emotional Stroop task, etc.) to assess the individual's decision-making and emotional control ability in stressful situations.

[0037] It should be understood that the large language model is pre-trained and fine-tuned using multimodal data, so that interactive questions and answers related to personality induction can be achieved. During the fine-tuning process, the description of the paradigm is input into the large language model, where the input should be a description of the role of the paradigm, and the output should be the name of the paradigm itself and the link to the paradigm port. In this way, in the process of model thinking chain reasoning, when the model generates a description of the subject's psychological state, it can naturally infer the required paradigm. In this way, during the test, the large language model obtains the subject's basic information, preference characteristics and preliminary psychological state through interaction with the subject, and can autonomously select a suitable stimulation paradigm based on the subject's real-time response, and actively induce the subject's personality traits.

[0038] It should be understood that the paradigm set specifically includes: a scale, an interactive game, a virtual reality video and a stimulation video, and the scale is a tool for measuring the emotional state of the subject.

[0039] The interactive game is to obtain the emotional response of the subject through human-computer interaction games, which may include but are not limited to memory games (digital sequence memory, emotional vocabulary memory, picture memory, etc.), decision-making interactive games (reward games, maze games), and attention games (Flanker task, Stroop task, etc.).

[0040] The virtual reality video provides an immersive experience through virtual reality to stimulate the emotions of the subjects, which may include but is not limited to social scenario simulation (simulating social interaction scenarios to evaluate the emotional responses and behaviors of the subjects in different social situations), horror and adventure scenarios (simulating horror or adventure scenarios to stimulate strong stress responses).

[0041] The stimulation videos are designed to guide the emotional changes of the subjects through video content, including but not limited to emotion-inducing videos (such as videos of joy, anger, sadness, fear, etc.), relaxation videos (such as natural scenes, natural landscapes, etc.), and emotional resonance videos (playing video clips that can cause emotional resonance, such as family, friendship and love stories, and observing the emotional reactions and empathy abilities of the subjects).

[0042] For example, the paradigm set is used to actively induce and detect the emotional state of the subject, such as happiness, anger, sadness, surprise, fear, disgust and neutral emotions. The paradigm set includes a variety of stimulation methods, such as scales, games, virtual reality and video stimulation. These paradigms can provide different subjects with emotion induction methods suitable for them, thereby improving the accuracy and effectiveness of emotion analysis.

[0043] First, the scale is a standardized emotion scale, including the Positive and Negative Affect Scale (PANAS), the Depression Anxiety Stress Scale (DASS), the Emotional Vocabulary, etc. The subject's current emotional state is assessed through self-report, and the system can automatically record and analyze these responses.

[0044] Secondly, specific game scenarios are designed to induce various emotions in the subjects, such as happiness, tension, anger, and surprise. Through interaction and feedback mechanisms, the emotional responses of the subjects are induced. The system monitors the physiological and behavioral data of the subjects in real time through multi-mode integrated equipment.

[0045] Again, virtual reality technology is used to create realistic virtual environments, such as horror scenes, relaxing natural landscapes, etc., to induce strong emotional responses through immersive experiences.

[0046] Finally, play specific emotional video clips, such as emotional resonance videos, thriller clips, relaxation videos, etc., to induce emotional responses in the subjects through visual and auditory stimulation, and record the physiological and behavioral responses of the subjects. Different paradigm sets can have different stimulation effects and should be selected according to the individual situation of the subjects.

[0047] For the task of organizing data for big model training, organize natural conversation data and emotion-labeled conversation data. This step can enable the big model to understand the language habits of the human world and conduct interactive conversations; collect data from public mental health forums or Q&A websites, such as the mental health consultation Q&A on the "Ding Xiang Doctor" website. This can give the big model a solid foundation in the field of psychological counseling and personality profiling, and better understand interview tasks.

[0048] Furthermore, the multimodal data generated by the subjects when using the corresponding paradigm set is obtained, and the multimodal data is preprocessed, wherein the multimodal data includes: expression data, text data and voice data; wherein the multimodal data is preprocessed, including: performing noise filtering processing, feature extraction processing, data normalization processing and facial expression data encoding processing on the expression data in sequence; and performing word form normalization processing, meaningless word removal processing and word segmentation processing on the text data in sequence.

[0049] It should be understood that multi-modal integrated devices including cameras, touch screens, microphones, etc. are used for physiological perception, real-time monitoring and collection of multi-modal data such as expressions, text, and voice of subjects when applying the stimulation paradigm, and pre-processing of the data, including noise filtering, feature extraction, and data standardization. For facial videos, FACS (Facial Action Coding System) is used to encode facial behavior; voice signals are enhanced and noise is eliminated; text information is cleaned, word form normalization, meaningless words are removed, analyzed, and segmented.

[0050] It should be understood that multimodal data of the subject, including physiological data and behavioral data, is collected in real time when different paradigms of stimulation are performed, and the collected data is preprocessed, such as noise filtering and feature extraction.

[0051] Furthermore, the preprocessed multimodal data is input into the trained emotion analysis model, and the emotion analysis result of the subject at the tth moment is output, which specifically includes: the trained emotion analysis model, the training process includes: constructing a third training set, the third training set is multimodal data with known emotion classification labels and emotion intensity scores; the emotion classification labels include: pleasure, anger or sadness; the emotion intensity score is divided into 1 to 10 points; the third training set is input into the emotion analysis model, the model is trained, and when the third loss function value of the model no longer decreases, or the number of iterations exceeds the set number, the training is stopped to obtain the trained emotion analysis model.

[0052] Furthermore, the trained emotion analysis model includes: a video encoder, a text encoder and a speech encoder; the input value of the video encoder is video data; the input value of the text encoder is text data; the input value of the speech encoder is speech data; the video encoder is used to encode the video data; the text encoder is used to encode the text data, and the speech encoder is used to encode the speech data; the output end of the video encoder, the output end of the text encoder and the output end of the speech encoder are all connected to the input end of the attention mechanism layer, the attention mechanism layer is used to output the weight of each input value, the output end of the attention mechanism layer is connected to the input end of the fusion unit, and the output end of the fusion unit is the output end of the emotion analysis model; the fusion unit performs weighted summation processing on the input values.

[0053] It should be understood that the trained emotion analysis model is used to classify and locate emotions in the preprocessed multimodal data, and to analyze the emotional response patterns of the subjects during paradigm stimulation (such as response to stress, emotion regulation ability, etc.) in real time to further improve the personality profile.

[0054] For the task of organizing data for sentiment analysis model training and for the task of organizing video data with sentiment labels, videos and their corresponding sentiment labels are obtained from Aff-Wild2. Aff-Wild2 is a large sentiment database that contains a variety of sentiment videos collected from YouTube, with 7 basic emotions and sentiment intensities annotated. This allows the large model to learn the correspondence between facial expressions and reactions and different emotions. For the task of organizing speech data with sentiment labels, speech data and their corresponding sentiment labels are obtained from the CASIA Chinese sentiment speech database. This step allows the large model to learn the correspondence between the speech signals input by the subjects and the sentiment labels. For the task of organizing text data with sentiment labels, text data and their corresponding sentiment labels are obtained from the NLPCC 2014 dataset.

[0055] The video, voice, and text data with emotion labels are used as the input of the emotion analysis model, aiming to use multimodal emotional data to comprehensively judge the emotional state and conduct accurate analysis of emotions. This step will use the multimodal Transformer (MuIT) as the model framework, which expands the standard Transformer network and can directly learn representations from unaligned multimodal streams. In the present invention, multimodal physiological and behavioral signals including video, voice, and text after preprocessing are used as model input. In the MuIT model, the present invention uses Inflated 3D ConvNet as a video encoder, BERT as a text encoder, and Wav2Vec as a voice encoder, which can convert voice signals into embedded representations. Use attention mechanisms (such as Cross Attention) to establish associations between different modalities and align the embedded representations of different modalities. Use the Concat method to fuse the aligned multimodal embeddings. Finally, based on pre-training, the model is fine-tuned to further improve its performance in emotion analysis tasks. Through model training and fine-tuning, multimodal data is learned and integrated, the performance of the sentiment analysis model in multimodal tasks is improved, and emotions are accurately identified and analyzed. This step enables the large model to analyze and locate emotions based on multimodal physiological data of subjects.

[0056] Furthermore, the model fine-tuning module is also configured to: determine whether the emotion analysis result of the subject at the tth moment is consistent with the emotion analysis result at the t-1th moment; if they are inconsistent, it indicates that the subject's emotions fluctuate; determine the amplitude of the emotion fluctuation according to the change amplitude of the emotion intensity score; when the emotion fluctuation amplitude exceeds the set threshold, it is considered that the subject's psychological defense mechanism has been broken; when the emotion fluctuation amplitude does not exceed the set threshold, it is considered that the subject's psychological defense mechanism has not been broken temporarily.

[0057] Furthermore, the determination of whether the emotion analysis result of the subject at the tth moment is consistent with the emotion analysis result at the t-1th moment includes: if the emotion analysis result at the tth moment is the A emotion classification label, and the emotion analysis result at the t-1th moment is the B emotion classification label, then the two are inconsistent; if the emotion analysis result at the tth moment is the A emotion classification label and the emotion intensity score is C, and the emotion analysis result at the t-1th moment is the A emotion classification label and the emotion intensity score is D, then the two are inconsistent; if the emotion analysis result at the tth moment is the A emotion classification label and the emotion intensity score is C, and the emotion analysis result at the t-1th moment is the A emotion classification label and the emotion intensity score is C, then the two are consistent.

[0058] Furthermore, determining the amplitude of emotion fluctuation according to the amplitude of change of the emotion intensity score includes: taking the change difference of the emotion intensity score as the amplitude of emotion fluctuation.

[0059] It should be understood that according to the results of sentiment analysis, the interview model is fine-tuned to dynamically adjust the conversation content and the selection of paradigm sets, optimize the active personality induction effect, and gradually form a comprehensive diagnosis of the subject's personality and psychology.

[0060] Furthermore, when the amplitude of emotional fluctuation exceeds the set threshold, it is considered to have broken through the subject's psychological defense mechanism. The set threshold refers to the emotional intensity score, which differs by 5 to 6 points.

[0061] Furthermore, according to the amplitude of emotional fluctuations, a new paradigm set is selected, specifically including: a new paradigm set including: video stimulation paradigm (including movie clips of joy, anger, sorrow, fear, etc.), music and sound stimulation paradigm (including natural sounds, red and white noise, etc.), and text paradigm (including reading aloud and memorizing stories and vocabulary with strong emotional colors).

[0062] When the fluctuation amplitude of emotional intensity is greater than the set threshold, a relatively gentle paradigm is selected. The large model uses gentle conversation and a relaxing paradigm (such as red noise) to ease the emotions of the subjects.

[0063] When the fluctuation amplitude of emotion intensity is less than the set threshold, other paradigms of the same elicitation task are reselected from the paradigm library. If more than three paradigms still cannot break through the psychological defense, the process should be terminated.

[0064] Furthermore, the system also includes: when the emotion classification label changes, it is considered that the psychological defense mechanism of the subject has been broken through, and a new paradigm set is selected according to the updated emotion classification label.

[0065] Furthermore, selecting a new paradigm set according to the updated emotion classification labels includes: for the updated emotion classification labels, when the emotion intensity of the subject exceeds a set threshold and the appearance order of the stimulus paradigm labels is consistent with the subject's emotion labels, entering the next emotion stimulation.

[0066] When the subject's emotions are consistent with the stimulus paradigm label and the emotional intensity exceeds the set threshold for more than four times, it is considered that the psychological defense has been broken through. At this time, psychological induction can improve reliability and validity.

[0067] After each paradigm is applied, the results generated by the emotion analysis model include: emotion categories (such as pleasure, anger, sadness, etc.) and emotion intensity information, which are passed to the interview model in a structured form for fine-tuning. The model iteratively adjusts according to the subject's response (including language and emotional changes), selects a new paradigm that is more suitable for the subject or continues to delve into the current topic. The subject's decision-making obtained through the interview model dialogue and the subject's emotions detected by the multimodal emotion classification model are combined to construct a comprehensive personality profile. The large language model uses this data to continuously adjust and optimize the dialogue strategy and paradigm selection, and can gradually deepen and comprehensively profile the subject's psychology to ensure the accuracy and effectiveness of psychological measurement.

[0068] Furthermore, the subject's emotions no longer fluctuate, which means that the emotion type no longer changes, and the fluctuation range of the emotion intensity value is within the set threshold range.

[0069] Furthermore, the method of obtaining all conversation records of the subject in the process of using the large language model and generating a psychological diagnosis report based on all conversation records specifically includes: inputting all conversation records into the trained psychological diagnosis report generation model, and outputting the psychological diagnosis report; the trained psychological diagnosis report generation model is implemented using ChatGPT.

[0070] The trained psychological diagnosis report generation model, the training process includes: constructing a fourth training set, the fourth training set is a conversation record of a known psychological diagnosis report; inputting the fourth training set into the psychological diagnosis report generation model, training the model, when the fourth loss function value of the model no longer decreases, or when the number of iterations exceeds the set number, stopping the training, and obtaining the trained psychological diagnosis report generation model.

[0071] For the implementation of the interview language model, ChatGLM will be used as the infrastructure. The datasets including natural conversations, emotion-labeled conversations, subject characteristics, public mental health forums, and website content should be collated, including the description of the data, the input of the model, and the corresponding expected output.

[0072] Among them, the present invention uses the CMRC 2018 Chinese machine reading comprehension evaluation dataset as a natural dialogue dataset; uses SMP2019-EWECT and Chinese Emotional Conversation Dataset (CECD) as emotion annotation dialogue datasets; uses public demographic data and psychological characteristic datasets as subject characteristic datasets; and uses the Ren-CECps Chinese emotion dataset as an emotion and personality research dataset. The above datasets are used to train and fine-tune the infrastructure model. The trained and fine-tuned model should be able to interact with people fluently in natural language, and be able to achieve preliminary personality diagnosis and psychological profiling of the subjects, and generate personalized paradigm selection for the subjects, actively inducing the personality traits of the subjects. Joint cross entropy loss function : ; N represents the number of tasks, in this task N is 2; is the weight coefficient of the i-th task, indicating the contribution of the task to the total loss, which can be 0.5; is the number of categories of the i-th task; is the one-hot encoding of the sample target value, which is 1 if the sample belongs to the cth class and 0 otherwise; is the probability of the cth class predicted by the model.

[0073] In addition, a joint cross entropy loss function is designed to use the output results of the sentiment analysis model for real-time model fine-tuning, combining the loss of the interview model with the loss of the sentiment analysis model for joint optimization. A cyclic training mode is designed. In each round of training, the interview model is adjusted according to the sentiment analysis results, and the updated model parameters are used in the next round of training. In this way, the model is continuously iterated to adjust and optimize the dialogue strategy and paradigm selection. In the actual dialogue process, the online learning method is used to dynamically adjust the parameters of the interview model so that it can continuously adapt to the emotional changes and dialogue needs of the subjects. Through multiple rounds of dialogue and sentiment analysis feedback, the interview model should be able to make paradigm selections that are more personalized and more suitable for the subjects, and gradually form a comprehensive diagnosis of the subject's personality and psychology.

[0074] Secondly, the construction of the sentiment analysis model includes the following specific requirements and solutions: Multimodal Transformer (MuIT) is used as the basic architecture of the model, and video, voice, and text data with emotion labels are used as input for model training.

[0075] During model training, the video data with emotion labels is encoded through the Inflated 3D ConvNet video encoder and mapped to the tensor space; the speech data with emotion labels is encoded through the Wav2Vec speech encoder and mapped to another tensor space; the text data with emotion labels is encoded through the BERT text encoder and mapped to another tensor space.

[0076] The features of each modality are processed through a convolutional layer (Conv1D) to extract local features and increase the representation capacity of the data. The processed features are added with position encoding to capture sequence information. Through the cross-modal Transformer layer, the features of different modalities pay attention to and interact with each other. After each modality is processed by the cross-modal attention module, the features of all modalities are concat processed. The concatenated features are processed through a self-attention Transformer layer to further capture the high-level features and relationships within the modality. After multiple layers of cross-modal and self-attention processing, the fused features are input to the prediction layer to generate the final prediction output. Finally, the model performance is further improved through retrieval enhancement fine-tuning. The model after this step should be able to perform real-time sentiment analysis and positioning for the multimodal input of the subject.

[0077] In actual use, the following steps are included: interact with the interview big model and select a paradigm: the subject interacts with the interview big model. Further, the interview big model conducts a preliminary personality analysis and generates a personalized paradigm selection; conducts experiments according to the selected paradigm and collects physiological data: the subject is experimented with according to the paradigm set selected in the previous step. During the experiment, multi-mode integrated devices such as cameras, microphones, and touch screens are used to collect video, voice, and text signals and perform preprocessing; the multi-modal data is input into the emotion analysis model: the pre-processed multi-modal data is used as the input of the emotion analysis model, and the emotion analysis model will generate a professional subject emotion classification and positioning result.

[0078] Joint training and optimization: Design a joint loss function, conduct joint training, and optimize emotion classification and dialogue generation. The interview model is fine-tuned each time it receives feedback from the emotion analysis, extracting the subject's features from the dialogue and emotion analysis results, including emotional fluctuations, semantic features, decision-making patterns, etc.; based on the extracted features, the subjects are classified into different personality types. This process requires continuous data collection, model training, and feedback adjustment to ensure that the model can accurately reflect the subject's emotions and personality traits. Finally, a detailed personality profile and psychological diagnosis report is generated by combining the interview records and emotion analysis results. Through multiple rounds of dialogue and emotion analysis feedback, the interview model gradually forms a comprehensive diagnosis of the subject's personality and psychology.

[0079] Embodiment 2: This embodiment provides a psychological state prediction method based on a large language model, including: (201) obtaining a conversation record generated during a conversation between a subject and a trained large language model, and confirming a preliminary personality analysis result of the subject based on the conversation record; selecting a corresponding paradigm set based on the preliminary personality analysis result of the subject; (202) obtaining multimodal data generated when the subject uses the corresponding paradigm set, and preprocessing the multimodal data; (203) inputting the preprocessed multimodal data into a trained emotion analysis model, and outputting the emotion analysis result of the subject at the tth moment, where t is a positive integer; (204) selecting a new paradigm set based on the amplitude of emotion fluctuations; (205) repeating steps (202) to (204) until the subject's emotions no longer fluctuate; obtaining all conversation records of the subject in the process of using the large language model, and generating a psychological diagnosis report based on all conversation records.

[0080] Furthermore, (204) also includes: determining whether the emotion analysis result of the subject at time t is consistent with the emotion analysis result at time t-1; if they are inconsistent, it indicates that the subject's emotions fluctuate; determining the emotion fluctuation amplitude according to the change amplitude of the emotion intensity score; when the emotion fluctuation amplitude exceeds the set threshold, it is considered that the subject's psychological defense mechanism has been broken; when the emotion fluctuation amplitude does not exceed the set threshold, it is considered that the subject's psychological defense mechanism has not been broken temporarily.

[0081] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A psychological state prediction system based on a large language model, characterized by: include: An initialization module is configured to: obtain a conversation record generated during a conversation between a subject and a trained large language model, and confirm a preliminary personality analysis result of the subject according to the conversation record; and select a corresponding paradigm set according to the preliminary personality analysis result of the subject; A real-time acquisition module is configured to: obtain multimodal data generated by the subject when using the corresponding paradigm set, and pre-process the multimodal data; The emotion analysis module is configured to: input the preprocessed multimodal data into the trained emotion analysis model, and output the emotion analysis result of the subject at the tth time, where t is a positive integer; A model fine-tuning module, which is configured to: select a new set of paradigms according to the amplitude of emotion fluctuations; The repetition module is configured to: repeat the work of the real-time acquisition module, the emotion analysis module and the model fine-tuning module until the subject's emotions no longer fluctuate; obtain all conversation records of the subject in the process of using the large language model, and generate a psychological diagnosis report based on all conversation records.

2. The psychological state prediction system based on a large language model as claimed in claim 1, characterized in that: The conversation record generated during the conversation between the subject and the trained large language model is obtained, and the trained large language model is implemented by using ChatGLM; the training process of the trained large language model includes: constructing a first training set, and the first training set is known conversation context data; The previous conversation is used as the input value of the large language model, and the following conversation is used as the output value of the large language model. The large language model is trained. When the first loss function value of the large language model no longer decreases, or the number of iterations exceeds the set number, the training is stopped to obtain the trained large language model.

3. The psychological state prediction system based on a large language model as claimed in claim 1, characterized in that: The preliminary personality analysis results of the subject are confirmed based on the conversation records, including: Input the conversation records into the trained personality classification model to obtain the personality classification results; The trained personality classification model is implemented using a support vector machine model; The training process of the trained personality classification model includes: constructing a second training set, which is a conversation record with known personality classification results; the personality classification results include scores of five dimensions, namely extraversion, openness, conscientiousness, agreeableness and neuroticism; inputting the second training set into the personality classification model, training the model, and stopping the training when the second loss function value of the model no longer decreases or the number of iterations exceeds the set number, thereby obtaining the trained personality classification model.

4. The psychological state prediction system based on a large language model as claimed in claim 1, characterized in that: The method obtains multimodal data generated by the subject when using the corresponding paradigm set, and pre-processes the multimodal data, wherein the multimodal data includes: expression data, text data and voice data; wherein the pre-processing of the multimodal data includes: performing noise filtering processing, feature extraction processing, data standardization processing and facial expression data encoding processing on the expression data in sequence; and performing word form normalization processing, meaningless word removal processing and word segmentation processing on the text data in sequence.

5. The psychological state prediction system based on a large language model as claimed in claim 1, characterized in that: The preprocessed multimodal data is input into the trained emotion analysis model, and the emotion analysis result of the subject at the tth moment is output, wherein the training process of the trained emotion analysis model includes: Construct a third training set, wherein the third training set is multimodal data with known emotion classification labels and emotion intensity scores; the emotion classification labels include: happiness, anger or sadness; the emotion intensity scores are divided into 1 to 10 points; the third training set is input into the emotion analysis model to train the model, and when the third loss function value of the model no longer decreases, or the number of iterations exceeds the set number, the training is stopped to obtain the trained emotion analysis model.

6. The psychological state prediction system based on a large language model as claimed in claim 1 or 5, characterized in that: The trained emotion analysis model includes: a video encoder, a text encoder and a speech encoder; The input value of the video encoder is video data; the input value of the text encoder is text data; the input value of the speech encoder is speech data; the video encoder is used to encode the video data; the text encoder is used to encode the text data, and the speech encoder is used to encode the speech data; The output end of the video encoder, the output end of the text encoder and the output end of the speech encoder are all connected to the input end of the attention mechanism layer. The attention mechanism layer is used to output the weight of each input value. The output end of the attention mechanism layer is connected to the input end of the fusion unit. The output end of the fusion unit is the output end of the emotion analysis model. The fusion unit performs weighted summation processing on the input values.

7. The psychological state prediction system based on a large language model as claimed in claim 1, characterized in that: The model fine-tuning module is further configured to: determine whether the emotion analysis result of the subject at the tth moment is consistent with the emotion analysis result at the t-1th moment; if they are inconsistent, it indicates that the subject's emotion fluctuates; determine the emotion fluctuation amplitude according to the change amplitude of the emotion intensity score; when the emotion fluctuation amplitude exceeds the set threshold, it is considered that the psychological defense mechanism of the subject has been broken; when the emotion fluctuation amplitude does not exceed the set threshold, it is considered that the psychological defense mechanism of the subject has not been broken temporarily; The determining whether the emotion analysis result of the subject at the t moment is consistent with the emotion analysis result at the t-1 moment includes: if the emotion analysis result at the t moment is the A emotion classification label, and the emotion analysis result at the t-1 moment is the B emotion classification label, then the two are inconsistent; if the emotion analysis result at the t moment is the A emotion classification label and the emotion intensity score is C, and the emotion analysis result at the t-1 moment is the A emotion classification label and the emotion intensity score is D, then the two are inconsistent; if the emotion analysis result at the t moment is the A emotion classification label and the emotion intensity score is C, and the emotion analysis result at the t-1 moment is the A emotion classification label and the emotion intensity score is C, then the two are consistent.

8. The psychological state prediction system based on a large language model as claimed in claim 1, characterized in that: According to the amplitude of emotional fluctuations, a new set of paradigms is selected, including: A new set of paradigms, including: video stimulation paradigm, music and sound stimulation paradigm, and text paradigm; When the fluctuation amplitude of emotional intensity is greater than the set threshold, the gentle paradigm is selected, and the large language model adopts a gentle conversation style and a relaxed paradigm to ease the subject's emotions; When the fluctuation amplitude of emotional intensity is less than the set threshold, other paradigms of the same induced task are re-selected from the paradigm library. If more than three paradigms still cannot break through the psychological defense, the process is terminated; or, The system further includes: when the emotion classification label changes, it is considered that the psychological defense mechanism of the subject has been broken, and a new paradigm set is selected according to the updated emotion classification label; the selecting a new paradigm set according to the updated emotion classification label includes: For the updated emotion classification labels, when the subject's emotion intensity exceeds the set threshold and the order of the stimulus paradigm labels is consistent with the subject's emotion labels, proceed to the next emotion stimulation; When the subject's emotions are consistent with the stimulus paradigm label and the emotional intensity exceeds the set threshold for more than four times, it is considered that the psychological defense has been broken.

9. The psychological state prediction system based on a large language model as claimed in claim 1, characterized in that: The method of obtaining all conversation records of the subject in the process of using the large language model and generating a psychological diagnosis report based on all conversation records specifically includes: inputting all conversation records into the trained psychological diagnosis report generation model to output the psychological diagnosis report; the trained psychological diagnosis report generation model is implemented using ChatGPT; the training process of the trained psychological diagnosis report generation model includes: constructing a fourth training set, the fourth training set being conversation records of known psychological diagnosis reports; inputting the fourth training set into the psychological diagnosis report generation model to train the model, and when the fourth loss function value of the model no longer decreases, or the number of iterations exceeds the set number, stopping the training to obtain the trained psychological diagnosis report generation model.

10. A method for predicting mental states based on a large language model, characterized in that: include: (1) Obtain the conversation records generated during the conversation between the subject and the trained large language model, and confirm the preliminary personality analysis results of the subject based on the conversation records; According to the results of the preliminary personality analysis of the subjects, the corresponding paradigm set was selected; (2) Obtaining the multimodal data generated by the subjects when using the corresponding paradigm set and preprocessing the multimodal data; (3) Input the preprocessed multimodal data into the trained sentiment analysis model and output the sentiment analysis result of the subject at time t, where t is a positive integer; (4) Select a new set of paradigms based on the magnitude of emotional fluctuations; (5) Repeat steps (2) to (4) until the subject's emotions no longer fluctuate; obtain all conversation records of the subject during the process of using the large language model, and generate a psychological diagnosis report based on all conversation records.

Images