749 results about "Mood" patented technology

The invention discloses an intelligent real-time emotion evaluation method for social media and online text data based on a multi-modal knowledge graph, and the method comprises the steps: firstly obtaining and preprocessing multi-modal original data; secondly, designing single-mode feature extraction of dialogue texts, voices and facial expressions based on BERT-BiLSTM, AST and ResNet-50 pre-training models; furthermore, sharing among multiple modes and private emotion semantic information are considered, and complementation of cross-mode emotion information is promoted by enhancing repeated emotion information and utilizing uniqueness of private information. Meanwhile, a multi-mode knowledge graph method is provided from bottom to top to carry out emotion real-time evaluation. Analyzing user behaviors by using DeepFM and generating user features, and capturing external knowledge layer features in combination with K-BERT; finally, on the basis of a TransformerEncoder structure, cross-modal emotion information interaction and feature fusion are achieved, real-time recognition and classification of multi-modal emotion perception are completed through a Softmax layer, and therefore the purpose of intelligent real-time emotion evaluation is achieved.

The present invention relates to a system and method for providing adaptive digital therapy, enhancing the emotional well-being of users through the analysis and translation of expressive and physiological states. Aspects of the invention are directed to identifying the neuroplastic window via user inputs, with or without the aid of psychedelics. This invention is particularly suited for supporting psychoactive therapies involving compounds such as 5-HT2A agonists, dissociatives, and empathogens, as well as applications beyond psychoactive treatments. By utilizing multi-modal deep learning methods, the system captures a wide array of user inputs, including text, speech, vision, symbols, and brain waves, to form a dynamic user representation. This representation is then used to provide real-time, tailored generative communications informed by established psychological therapeutic frameworks. The invention features a closed-loop feedback system through an access module, enabling continuous data relay, patient response metric derivation, and transmission of emotional optimization communications. The system also introduces novel communication methods, including symbolic language input, to articulate complex experiences and extends its capabilities to include natural emotional states, broadening its therapeutic applicability.

Aiming at the problems of insufficient language understanding depth, weak personalized dialogue generation ability, lack of continuous learning and long-term user state modeling and the like in the current emotion recognition and psychological intervention technology, the invention provides a user emotion recognition and psychological intervention method combined with a large language model (LLM). According to the method, the potential emotional state is identified by analyzing free text information input by a user by utilizing the powerful capabilities of a large language model in the aspects of natural language understanding, emotional modeling and text generation; constructing a multi-round dialogue context, and reasoning a psychological change trend of the user; in combination with a psychological knowledge base, personalized and mild psychological intervention dialogue content with a dredging effect is generated. The system supports recognition and classification of various emotional states such as depression, anxiety and alonity, is suitable for various interaction scenes (such as APPs, webpages and social robots), and can greatly improve the precision of emotion recognition and the timeliness and effectiveness of psychological intervention. The emotion recognition and psychological intervention method based on the large language model provides solid technical support for constructing an intelligent, continuous and personalized psychological health management system, and has wide application prospects and profound social significance.

The invention, which relates to the technical field of social emotion analysis, discloses a university social negative emotion characterization analysis system comprising a data acquisition and preprocessing module, a feature extraction and anomaly detection module, an intelligent perception and negative emotion characterization module, and an anomaly risk assessment and response module. Emotional feature information is acquired in real time based on a multi-dimensional data source, and the acquired emotional feature information is cleaned and preprocessed. According to the method, by calculating the data consistency offset coefficient and the strong correlation coefficient, the multi-collinearity problem among multi-dimensional data features is solved, and instability of the model regression coefficient is avoided. And the system identifies abnormal correlation in real time and combines risk threshold classification processing to ensure robust model output. An analysis set is established by continuously collecting data, long-term emotion trend analysis is supported, personalized responses such as reminding, intervention or emergency measures are provided according to risk levels, and accurate management and timely adjustment of student emotional states are achieved.

The invention discloses a multi-modal XR emotion interaction method, system and device and a storage medium. The method comprises the steps of collecting multi-modal data of a user through a multi-modal perception interface of an XR device; preprocessing the collected multi-modal data, and performing emotion recognition on the preprocessed multi-modal data by adopting a deep neural network to obtain an emotion recognition result; constructing an emotion knowledge base, mapping the emotion recognition result into the emotion knowledge base, and reasoning and analyzing the current emotion; and automatically classifying the emotion according to the personalized emotion model of the user, and intervening and regulating the emotion of the user according to the current emotion of the user, the personalized emotion model and an emotion regulation reinforcement learning algorithm. Omnibearing, accurate, dynamic and personalized perception and quantitative analysis of the emotional state of the user are realized, and the emotional interaction experience in an XR scene can be remarkably improved.

The invention discloses a pet emotion recognition system and method, and relates to the technical field of pet emotion recognition, and the system comprises a data acquisition module, a data analysis module, an environment analysis module, an emotion analysis module, an emotion comparison module, and an abnormality processing module. The data acquisition module is used for acquiring classified pet historical emotion data, pet emotion data acquired in real time and environment data acquired in real time, and then sorting the classified pet historical emotion data; the technical key points are as follows: analyzing and processing historical emotion data of the pet, identifying the emotion of the pet from two perspectives of sound and physiology, knowing the relationship between the emotion of the pet and the sound and physiology, and then identifying the emotion of the pet in real time based on the relationship between the emotion of the pet and the sound and physiology. The external environment and the state of the pet are analyzed in the pet emotion recognition process, the pet emotion recognition accuracy can be improved, and the good use prospect is achieved.

A system for real-time analysis of emotional feedback during motivational speeches, consisting of: a series of multimodal sensors, including at least one visual sensor configured to capture facial expressions of spectators, at least one directional microphone configured to capture the audio responses of the audience, and optionally one or more physiological sensors configured to capture biometric signals from spectators; an edge-based processing unit that is communicatively coupled to the arrangement of multimodal sensors, wherein the edge-based processing unit comprises the following: (a) a feature extraction module configured to extract visual features from captured facial images, acoustic features from voice responses, and physiological features from biometric signals; (b) an emotion inference machine configured to process the features using a deep learning-based emotion recognition model comprising a convolutional neural network (CNN) for classifying facial expressions, a recurrent neural network (RNN) for classifying voice emotions, and a multimodal late fusion layer configured to compute a composite emotion state vector representing the aggregated emotions of the audience; (c) a timestamp and speech alignment module configured to correlate the calculated composite emotion state vector with segmented portions of a live motivational speech based on real-time speech-to-text transcription and semantic analysis; and (d) a session-based storage unit configured to log time-indexed emotional state vectors and corresponding speech segments for post-event analysis; A speaker feedback interface comprising a portable display or a podium-mounted visualization panel, wherein the interface is configured to display visual indicators of emotional feedback in real time, the indicators being derived from the emotional state vector and including at least emotional trend graphs, threshold alerts, or engagement indices.

The invention provides a multi-modal multi-factor depression recognition system fusing emotion information, relates to the technical field of multi-modal fusion, and provides a transfer learning and multi-dataset training method to extract more accurate emotion features so as to solve the problem of insufficient emotion recognition of the existing method. According to the method, feature extraction is carried out by taking clauses as units, and complete context information is reserved, so that language and emotion fluctuation of the user can be better understood. In order to optimize multi-modal data fusion, the invention adopts a fusion method based on an attention mechanism, which not only can more effectively capture the dependency relationship among the modals, but also can fully extract key information in each modality, thereby improving the overall detection effect. Meanwhile, the influence of gender difference and external knowledge is also considered, the accurate recognition and understanding ability of the model on the emotions of the users with different genders is improved by introducing the gender difference and an external common knowledge base, and the adaptability and accuracy of the model are further improved.

An artificial intelligence-driven sentiment analysis system for social media monitoring, comprising: a data collection module configured to retrieve user-generated content from at least one social media platform via application programming interfaces (APIs); a preprocessing module configured to process and clean the collected data by removing irrelevant content, tokenizing the text, removing stop words, and assigning sentiment indicators to emojis, slang, and abbreviations; a sentiment analysis engine comprising one or more deep learning models, wherein the deep learning models include recurrent neural networks (RNNs), long-short-term memory networks (LSTM), and transformer-based models configured to analyze the preprocessed data and classify the content into one or more sentiment categories; an emotion recognition module configured to detect emotional tones within the content and classify them into predefined emotional categories; a real-time data processing unit configured to continuously monitor social media content and provide real-time insights into moods and emotions; a feedback loop module configured to update and refine the deep learning models through reinforcement learning based on new data inputs and sentiment classification results; and a user interface module configured to visualize sentiment and emotion analysis results in real time, including heatmaps, sentiment trend analysis, and emotional breakdowns.

The present invention discloses a method and system for driving digital human actions using a large language model, the method comprising calling a large model to identify the current and previous conversation content, predicting the character's emotions and actions, and outputting coherent emotion and action labels; according to the mapping relationship between different emotions and actions, combining deep learning with a rule base construction method, establishing a digital human rule base covering various behavior patterns; according to the identified emotion and action labels of the current conversation, calling the bound expression and action rendering logic in the digital human rule base to generate a real-time conversation video stream; playing the generated digital human conversation video stream frame by frame in real time. The advantages are: it allows the digital human to combine the conversation content more naturally, generate a video stream with expressions and actions, and achieve the effect of playing the digital human conversation video stream in real time through loop execution, reducing the possibility of stiff expressions and actions or mismatch with the conversation content in traditional methods.

The invention relates to the technical field of health management, in particular to a psychological risk early warning method and system based on multi-dimensional emotion data of a user. The method comprises the following steps: acquiring user multi-dimensional emotion data corresponding to a user emotion expression image, a user emotion voice signal and a user emotion physiological index, and performing multi-source time sequence alignment processing on the user multi-dimensional emotion data to obtain user multi-dimensional time sequence synchronous emotion data; an emotion three-level feature extraction framework is constructed, modal emotion feature extraction and cross-modal fusion are carried out, and cross-modal fusion psychological features of the user are obtained; and obtaining user environment data and user social network data, and performing emotional psychological dynamic graph reasoning and psychological risk early warning on the user cross-modal fusion psychological features based on the user environment data and the user social network data to generate corresponding user emotional psychological risk early warning response measures. According to the invention, early recognition and intervention of psychological health problems can be realized.

A system for AI-based analysis of student behavior, consisting of: a variety of data collection devices, including cameras, microphones and wearable sensors; a central processing unit (CPU) that is communicatively connected to the data acquisition devices; and a behavioral analysis engine, wherein: (a) the cameras are configured to capture visual data indicating student behavior, including facial expressions, body language and eye movements, with the captured visual data being processed using computer vision techniques to detect certain emotional and engagement-related indicators such as stress, boredom, attention or excitement; (b) the microphones are configured to capture audio data, including speech patterns, tone of voice, volume and level of participation, the captured audio data being processed using natural language processing (NLP) techniques to assess the mood, engagement and verbal participation of students in classroom activities; (c) the wearable sensors are configured to collect biometric data, including heart rate, skin temperature and movement, with the biometric data being analyzed to detect physiological signs of emotional stress, fatigue or engagement; (d) The behavioral analysis engine is configured to process the captured visual, auditory, and biometric data in real time to create a behavioral profile of each student. It uses machine learning models to correlate certain patterns with academic performance, emotional well-being, and social engagement. (e) the system is also configured to create predictive models based on historical data to identify pupils at risk of declining performance, social isolation or emotional distress, and to send alerts to teachers, counselors or the administration to enable timely intervention to address the identified problems.

A method of predicting neural activation and psychopathology, indicative of ADHD and mood and anxiety disorders in a subject, such as children, is disclosed. The method may be implemented via computer software on a tablet computer, laptop or desktop computer, an input device, such as touchscreen, keyboard or mouse, and a camera. The camera is positioned to capture the subject's face in the frame of the camera. A frustration-inducing activity, such as a game, to the subject, where the subject received positive or negative feedback based on the selections they make in a game. The subject's reactions to the feedback are captured and parsed by the software, to determine the subject's emotional reaction to feedback. Reactions such as eye gaze, head pose, and facial expression may be used. Over a number of trials of the game, a prediction may be made for the neural activation and psychopathology of the subject.

The invention relates to the field of data processing, and particularly discloses a user emotion recognition method based on AI and voice data, which obtains passive data of a user through natural language dialogue or voice interaction, and dynamically analyzes an emotion state in combination with an emotion recognition model. When the model detects that the emotion does not reach a discrimination threshold value, three levels of emotion induction mechanisms are triggered in sequence: in the primary stage, basic emotion data are supplemented by adopting a general guide verbal skill; in the intermediate stage, a personalized induction strategy is adapted based on user portrait features; and in the advanced stage, multi-modal auxiliary analysis is started. According to the technology, recessive emotions are effectively mined through a progressive data enhancement strategy, the proportion of effective emotion samples in customer service, psychological counseling and other scenes is increased, the emotion recognition accuracy is improved through a three-layer data compensation mechanism especially for the scene that a user actively hides the emotions, and deep emotion interaction and accurate recognition are achieved.

The invention discloses a method, a system, equipment, a medium and a program for identifying the mood and age of a tourist in a village space environment based on machine learning, and belongs to the technical field of tourism service. The method comprises the steps of preprocessing to-be-identified tourist emotion and age label data to obtain a pre-training data set of the emotion and age label data; and inputting the pre-trained data set of the emotion and age label data into a pre-trained deep learning model, and performing identification to obtain an identification result. According to the invention, the emotion and age of the tourist for the street space environment can be identified in high precision and automatically in real time, and the emotion and age index data result for the space environment can be calculated.

The invention discloses a multi-dimensional personality evaluation method for dynamic emotion changes, and relates to the technical field of personality evaluation, the system is composed of a plurality of functional modules, and the method comprises the following steps: step 1, recording physiological data in real time through a wearable device, capturing micro-reactions by using a computer vision technology, and determining whether the micro-reactions exist or not; the micro-reactions comprise facial micro-expressions, limb actions and voice and intonation changes; based on a preset dynamic simulation social scene, observing an emotion regulation strategy of the subject in pressure; step 2, using an LSTM model to carry out time sequence modeling on the multi-modal acquisition data, capturing dynamic evolution of emotions, and combining an attention mechanism to dynamically adjust the weight of each modal; based on six basic emotions and nine mental disorder tendencies, establishing a classification model, and capturing association between emotions and personalities; reinforcement learning is introduced, and model parameters are dynamically optimized according to feedback of a subject; and predicting the emotion trend of the subject in a period of time in the future through an autoregressive integral moving average model.

The invention relates to the technical field of service robots, and discloses an elderly care service robot based on an emotion adjustment system, and the robot comprises a multi-mode emotion recognition module which is used for carrying out the comprehensive perception of the emotional state of the elderly through combining various emotion signals of voice, facial expression and limb movement; the personalized emotion adjusting module is used for generating personalized emotion responses meeting the requirements of the old people; the emotion memory module is used for recording emotion changes, important life events and personal preferences of the elderly; the scene self-adaption module is used for sensing the living environment of the old people in real time and adjusting the emotional expression mode of the robot according to the environment information; the health intervention module is used for generating a health management scheme through long-term monitoring of the emotional state and health of the old people; and a safety monitoring module. According to the invention, the emotion state of the elderly is analyzed through the multi-modal fusion network, the emotion misjudgment probability is reduced, and the man-machine interaction experience and the emotion support effect of the elderly are remarkably improved.

The invention relates to the technical field of intelligent assessment of psychological states, and discloses an intelligent dynamic assessment method for psychological states of teenagers based on multi-modal data. The method comprises the steps that multi-modal psychological data of a target assessment object is acquired, the multi-modal psychological data comprises voice emotion information and expression behavior relations, and dynamic features are extracted from the multi-modal psychological data; performing feature fusion processing on the psychological data by adopting a trained machine learning model to generate a multi-modal feature set containing a mapping relation between modal type identifiers and time sequence codes; based on the multi-modal feature set, performing time sequence correlation analysis on the dynamic features through a state analysis network to obtain a fusion result containing emotion dimension features and cognitive mode features; and inputting a fusion result into an evaluation model for dynamic state deduction processing, and outputting psychological state evaluation elements. According to the invention, comprehensive and dynamic assessment of the psychological state can be realized.

The invention relates to the field of psychological health, and provides a personalized mind-correcting and pressure-reducing system and method based on multi-modal physiological signals. The system comprises a multi-mode physiological signal acquisition module, an emotion recognition module, a mind content generation module and a virtual / augmented reality presentation module. The acquisition module is used for acquiring and preprocessing various physiological signals by using an advanced sensor; the emotion recognition module establishes a signal and emotion mapping relation according to a deep learning algorithm and classifies the signal and emotion mapping relation; the content generation module generates content according to the emotion and preference of the user by means of a big language model and a retrieval enhancement technology; and the presentation module is used for providing immersive experience by combining Unity 3D with VR / AR (Virtual Reality / Augmented Reality) equipment. The system has the advantages of being capable of accurately monitoring emotions in real time, intelligently matching contents, providing immersion experience and forming closed-loop feedback, having expansibility, achieving the emotion recognition accuracy rate of 95%, and being capable of effectively improving mental health and life quality.

The invention relates to the technical field of emotion regulation and control, in particular to an emotion regulation and control system based on a self-adaptive virtual reality scene, which comprises an emotion monitoring module, a virtual reality device, a self-adaptive regulation and control module, a feedback module, an emotion guiding module and an effect evaluation module, the emotion category and the emotion intensity are analyzed and judged; the virtual reality equipment is used for providing virtual reality experience for a user; the self-adaptive regulation and control module is used for adjusting elements and parameters of the virtual scene; the feedback module is used for collecting evaluation on the emotion regulation effect, analyzing the evaluation and generating a feedback report; the emotion guiding module is used for providing emotion guiding suggestions; and the effect evaluation module is used for quantifying the effect of emotion regulation and control and evaluating the effect of emotion regulation and control. According to the invention, real-time monitoring and dynamic regulation and control of the emotion of the user can be realized, personalized virtual reality experience is provided, and more effective mental health support is provided for the user.

The invention relates to the technical field of psychological analysis, in particular to a student psychological interview assisting method and system based on an AI large model, and the method comprises the following steps: obtaining a student answer text, carrying out emotion vocabulary marking by using the AI large model, recognizing a grammatical role and an emotion direction of emotion vocabularies in the text, and carrying out the quantitative analysis of the intensity of each emotion word. And evaluating the emotion direction and intensity of each sentence of the student to obtain emotion state information. According to the method, the AI large model is introduced to perform emotion analysis on the answer text of the student, so that emotion evaluation is more comprehensive and detailed, the rate and amplitude of emotion fluctuation of the student are monitored in real time, the emotion fluctuation curve is drawn, the duration time is recorded, dynamic data support is provided for psychological health evaluation, the accuracy and real-time performance of psychological evaluation are improved, and the psychological evaluation efficiency is improved. The psychological health risk of the student is accurately identified by combining emotion fluctuation and psychological pressure analysis, so that the evaluation result is more personalized, and targeted and prospective support is provided for psychological experts.

The invention provides an intelligent virtuality and reality combined psychological health service device based on digital elements, and aims to improve the accuracy and individuation level of psychological health management. The device firstly obtains the physiological indexes, behavior data, environmental factor data and social economic data of an individual, and carries out multi-modal fusion to form comprehensive feature data. Based on this, a psychological health environment factor model is constructed to quantify the influence of the external environment on the individual psychological state, and the calculation weight and prediction logic of the negative emotion large model are optimized. And the optimized negative emotion large model is used for identifying an individual emotion state, analyzing factors such as social environment, economic pressure and life events in combination with the mental health environment factor model, and generating an individual mental health assessment result. And according to an evaluation result, the virtual digital doctor provides intelligent pre-inquiry and other services. Through data-driven intelligent analysis and virtual-real combined intervention means, the accessibility, accuracy and intervention effect of psychological health services are improved.

The invention belongs to the technical field of emotion recognition, and discloses a graph confrontation emotion recognition method and system for dialogue analysis in an Internet of Things environment, and the method comprises the steps: constructing a context analysis module and a causal perception module, analyzing a structural dependency relationship in a dialogue, and learning context emotion features between dialogues through a structural dependency relationship graph; meanwhile, causal relationships among dialogues of different speakers are captured, weights of causal clues in the dialogues are dynamically adjusted in combination with a self-attention mechanism, the causal relationships which have the most influence on emotional changes are obtained, and the emotions contained in the dialogues are more accurately understood; and finally, introducing a difference regularization item, measuring the difference between the output of the context analysis module and the output of the causal perception module, promoting information interaction between the context analysis module and the causal perception module, and obtaining more comprehensive emotion information. According to the method, better emotion recognition processing is carried out on a complex dialogue scene in an Internet of Things environment, and the recognition precision of an emotion recognition task is improved.

The invention provides a psychological tutoring system based on a digital human technology, and relates to the technical field of psychological health. The system captures the emotional state of a user in real time in a multi-modal interaction mode, generates a real-time emotional state group of the user by combining multi-dimensional data fusion of facial micro-expressions, voice spectrums and dialogue texts, and maps the real-time emotional state group to a three-dimensional emotional space for accurate emotion recognition. Based on an emotion-behavior mapping rule, the system dynamically adjusts the digital person and provides an intervention scheme. And when the emotion fluctuation of the user is detected to exceed a preset threshold value, the system automatically triggers a digital human hiding mechanism, gradually weakens visual, auditory and action stimulation, and provides psychological pacification and guidance. According to the method, the accuracy of psychological state recognition and the pertinence of intervention are remarkably improved, real-time and continuous psychological health support is provided for the user, and the method is suitable for psychological health assistance and emotional state analysis.

The invention discloses a three-dimensional emotion resonance interaction system fused with multi-modal perception, and belongs to the field of artificial intelligence. The emotion micro-expression fine-grained recognition module is used for extracting facial local muscle micro-motion features and amplifying and enhancing micro-expression detail changes through sub-pixel-level motion; the voice emotion spectrum dynamic modeling module is used for analyzing an audio emotion change trend; the individual emotional process depth tracking module is used for modeling an individual emotional state evolution track according to historical interaction records; the three-dimensional emotion situation synchronous rendering module is used for inputting and synchronously displaying emotion changes in a virtual environment; the self-adaptive personalized emotion modeling module dynamically learns an individual emotion response mode and constructs an emotion preference map; the interactive memory evolution module evolves and updates the individual emotion model; and the emotion resonance enhancement regulation and control module is used for dynamically regulating interaction contents and expression modes based on resonance intensity. The method has the beneficial effects that fine-grained perception, accurate understanding and natural expression of three-dimensional emotion resonance interaction are realized.

The present invention proposes an AI dynamic emotion recognition multi-dimensional targeted training system, which relates to the fields of computer vision technology and artificial intelligence technology. It includes multiple emotion collection and training terminals at the front end and a background server. The emotion collection and training terminals are used to detect the emotions of the tested personnel. The terminals analyze the face images to obtain the multi-dimensional emotion data of the tested personnel. The multi-dimensional emotion data is uploaded to the background server. The background server analyzes the current emotion state of the tested personnel according to the received multi-dimensional emotion data, and pushes different targeted training tasks to the corresponding emotion collection and training terminals for the tested personnel in different states. After receiving the targeted training tasks, the emotion collection and training terminals, the tested personnel complete the corresponding training content according to the guidance. The present invention can specifically improve the psychological quality of the tested personnel or improve the bad emotion state.

The invention discloses an elderly emotion evaluation system based on multi-modal physiological and behavior signals. The elderly emotion evaluation system is composed of a multi-modal signal acquisition module, an emotion induction task design module and a multi-modal feature processing module. By collecting multi-mode signals such as electroencephalogram, electrocardio, voice and facial expressions, the system can synchronously record physiological and behavior changes of a user in a specific situation. The emotion induction task adopts scientifically designed audio and video contents, and six typical emotions such as happiness, tension, boring, anger, calm and sadness are guided. Then, the system extracts emotion related features from the multi-modal signals by using a deep learning method, models a cooperative relationship among the multi-modal signals by using an attention mechanism fusion technology, and generates unified emotion feature representation; by comprehensively analyzing and identifying the emotional state of the elderly user, the system provides important reference for early monitoring of emotional health of the elderly, psychological state evaluation and personalized intervention, and has social and clinical values.

To provide a system.SOLUTION: A system includes: means for analyzing numerical data included in the result of health screening provided by a user that indicates the health state of the user; means for inputting a specific prompt sentence to a generative AI model and thereby providing, on the basis of the analysis result, advice on the health habit and meal of the user and the exercise performed by the user, as advice on the health of the user; means for providing, on the basis of the analysis result, the user with a one-week recipe customized in conformity with the heath state of the user; means for analyzing the photographs of the meal registered by the user and proposing a plan for improving the meal of the user; and optimization means for optimizing the advice on health and the recipe on the basis of emotion by using an emotion engine that recognizes the emotion of the user and the generative AI model.SELECTED DRAWING: Figure 1

The invention discloses a cognitive training emotion interaction method and system based on a large model. The method comprises the following steps: acquiring a cognitive evaluation result of a patient, and pushing a corresponding cognitive training scheme to the patient; before cognitive training, acquiring the current emotional psychological state of the user to judge whether the cognitive training requirement is met or not, and if not, performing emotion regulation until the cognitive training requirement is met; during cognitive training, acquiring multi-modal data of the patient and inputting the multi-modal data into a preset large model to perform emotion calculation, so as to identify the current emotional psychological state of the patient again; and based on the current emotional psychological state of the patient, carrying out interaction feedback of a corresponding mode on the patient so as to carry out personalized emotional interaction with the patient. According to the method, the emotional state of the user is recognized in advance before cognitive training, emotion feedback of different modes can be performed according to the current emotional psychological state of the patient in the cognitive training, so that the training compliance of the patient is improved, and the cognitive training effect is improved.

The invention discloses a depression state detection system based on multiple modes, which belongs to the technical field of psychological counseling and comprises an information acquisition module, a preprocessing module, a feature extraction module, a depression state detection module and a depression state evaluation module. According to the method, questionnaire evaluation, semantic evaluation and mood emotion are combined, depression state evaluation of multi-modal information of the user is achieved, the depression state self-test threshold is lowered, and doctors can conveniently and comprehensively know the depression state of the patient. The depression self-evaluation scale is combined with the voice information under the stimulation task, so that the system can provide more accurate evaluation for the depression state of the user, and the accuracy and convenience of traditional depression state self-test are improved. The user self-detection design of the system improves the instantaneity and individuation of depression state detection, and the output depression state evaluation score can be used as an important basis for assisting a doctor in diagnosis and mastering the depression state change of a patient in real time.