Psychological assessment and emotion regulation system and method based on multiple modes
Through the multimodal psychological assessment and emotional regulation system, combined with language, vision, auditory and action assessment, the problem of mental health assessment for the elderly is solved, accurate assessment and emotional regulation of the mental health status of the elderly is achieved, and the evaluation is improved. Scientificity and effectiveness of regulation.
Patent Information
- Application Number
- CN202411271595.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-11
- Publication Date
- 2024-11-22
AI Technical Summary
There are fewer psychological health perception assessments for the elderly, and it is difficult to accurately and effectively obtain the mental health status of the elderly.
Using a multimodal psychological assessment and emotion regulation system, including a multimodal assessment module, an evaluation result analysis module and an emotion regulation module, through language, vision, auditory and action evaluation, combined with database construction and digital twin universe scenes, Comprehensive mental health assessment and emotional regulation.
It improves the pertinence, effectiveness and scientific nature of mental health assessment, can fully and effectively grasp the mental health status of users to be tested, and effectively regulate bad emotions through the emotional adjustment module.
Smart Images

Figure CN119007945A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of mental health technology, and in particular to a multimodal-based psychological assessment and emotion regulation system and method thereof. Background Art
[0002] With the increase of the aging population in my country, the examination and diagnosis of the physical and mental health of the elderly are receiving more and more attention. While paying attention to the physical health of the elderly, the mental health of the elderly also needs to be taken seriously. Real-time perception and evaluation of the mood of the elderly can help understand the changes in the psychological activities of the elderly, so as to provide targeted health examination guidance.
[0003] There are few existing perceptual assessments of the mental health of the elderly, making it difficult to accurately and effectively obtain information about the mental health of the elderly.
[0004] Therefore, it is necessary to provide a multimodal-based psychological assessment and emotion regulation system and method. Summary of the invention
[0005] The present invention provides a multi-modal psychological assessment and emotion regulation system and method thereof, which can improve the pertinence, effectiveness and scientificity of the regulation by conducting psychological assessment based on multiple modalities and obtaining negative emotions and performing emotion regulation according to the analysis of the psychological assessment results, and facilitate comprehensive and effective grasp of the mental health status of the user to be tested.
[0006] The present invention provides a multimodal psychological assessment and emotion regulation system, comprising:
[0007] Multimodal assessment module, used to conduct psychological assessment based on multiple modalities and obtain psychological assessment results;
[0008] The evaluation result analysis module is used to analyze the psychological evaluation results and obtain the emotion analysis results and negative emotions;
[0009] The emotion regulation module is used to regulate emotions against negative emotions.
[0010] Further, the multimodal assessment module includes a database construction unit, an assessment setting unit, and an assessment implementation unit;
[0011] A database construction unit is used to construct a database based on mental health big data and historical assessment implementation data, and to classify and quantify the data, and configure it in a preset cloud platform;
[0012] The evaluation setting unit is used to set the implementation type and implementation time of the evaluation; the implementation types include quick evaluation and system evaluation; the implementation time of the quick evaluation is set to 15 minutes, and the implementation time of the system evaluation is set to 30 minutes;
[0013] The assessment implementation unit is used to implement the assessment based on the database, implementation type and implementation duration.
[0014] Further, the assessment implementation unit includes a language assessment implementation subunit, a visual assessment implementation subunit, an auditory assessment implementation subunit, and a movement assessment implementation subunit;
[0015] The language assessment implementation subunit is used to randomly select three types of question banks based on the database and conduct assessment in the form of voice reading or interactive answering, obtain three types of assessment results, and use the average of the three types of assessment results as the language assessment score;
[0016] The visual evaluation implementation subunit is used to use the evaluation robot to play videos or pictures for the user to be tested to watch, and then ask questions to the user to be tested, and obtain a first visual evaluation result according to the result of the user to be tested answering the question; and use the camera configured on the evaluation robot to shoot the facial expression of the user to be tested when answering the question, and evaluate the emotion reflected by the facial expression according to the set observation algorithm to obtain a first emotion evaluation score; or use the evaluation robot to issue an action command, and the user to be tested makes a corresponding action gesture according to the action command, and then obtains an action posture test score according to the standard degree of the action gesture after observing with the camera configured on the evaluation robot; obtain a visual evaluation score according to the first visual evaluation result, the first emotion evaluation score and the posture test score;
[0017] The auditory assessment implementation subunit is used to play music using multiple different musical instruments, and let the user to be tested answer questions about music, and obtain the auditory assessment results according to the results of the user to be tested answering the questions; and use the camera configured on the assessment robot to shoot the facial expression of the user to be tested when answering the questions, and evaluate the emotions reflected by the facial expression according to the set observation algorithm to obtain a second emotion evaluation score; according to the auditory assessment result and the second emotion evaluation score, obtain the auditory assessment score;
[0018] The action assessment implementation sub-unit is used to use the assessment robot to play videos. During the video playback, the user to be tested touches the head of the assessment robot with his hand. The pressure sensor configured on the head of the assessment robot collects the touch pressure value of the user to be tested's hand touching the head of the assessment robot, and the action assessment score of the user to be tested is obtained based on the touch pressure value evaluation and calculation.
[0019] Furthermore, the voice reading method includes: asking the user to be tested to read the contents of the three types of question banks on the display terminal device within a specified time, and using the set scoring model to measure the reading tone, accuracy, and speed rhythm of the user to be tested to obtain the voice reading method measurement score;
[0020] The interactive answering method includes: using the evaluation robot to randomly ask multiple test questions from three types of question banks to the user to be tested, and obtaining the first answer result; and using the evaluation robot to ask the user to be tested about his or her interests and hobbies, and obtaining the second answer result.
[0021] Furthermore, the language assessment implementation subunit also includes an auxiliary observation molecular unit and an assessment result acquisition molecular unit; the auxiliary observation molecular unit is used to use the camera configured on the assessment robot to capture the facial expression of the user to be tested when the user to be tested is interactively answering questions, and to evaluate the emotions reflected by the facial expression according to the set observation algorithm to obtain a fourth emotion evaluation score value;
[0022] The evaluation result acquisition numerator unit is used to obtain the interactive question test score after comprehensive evaluation based on the first answer result and the second answer result; the fourth emotion evaluation score is used as the weight of the interactive question test score, and multiplied by the interactive question test score to obtain the interactive question comprehensive test score; the interactive question comprehensive test score and the voice reading method measurement score are added together to obtain the average value to obtain 3 types of evaluation results corresponding to the 3 types of question banks.
[0023] Furthermore, the evaluation result analysis module includes: according to the language evaluation score, visual evaluation score, auditory evaluation score and action evaluation score, multiplying them by the set score weight coefficient and then accumulating them to obtain a comprehensive evaluation result; according to the comprehensive evaluation result, using the set emotion evaluation rules, the emotion of the user to be tested is evaluated, and according to the emotion classification, negative emotions are obtained.
[0024] Furthermore, the emotion regulation module includes: for negative emotions, according to the preset emotion regulation knowledge graph, the regulation content is recommended; the regulation content includes pushing relevant emotion training, pushing relevant videos, pushing relevant text and voice, pushing relevant personnel to comfort the users to be evaluated, pushing professional regulation measures, and using the constructed digital twin metaverse scene for regulation.
[0025] Furthermore, it also includes an evaluation result processing unit, which is used to store, retrieve and process the changes of the evaluation results, including:
[0026] According to the evaluation results, set the viewing method and transmission method of the evaluation results. The viewing method includes printing or real-time viewing. The transmission method includes transmitting to the designated receiving location through QQ program, WeChat APP, and email;
[0027] According to the evaluation results, set the analysis and processing method of the evaluation results; the analysis and processing method is: based on the evaluation results, generate a data change trend graph, and according to the data change trend graph, use the artificial intelligence analysis model to analyze and judge the various possible emotional changes of the user to be evaluated, as well as the short-term probability, medium-term probability and long-term probability of the occurrence of various emotional changes, and according to the short-term probability, medium-term probability and long-term probability, use the preset data mutation monitoring model to monitor the mutation of the short-term probability, medium-term probability and long-term probability; if the mutation value reflecting the mutation is greater than the set mutation threshold, an early warning prompt of emotional mutation is issued.
[0028] Furthermore, the constructed digital twin metaverse scenario is used for adjustment, including:
[0029] Based on digital twin technology, a virtual user is constructed according to the information of the user to be tested;
[0030] Based on the evaluation results, obtain the test scenario for the user to be tested, as well as the expression, action, and posture of the user to be tested, and construct the metaverse of the user to be tested according to the test scenario and the expression, action, and posture;
[0031] Based on a Metaverse server connected to the Metaverse, multiple emotion regulation scenarios are obtained;
[0032] Acquire emotion regulation data of the virtual user according to the emotion regulation scenario;
[0033] The emotion regulation data is used to regulate the emotions of the users to be tested.
[0034] A multimodal psychological assessment and emotion regulation method, comprising:
[0035] Conduct psychological assessment based on multiple modalities and obtain psychological assessment results;
[0036] Analyze the psychological assessment results to obtain emotional analysis results and negative emotions;
[0037] Carry out emotional regulation for negative emotions.
[0038] Compared with the prior art, the present invention has the following advantages and beneficial effects: by conducting psychological assessments based on multiple modalities, negative emotions are obtained and emotional regulation is performed according to the analysis of the psychological assessment results, which can improve the pertinence, effectiveness and scientificity of the regulation, and facilitate comprehensive and effective grasp of the mental health status of the user to be tested.
[0039] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.
[0040] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:
[0042] Figure 1 This is a schematic diagram of the structure of a multimodal psychological assessment and emotion regulation system;
[0043] Figure 2 This is a schematic diagram of the structure of the multimodal evaluation module;
[0044] Figure 3 Schematic diagram of the steps of multimodal psychological assessment and emotion regulation methods. DETAILED DESCRIPTION
[0045] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.
[0046] The present invention provides a multi-modal psychological assessment and emotion regulation system, such as Figure 1 As shown, including:
[0047] Multimodal assessment module, used to conduct psychological assessment based on multiple modalities and obtain psychological assessment results;
[0048] The evaluation result analysis module is used to analyze the psychological evaluation results and obtain the emotion analysis results and negative emotions;
[0049] The emotion regulation module is used to regulate emotions against negative emotions.
[0050] The working principle of the above technical scheme is: in order to realize a multimodal psychological assessment and emotion regulation system, the present invention proposes a multimodal assessment module, which is used to conduct psychological assessment based on multiple modalities and obtain psychological assessment results; proposes an assessment result analysis module, which is used to analyze the psychological assessment results and obtain emotion analysis results and negative emotions; and proposes an emotion regulation module, which is used to perform emotion regulation on negative emotions.
[0051] The beneficial effect of the above technical solution is: by adopting the solution provided by this embodiment, by conducting psychological assessment based on multiple modalities, obtaining negative emotions and conducting emotional regulation according to the analysis of the psychological assessment results, the pertinence, effectiveness and scientificity of the regulation can be improved, and it is convenient to comprehensively and effectively grasp the mental health status of the user to be tested.
[0052] In one embodiment, Figure 2 As shown, the multimodal evaluation module includes a database construction unit, an evaluation setting unit and an evaluation implementation unit;
[0053] A database construction unit is used to construct a database based on mental health big data and historical assessment implementation data, and to classify and quantify the data, and configure it in a preset cloud platform;
[0054] The evaluation setting unit is used to set the implementation type and implementation time of the evaluation; the implementation types include quick evaluation and system evaluation; the implementation time of the quick evaluation is set to 15 minutes, and the implementation time of the system evaluation is set to 30 minutes;
[0055] The assessment implementation unit is used to implement the assessment based on the database, implementation type and implementation duration.
[0056] The working principle of the above technical solution is: in order to realize multimodal assessment, the present invention proposes a database construction unit, an assessment setting unit and an assessment implementation unit; wherein, the database construction unit is used to construct a database based on mental health big data and historical assessment implementation data, and classify and quantify the annotations, and configure it in a preset cloud platform; the assessment setting unit is used to set the implementation type and implementation duration of the assessment; the implementation types include quick assessment and system assessment; the quick assessment is set to a 15-minute implementation duration, and the system assessment is set to a 30-minute implementation duration; the assessment implementation unit is used to implement the assessment based on the database, implementation type and implementation duration.
[0057] The beneficial effect of the above technical solution is: by adopting the solution provided in this embodiment, comprehensive evaluation data can be obtained by performing multi-modal evaluation.
[0058] In one embodiment, the assessment implementation unit includes a language assessment implementation subunit, a visual assessment implementation subunit, an auditory assessment implementation subunit, and a movement assessment implementation subunit;
[0059] The language assessment implementation subunit is used to randomly select three types of question banks based on the database and conduct assessment in the form of voice reading or interactive answering, obtain three types of assessment results, and use the average of the three types of assessment results as the language assessment score;
[0060] The visual evaluation implementation subunit is used to use the evaluation robot to play videos or pictures for the user to be tested to watch, and then ask questions to the user to be tested, and obtain a first visual evaluation result according to the result of the user to be tested answering the question; and use the camera configured on the evaluation robot to shoot the facial expression of the user to be tested when answering the question, and evaluate the emotion reflected by the facial expression according to the set observation algorithm to obtain a first emotion evaluation score; or use the evaluation robot to issue an action command, and the user to be tested makes a corresponding action gesture according to the action command, and then obtains an action posture test score according to the standard degree of the action gesture after observing with the camera configured on the evaluation robot; obtain a visual evaluation score according to the first visual evaluation result, the first emotion evaluation score and the posture test score;
[0061] The auditory assessment implementation subunit is used to play music using multiple different musical instruments, and let the user to be tested answer questions about music, and obtain the auditory assessment results according to the results of the user to be tested answering the questions; and use the camera configured on the assessment robot to shoot the facial expression of the user to be tested when answering the questions, and evaluate the emotions reflected by the facial expression according to the set observation algorithm to obtain a second emotion evaluation score; according to the auditory assessment result and the second emotion evaluation score, obtain the auditory assessment score;
[0062] The action assessment implementation sub-unit is used to use the assessment robot to play videos. During the video playback, the user to be tested touches the head of the assessment robot with his hand. The pressure sensor configured on the head of the assessment robot collects the touch pressure value of the user to be tested's hand touching the head of the assessment robot, and the action assessment score of the user to be tested is obtained based on the touch pressure value evaluation and calculation.
[0063] The working principle of the above technical solution is as follows: in order to implement the evaluation, the present invention proposes a language evaluation implementation subunit, a visual evaluation implementation subunit, an auditory evaluation implementation subunit and an action evaluation implementation subunit; wherein the language evaluation implementation subunit is used to randomly select 3 types of question banks based on the database to conduct evaluation in the form of voice reading or interactive answering, obtain 3 types of evaluation results, and take the average of the 3 types of evaluation results as the language evaluation score; the visual evaluation implementation subunit is used to use the evaluation robot to play videos or pictures for the user to be tested to watch, and then ask questions to the user to be tested, and obtain the first visual evaluation result according to the result of the user to be tested answering the question; and use the camera configured on the evaluation robot to shoot the facial expression of the user to be tested when answering the question, and evaluate the emotions reflected by the facial expression according to the set observation algorithm to obtain the first emotion evaluation score; or use the evaluation robot to issue an action command, and the user to be tested makes a corresponding action gesture according to the action command, and then uses the camera configured on the evaluation robot to observe, and then obtains the first emotion evaluation score according to the action gesture. The standard degree of posture is used to obtain the action posture test score; the visual evaluation score is obtained according to the first visual evaluation result, the first emotion evaluation score and the posture test score; the auditory evaluation implementation subunit is used to play music using multiple different musical instruments, let the user to be tested answer questions about music, and obtain the auditory evaluation result according to the results of the user to be tested answering the questions; and use the camera configured on the evaluation robot to shoot the facial expression of the user to be tested when answering the questions, and evaluate the emotions reflected by the facial expression according to the set observation algorithm to obtain the second emotion evaluation score; according to the auditory evaluation result and the second emotion evaluation score, obtain the auditory evaluation score; the action evaluation implementation subunit is used to use the evaluation robot to play the video, during the video playback process, let the user to be tested touch the head of the evaluation robot with his hand, use the pressure sensor configured on the head of the evaluation robot to collect the touch pressure value of the user to be tested touching the head of the evaluation robot with his hand, and obtain the action evaluation score of the user to be tested according to the touch pressure value evaluation and measurement.
[0064] The beneficial effect of the above technical solution is: by adopting the solution provided by this embodiment, by distinguishing language evaluation, visual evaluation, auditory evaluation and action evaluation, it is possible to ensure that comprehensive evaluation results are obtained.
[0065] In one embodiment, the voice reading method includes: asking the user to be tested to read the contents of the three types of question banks on the display terminal device within a specified time, and using a set scoring model to measure the reading tone, accuracy, and speed of the user to be tested to obtain a voice reading method measurement score;
[0066] The interactive answering method includes: using the evaluation robot to randomly ask multiple test questions from three types of question banks to the user to be tested, and obtaining the first answer result; and using the evaluation robot to ask the user to be tested about his or her interests and hobbies, and obtaining the second answer result.
[0067] The working principle of the above technical solution is as follows: in order to realize voice reading, the present invention proposes to let the user to be tested read aloud the contents of the three types of question banks on the display terminal device within a specified time, and calculate the reading tone, accuracy, and fast and slow rhythm of the user to be tested using a set scoring model to obtain the voice reading method calculation score; in order to realize interactive answering, the present invention uses an evaluation robot to randomly ask multiple test questions from the three types of question banks to the user to be tested, and obtain a first answer result; and uses the evaluation robot to ask the user to be tested about his interests and hobbies, and obtain a second answer result.
[0068] The beneficial effect of the above technical solution is: by adopting the solution provided in this embodiment, voice evaluation can be effectively completed through voice reading and interactive answering.
[0069] In one embodiment, the language assessment implementation subunit further includes an auxiliary observation molecular unit and an assessment result acquisition molecular unit; the auxiliary observation molecular unit is used to use a camera configured on the assessment robot to capture the facial expression of the user to be tested when the user to be tested is interactively answering questions, and to evaluate the emotions reflected by the facial expression according to a set observation algorithm to obtain a fourth emotion evaluation score value;
[0070] The evaluation result acquisition numerator unit is used to obtain the interactive question test score after comprehensive evaluation based on the first answer result and the second answer result; the fourth emotion evaluation score is used as the weight of the interactive question test score, and multiplied by the interactive question test score to obtain the interactive question comprehensive test score; the interactive question comprehensive test score and the voice reading method measurement score are added together to obtain the average value to obtain 3 types of evaluation results corresponding to the 3 types of question banks.
[0071] The working principle of the above technical solution is: in order to implement language assessment, the present invention also proposes auxiliary observation and summarizing the assessment results; wherein, the auxiliary observation is: when the user to be tested is answering questions interactively, the facial expression of the user to be tested is photographed by the camera configured on the assessment robot, and the emotions reflected by the facial expressions are evaluated and tested according to the set observation algorithm to obtain a fourth emotion evaluation score; the assessment result is obtained as follows: based on the first answer result and the second answer result, an interactive question test score is obtained after comprehensive evaluation; the fourth emotion evaluation score is used as the weight of the interactive question test score, and multiplied by the interactive question test score to obtain the interactive question comprehensive test score; the interactive question comprehensive test score and the voice reading method measurement score are added and averaged to obtain 3 types of assessment results corresponding to 3 types of question banks.
[0072] The beneficial effect of the above technical solution is: by adopting the solution provided in this embodiment, the necessary conditions for the implementation of speech evaluation are provided through auxiliary observation and summary of evaluation results.
[0073] In one embodiment, the evaluation result analysis module includes: according to the language evaluation score, the visual evaluation score, the auditory evaluation score and the action evaluation score, multiplying them by the set score weight coefficients respectively and then accumulating them to obtain a comprehensive evaluation result; according to the comprehensive evaluation result, using the set emotion evaluation rules, the emotion of the user to be tested is evaluated, and according to the emotion classification, negative emotions are obtained.
[0074] The working principle of the above technical solution is: in order to realize the analysis of the evaluation results, the present invention first multiplies the language evaluation score, visual evaluation score, auditory evaluation score and action evaluation score by the set score weight coefficient and then accumulates them to obtain a comprehensive evaluation result; then, based on the comprehensive evaluation result, the set emotion evaluation rules are used to evaluate the emotions of the user to be tested, and according to the emotion classification, the negative emotions are obtained.
[0075] The beneficial effect of the above technical solution is: by adopting the solution provided by this embodiment, the emotions of the user to be tested can be obtained through analysis of the evaluation results, and negative emotions can be obtained therefrom.
[0076] In one embodiment, the emotion regulation module includes: for negative emotions, according to a preset emotion regulation knowledge graph, recommending regulation content; the regulation content includes pushing relevant emotion training, pushing relevant videos, pushing relevant text and voice, pushing relevant personnel to comfort the users to be evaluated, pushing professional regulation measures, and using the constructed digital twin metaverse scene for regulation.
[0077] The working principle of the above technical solution is: in order to achieve emotional regulation of negative emotions, the present invention recommends regulation content based on a preset emotion regulation knowledge graph; the regulation content includes pushing relevant emotion training, pushing relevant videos, pushing relevant text and voice, pushing relevant personnel to comfort the users to be evaluated, pushing professional regulation measures, and using the constructed digital twin metaverse scene for regulation.
[0078] The beneficial effect of the above technical solution is: by adopting the solution provided by this embodiment, by utilizing the emotion regulation knowledge graph to recommend regulation content, targeted emotion regulation can be provided.
[0079] In one embodiment, an evaluation result processing unit is further included, which is used to process the storage, review and change of the evaluation results, specifically including:
[0080] According to the evaluation results, set the viewing method and transmission method of the evaluation results. The viewing method includes printing or real-time viewing. The transmission method includes transmitting to the designated receiving location through QQ program, WeChat APP, and email;
[0081] According to the evaluation results, set the analysis and processing method of the evaluation results; the analysis and processing method is: based on the evaluation results, generate a data change trend graph, and according to the data change trend graph, use the artificial intelligence analysis model to analyze and judge the various possible emotional changes of the user to be evaluated, as well as the short-term probability, medium-term probability and long-term probability of the occurrence of various emotional changes, and according to the short-term probability, medium-term probability and long-term probability, use the preset data mutation monitoring model to monitor the mutation of the short-term probability, medium-term probability and long-term probability; if the mutation value reflecting the mutation is greater than the set mutation threshold, an early warning prompt of emotional mutation is issued.
[0082] The working principle of the above technical scheme is: in order to realize the processing of the evaluation results, the present invention proposes to store, review and analyze the changes of the evaluation results, specifically including: according to the evaluation results, setting the viewing method and transmission method of the evaluation results, the viewing method includes printing or real-time viewing, and the transmission method includes transmitting to a designated receiving location through a QQ program, WeChat APP, or email; according to the evaluation results, setting the analysis and processing method of the evaluation results; the analysis and processing method is: based on the evaluation results, generating a data change trend graph, and according to the data change trend graph, using an artificial intelligence analysis model to analyze and judge the various possible emotional changes of the user to be evaluated, as well as the short-term probability, medium-term probability and long-term probability of the occurrence of various emotional changes, and according to the short-term probability, medium-term probability and long-term probability, using a preset data mutation monitoring model to monitor the mutation of the short-term probability, medium-term probability and long-term probability. If the mutation value reflecting the mutation is greater than the set mutation threshold, an early warning prompt of the emotional mutation is issued.
[0083] The beneficial effect of the above technical solution is: by adopting the solution provided in this embodiment, it is easy to grasp and use the evaluation result data through processing the evaluation results.
[0084] In one embodiment, the constructed digital twin metaverse scenario is used for adjustment, including:
[0085] Based on digital twin technology, a virtual user is constructed according to the information of the user to be tested;
[0086] Based on the evaluation results, obtain the test scenario for the user to be tested, as well as the expression, action, and posture of the user to be tested, and construct the metaverse of the user to be tested according to the test scenario and the expression, action, and posture;
[0087] Based on a Metaverse server connected to the Metaverse, multiple emotion regulation scenarios are obtained;
[0088] Acquire emotion regulation data of the virtual user according to the emotion regulation scenario;
[0089] The emotion regulation data is used to regulate the emotions of the users to be tested.
[0090] The working principle of the above technical solution is: in order to realize the adjustment using the constructed digital twin metaverse scene, the present invention first constructs a virtual user based on the digital twin technology according to the information of the user to be tested; then, based on the evaluation results, the test scene for the user to be tested, as well as the expression, action, and posture of the user to be tested are obtained, and the metaverse of the user to be tested is constructed according to the test scene and the expression, action, and posture; then, based on the metaverse server connected to the metaverse, multiple emotion regulation scenes are obtained; according to the emotion regulation scene, emotion regulation data for the virtual user is obtained; finally, the emotion regulation data is used for emotion regulation of the user to be tested.
[0091] The beneficial effect of the above technical solution is: by adopting the solution provided in this embodiment, the comprehensiveness and effectiveness of emotional regulation can be improved by utilizing the constructed digital twin metaverse scene for adjustment.
[0092] A multimodal psychological assessment and emotion regulation method, such as Figure 3 As shown, including:
[0093] Conduct psychological assessment based on multiple modalities and obtain psychological assessment results;
[0094] Analyze the psychological assessment results to obtain emotional analysis results and negative emotions;
[0095] Carry out emotional regulation for negative emotions.
[0096] The working principle of the above technical solution is: in order to realize the multi-modal psychological assessment and emotion regulation method, the present invention first conducts psychological assessment based on multiple modalities to obtain psychological assessment results; then analyzes the psychological assessment results to obtain emotion analysis results and negative emotions; finally, emotion regulation is performed for negative emotions.
[0097] The beneficial effect of the above technical solution is: by adopting the solution provided by this embodiment, by conducting psychological assessment based on multiple modalities, obtaining negative emotions and conducting emotional regulation according to the analysis of the psychological assessment results, the pertinence, effectiveness and scientificity of the regulation can be improved, and it is convenient to comprehensively and effectively grasp the mental health status of the user to be tested.
[0098] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A multimodal psychological assessment and emotion regulation system, characterized in that: include: Multimodal assessment module, used to conduct psychological assessment based on multiple modalities and obtain psychological assessment results; The evaluation result analysis module is used to analyze the psychological evaluation results and obtain the emotion analysis results and negative emotions; The emotion regulation module is used to regulate emotions against negative emotions.
2. A multimodal psychological assessment and emotion regulation system according to claim 1, characterized in that: The multimodal assessment module includes a database construction unit, an assessment setting unit, and an assessment implementation unit; A database construction unit is used to construct a database based on mental health big data and historical assessment implementation data, and to classify and quantify the data, and configure it in a preset cloud platform; The evaluation setting unit is used to set the implementation type and implementation time of the evaluation; the implementation types include quick evaluation and system evaluation; the implementation time of the quick evaluation is set to 15 minutes, and the implementation time of the system evaluation is set to 30 minutes; The assessment implementation unit is used to implement the assessment based on the database, implementation type and implementation duration.
3. The multimodal psychological assessment and emotion regulation system according to claim 1, characterized in that: The assessment implementation unit includes a language assessment implementation subunit, a visual assessment implementation subunit, an auditory assessment implementation subunit, and a movement assessment implementation subunit; The language assessment implementation subunit is used to randomly select three types of question banks based on the database and conduct assessment in the form of voice reading or interactive answering, obtain three types of assessment results, and use the average of the three types of assessment results as the language assessment score; The visual evaluation implementation subunit is used to use the evaluation robot to play videos or pictures for the user to be tested to watch, and then ask questions to the user to be tested, and obtain a first visual evaluation result according to the result of the user to be tested answering the question; and use the camera configured on the evaluation robot to shoot the facial expression of the user to be tested when answering the question, and evaluate the emotion reflected by the facial expression according to the set observation algorithm to obtain a first emotion evaluation score; or use the evaluation robot to issue an action command, and the user to be tested makes a corresponding action gesture according to the action command, and then obtains an action posture test score according to the standard degree of the action gesture after observing with the camera configured on the evaluation robot; obtain a visual evaluation score according to the first visual evaluation result, the first emotion evaluation score and the posture test score; The auditory assessment implementation subunit is used to play music using multiple different musical instruments, let the user to be tested answer questions about the music, and obtain the auditory assessment results according to the results of the user to be tested answering the questions; The camera configured on the evaluation robot is used to capture the facial expression of the user to be tested when answering questions, and the emotion reflected by the facial expression is evaluated and tested according to the set observation algorithm to obtain a second emotion evaluation score; the hearing evaluation score is obtained according to the hearing evaluation result and the second emotion evaluation score; The action assessment implementation sub-unit is used to use the assessment robot to play videos. During the video playback, the user to be tested touches the head of the assessment robot with his hand. The pressure sensor configured on the head of the assessment robot collects the touch pressure value of the user to be tested's hand touching the head of the assessment robot, and the action assessment score of the user to be tested is obtained based on the touch pressure value evaluation and calculation.
4. A multimodal psychological assessment and emotion regulation system according to claim 3, characterized in that: The voice reading method includes: asking the user to be tested to read the contents of the three types of question banks on the display terminal device within a specified time, and using the set scoring model to measure the reading tone, accuracy, speed and rhythm of the user to be tested to obtain the voice reading method measurement score; The interactive answering method includes: using the evaluation robot to randomly ask multiple test questions from three types of question banks to the user to be tested, and obtaining the first answer result; and using the evaluation robot to ask the user to be tested about his or her interests and hobbies, and obtaining the second answer result.
5. A multimodal psychological assessment and emotion regulation system according to claim 4, characterized in that: The language assessment implementation sub-unit also includes an auxiliary observation molecular unit and an assessment result acquisition molecular unit; The auxiliary observation molecular unit is used to use the camera configured on the evaluation robot to capture the facial expression of the user to be tested when the user to be tested is interactively answering questions, and to evaluate the emotion reflected by the facial expression according to the set observation algorithm to obtain a fourth emotion evaluation score value; The evaluation result acquisition molecular unit is used to obtain the interactive question test score after comprehensive evaluation based on the first answer result and the second answer result; The fourth emotion assessment score is used as the weight of the interactive question answering test score, and multiplied by the interactive question answering test score to obtain the interactive question answering comprehensive test score; the interactive question answering comprehensive test score and the voice reading method measurement score are added together to calculate the average value, and 3 types of evaluation results corresponding to the 3 types of question banks are obtained.
6. The multimodal psychological assessment and emotion regulation system according to claim 3, characterized in that: The evaluation result analysis module includes: according to the language evaluation score, visual evaluation score, auditory evaluation score and action evaluation score, the set score weight coefficient is used to multiply them respectively and then accumulate them to obtain the comprehensive evaluation result; according to the comprehensive evaluation result, the set emotion evaluation rules are used to evaluate the emotion of the user to be tested, and the negative emotion is obtained according to the emotion classification.
7. The multimodal psychological assessment and emotion regulation system according to claim 2, characterized in that: The emotion regulation module includes: for negative emotions, according to the preset emotion regulation knowledge graph, the regulation content is recommended; the regulation content includes pushing relevant emotion training, pushing relevant videos, pushing relevant text and voice, pushing relevant personnel to comfort the users to be evaluated, pushing professional regulation measures, and using the constructed digital twin metaverse scene for regulation.
8. The multimodal psychological assessment and emotion regulation system according to claim 1, characterized in that: It also includes an evaluation result processing unit, which is used to store, retrieve and process changes in the evaluation results, including: According to the evaluation results, set the viewing method and transmission method of the evaluation results. The viewing method includes printing or real-time viewing. The transmission method includes transmitting to the designated receiving location through QQ program, WeChat APP, and email; According to the evaluation results, set the analysis and processing method of the evaluation results; the analysis and processing method is: based on the evaluation results, generate a data change trend graph, and according to the data change trend graph, use the artificial intelligence analysis model to analyze and judge the various possible emotional changes of the user to be evaluated, as well as the short-term probability, medium-term probability and long-term probability of the occurrence of various emotional changes, and according to the short-term probability, medium-term probability and long-term probability, use the preset data mutation monitoring model to monitor the mutation of the short-term probability, medium-term probability and long-term probability; if the mutation value reflecting the mutation is greater than the set mutation threshold, an early warning prompt of emotional mutation is issued.
9. The multimodal psychological assessment and emotion regulation system according to claim 7, characterized in that: Use the constructed digital twin metaverse scenario for adjustment, including: Based on digital twin technology, a virtual user is constructed according to the information of the user to be tested; Based on the evaluation results, obtain the test scenario for the user to be tested, as well as the expression, action, and posture of the user to be tested, and construct the metaverse of the user to be tested according to the test scenario and the expression, action, and posture; Based on a Metaverse server connected to the Metaverse, multiple emotion regulation scenarios are obtained; Acquire emotion regulation data of the virtual user according to the emotion regulation scenario; The emotion regulation data is used to regulate the emotions of the users to be tested.
10. A multimodal psychological assessment and emotion regulation method, characterized in that: include: Conduct psychological assessment based on multiple modalities and obtain psychological assessment results; Analyze the psychological assessment results to obtain emotional analysis results and negative emotions; Carry out emotional regulation for negative emotions.
Citation Information
Patent Citations
Psychological health monitoring system based on multi-modal data analysis
CN116072264A
Emotion regulation method and system based on multi-modal emotion interaction
CN116661607A
Non-contact psychological state assessment method and system based on multi-modal fusion technology
CN117936032A
Method for using artificial intelligence models to provide conversational outfit suggestions
KR102777049B1
