Psychological assessment method based on large model and related device

Abstract

The invention relates to the field of data processing, in particular to a psychological assessment method based on a large model and a related device. The method comprises the following steps: generating a psychological evaluation strategy according to a current interview task and a corresponding psychological evaluation index; generating multi-round dialogue information through an evaluation interaction model based on a psychological evaluation strategy, and interacting with the target object through the multi-round dialogue information to complete a psychological evaluation process corresponding to the current interview task; performing multi-modal interaction mode analysis on the target object based on the multi-round dialogue information and the corresponding response information to obtain a psychological assessment report of the target object; the psychological assessment report at least comprises a quantitative analysis result and a qualitative psychological analysis result aiming at the dialogue content. According to the method, by accurately guiding dialogue content generation and adjusting the evaluation strategy in real time, the psychological state of the target object is more accurately touched, individual differences are adapted, related personnel are assisted in completing psychological evaluation tasks, the accuracy of the evaluation result is improved, and the psychological evaluation efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a psychological assessment method based on a large model and related devices. Background Art

[0002] In psychological assessment, self-assessment and peer assessment are the two main assessment methods. Self-assessment refers to the test-taker filling out a questionnaire or assessment form to directly report his or her feelings, behaviors, attitudes or symptoms. This method relies on the individual's self-observation and self-expression ability. Peer assessment refers to the assessment of the test-taker by others (such as psychologists, doctors, teachers, relatives, etc.) based on observation or other sources of information. This method usually involves more objective data and professional judgment.

[0003] Self-assessment is generally conducted through a scale, which is direct and convenient, but it may not be sufficiently accurate due to the influence of individual maturity and social expectations. Peer evaluation can reduce the error of self-assessment through objective evaluation, and evaluation by professionals can also be more professional. However, due to the involvement of professionals, peer evaluation involves higher costs, and there are differences in the ability level and evaluation criteria between different evaluators, which may affect the consistency and repeatability of the evaluation. At present, the evaluation quality of peer evaluation psychological assessment mainly depends on the professional level and experience of the evaluator. The differences in background and methods of different evaluators may lead to inconsistent results, affecting the accuracy and reliability of the evaluation.

[0004] Therefore, it is urgent to propose a new technical solution to solve at least one of the above technical solutions. Summary of the invention

[0005] In response to the technical problems existing in the prior art, the present application provides a psychological assessment method and related devices based on a large model to solve at least one technical problem existing in the prior art.

[0006] In a first aspect, an embodiment of the present application provides a psychological assessment method based on a large model, the method comprising:

[0007] For the target object to be evaluated, a psychological evaluation strategy is generated according to the current interview task and the corresponding psychological evaluation index; the psychological evaluation strategy is used to guide the large model to generate dialogue content that meets the current interview task and the psychological evaluation index;

[0008] Based on the psychological assessment strategy, multiple rounds of dialogue information are generated through the assessment interaction model, and the target object is interacted with through the multiple rounds of dialogue information to complete the psychological assessment process corresponding to the current interview task; the questioning strategy in the psychological assessment strategy is obtained by real-time adjustment based on the response information of the target object during the interaction process;

[0009] Based on the multi-round dialogue information, a multimodal interaction mode analysis is performed on the target object to obtain a psychological assessment report of the target object; the psychological assessment report at least includes: quantitative analysis results and qualitative psychological analysis results for the dialogue content; the qualitative psychological analysis results are associated with the psychological assessment indicators.

[0010] In a second aspect, an embodiment of the present application provides a psychological assessment device based on a large model, the device comprising at least the following units:

[0011] A generating unit is configured to generate a psychological assessment strategy for a target object to be assessed according to a current interview task and a corresponding psychological assessment indicator; the psychological assessment strategy is used to guide the large model to generate dialogue content that meets the current interview task and the psychological assessment indicator;

[0012] The interaction unit is configured to generate multiple rounds of dialogue information through the evaluation interaction model based on the psychological evaluation strategy, and interact with the target object through the multiple rounds of dialogue information to complete the psychological evaluation process corresponding to the current interview task; the questioning strategy in the psychological evaluation strategy is obtained by real-time adjustment based on the response information of the target object during the interaction process;

[0013] The evaluation unit is configured to perform a multimodal interaction mode analysis on the target object based on the multi-round dialogue information to obtain a psychological evaluation report of the target object; the psychological evaluation report at least includes: a quantitative analysis result and a qualitative psychological analysis result for the dialogue content; the qualitative psychological analysis result is associated with the psychological evaluation index.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising:

[0015] at least one processor, memory, and input-output unit;

[0016] The memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute the large model-based psychological assessment method of the first aspect.

[0017] In a fourth aspect, a computer-readable storage medium is provided, which includes instructions, and when the instructions are executed on a computer, the computer executes the large model-based psychological assessment method of the first aspect.

[0018] The beneficial effects of this application are: by accurately guiding the generation of conversation content and adjusting the evaluation strategy in real time, the psychological state of the target object can be more accurately reached, and individual differences can be adapted to, which helps to improve the accuracy of the evaluation results, assist relevant personnel in completing psychological evaluation tasks, and improve the efficiency of psychological evaluation. The evaluation strategy of this solution fits the interview task and can change in real time with the response of the target object, so that the interaction can be carried out in an orderly manner and more adaptable to changes in the target object, improving the flexibility and adaptability of the interaction. This solution is based on multimodal interaction analysis to gain insight into the psychological state from multiple dimensions, and combines qualitative and quantitative analysis to generate a comprehensive psychological assessment report, presenting the evaluation situation in a more in-depth and comprehensive manner. In addition, the solution orderly advances the evaluation process to improve efficiency, uses automated processes to reduce the burden of manpower, and makes the auxiliary customization of psychological evaluation simpler and more feasible in more scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flowchart of a psychological assessment method based on a large model in an embodiment of the present application;

[0020] Figure 2 It is a structural schematic diagram of a psychological assessment device based on a large model according to an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of the structure of an electronic device according to an embodiment of the present application;

[0022] Figure 4 It is a structural schematic diagram of a medium device according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0024] In order to solve at least one technical problem in the related art, the embodiment of the present application provides a psychological assessment method based on a large model and a related device. In this technical solution, first, for the target object to be evaluated, a psychological assessment strategy is generated according to the current interview task and the corresponding psychological assessment index; the psychological assessment strategy is used to guide the large model to generate dialogue content that conforms to the current interview task and the psychological assessment index. Then, based on the psychological assessment strategy, multiple rounds of dialogue information are generated through the assessment interaction model, and the target object is interacted with through the multiple rounds of dialogue information to complete the psychological assessment process corresponding to the current interview task; in the non-first round of interaction process, the psychological assessment strategy is obtained by real-time adjustment based on the response information of the target object during the interaction process. Finally, based on the multiple rounds of dialogue information and the corresponding response information, a multimodal interaction mode analysis is performed on the target object to obtain a psychological assessment report of the target object; the psychological assessment report at least includes: quantitative analysis results and qualitative psychological analysis results for the dialogue content; the qualitative psychological analysis results are associated with the psychological assessment index.

[0025] In the technical solution provided by this application, on the one hand, by generating a psychological assessment strategy based on the current interview task and the corresponding psychological assessment indicators, the large model can be guided to output dialogue content that meets the requirements, ensuring that the topics discussed are closely centered around the core of the assessment, avoiding irrelevant or inefficient conversations, and thus more accurately touching on the relevant aspects of the target object's psychological state, which helps to accurately obtain the information required for the assessment and improve the accuracy of the assessment results. In the non-first round of interaction process, the psychological assessment strategy is adjusted in real time based on the response information of the target object, which enables the assessment to dynamically optimize the direction of the conversation based on the unique response of each target object. For example, some target objects may be more sensitive to certain types of questions and give obscure answers. By adjusting the strategy in real time, a more appropriate questioning angle can be used to continue to mine information, effectively adapting to the characteristics of different individuals and further improving the accuracy of the assessment.

[0026] Secondly, the psychological assessment strategy is closely integrated with the interview task to ensure that the entire interaction process is always carried out in an orderly manner along the direction of completing the task. Whether it is a short-term single psychological state screening task or a long-term complex task such as personality trait assessment, it can be carried out in an orderly manner, and the dialogue content can be flexibly adjusted according to the specific needs of different stages, so that the interaction is more in line with the actual requirements of the task. As the target object's response changes in the interaction, the assessment strategy changes in real time, which means that the system does not communicate in a fixed and rigid mode, but like an experienced assessor, it can change the questioning and guidance methods at any time according to the other party's emotions, attitudes, etc., which greatly enhances the adaptability of interaction with the target object and makes it easier for the target object to accept and cooperate with the assessment process. The use of multi-round dialogue information and corresponding response information for multimodal interaction mode analysis is no longer limited to a single text question and answer content. For example, in addition to analyzing the discourse itself, it will also pay attention to non-verbal information such as voice intonation and expression, and comprehensively consider the psychological state of the target object from multiple dimensions, which can more comprehensively and deeply understand the target object's true psychological characteristics, emotional changes, etc., so that the final psychological assessment report is more valuable for reference.

[0027] Thirdly, the psychological assessment report contains both quantitative analysis results for the content of the conversation, such as the specific numerical scores of various psychological indicators, and qualitative psychological analysis results related to psychological assessment indicators, such as a textual description of the psychological state of the target object, analysis of potential psychological problems, etc. This combination of qualitative and quantitative methods provides a comprehensive and multi-level understanding perspective for both professionals and the target object itself, which helps to accurately grasp the assessment situation and make reasonable decisions or take corresponding intervention measures accordingly. The entire process conducts multiple rounds of interaction around a clear psychological assessment strategy, making the assessment process clear and compact, avoiding unnecessary repetition or invalid dialogue, and being able to more efficiently complete the psychological assessment process corresponding to the corresponding interview task. Especially when facing a large number of target objects to be assessed, the efficiency of the overall assessment can be improved while ensuring quality.

[0028] Fourthly, the use of automated means such as assessment interaction models to achieve multiple rounds of interaction and subsequent analysis and report generation processes reduces dependence on a large number of professional manpower, making psychological assessment feasible in more scenarios. For example, in some areas with limited resources and insufficient professional personnel or in large-scale screening scenarios, psychological assessment work can also be carried out smoothly.

[0029] Combining the above aspects, the solution accurately guides the generation of conversation content and adjusts the evaluation strategy in real time, so as to more accurately reach the psychological state of the target object, adapt to individual differences, help improve the accuracy of the evaluation results, assist relevant personnel in completing psychological evaluation tasks, and improve the efficiency of psychological evaluation. The evaluation strategy of this solution fits the interview task and can change in real time with the response of the target object, so that the interaction can be carried out in an orderly manner and more adaptable to the changes of the target object, improving the flexibility and adaptability of the interaction. Based on multimodal interaction analysis, this solution has insights into the psychological state from multiple dimensions, combines qualitative and quantitative analysis to generate a comprehensive psychological assessment report, and presents the evaluation situation more deeply and comprehensively. In addition, the solution orderly promotes the evaluation process to improve efficiency, uses automated processes to reduce the burden of manpower, and makes the auxiliary customization of psychological evaluation simpler and more feasible in more scenarios.

[0030] The technical solution of the present application and the psychological evaluation scheme based on the large model provided by the embodiment of the present application can also be performed by an electronic device, which can be a server, a server cluster, or a cloud server. The electronic device can also be a terminal device such as a mobile phone, a computer, a tablet computer, a wearable device, or a special device (such as a special terminal device with a psychological evaluation method system based on a large model). The chips or other hardware processing units introduced in the above embodiments can also be carried in these electronic devices. Alternatively, these electronic devices can also be installed with a service program for executing a psychological evaluation scheme based on a large model.

[0031] Figure 1 A flowchart of a psychological assessment method based on a large model provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method comprises the following steps:

[0032] 101, for the target object to be evaluated, generate a psychological evaluation strategy according to the current interview task and the corresponding psychological evaluation index; 102, based on the psychological evaluation strategy, generate multiple rounds of dialogue information through the evaluation interaction model, and interact with the target object through the multiple rounds of dialogue information to complete the psychological evaluation process corresponding to the current interview task;

[0033] 103. Perform multimodal interaction mode analysis on the target object based on the multiple rounds of dialogue information and the corresponding response information to obtain a psychological assessment report of the target object.

[0034] In an embodiment of the present application, the psychological assessment strategy is used to guide the large model to generate dialogue content that meets the current interview task and the psychological assessment indicators. Exemplarily, the psychological assessment strategy includes at least one of the following: key psychological assessment indicators, key information types, questioning strategies, and reply skills guidance. The key psychological assessment indicators include at least one of the following: emotional response, behavioral performance, and physiological response; the questioning strategy includes at least one of the following: open-ended questions, closed-ended questions, situational simulation questions, and emotional support; the reply skills guidance includes at least one of the following: follow-up questions, switching topics, empathizing with users, and clarifying methods.

[0035] Specifically, the psychological assessment strategy is the guide for the entire psychological assessment process, aiming to enable the large model to generate conversation content that closely matches the current interview task and psychological assessment indicators, ensuring the accuracy and effectiveness of the assessment direction, and covering multiple key elements. Emotional response mainly focuses on the emotional states such as joy, sadness, anger, anxiety, etc. shown by the assessment subject in the conversation, and understands their psychological state by capturing and analyzing these emotions. For example, frequent sighing may indicate anxiety. Behavioral performance, pay attention to the daily behavioral habits and special behavioral actions mentioned by the assessment subject, such as whether there are compulsive behaviors such as repeated checking of doors and windows, which may reflect specific psychological traits. Physiological reactions include whether physical reactions such as sleep disorders, headaches, and accelerated heartbeats are mentioned. These physiological phenomena may be associated with psychological stress or emotional problems. Open-ended questions such as "Can you talk about things that make you feel stressed in your life recently?" encourage the assessment subject to express his thoughts and feelings freely and obtain richer and more detailed information. Closed-ended questions such as "Do you often suffer from insomnia? Yes / No" are used to quickly confirm whether specific facts or behaviors exist, obtain clear answers, and facilitate accurate judgment. Situational simulation questions, such as "What would you do if you suddenly encounter a malfunction in the elevator?", set up specific situations, observe the coping methods of the test subjects, and understand their psychological characteristics and behavior patterns. Emotional support, mainly reflects the understanding and acceptance of the test subject's emotions in the questions, such as "I can understand your current troubles, are you willing to tell me how you deal with these difficulties?", which helps to establish a trusting relationship and make the test subject more willing to open up. Follow-up questions are mainly used to further inquire about details when the test subject's answer is vague or incomplete, such as "You said that your work is not going well, can you tell me specifically what the problem is?", in order to obtain more in-depth and accurate information. Switching topics: If the current topic is deadlocked or enough information has been obtained, switch to other related topics in a timely manner to keep the conversation smooth and comprehensive, such as switching from work pressure to hobbies, so as to understand the test subject from different angles. Empathy users are used to express their recognition and understanding of the test subject's feelings, such as "It sounds like you have been through a difficult time, which must be very uncomfortable for you", to enhance the emotional connection between each other and improve the participation and comfort of the test subject. The clarification method is mainly to confirm the true intention of the evaluation subject when there is ambiguous answer by asking "Do you mean...?" to avoid misunderstanding and ensure the accuracy of information.

[0036] In the embodiment of the present application, the psychological assessment strategy in the non-first round of interaction is adjusted in real time based on the response information of the target object during the interaction. The specific adjustment method is described below and will not be expanded here.

[0037] This dynamic adjustment method can make the assessment process highly fit the individual situation of the target object, as if it is tailor-made for them. Flexible changes in questions and guidance directions based on the target object's response avoid rigid processes, more accurately tap into deep psychological information, and improve assessment accuracy. At the same time, it makes the conversation natural and smooth, enhances the target object's sense of participation and cooperation, and reduces the resistance caused by fixed patterns, thereby optimizing the overall assessment effect and making psychological assessment more scientific, effective, and humane.

[0038] In the embodiment of the present application, the psychological assessment report includes at least: a quantitative analysis result and a qualitative psychological analysis result for the conversation content. Further optionally, the qualitative psychological analysis result is associated with the psychological assessment index.

[0039] For example, in a psychological assessment of a professional, the quantitative analysis results may be specific values ​​such as 70 (out of 100) for anxiety and 40 for stress coping ability, which are obtained through scoring on a specific scale. These values ​​directly present the degree of their psychological state. The qualitative psychological analysis results may be described as "In the conversation, the person frequently mentioned concerns about the amount of work tasks, and showed obvious tension when describing the work scene. Combined with his language expression and emotional response, it can be judged that he is in a high state of anxiety, which may have a certain impact on his work efficiency. It is necessary to pay attention to and appropriately intervene in his psychological state." Such psychological assessment reports are effective. On the one hand, the quantitative analysis results provide an accurate quantitative presentation of the psychological state, which is convenient for horizontal and vertical comparisons. On the other hand, qualitative analysis is associated with psychological assessment indicators, which deeply interprets the psychological reasons and potential impacts behind the values, and provides a detailed and targeted basis for the subsequent psychological counseling and intervention measures. Both professional psychological counselors and the subjects themselves can clearly understand the overall picture of their psychological state, so as to better take action to improve their psychological state.

[0040] Optionally, in the psychological assessment report generation stage, with the help of the data integration analysis and report generation module based on the big model, the results obtained from the facial micro-expressions in the video, the pitch rhythm of the voice, and the text semantics are summarized and integrated, and the internal connections and patterns between these different modal signals are deeply excavated and identified. According to the pre-set interview indicator system, the system uses scientific methods to carry out a comprehensive analysis combining qualitative and quantitative analysis. In terms of quantitative analysis, specific values ​​such as the emotional intensity index and the mental health score will be obtained, which intuitively presents the degree of the psychological state of the subject; in terms of qualitative analysis, the psychological condition of the subject is elaborated in detail, including the analysis of possible psychological problems and targeted recommended intervention measures, such as describing the emotional fluctuations and thinking characteristics shown by the subject in the communication, and pointing out the potential sources of psychological distress. Finally, the information collected and analyzed by all modules is integrated to generate a detailed evaluation report that covers comprehensive analysis conclusions and practical suggestions. This report can not only provide key decision-making basis for professional psychological practitioners in diagnosis and treatment plan formulation, but also enable the subjects to clearly understand their own mental state, thereby providing direction and guidance for their self-adjustment and improvement, helping them to better cope with psychological problems and improve their mental health level.

[0041] Further optionally, in the non-first-round interaction process, a multimodal interaction mode analysis can be performed on the target object based on the multi-round dialogue information and the corresponding response information to obtain the target object's real-time psychological evaluation information; and the psychological evaluation strategy in the non-first-round interaction process can be dynamically adjusted based on the real-time psychological evaluation information. In this way, the evaluation strategy and difficulty can be automatically adjusted according to different user personal information, reactions and progress in the dialogue process, so as to ensure the personalization and optimal effect of the subsequent evaluation process, and prepare for the next round of interaction or evaluation tasks.

[0042] In the above or following embodiments, in 101, according to the current interview task and the corresponding psychological assessment index, a first psychological assessment strategy for the target object is created through a strategy creation model. Further, in 102, based on the first psychological assessment strategy, a first dialogue information is generated through an assessment interaction model; the first dialogue information is a first round of assessment dialogue with the target object; the first response information fed back by the target object based on the first dialogue information is obtained; based on the dialogue information, response information and the psychological assessment index of the current round, the psychological assessment strategy of the current round is adjusted through an interaction strategy model; the second dialogue information is generated through an assessment interaction model using the adjusted second psychological assessment strategy; the second dialogue information is a second round of assessment dialogue with the target object generated after real-time adjustment of the questioning strategy; the second response information fed back by the target object based on the second dialogue information is obtained; jump to the dialogue information, response information and the psychological assessment index of the current round, the psychological assessment strategy of the current round is adjusted through an interaction strategy model, and the adjusted psychological assessment strategy is used to execute the next round of assessment dialogue until the current interview task is completed or the target object ends the interaction operation.

[0043] For example, taking the evaluation of an employee who experiences work stress as an example, in the psychological evaluation process, first based on the interview task of evaluating his work stress and the corresponding psychological evaluation indicators, the strategy creation model generates the first psychological evaluation strategy for the employee, such as asking open-ended questions such as whether there are any tasks at work that make him feel overwhelmed, starting the first round of evaluation dialogue, and obtaining his first response information. Then, through the interactive strategy model, the system combines the dialogue and response information and the evaluation indicators to adjust the strategy. For example, if it is found that the employee is sensitive to workload, the next round will focus on asking about the details related to the workload, generate the second dialogue information, and repeat the adjustment and dialogue steps after the employee feedback, and repeat this cycle until the work stress evaluation task is completed or the employee stops interacting.

[0044] This method is very effective and can optimize dialogue strategies in real time based on individual employee situations, accurately explore the details of their psychological states, avoid stereotyped questions, make assessments more targeted and accurate, effectively improve assessment quality, and allow the final psychological assessment results to better reflect employees' true psychological conditions, enhance their sense of participation, and improve the scientificity and practicality of the entire psychological assessment process.

[0045] Further optionally, in 101, the step of creating a first psychological assessment strategy for the target object through a strategy creation model according to the current interview task and the corresponding psychological assessment indicators can be implemented as follows: obtaining the current interview task; performing information demand analysis on the current interview task through a strategy creation model to obtain key information types and key psychological assessment indicators that match the current interview task; determining questioning strategies for key psychological assessment indicators; and generating a dialogue flow control method corresponding to the current interview task based on the key psychological assessment indicators, key information types, and questioning strategies. In 102, the step of generating the first dialogue information through a strategy creation model based on the first psychological assessment strategy can be implemented as follows: generating the first dialogue information based on key psychological assessment indicators, key information types, questioning strategies, and corresponding dialogue flow control methods.

[0046] For example, suppose a person who is facing career transition confusion needs to be psychologically assessed. First, the system obtains the interview task of career transition, and then the strategy creation model conducts information demand analysis. For example, the key information types are determined to be the specific manifestations of career confusion, personal strengths and concerns, etc.; the key psychological assessment indicators include anxiety, self-awareness clarity, decision-making tendency, etc. According to these key indicators, the questioning strategy is determined, such as using open-ended questions to ask "Can you tell me in detail how you feel when considering career transition?" to understand anxiety, and using situational simulation questions such as "If there are two career opportunities in different directions, what will you base your choice on?" to examine decision-making tendencies. Then, based on these elements, a dialogue process control method is generated, starting with a broad introduction to career confusion, and then gradually deepening into psychological state and decision-making thinking. Based on the first dialogue information generated in this way, the assessment can be started naturally, smoothly and in a targeted manner.

[0047] The effect of this approach is that it allows the assessment to accurately locate the required information from the very beginning, and through a scientific combination of strategies, the conversation is closely centered around key points, which not only improves efficiency but also ensures that the psychological state of the target object in a specific situation can be fully and deeply explored, providing a solid foundation for subsequent assessment adjustments and final result determination, making the entire psychological assessment process more systematic, professional, and effective, and better serving the target object's psychological state analysis and response strategy formulation.

[0048] It is worth noting that in some optional examples, first of all, the current interview task needs to be obtained, which is the starting point for the formulation of the entire psychological assessment strategy. For example, if the work stress assessment is conducted on professionals, then this task clarifies the direction of subsequent work. Then, the task is deeply analyzed through the strategy creation model to conduct information needs analysis. In this process, the key information types that match the current interview task are determined. For example, for work stress assessment, the key information types may include information on the amount of work tasks, working hours, and relationships with colleagues or superiors. At the same time, key psychological assessment indicators are clarified, such as anxiety, stress coping ability, and degree of professional burnout. The determination of these key information types and key indicators lays the foundation for subsequent accurate questioning and effective assessment, ensuring that information reflecting the psychological state of the target object can be collected comprehensively and targetedly, avoiding blind questioning and information omissions. After determining the key psychological assessment indicators, it is crucial to formulate appropriate questioning strategies for these indicators.

[0049] For different indicators, a variety of questioning methods are used. For example, for more abstract and subjective indicators such as understanding the emotional state of the target object, open-ended questions can allow them to fully express their feelings, such as "Can you tell me in detail what the most stressful moment was at work?" For some behavioral frequency or factual indicators, closed-ended questions can quickly get clear answers, such as "Do you often suffer from insomnia due to work pressure? Yes / No"; situational simulation questions are helpful to examine the target object's response method in a specific work pressure scenario, such as "If a team member suddenly falls ill and cannot work before the deadline of an important project, what measures will you take?" In addition, appropriate emotional support questions, such as "I understand your hard work now, are you willing to tell me how you persevered?" can help establish a good communication atmosphere, make the target object more willing to cooperate in answering questions, and improve the quality and efficiency of information collection.

[0050] Through this targeted combination of questioning strategies, the psychological state of the target object can be deeply explored from multiple angles, ensuring that the information obtained is rich and accurate, providing strong support for accurate evaluation. Based on the key psychological assessment indicators, key information types and questioning strategies determined previously, the dialogue process control method corresponding to the current interview task is further generated. This includes planning the order of questions, deciding whether to continue to dig deeper into the current indicator information or switch to the next indicator based on the target object's answer, or backtracking and supplementing questions when necessary. For example, if the target object is emotionally excited and mentions cooperation problems with colleagues when answering questions about the amount of work tasks, then the dialogue process control may decide to suspend the questioning of the amount of tasks and instead ask questions related to the relationship with colleagues to better understand the root of their emotions and the overall picture of work pressure.

[0051] Finally, based on these key elements and the dialogue process control method, the first dialogue information is generated through the strategy creation model. For example, based on the preliminary analysis of the work stress assessment of professionals in the workplace, the first dialogue information generated may be "Hello, let's talk about your current workload first. Have you found any tasks particularly heavy or difficult to complete recently?" Such dialogue information not only revolves around key psychological assessment indicators, but also follows a reasonable questioning strategy and dialogue process control. It can naturally and smoothly start a psychological assessment dialogue with the target object, guide them to gradually reveal more information about their own psychological state, and lay a good foundation for subsequent in-depth assessment and accurate evaluation, ensuring that the entire psychological assessment process is carried out scientifically, orderly, and efficiently, and ultimately achieves accurate grasp and comprehensive evaluation of the psychological state of the target object.

[0052] Further optionally, in 102, the step of adjusting the psychological assessment strategy of the current round through the interaction strategy model based on the dialogue information, response information and the psychological assessment index of the current round can be implemented as follows:

[0053] Through the emotion analysis layer in the interaction strategy model, emotion analysis is performed on the target object based on the first dialogue information and the first response information to obtain the real-time emotion characteristics of the target object during the interaction process; through the intention recognition layer in the interaction strategy model, user intention is recognized on the target object based on the first dialogue information and the first response information to obtain the behavior pattern characteristics of the target object during the interaction process; through the adversarial sample generation layer in the interaction strategy model, diversified dialogue scene data corresponding to the target object are constructed; through the adjustment layer in the interaction strategy model, the first psychological assessment strategy is dynamically adjusted according to the real-time emotion characteristics, behavior pattern characteristics and the diversified dialogue scene data of the target object during the interaction process to obtain the second psychological assessment strategy.

[0054] Among them, the interactive strategy model is the key mechanism used to dynamically adjust the assessment strategy according to the feedback from the target object in the entire psychological assessment process. It consists of multiple functional layers that work together to enable the assessment strategy to adapt to the status of the target object in real time, making the entire assessment dialogue more targeted and effective.

[0055] Emotional analysis layer: During the interaction process, the target object's emotions are an important clue to understanding their psychological state. The emotional analysis layer works based on the first dialogue information and the corresponding first response information. For example, features are extracted from the vocabulary, tone, intonation (if there is voice information) used by the target object in the conversation, and natural language processing technology and sentiment analysis algorithms are used to identify the emotional information contained therein, such as different emotional states such as happiness, sadness, anxiety, anger and their intensity, and then summarize the real-time emotional characteristics of the target object during the interaction process. This helps the evaluator understand the emotional impact of the current conversation on the target object, provides a basis for subsequent adjustment strategies, and avoids the target object from having resistance or negative emotions due to improper questions, which affects the evaluation effect.

[0056] Intent recognition layer: Also based on the first dialogue information and the first response information, the intent recognition layer is dedicated to analyzing the target object's true intentions and behavioral pattern characteristics. It attempts to understand the target object's intrinsic motivation for participating in the assessment, what he wants to express or hide, and the behavioral patterns he exhibits when facing different questions by analyzing factors such as the angle from which the target object answers questions, the focus of the information provided, and the attention paid to different topics. For example, if the target object always avoids certain topics but elaborates on specific types of topics, the intent recognition layer can capture this behavior pattern and infer its potential intentions, whether it is due to shyness, worry, or other psychological factors, thereby clarifying the direction for subsequent strategy adjustments and ensuring that the assessment can be carried out in depth along the psychological trajectory of the target object and dig out more real and accurate psychological information.

[0057] Adversarial sample generation layer: The core of this layer is to construct diversified dialogue scenario data corresponding to the target object. Considering that each person's psychological state and reaction will be different in different scenarios, rich and diverse sample data is generated by simulating various possible dialogue scenarios. For example, for a person who is undergoing a professional psychological assessment, the adversarial sample generation layer may simulate the corresponding dialogue content in different scenarios such as facing an urgent project at work, conflict with colleagues, and being praised by leaders. These scenario data cover a variety of situations and emotional triggers, providing a reference for a comprehensive understanding of the psychological reactions of the target object in different situations, helping to more flexibly respond to various complex situations that may arise in the actual assessment, and making the subsequent adjustment of the assessment strategy more adaptable and comprehensive.

[0058] Adjustment layer: The adjustment layer plays an integrating role. It dynamically adjusts the first psychological assessment strategy based on the real-time emotional features obtained by the previous emotional analysis layer, the behavioral pattern features obtained by the intention recognition layer, and the diversified dialogue scene data constructed by the adversarial sample generation layer. For example, if the emotional analysis shows that the target object is nervous and anxious, and the intention recognition finds that it avoids certain key questions, and combined with the effective coping methods in similar scenarios in the diversified scene data, the adjustment layer may change the questioning strategy, changing some direct closed questions to more euphemistic open questions, or temporarily switching to a lighter related topic to relieve tension, so as to generate a second psychological assessment strategy that is more in line with the current state of the target object, ensure that the assessment dialogue can be carried out continuously, smoothly and efficiently, and maximize the excavation of the target object's true and comprehensive psychological condition, and improve the accuracy and scientificity of the assessment.

[0059] Based on the same principle, other exemplary methods and improvement schemes for implementing the interaction strategy model are introduced below.

[0060] First of all, it needs to be explained that the intention recognition algorithm in the embodiment of the present application aims to accurately capture and analyze the user's intentions, psychological state and required psychological indicators during the psychological assessment interaction process through a variety of advanced deep learning technologies, and then dynamically adjust the dialogue strategy to achieve efficient, personalized and accurate psychological assessment.

[0061] For example, recurrent neural networks (RNNs) and long short-term memory networks (LSTMs). In psychological assessment conversations, users' answers are often contextual and time-dependent. For example, when discussing work stressors, the previous answer may affect the direction and content of subsequent answers. RNNs and LSTMs can handle this kind of sequence data well. By processing the input (words, sentences, etc. in the user's answer) at each time step and combining the hidden state of the previous moment, the understanding of the entire conversation context is continuously updated, thereby more accurately capturing user intentions and complex conversation patterns. For example, when a user is describing an event, the model can remember the key information mentioned previously, better understand the meaning of the user's current expression, improve the accuracy of intent recognition, and provide a solid foundation for the formulation of appropriate subsequent conversation strategies.

[0062] Alternatively, transformer models such as BERT and GPT have powerful language understanding and generation capabilities. In psychological assessment scenarios, they can deeply analyze the text input by users and capture fine language structures and rich contextual relationships. For example, when users use metaphors or euphemisms to mention their emotional problems, these models can rely on pre-trained knowledge of large amounts of text data to understand their underlying intentions, and then generate more contextual and targeted conversation content, guiding users to express their mental state more clearly and improving the effectiveness of the assessment.

[0063] In addition, in psychological assessment intention recognition, Generative Adversarial Networks (GANs) are mainly used to generate diverse dialogue strategies. Through the adversarial training process of the generator and the discriminator, it continuously simulates real user interaction scenarios and generates various possible dialogue content and strategies. For example, when facing users with different personalities and psychological states, GANs can provide a variety of coping methods to help the model better identify user intentions, while improving the quality and diversity of generated dialogue content, making dialogue strategies more flexible and adaptable to different assessment scenarios, thereby more accurately mining users' psychological information.

[0064] In the process of adjusting the strategy, sentiment analysis technology is used to judge the emotional state of the user's answer. For example, if the user uses a lot of negative words and a heavy tone when describing his work experience, the system can judge that his mood is relatively low, and thus adjust the tone of the conversation to a more gentle, soothing and supportive language style, such as "It sounds like you have experienced a lot of difficulties. If you are willing, you can tell me in detail. Maybe it will make you feel better." This helps to establish a good communication atmosphere, making users more willing to open up, and avoid aggravating users' negative emotions due to improper conversation methods, affecting the accuracy and smooth progress of the evaluation.

[0065] Through the real-time intention recognition model, the user's current needs and goals can be quickly analyzed. For example, when a user mentions concerns about future career development and expresses anxiety, the system can keenly capture this intention and promptly guide the user to conduct relevant psychological assessments, such as the question-and-answer session of the career anxiety assessment index, so that the conversation direction is closely centered on the user's psychological needs, and more accurately obtain information related to the user's psychological state, providing strong support for comprehensive assessment.

[0066] By monitoring the user's interaction frequency and response time to answer questions, the system can roughly judge the user's participation and attention status. If the user responds slowly, it may mean that he is thinking or is not familiar with the current topic, or has resistance, etc. At this time, the system can reduce the complexity of the question accordingly, split the complex question into simpler and easier to understand small questions, or add some encouraging words, such as "Don't worry, answer slowly, you have done a good job." to increase the user's participation and enthusiasm and ensure that the conversation can continue effectively.

[0067] Analyzing the behavioral characteristics of users in conversations, such as changes in tone, pause time, word usage habits, etc., can provide more clues for understanding the user's psychological state. For example, if a user suddenly raises his voice or pauses frequently, it may indicate his inner tension or emotional fluctuations. The system can adjust the conversation strategy accordingly, such as pausing the current sensitive topic, chatting lightly or soothing the user's emotions, and then continuing to discuss key issues in depth after the user's emotions have stabilized, so as to more accurately capture the user's psychological state and improve the scientificity and effectiveness of the evaluation.

[0068] In psychological assessment interactions, making full use of multimodal data can provide a more comprehensive understanding of the user's status. For example, by using speech recognition technology to analyze the user's tone, speaking speed, volume and other characteristics, we can determine their emotional fluctuations; combined with facial expression recognition, we can capture the user's micro-expressions, such as frowning and smiling, to further verify their emotional state; and then combine it with the text content to use a unified algorithm framework for comprehensive analysis. For example, when the user's voice trembles and facial expression is tense when answering questions, and the text content also reveals anxiety, the system can more accurately judge the user's psychological state, and adjust the dialogue strategy accordingly, and continue the dialogue in a gentler and more soothing way, ensuring that the user's psychological state is more accurately and comprehensively grasped during the assessment process, thereby improving the reliability of the assessment.

[0069] Based on the real-time feedback mechanism of multimodal data, the system can dynamically adjust the content and tone of the conversation to ensure the continuity of the conversation and a good user experience. For example, when the user is found to be impatient through voice and expression, the system can immediately adjust the rhythm of the conversation, speed up the switching of questions or change the way of asking questions to avoid users from getting bored due to long waiting times or complex questions, making the conversation more natural and smooth, and improving the user's willingness and cooperation in participating in the assessment, thereby improving the efficiency and quality of the entire assessment process.

[0070] Using reinforcement learning algorithms, the system continuously tries different dialogue strategies during the interaction with users, and gives corresponding rewards or penalties based on user feedback (such as the level of detail of answers, improvement in emotional state, etc.), thereby gradually optimizing dialogue strategies. For example, if the system uses a new way of asking questions to guide users to explain their psychological problems in more depth and obtain more valuable information, then this strategy will be strengthened and more likely to be adopted in subsequent dialogues, thereby continuously improving the effectiveness of dialogue strategies, better meeting users' psychological assessment needs, and achieving more accurate psychological assessments.

[0071] Based on the user's feedback and behavior data during the assessment process, the user model is updated in real time, making the system's understanding of the user more in-depth and accurate. For example, if the user shows psychological characteristics or behavior patterns that are different from previous expectations when answering questions, the system will promptly update the relevant parameters in the user model to provide a more personalized and accurate conversation experience in subsequent conversations, better adapt to the individual differences of users, and improve the pertinence and accuracy of psychological assessments.

[0072] Exemplarily, the embodiment of the present application also provides a process control center, which mainly judges and controls the entire dialogue process in a macroscopic manner based on the current user input (including multimodal information such as text and voice), the previous round of input of the model, and the state of psychological indicators, and determines the direction of generating content for the next round of models. For example, by comprehensively analyzing the user's current answer to a psychological indicator question, combined with the previous dialogue history and the unfinished psychological indicator tasks, it is judged whether to continue to ask questions based on the current indicator to obtain more detailed information, or to return to a previous indicator for additional inquiries, or to end the current indicator and enter the next more relevant indicator for evaluation, or even suspend the current indicator to handle some special situations (such as the user's emotional instability needs to be comforted). The purpose is to ensure that the dialogue can be carried out in an orderly manner according to the requirements of the psychological assessment, efficiently complete the comprehensive assessment of the user's psychological state, and optimize the dialogue process to improve the efficiency and accuracy of the assessment.

[0073] Exemplarily, multimodal data such as text, voice or image currently input by the user is obtained through corresponding sensors and data acquisition technologies. For example, voice data is collected using a microphone and converted into text, or text information input by the user in the interactive interface is directly obtained, and the user's facial expression image is captured by a camera, etc. These data are the direct source of understanding the user's current psychological state and intention.

[0074] The user's past input data will be continuously collected and organized, including contextual information such as the content of previous conversations, the order of questions asked, and the user's answers. These historical data are important for understanding the user's thought process, emotional trends, and the focus of previously discussed topics. They can help the model better grasp the user's overall psychological state and make more appropriate strategic choices and responses in the current conversation.

[0075] The user's psychological assessment index data is obtained and updated in real time from the task setting of the psychological assessment. These indicators cover multiple aspects such as emotional state, stress level, anxiety level, personality traits, etc., and each indicator may have different quantification methods and evaluation criteria. For example, emotional state can be quantified by the score of sentiment analysis, and stress level can be comprehensively judged and assigned based on factors such as workload and life events mentioned by the user. These indicator data provide clear goals and directions for the entire intention recognition and dialogue strategy adjustment process, ensuring that the dialogue revolves around the accurate evaluation of these psychological indicators, and ultimately achieving comprehensive and accurate psychological assessment.

[0076] For data of different modalities, a special algorithm is used to convert them into a unified vectorized feature representation to facilitate subsequent model processing. For example, for text data, word embedding technology (such as Word2Vec, GloVe, etc.) can be used to map words to a low-dimensional vector space; for speech data, its audio features (such as Mel frequency cepstral coefficients, etc.) can be extracted and quantized; for image data (such as facial expression images), convolutional neural networks can be used to extract facial feature vectors. In this way, multimodal data is converted into a unified format that the model can understand and process, giving full play to the advantages of data of different modalities and improving the comprehensive judgment ability of the user's psychological state.

[0077] From the acquired psychological indicator status data, key psychological indicators are extracted and quantified to form numerical features that can be directly used by the model. For example, the anxiety level is numerically encoded according to different levels such as mild, moderate, and severe, or the positive and negative levels of the emotional state are numerically quantified, so that psychological indicators can be effectively integrated and calculated with other input features, providing an accurate basis for the model's decision-making, helping the model to better understand the user's status in various psychological dimensions, and thus formulate more accurate dialogue strategies.

[0078] The feature vectors of the current user input, historical input data, and psychological indicator status are integrated to form a comprehensive feature vector as the input of the deep learning model. This comprehensive feature vector integrates the user's current expression, past conversation history, and psychological assessment target information, allowing the model to fully understand the user's situation, better capture the user's intentions and psychological state changes, and provide strong support for subsequent dialogue strategy generation, ensuring that the dialogue can be closely centered on the user's psychological assessment needs, and improving the efficiency and accuracy of the assessment.

[0079] Use deep learning models such as recurrent neural networks (RNN) and long short-term memory networks (LSTM) to process the fused sequence data. These models can effectively process dialogue data with time series properties, capture contextual information and time dependencies in user answers, and better understand the evolution of user intentions. At the same time, through the attention mechanism, the input data is contextually modeled based on the Transformer model, so that the model can pay more attention to key information in the input data, such as key words when users describe psychological problems, key frames of expression changes, etc., so as to more accurately capture the potential intentions and emotional changes of user input, provide a more accurate basis for generating appropriate dialogue strategies, and improve the performance and effect of the entire intention recognition and dialogue strategy adjustment process.

[0080] The model first outputs its understanding and judgment of the conversation status, that is, it analyzes the current user's psychological state, intention, and the stage and situation of the conversation based on the input comprehensive feature vector. For example, it determines whether the user has fully expressed his views on a certain psychological indicator, or whether he has resistance to the current topic. Then, based on the judgment of the dialogue state, the strategy for generating the next round of content is determined, including but not limited to continuing to ask questions about the current topic to obtain more detailed information (such as when the user mentions the stress source at work but the description is not detailed enough, continue to ask about specific events and feelings), return to a previously discussed indicator for supplementary inquiries (such as when it is found that the current topic is related to a psychological indicator that has not been discussed in depth before, return to re-ask the relevant details), enter the next psychological indicator for evaluation (when the information collection of the current indicator is basically completed and meets the conditions for entering the next indicator, naturally transition to the dialogue of the next psychological indicator), enter the most relevant indicator (based on the user's current psychological state and input content, select the psychological indicator that is most closely related to the current emotion or problem for priority discussion to improve the pertinence and efficiency of the evaluation), and suspend the current indicator (when the user's emotions are unstable or special circumstances occur in the dialogue, suspend the current topic and take corresponding measures to adjust it, such as soothing the user's emotions, waiting for the user to further input, etc.). Through this staged output and strategy adjustment, the dialogue can be carried out flexibly and efficiently according to the user's actual situation, ensuring that the psychological evaluation process can comprehensively and accurately obtain the user's psychological state information, providing a solid foundation for generating accurate psychological assessment reports.

[0081] In actual applications, the intent recognition algorithm code will not be updated in real time. A wealth of data has been collected in a large number of previous psychological assessment projects, and the intent recognition algorithm has been fully trained using this data. Such a trained algorithm has a strong generalization ability and can adapt to the intent recognition needs of different users in various psychological assessment scenarios. By learning a large amount of user data of different ages, genders, occupations, and psychological states, the algorithm can learn various common conversation patterns, intent expressions, and psychological characteristics, so that in new assessment tasks, it can quickly and accurately identify user intentions and adjust conversation strategies without the need for real-time updates during each assessment process, which improves the operating efficiency and stability of the system, while also ensuring the accuracy and reliability of the assessment results.

[0082] In some specific cases, if the algorithm needs to be updated based on new psychological indicators or special assessment needs, the algorithm can be trained by automatically synthesizing data. For example, when introducing new psychological assessment dimensions or conducting assessments on specific groups (such as patients with rare mental illnesses), it may be necessary to generate some specific data to train the algorithm so that it can better identify and process user intentions and dialogue strategies related to these new situations. This approach can expand the applicability and accuracy of the algorithm to a certain extent, but it requires relatively more computing resources and data processing capabilities, and the method of synthesizing data needs to be carefully designed to ensure that it can truly reflect user behavior and psychological characteristics in actual assessment scenarios, and avoid algorithm performance degradation due to data bias.

[0083] To sum up, through the above series of complex and sophisticated intention recognition algorithms, dynamic dialogue strategy adjustment mechanisms and specific implementation steps, the system can realize a highly intelligent, personalized and precise psychological assessment interaction process, effectively improve the quality and efficiency of psychological assessment, and provide users with more accurate and comprehensive psychological assessment services.

[0084] In the above or following embodiments, in 103, the step of performing a multimodal interaction mode analysis on the target object based on the multi-round dialogue information and the corresponding response information to obtain a psychological assessment report of the target object can be implemented as the following steps:

[0085] 301. Performing multimodal feature extraction on the multi-round dialogue information and the corresponding response information through a feature extraction layer in a multimodal analysis model to obtain multimodal psychological features;

[0086] 302. Performing feature evaluation on the multimodal psychological features respectively through multiple psychological feature evaluation layers in the multimodal analysis model to obtain a multimodal psychological feature evaluation result;

[0087] 303. Through the fusion layer in the multimodal analysis model, the multimodal psychological feature assessment results are fused to obtain the psychological assessment report.

[0088] The multimodal psychological features include at least: conversation text features, conversation audio features, and facial image features. Accordingly, corresponding feature change analysis strategies are configured in each of the multiple psychological feature evaluation layers.

[0089] Specifically, the feature extraction layer (301) uses natural language processing technology to analyze the vocabulary, grammar, semantics, etc. in the multi-round dialogue information. For example, through word frequency statistics, we can understand the topics or emotional words frequently mentioned by the target object, and use semantic analysis to determine the emotional tendency of the sentence (positive, negative or neutral). Syntactic structure analysis helps to understand the thinking logic and expression habits of the target object, so as to extract text features that reflect their psychological state, such as whether there is a negative thinking pattern, the fluency and logical changes of expression, etc. These features can indicate their psychological pressure, emotional stability and other aspects.

[0090] For example, features such as pitch, volume, speaking speed, and timbre can be extracted from the conversation audio. For example, when the target subject is in a tense state, the pitch may rise, the speaking speed may increase, and the volume may be unstable; while in a relaxed state, the audio features are often relatively stable and soothing. By extracting and analyzing these audio features, the emotional fluctuations and changes in the psychological state of the target subject during the conversation can be judged, and information such as emotional intensity and immediate reaction that may be missing in the text information can be supplemented.

[0091] For example, computer vision technology is used to analyze facial expressions, eye contact, facial muscle movements, etc. in facial images. For example, facial features such as smiles, frowns, wandering eyes, focused eyes, and the degree of tension of facial muscles can be identified. These facial features are closely related to emotional states. For example, frowns may indicate confusion, anxiety, or dissatisfaction, and smiles may indicate relaxation, pleasure, or friendliness. By extracting these features, intuitive visual information is provided to understand the psychological state of the target object.

[0092] Multiple psychological characteristics assessment layers (302) use a strategy based on the combination of psychological theory and language analysis for the dialogue text characteristics assessment. For example, based on the theory of cognitive behavioral therapy, the automatic thinking patterns of the target object are analyzed, and the potential risk level of psychological problems, such as depression tendency, anxiety level, etc., is assessed by identifying and counting the sentences such as self-denial and catastrophic thinking that appear in their speech. At the same time, combined with language style analysis, for example, a concise and clear language style may imply self-confidence and decisiveness, while a lengthy, complex, and ambiguous language style may reflect inner contradictions and uncertainty, so as to comprehensively judge their psychological characteristics.

[0093] For the evaluation of the characteristics of the audio of the conversation, the principles of acoustic psychology and the emotion recognition algorithm are applied. According to the statistical laws and models of the audio characteristics under different emotional states, such as by establishing the mapping relationship between emotions such as anger, sadness, and joy and audio characteristic parameters (such as fundamental frequency, resonance peak, etc.), the emotional category and intensity of the target object can be judged. At the same time, the rhythmic characteristics of the audio (such as pauses, stress, etc.) are combined to analyze the key points of expression and the ups and downs of emotions, and further understand the dynamic process of its psychological state.

[0094] For facial image feature assessment, we rely on theories and technologies such as the Facial Action Coding System (FACS). By analyzing the combination and intensity of specific facial muscle movements, we can identify the expression of various basic emotions (such as happiness, sadness, fear, anger, disgust, surprise, etc.) and changes in micro-expressions. These micro-expressions can often reveal the subconscious emotional reactions of the target object in an instant, providing more delicate and real information for psychological assessment. Combined with factors such as the duration and frequency of expression, we can comprehensively assess the stability of their psychological state and potential emotional problems.

[0095] The fusion layer (303) fuses the multimodal psychological feature evaluation results obtained by each psychological feature evaluation layer. Data fusion technology, such as weighted average, decision-level fusion, etc., is used to assign corresponding weights according to the reliability and importance of different modal information. For example, in some cases, facial image features may be more intuitive and accurate for emotion recognition, and a higher weight is assigned; while in other cases, dialogue text features are more important for in-depth understanding of the target object's thinking mode and the root cause of psychological problems, and the weight is adjusted accordingly. Through this fusion method, multimodal information is integrated into an organic whole, and a psychological evaluation report containing quantitative analysis results (such as scores of various psychological indicators) and qualitative psychological analysis results (such as text descriptions of the target object's psychological state, diagnosis and suggestions for potential psychological problems, etc.) is generated, which comprehensively and systematically presents the psychological state and characteristics of the target object, provides decision support for professionals, and provides self-improvement guidance for the target object.

[0096] In this way, by comprehensively analyzing multimodal information such as conversation text features, conversation audio features, and facial image features, the psychological state and characteristics of the target object can be captured more comprehensively, avoiding the limitations of single-modal information. For example, judging from the text alone, the target object may express a relatively calm state, but from the trembling sound in the audio features and the tense expression in the facial image features, its potential anxiety can be found, making the psychological assessment more accurate and providing a more reliable basis for subsequent intervention and treatment.

[0097] This multimodal analysis method can deeply explore the target object's true feelings and thoughts. Information from different modes complements and confirms each other, which helps to discover some psychological problems hidden beneath the surface. For example, in the occupational stress assessment, the target object mentioned in the text that the work is busy but can still cope with it, but through the tired tone in the audio and the dim eyes in the facial image, it can be further judged that his psychological stress may have reached a high level, which has a potential negative impact on his work and life, thus gaining a deeper understanding of his psychological state.

[0098] Comprehensive evaluation based on multimodal features can achieve personalized analysis of each target object. Different people may express emotions and psychological states differently in different modalities. By extracting and analyzing these unique multimodal features, a unique psychological assessment report is generated for each target object to better meet the individual's psychological assessment needs and provide strong support for the development of targeted psychological counseling or treatment plans.

[0099] Further optionally, for facial image features, in one implementation of 302, feature evaluation is performed on the multimodal psychological features respectively through multiple psychological feature evaluation layers in the multimodal analysis model to obtain a multimodal psychological feature evaluation result, which can be implemented as follows:

[0100] Perform facial tracking detection on the facial image features to obtain visual change features of facial key points; perform micro-expression detection based on the visual change features to obtain the micro-expression type of the target object; and identify the emotional state of the target object based on the micro-expression type to obtain emotional assessment information of the target object.

[0101] In an embodiment of the present application, the emotion assessment information is used to indicate the emotional state of the target object displayed through facial expressions.

[0102] For example, advanced facial recognition and image processing technologies are used to analyze the facial micro-expressions of the subject (i.e., the target object) in the video. By identifying micro-expressions that appear quickly and last for a short time, the system can reveal the subject's true emotions that may be hidden or unable to be expressed through words during the interview.

[0103] Further optionally, in the above steps, micro-expression detection is performed based on the visual change feature to obtain the micro-expression type of the target object, including: extracting the corresponding facial action unit from the visual change feature; encoding the facial action unit, and performing change feature recognition on the encoding sequence of the facial action unit to obtain a first expression feature; performing tiny movement recognition on the visual change feature using an optical flow method to obtain a second expression feature; determining the micro-expression type based on the first expression feature and the second expression feature; wherein the micro-expression type is at least one of a plurality of micro-expression types pre-set based on a plurality of emotional states.

[0104] Here, expression features are extracted and integrated in different ways to identify the target object’s micro-expression type, thereby gaining insight into the individual’s psychological state and emotional fluctuations.

[0105] In the process of facial expression formation, different muscle movement combinations correspond to different expression changes, and facial action units (AUs) are the division of the basic elements of facial muscle movement. Extracting the corresponding facial action units from visual change features (such as facial dynamic images captured in the video) is to identify which muscles produce the action and the specific form of the action. This is similar to the basic link of using the Facial Action Coding System (FACS) for analysis mentioned in the reference content. For example, the action of frowning may involve several specific facial action unit combinations. By analyzing the facial visual change features in the video screen, the relevant action units are extracted to prepare for subsequent analysis.

[0106] This is consistent with the commonly used method of action unit (AUs) analysis in micro-expression detection mentioned in the reference content. Both methods use facial action units as the entry point to deeply analyze expression changes.

[0107] Encoding the extracted facial action units is to assign specific codes to each action unit according to certain standards and rules (such as the encoding rules set by FACS) so that it can be more clearly identified and processed by the computer system. Then, by observing the changing characteristics of these coding sequences in the time dimension, such as the activation order, duration, and intensity changes of the action units, a feature that can reflect the characteristics of the expression is summarized, namely the first expression feature. For example, a certain action unit is activated first, and then several other action units are activated in turn, and the intensity gradually increases. This specific coding sequence change feature may represent a unique expression trend, which can be used as an important basis for judging emotions.

[0108] This is also an extension of action unit analysis. The reference content emphasizes the use of FACS to encode and analyze facial action units. Here, expression features are obtained by further identifying the changing characteristics of the encoding sequence. This is a more detailed operation based on it, and the purpose is to explore the emotional information behind the expression.

[0109] The optical flow method can capture the tiny movements of an object (here refers to the facial area) by analyzing the movement of pixels in an image sequence. In facial expression analysis, even very subtle muscle movements will cause corresponding flow changes in facial pixels. By processing the visual change features using the optical flow method, the expression information contained in these tiny movements can be extracted to form a second expression feature. For example, the subtle movement of the corners of the mouth slightly raising can be tracked by the optical flow method to the specific movement pattern of the pixels around the corners of the mouth, which can be used as a feature dimension to reflect the expression state.

[0110] The reference content clearly points out that the optical flow method is one of the commonly used methods for capturing tiny movements of the facial area in micro-expression detection. The use of the optical flow method here to obtain the second expression feature is to take advantage of its advantages in this regard. It is consistent with the technical application ideas in the reference content, and both use the optical flow method to explore the expressions reflected by subtle facial changes.

[0111] Since different emotional states correspond to specific micro-expression types, and these micro-expression types can be reflected by different expression features. The first expression feature reflects the expression characteristics from the perspective of the changes in the facial action unit coding sequence, and the second expression feature reflects the expression from the perspective of the micro-movement captured by the optical flow method. These two expression features extracted from different dimensions are combined for analysis and matched with the pre-set multiple micro-expression types corresponding to multiple emotional states to determine the current micro-expression type. For example, if the first expression feature shows that the changes in the specific action unit coding sequence are consistent with the common pattern under the "surprise" emotion, and the second expression feature obtained by the optical flow method also reflects the corresponding facial micro-movement characteristics and the "surprise" micro-expression characteristics, then the current micro-expression type can be determined as "surprise".

[0112] This step is equivalent to the specific operational embodiment of the emotion classification link in the reference content. The reference content mentions the use of pre-trained emotion recognition models for emotion classification. Here, it is based on the two expression features extracted and analyzed previously. According to a similar classification idea, it corresponds to the pre-set micro-expression types, and then realizes the judgment of the emotional state. All of this is to ultimately accurately identify the individual's emotions and reflect his or her psychological state.

[0113] Through the above steps, a variety of technical means are used to analyze facial features from different angles, and finally the type of micro-expression is determined to interpret the emotional state of the individual at a specific moment, providing a strong basis for further understanding their psychological state and emotional fluctuations. This is also one of the key links in the entire psychological assessment process based on facial analysis.

[0114] Further optionally, for the dialogue audio features, in one implementation of 302, key features are first extracted from the dialogue audio data. For example, the fundamental frequency in the speech is detected by a corresponding algorithm to analyze the pitch change, the pauses, beats and speed of the speech are observed to extract the rhythm features, the volume change is obtained by amplitude analysis, and the number of speech segments per minute is calculated to determine the speech rate. In addition, the speech signal is converted into a frequency domain signal by using a short-time Fourier transform (STFT), and then the frequency domain signal is processed by a Mel filter bank to simulate the auditory characteristics of the human ear, and then the logarithmic power spectrum is calculated and a discrete cosine transform (DCT) is performed to obtain the Mel frequency spectrum coefficients (MFCCs), etc. These features can reflect the information contained in the audio from different angles.

[0115] Specifically, the collected conversation audio is first preprocessed, such as removing background noise and enhancing the quality of the voice signal. Then, voice features such as pitch, rhythm, volume, speaking rate, and MFCCs are extracted according to the above principles to prepare the data foundation for subsequent evaluation work such as emotion recognition.

[0116] Combining the above-mentioned various types of extracted speech features (including MFCCs) into a feature vector is like building a comprehensive "data package" that describes the audio features. Next, using a large amount of speech data that has been labeled with emotional labels (such as nervous, anxious, calm, positive, etc.), a machine learning or deep learning algorithm is used to train the model, such as support vector machine (SVM), random forest, convolutional neural network (CNN) or recursive neural network (RNN), etc., so that the model can learn the speech feature patterns corresponding to different emotional states. For example, select a suitable machine learning or deep learning algorithm, input the extracted feature vector into the model training, and continuously optimize the model through cross-validation, hyperparameter tuning and other methods to improve its accuracy and robustness in classifying different emotional states, so that it can accurately classify emotions when facing new conversation audio data.

[0117] Deploy the trained emotion recognition model to the actual multimodal analysis scenario, analyze the real-time speech produced by the subject during the conversation, and evaluate the subject's emotional state and the psychological characteristics that may be reflected based on the emotion classification results output by the model. For example, by identifying nervous emotions, it is inferred that the subject may be under great psychological pressure. In actual use, let the model process the real-time incoming conversation audio, and generate an evaluation report on the conversation audio features based on the results of the emotional state obtained by the analysis, as an important part of the audio part of the multimodal psychological feature evaluation results.

[0118] Further optionally, for the dialogue text features, in one implementation of 302, the words selected by the subject in the dialogue can often directly reflect his current psychological state and ideological tendency. For example, frequent use of negative words such as "sad", "depressed", "helpless", etc. may suggest that he has a tendency to depression or is in a negative mood; while more mentions of positive words such as "happy", "expectation", "satisfied", etc. may reflect a more optimistic mentality. At the same time, the frequency of occurrence of professional words related to specific psychological topics, such as "worry", "uneasiness", "panic" and so on related to anxiety, can also be used as a basis for judging the anxiety level. For example, the word frequency statistics tools and algorithms in natural language processing are used to scan and count the dialogue text, sort out the frequency of occurrence of various psychologically suggestive words, focus on those words associated with common psychological characteristics (such as depression, anxiety, optimism, etc.), and form text feature data on the vocabulary level.

[0119] Sentence analysis can be achieved by reflecting the expression habits and thinking logic of the test subject (i.e. the target object) through sentence structure, thereby reflecting their psychological characteristics. For example, the frequent use of simple and direct sentences may imply a more straightforward and decisive personality; while the frequent use of complex, lengthy sentences with many modifiers and clauses may reflect more complex inner thoughts, careful consideration of problems, or certain entangled emotions. In addition, some special sentence structures are also related to specific psychological states. For example, excessive use of rhetorical questions may reflect a strong self-defense mentality or a questioning attitude towards certain viewpoints.

[0120] For example, with the help of syntactic analysis technology in natural language processing, the sentence structure in the text is analyzed, the frequency and distribution of use of different sentence patterns are identified, and the text features displayed at the sentence level are summarized, which is used as a dimension to judge the psychological characteristics of the subjects.

[0121] Language style analysis can be realized as follows: language style covers multiple aspects such as the formality, vividness, and emotional color of the expression, and is an important clue to understanding the psychological state and personality characteristics of the subject. For example, a more formal and rigorous language style may imply that the person has a strong sense of responsibility and pays attention to norms in doing things; while an expression full of vivid images and rich emotional colors may reflect a personality trait of high openness, rich emotions, and good at expressing inner feelings. If the overall language style is negative and bland, lacking emotional ups and downs, it may indicate potential psychological problems, such as depression caused by depression and lack of interest in things.

[0122] For example, by conducting qualitative and quantitative analysis on the overall language style of the text, such as counting the number of times rhetorical devices are used, the richness of emotional vocabulary, the formality and casualness of expression, and other indicators, characteristic information on the language style can be extracted, laying the foundation for a comprehensive assessment of psychological characteristics.

[0123] Specifically, based on the previously extracted text features such as vocabulary, sentence patterns, and language style, they are quantitatively compared with various established psychological characteristic models or standards. For example, for depressive tendencies, there are professional psychological scales that set typical vocabulary, sentence patterns, and language style expressions corresponding to different degrees of depression. By comparing the text features of the subject with them, the degree of his or her depressive tendency can be roughly judged; for openness in personality traits, there is also a corresponding theoretical framework that describes the characteristics of language expression of individuals with high openness, and based on this, the level of the subject in the personality dimension of openness can be measured. In the same way, psychological characteristics such as anxiety level, optimism, and sense of responsibility can be quantitatively analyzed and matched accordingly.

[0124] For example, with the help of existing psychological research results and quantitative standards, data analysis and matching algorithms are used to transform and compare the feature data extracted from the text to obtain quantitative evaluation results of specific psychological characteristics such as depressive tendencies, anxiety levels, and optimism, as well as personality characteristics such as openness and responsibility, thus forming a psychological characteristic evaluation result corresponding to the text semantic information.

[0125] Considering that psychological characteristics are interrelated and influence each other, the evaluation results of a single characteristic cannot be viewed in isolation, and it is necessary to make an overall judgment based on all aspects. For example, although a subject shows a certain positive tendency in vocabulary, his sentence structure is complex and his language style is negative and restrained. At this time, it is necessary to comprehensively analyze these seemingly contradictory characteristics, combine psychological knowledge and experience, and infer his possible deep psychological state, such as whether he is optimistic on the surface but depressed in his heart, or whether he has contradictory psychological manifestations in specific situations, so as to form a more accurate and comprehensive evaluation conclusion on his overall psychological characteristics.

[0126] For example, the quantitative evaluation results of various psychological characteristics are integrated, and through professional psychological analysis methods and logical reasoning, combined with the overall context of multiple rounds of dialogue texts, a comprehensive judgment is made on the psychological characteristics and attitudes of the subjects, generating complete psychological characteristic evaluation results based on text semantic information as an important part of the multimodal psychological characteristic evaluation results.

[0127] Further optionally, in 303, a detailed description of the process of fusing the multimodal psychological feature assessment results to obtain the psychological assessment report is provided:

[0128] In the input stage, corresponding preprocessing operations are performed for data of different modalities, that is, video, voice and text-related data. For example, for video data, image normalization, cropping and other processing may be involved to ensure the consistency of image quality and size; for voice data, preprocessing such as noise reduction and volume normalization is required to improve the quality of voice signals; for text data, cleaning, word segmentation and other operations are performed. Then, feature extraction is performed using the respective adapted technologies, such as using convolutional neural networks to extract image features from video frames, extracting voice features such as pitch, rhythm, and Mel-spectrogram coefficients from audio signals, and obtaining text features such as vocabulary, sentence structure, and semantics from texts through natural language processing technology to form feature vectors corresponding to each modality.

[0129] The feature vectors extracted from different modalities are connected or spliced ​​in a certain way, for example, simply arranging the video feature vector, speech feature vector and text feature vector in sequence to form a joint feature representation. The purpose of this is to integrate the information of multiple modalities into a unified data structure, which is convenient for subsequent processing with a unified model, so that the model can simultaneously consider the association between the features of different modalities and mine more comprehensive information.

[0130] These joint features are processed and analyzed using a unified model such as a deep neural network. During the training process, the model learns the patterns and rules contained in the combination of different modal features. For example, it learns the association between specific facial expression features in the video and the corresponding voice intonation features and text emotional tendency features. Based on this association analysis, it comprehensively judges the psychological state of the subject and provides a basis for the generation of the final psychological assessment report.

[0131] Furthermore, specially adapted models are trained for the three different modalities of data: video, speech, and text. For example, convolutional neural networks (CNNs) are used to process video image data because they are good at capturing spatial features in images and can analyze visual information such as facial expressions. Recurrent neural networks (RNNs) or long short-term memory networks (LSTMs) are used to process text data to cope with the sequential characteristics of text and mine the semantic and logical relationships therein. Temporal convolutional networks (TCNs) are used to process audio data to effectively extract time series features in speech. Each model focuses on learning the feature representation and internal rules of the corresponding modal data and outputs the feature information of its own modality.

[0132] For example, the features output by these different modal models can be fused through weighted summation, concatenation, and attention mechanisms. For example, weights can be assigned to different modalities according to their importance in reflecting psychological characteristics, and weighted sums can be performed on the output features of each modality; or the feature vectors of each modality can be directly concatenated together; the attention mechanism can also be used to allow the model to automatically learn the importance differences between features of different modalities, selectively focus on key features for fusion, and enhance the effectiveness and representativeness of the fused features.

[0133] Next, a comprehensive model is used to further process and analyze the fused features. Based on the fused multimodal features, this comprehensive model continues to explore the deep-level relationships and patterns, and ultimately outputs relevant results for generating a psychological assessment report, enabling the assessment to comprehensively consider multimodal information and more comprehensively and accurately reflect the psychological state of the subject.

[0134] Then, in the fusion decision-making process, firstly, the data of each modality of video, voice and text are analyzed separately and in depth, and independent prediction results are obtained based on the existing features of each modality and the corresponding analysis methods. For example, by analyzing the changes in facial expressions in the video, the emotional state of the subject is predicted; the emotional intensity is judged based on the voice features; the psychological tendency is analyzed from the text semantics, etc. Each modality forms its own judgment and prediction about the psychological characteristics of the subject. The independent prediction results of these modalities are combined, and a simple voting mechanism, weighted average or other combination strategies are used to obtain the final prediction result. For example, if both the video and text modality analysis show that the subject is in a positive emotional state, but the voice modality analysis result is not clear, the overall emotional state is comprehensively judged by weighted average (giving higher weights to the video and text modalities), so as to integrate multimodal information and provide a comprehensive decision-making basis for psychological assessment.

[0135] Optionally, after extracting features from video, audio, and text respectively, the attention mechanism can be used to fuse these features. For example, the features of one modality are selected as the query vector (Query), and the features of other modalities are selected as the key vector (Key) and the value vector (Value). The attention weight is determined by calculating the similarity between the Query and the Key. This weight reflects the relevance and importance between the features of different modalities. For example, using text features as the Query, and video and audio features as the Key and Value, the model can automatically focus on important cross-modal features such as the corresponding facial expressions in the video and the corresponding changes in tone in the audio based on the emotional keywords mentioned in the text, to achieve targeted feature fusion, making the fused features more relevant and valuable.

[0136] For example, the attention mechanism is used to capture the interdependencies between features of different modalities. In actual situations, some frames in a video may be highly correlated with specific audio signals or text content. For example, when a subject is talking about something happy, he or she will smile and speak in a brisk tone. The attention mechanism can automatically identify and strengthen these cross-modal associations, and dig out the psychological state information hidden behind different modalities, providing support for a more comprehensive and accurate assessment of psychological conditions.

[0137] In the fusion process, the weighted summation of the attention mechanism can obtain a richer and more accurate joint feature representation. For example, in the fusion of speech and text, the attention mechanism can help the model better understand the subtle differences between the speech content and the corresponding text description, such as whether the key parts emphasized in the speech are consistent with the content emphasized in the text. Through this optimized representation of features, the overall performance of the model is improved, thus laying the foundation for generating high-quality psychological assessment reports.

[0138] Use pre-trained models or dedicated networks to extract feature vectors of video, audio, and text, respectively, to ensure that the features of each modality can be accurately and effectively extracted, and to prepare data for subsequent fusion and analysis. For example, use convolutional neural networks pre-trained on large-scale image datasets to extract image features such as facial expressions in videos, use specialized speech feature extraction tools and networks to obtain audio features, and rely on mature models and algorithms in the field of natural language processing to extract text features.

[0139] Exemplarily, a multi-head attention layer is defined, and the feature vector of each modality is input into the attention layer as Query, Key and Value respectively. The multi-head attention mechanism calculates multiple attention heads in parallel, each head calculates attention independently in different subspaces, and then splices the results together, which can more comprehensively capture the feature relationship of different dimensions in the input data, and help to mine the complex interdependence between multimodal features. The attention weight is obtained by calculating the similarity between Query and Key, and then the weight is normalized using the softmax function so that the sum of the weights is 1, and each weight value represents the relative importance of the corresponding feature in the fusion process, so that the model can focus on the features of different modalities according to these weights, and focus on more critical information for subsequent analysis. The normalized attention weight is used to perform weighted summation on Value to obtain the fused feature representation. This fused feature integrates the important information of different modalities and the correlation between them. Compared with the single modality feature, it can more comprehensively reflect the psychological state and characteristics of the subject, and provide strong data support for the accurate generation of psychological assessment reports. The integrated model is trained using the fused features to perform classification tasks and other analyses, such as determining the emotional category of the subject, the degree of psychological problems, etc. Through a large amount of data training, the model learns the mapping relationship between the fused features and the actual psychological assessment results, so that in actual applications, the corresponding assessment results can be accurately output based on the input multimodal fusion features, providing specific quantitative and qualitative analysis content for the psychological assessment report.

[0140] In particular, in terms of data statistics for quantitative analysis, based on the fused multimodal features, the statistical data of each modal feature can be calculated, such as the frequency of facial expressions, the distribution of speech emotions corresponding to different emotional states, the proportion of text emotional tendencies (positive, negative, neutral, etc.), etc. These statistical data intuitively present the psychological performance of the subjects in different modalities in a quantitative form, providing an objective numerical basis for psychological assessment.

[0141] The fused multimodal features are used for predictive analysis, such as predicting the emotional state of the subject (such as predicting whether he is in an emotional state such as anxiety, depression, etc. and to what extent), and predicting behavioral tendencies (such as predicting how to cope in a specific situation, etc.). The specific numerical results output by the model (such as the score of the emotional state, the probability of behavioral tendencies, etc.) can further enrich the content of the quantitative analysis and make psychological assessment more accurate and scientific.

[0142] In terms of qualitative analysis, the focus is mainly on effect verification and consistency testing, and whether the evaluation is consistent with the test set in terms of indicators, themes, actual performance, etc. For example, check whether the psychological state of the subject obtained based on the fusion feature analysis is consistent with the expected results of the pre-set interview indicators, whether the psychological themes involved (such as anxiety, optimism, etc.) are accurately reflected in the analysis results, and whether the actual psychological characteristics are consistent with the model predictions in logic and actual conditions. Through such qualitative analysis, the results of the entire multimodal fusion analysis are verified and calibrated to ensure the accuracy and reliability of the evaluation, providing a basis for the subsequent adoption of these results.

[0143] Finally, according to the pre-set interview indicators, the system will not only provide numerical evaluation results (such as emotional intensity index, mental health score and other specific values), but also include a detailed text description of the psychological condition of the subject, explaining possible psychological problems and recommended intervention measures. The numerical evaluation results are based on the indicators. The measurement standards are clearly defined in advance. For example, the total score of anxiety is set to 25 points, and the specific score is obtained based on the model analysis; the model gives a comprehensive analysis report based on the previous multimodal fusion analysis and the input of each module; and the framework of the detailed text description is prepared in advance by the psychology teacher, by building a set of prompt projects suitable for large language models, covering role definition, task definition, precautions, work flow, samples and other generation modules, and ultimately providing decision support for professionals, and providing the subject with a basis for self-improvement, generating a detailed, comprehensive and instructive psychological assessment report.

[0144] Through the above-mentioned multimodal fusion operation in the fusion layer, and the subsequent application in quantitative and qualitative analysis, a psychological assessment report is finally generated that comprehensively considers multimodal information, is scientifically accurate and has practical value, and achieves a comprehensive and in-depth assessment of the psychological state of the subject.

[0145] In the technical solution of the present application, the embodiment of the present application more accurately reaches the psychological state of the target object, adapts to individual differences, assists relevant personnel in completing psychological assessment tasks, improves the accuracy of assessment results, and improves the efficiency of psychological assessment by accurately guiding the generation of dialogue content and adjusting the assessment strategy in real time.

[0146] In another embodiment of the present application, a psychological assessment device based on a large model is also provided. Figure 2 The device comprises the following units:

[0147] A generating unit is configured to generate a psychological assessment strategy for a target object to be assessed according to a current interview task and a corresponding psychological assessment indicator; the psychological assessment strategy is used to guide the large model to generate dialogue content that meets the current interview task and the psychological assessment indicator;

[0148] The interaction unit is configured to generate multiple rounds of dialogue information through the evaluation interaction model based on the psychological evaluation strategy, and interact with the target object through the multiple rounds of dialogue information to complete the psychological evaluation process corresponding to the current interview task; in the non-first round of interaction process, the psychological evaluation strategy is adjusted in real time based on the response information of the target object during the interaction process;

[0149] The evaluation unit is configured to perform a multimodal interaction mode analysis on the target object based on the multiple rounds of dialogue information and the corresponding response information to obtain a psychological evaluation report of the target object; the psychological evaluation report at least includes: a quantitative analysis result and a qualitative psychological analysis result for the dialogue content; the qualitative psychological analysis result is associated with the psychological evaluation indicator.

[0150] Further optionally, the generating unit generates a psychological assessment strategy according to the current interview task and the corresponding psychological assessment indicators, and is configured to: create a first psychological assessment strategy for the target object through a strategy creation model according to the current interview task and the corresponding psychological assessment indicators; the interacting unit generates multiple rounds of dialogue information through an assessment interaction model based on the psychological assessment strategy, and interacts with the target object through the multiple rounds of dialogue information to complete the psychological assessment process corresponding to the current interview task, and is configured to: generate first dialogue information through an assessment interaction model based on the first psychological assessment strategy; the first dialogue information is a first round of assessment dialogue with the target object; obtain first response information fed back by the target object based on the first dialogue information; based on Based on the current round of dialogue information, response information and the psychological evaluation indicators, the psychological evaluation strategy of the current round is adjusted through the interactive strategy model; the second dialogue information is generated through the evaluation interaction model using the adjusted second psychological evaluation strategy; the second dialogue information is the second round of evaluation dialogue with the target object generated after the questioning strategy is adjusted in real time; the second response information fed back by the target object based on the second dialogue information is obtained; the process jumps to the current round of dialogue information, response information and the psychological evaluation indicators, the psychological evaluation strategy of the current round is adjusted through the interactive strategy model, and the adjusted psychological evaluation strategy is used to execute the next round of evaluation dialogue, until the current interview task is completed or the target object ends the interaction operation.

[0151] Further optionally, the psychological assessment strategy includes at least one of the following: key psychological assessment indicators, key information types, questioning strategies, and response skills guidance; the key psychological assessment indicators include at least one of the following: emotional response, behavioral performance, and physiological response; the questioning strategy includes at least one of the following: open-ended questions, closed-ended questions, situational simulation questions, and emotional support; the response skills guidance includes at least one of the following: follow-up questions, topic switching, empathy with users, and clarification methods.

[0152] Further optionally, the generating unit creates a first psychological assessment strategy for the target object through a strategy creation model according to the current interview task and the corresponding psychological assessment indicators, and is configured to: obtain the current interview task; perform information demand analysis on the current interview task through the strategy creation model to obtain key information types and key psychological assessment indicators that match the current interview task; determine the questioning strategy for the key psychological assessment indicators; and generate a dialogue flow control method corresponding to the current interview task based on the key psychological assessment indicators, key information types, and questioning strategies; the generating unit generates the first dialogue information through the strategy creation model based on the first psychological assessment strategy, including: generating the first dialogue information based on the key psychological assessment indicators, key information types, questioning strategies, and the corresponding dialogue flow control method.

[0153] Further optionally, the interaction unit adjusts the psychological assessment strategy of the current round through the interaction strategy model based on the dialogue information, response information and the psychological assessment index of the current round, and is configured as follows: through the emotion analysis layer in the interaction strategy model, based on the first dialogue information and the first response information, emotion analysis is performed on the target object to obtain the real-time emotion characteristics of the target object during the interaction process; through the intention recognition layer in the interaction strategy model, user intention is recognized on the target object based on the first dialogue information and the first response information to obtain the behavior pattern characteristics of the target object during the interaction process; through the adversarial sample generation layer in the interaction strategy model, diversified dialogue scene data corresponding to the target object is constructed; through the adjustment layer in the interaction strategy model, the first psychological assessment strategy is dynamically adjusted according to the real-time emotion characteristics, behavior pattern characteristics and the diversified dialogue scene data of the target object during the interaction process to obtain a second psychological assessment strategy.

[0154] Further optionally, the evaluation unit performs a multimodal interaction pattern analysis on the target object based on the multiple rounds of dialogue information and the corresponding response information to obtain a psychological evaluation report of the target object, and is configured as follows: performing multimodal feature extraction on the multiple rounds of dialogue information and the corresponding response information through a feature extraction layer in a multimodal analysis model to obtain multimodal psychological features; the multimodal psychological features at least include: dialogue text features, dialogue audio features, and facial image features; performing feature evaluation on the multimodal psychological features through multiple psychological feature evaluation layers in the multimodal analysis model to obtain multimodal psychological feature evaluation results; corresponding feature change analysis strategies are respectively configured in the multiple psychological feature evaluation layers; and performing fusion based on the multimodal psychological feature evaluation results through a fusion layer in the multimodal analysis model to obtain the psychological evaluation report.

[0155] Further optionally, the evaluation unit performs feature evaluation on the multimodal psychological features respectively through multiple psychological feature evaluation layers in the multimodal analysis model to obtain multimodal psychological feature evaluation results, and is configured to: perform facial tracking detection on the facial image features to obtain visual change features of facial key points; perform micro-expression detection based on the visual change features to obtain the micro-expression type of the target object; identify the emotional state of the target object according to the micro-expression type to obtain emotion evaluation information of the target object; the emotion evaluation information is used to indicate the emotional state of the target object displayed through facial expressions.

[0156] Further optionally, the evaluation unit performs micro-expression detection based on the visual change feature to obtain the micro-expression type of the target object, and is configured to: extract the corresponding facial action unit from the visual change feature; encode the facial action unit, and perform change feature recognition on the encoding sequence of the facial action unit to obtain a first expression feature; perform tiny movement recognition on the visual change feature using an optical flow method to obtain a second expression feature; determine the micro-expression type based on the first expression feature and the second expression feature; wherein the micro-expression type is at least one of a plurality of micro-expression types pre-set based on a plurality of emotional states.

[0157] Further optionally, the evaluation unit performs a multimodal interaction pattern analysis on the target object based on the multiple rounds of dialogue information and the corresponding response information to obtain a psychological evaluation report of the target object, and is also configured to: in a non-first-round interaction process, perform a multimodal interaction pattern analysis on the target object based on the multiple rounds of dialogue information and the corresponding response information to obtain real-time psychological evaluation information of the target object; and dynamically adjust the psychological assessment strategy in the non-first-round interaction process based on the real-time psychological evaluation information.

[0158] The system can implement various steps in the above method embodiment, which will not be expanded here.

[0159] See also Figure 3 , Figure 3 This is a schematic diagram of an electronic device provided in an embodiment of the present application. Figure 3 As shown, an embodiment of the present application provides an electronic device 500, including a memory 510, a processor 520, and a computer program 511 stored in the memory 510 and executable on the processor 520. When the processor 520 executes the computer program 511, a psychological assessment method based on a large model is implemented.

[0160] See also Figure 4 , Figure 4 A schematic diagram of an embodiment of a computer-readable storage medium provided in an embodiment of the present application. Figure 4As shown, this embodiment provides a computer-readable storage medium 600, on which a computer program 611 is stored, and when the computer program 611 is executed by a processor, a psychological assessment method based on a large model is implemented. It should be noted that in the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments. It should be understood by those skilled in the art that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. In this way, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these changes and variations.

Claims

1. A psychological assessment method based on a large model, characterized in that: The method comprises: For the target object to be evaluated, a psychological evaluation strategy is generated according to the current interview task and the corresponding psychological evaluation index; the psychological evaluation strategy is used to guide the large model to generate dialogue content that meets the current interview task and the psychological evaluation index; Based on the psychological assessment strategy, multiple rounds of dialogue information are generated through the assessment interaction model, and the target object is interacted with through the multiple rounds of dialogue information to complete the psychological assessment process corresponding to the current interview task; in the non-first round of interaction process, the psychological assessment strategy is adjusted in real time based on the response information of the target object during the interaction process; Based on the multiple rounds of dialogue information and the corresponding response information, a multimodal interaction pattern analysis is performed on the target object to obtain a psychological assessment report of the target object; the psychological assessment report at least includes: quantitative analysis results and qualitative psychological analysis results for the dialogue content; the qualitative psychological analysis results are associated with the psychological assessment indicators.

2. The psychological assessment method based on a large model according to claim 1, characterized in that: The psychological assessment strategy is generated according to the current interview task and the corresponding psychological assessment indicators, including: According to the current interview task and the corresponding psychological assessment indicators, the first psychological assessment strategy for the target object is created through the strategy creation model; The method of generating multiple rounds of dialogue information through the evaluation interaction model based on the psychological evaluation strategy, and interacting with the target object through the multiple rounds of dialogue information to complete the psychological evaluation process corresponding to the current interview task includes: Generate first dialogue information through the evaluation interaction model based on the first psychological evaluation strategy; the first dialogue information is the first round of evaluation dialogue with the target object; Obtaining first response information fed back by the target object based on the first dialogue information; Based on the conversation information, response information and the psychological assessment index of the current round, adjusting the psychological assessment strategy of the current round through the interaction strategy model; The second psychological assessment strategy after adjustment is used to generate second dialogue information through the assessment interaction model; the second dialogue information is a second round of assessment dialogue with the target object generated after the questioning strategy is adjusted in real time; Obtaining second response information fed back by the target object based on the second dialogue information; Jump to the current round of dialogue information, response information and the psychological assessment indicators, adjust the current round of psychological assessment strategy through the interactive strategy model, and use the adjusted psychological assessment strategy to execute the next round of assessment dialogue until the current interview task is completed or the target object ends the interactive operation.

3. The psychological assessment method based on a large model according to claim 2, characterized in that: The psychological assessment strategy includes at least one of the following: key psychological assessment indicators, key information types, questioning strategies, and response skills guidance; The key psychological assessment indicators include at least one of the following: emotional response, behavioral performance, and physiological response; the questioning strategies include at least one of the following: open-ended questions, closed-ended questions, situational simulation questions, and emotional support; the response skills guidance includes at least one of the following: follow-up questions, topic switching, empathy with users, and clarification methods.

4. The psychological assessment method based on a large model according to claim 3, characterized in that: The step of creating a first psychological assessment strategy for the target object through a strategy creation model according to the current interview task and the corresponding psychological assessment index includes: Get the current interview task; Conduct information demand analysis on the current interview task through the strategy creation model to obtain key information types and key psychological assessment indicators that match the current interview task; Determine questioning strategies for key psychometric indicators; Generate the dialogue process control method corresponding to the current interview task based on key psychological assessment indicators, key information types, and questioning strategies; The generating the first dialogue information by using the strategy creation model based on the first psychological assessment strategy includes: The first dialogue information is generated based on key psychological assessment indicators, key information types, questioning strategies, and corresponding dialogue process control methods.

5. The psychological assessment method based on a large model according to claim 4, characterized in that: The step of adjusting the psychological assessment strategy of the current round through the interactive strategy model based on the dialogue information, response information and the psychological assessment index of the current round includes: Through the emotion analysis layer in the interaction strategy model, emotion analysis is performed on the target object based on the first dialogue information and the first response information to obtain the real-time emotion characteristics of the target object during the interaction process; Through the intention recognition layer in the interaction strategy model, based on the first dialogue information and the first response information, the user intention of the target object is recognized to obtain the behavior pattern characteristics of the target object in the interaction process; Through the adversarial sample generation layer in the interaction strategy model, we build diverse conversation scenario data corresponding to the target object. Through the adjustment layer in the interaction strategy model, the first psychological assessment strategy is dynamically adjusted according to the real-time emotional characteristics, behavioral pattern characteristics, and the diversified dialogue scenario data of the target object during the interaction process to obtain the second psychological assessment strategy.

6. The psychological assessment method based on a large model according to claim 1, characterized in that: The multimodal interaction mode analysis of the target object based on the multiple rounds of dialogue information and the corresponding response information to obtain a psychological assessment report of the target object includes: Through the feature extraction layer in the multimodal analysis model, multimodal feature extraction is performed on the multi-round dialogue information and the corresponding response information to obtain multimodal psychological features; the multimodal psychological features at least include: dialogue text features, dialogue audio features, and facial image features; Through multiple psychological feature evaluation layers in the multimodal analysis model, feature evaluation is performed on the multimodal psychological features to obtain multimodal psychological feature evaluation results; corresponding feature change analysis strategies are respectively configured in the multiple psychological feature evaluation layers; The psychological assessment report is obtained by fusing the multimodal psychological feature assessment results through the fusion layer in the multimodal analysis model.

7. The psychological assessment method based on a large model according to claim 6, characterized in that: The method of performing feature evaluation on the multimodal psychological features respectively through the multiple psychological feature evaluation layers in the multimodal analysis model to obtain the multimodal psychological feature evaluation results includes: Performing facial tracking detection on the facial image features to obtain visual change features of facial key points; Performing micro-expression detection based on the visual change features to obtain the micro-expression type of the target object; According to the micro-expression type, the emotional state of the target object is identified to obtain the emotional evaluation information of the target object; the emotional evaluation information is used to indicate the emotional state displayed by the target object through facial expressions.

8. The psychological assessment method based on a large model according to claim 7, characterized in that: The micro-expression detection based on the visual change feature to obtain the micro-expression type of the target object includes: Extracting corresponding facial action units from the visual change features; Encoding the facial action unit and performing change feature recognition on the encoding sequence of the facial action unit to obtain a first expression feature; Using an optical flow method to perform micro-motion recognition on the visual change feature to obtain a second expression feature; Based on the first expression feature and the second expression feature, the micro-expression type is determined; wherein the micro-expression type is at least one of a plurality of micro-expression types pre-set based on a plurality of emotional states.

9. The psychological assessment method based on a large model according to claim 1, characterized in that: The method of performing a multimodal interaction mode analysis on the target object based on the multiple rounds of dialogue information and the corresponding response information to obtain a psychological assessment report of the target object also includes: In a non-first-round interaction process, a multimodal interaction mode analysis is performed on the target object based on the multi-round dialogue information and the corresponding response information to obtain real-time psychological evaluation information of the target object; Dynamically adjust the psychological assessment strategy during non-first-round interaction based on the real-time psychological assessment information.

10. A psychological assessment device based on a large model, characterized in that: The device comprises the following units, wherein: A generating unit is configured to generate a psychological assessment strategy for a target object to be assessed according to a current interview task and a corresponding psychological assessment indicator; the psychological assessment strategy is used to guide the large model to generate dialogue content that meets the current interview task and the psychological assessment indicator; The interaction unit is configured to generate multiple rounds of dialogue information through the evaluation interaction model based on the psychological evaluation strategy, and interact with the target object through the multiple rounds of dialogue information to complete the psychological evaluation process corresponding to the current interview task; in the non-first round of interaction process, the psychological evaluation strategy is adjusted in real time based on the response information of the target object during the interaction process; The evaluation unit is configured to perform a multimodal interaction mode analysis on the target object based on the multiple rounds of dialogue information and the corresponding response information to obtain a psychological evaluation report of the target object; the psychological evaluation report at least includes: a quantitative analysis result and a qualitative psychological analysis result for the dialogue content; the qualitative psychological analysis result is associated with the psychological evaluation indicator.

Images