A mental health analysis method and system based on a large model and multi-modal input

Abstract

This application relates to the fields of computational psychology and affective computing, and in particular to a method and system for mental health analysis based on a large model and multimodal input. The method includes acquiring user data to be analyzed and preset mental health assessment goals; analyzing the data to be analyzed based on the preset mental health assessment goals to obtain initial psychological data; generating multimodal interactive psychological stimulus scenarios based on the initial psychological data and a multimodal model; acquiring multimodal response data of the user during the interaction with the psychological stimulus scenarios; performing spatiotemporal alignment and correlation analysis on the multimodal response data and the stimulus timelines corresponding to the psychological stimulus scenarios to obtain correlation analysis results; calculating the user's psychological schema parameters based on the correlation analysis results and an inverse mapping model; and generating a mental state assessment report based on the psychological schema parameters. This method has the advantages of enabling dynamic interaction in the mental assessment process, in-depth exploration of analytical dimensions, computable transformation of psychological patterns, and improved ecological validity.

Description

Technical Field

[0001] This application relates to the fields of computational psychology and affective computing, and in particular to a method and system for mental health analysis based on large models and multimodal inputs. Background Technology

[0002] With the development of artificial intelligence technology, its application in fields such as mental health, human-computer interaction and talent assessment is becoming increasingly widespread. Traditional psychological assessment mainly relies on scales, interviews or behavioral observation. These methods are limited by problems such as subjectivity, low standardization and insufficient ecological validity. In recent years, the fields of computational psychology and affective computing have attempted to use artificial intelligence technology, especially natural language processing and computer vision technology, to automate and objectively analyze users' psychological states.

[0003] Currently, AI-assisted psychological state assessment mainly relies on text-based sentiment and topic analysis techniques. By analyzing text or speech-transcribed text uploaded by users, sentiment dictionaries, machine learning, or pre-trained language models are used to classify sentiment polarity, identify emotion categories, or mine topics, thereby determining the user's emotional state or focus of attention.

[0004] However, the aforementioned existing technologies have obvious limitations. Their analysis is passive and limited by expression, relying entirely on the text content that users spontaneously select and express. They cannot actively explore or touch the deep psychological structures that users do not explicitly state, are unconscious, or are difficult to describe precisely in language. Furthermore, the analytical dimensions are shallow and isolated, focusing on the emotional labels or thematic classifications of the expressed content, and lacking the exploration of the deep cognitive logic and behavioral motivations behind psychological states.

[0005] In summary, existing AI-assisted psychological assessment technologies are limited by their passivity, superficiality, and insufficient adaptability to different scenarios, making it difficult to break through the bottlenecks of traditional assessments. There is an urgent need for a solution that can proactively explore deeper psychological states and assess complex cognitive behaviors in dynamic situations, thereby improving the comprehensiveness and accuracy of assessments. Summary of the Invention

[0006] This application provides a method and system for psychological health analysis based on large models and multimodal inputs, which solves the problems of passive assessment, shallow analysis and poor adaptability to dynamic scenarios in the prior art, and realizes proactive, in-depth and situation-adaptive psychological assessment.

[0007] On the one hand, embodiments of this application provide a mental health analysis method based on large models and multimodal inputs, including:

[0008] The system acquires user data to be analyzed and preset mental health assessment goals, wherein the data to be analyzed includes at least one of facial images, voice data, or text data.

[0009] The data to be analyzed is analyzed based on the preset mental health assessment goals to obtain initial psychological data;

[0010] Based on the initial psychological data and the multimodal model, a multimodal interactive psychological stimulation scenario is generated;

[0011] Acquire multimodal response data during the user's interaction with the psychological stimulus scenario;

[0012] The multimodal response data and the stimulus timeline corresponding to the psychological stimulus scenario are spatiotemporally aligned and correlated to obtain correlation analysis results. The stimulus timeline includes the stimulus event types, temporal relationships and logical associations used to induce the target psychological response.

[0013] Based on the correlation analysis results and the inverse mapping model, the user's mental schema parameters are calculated, and the mental schema parameters are used to characterize the user's potential psychological patterns.

[0014] A psychological state assessment report is generated based on the aforementioned psychological schema parameters.

[0015] Furthermore, the generation of multimodal interactive psychological stimulus scenarios based on the initial psychological data and the multimodal model includes:

[0016] At least one target psychological construct that matches the preset mental health assessment goal is selected from the psychological schema library, and the target psychological construct is a psychological model constructed based on psychological theory;

[0017] Based on the initial psychological data, personalized situational elements corresponding to the target psychological construct are generated;

[0018] Based on the target mental construct, a scenario logic containing at least one situational variable is constructed. The situational variable is used to actively induce externally observed behavioral patterns corresponding to the target mental construct during multimodal interaction.

[0019] Based on the multimodal model, the personalized contextual elements and the scene logic are fused to obtain a dynamic scene of multimodal interaction between the personalized contextual elements and the scene logic;

[0020] The dynamic scene is used as the psychological stimulus scene.

[0021] Furthermore, the construction of scene logic containing at least one situational variable based on the target mental construct includes:

[0022] The core cognitive-emotional conflict pattern was obtained by analyzing the target psychological constructs.

[0023] Based on the conflict pattern, construct a pair of related variables consisting of a first situational variable and a second situational variable;

[0024] The scenario logic is generated based on the associated variables and preset rules;

[0025] Wherein, the first situational variable is used to actively induce intuitive responses corresponding to the conflict mode in multimodal interaction, and the second situational variable is used to apply cognitive load or present contradictory information to stimulate cognitive adjustment behavior after the intuitive response is induced. The preset rules include the triggering conditions, timing of action and response relationship of the first situational variable and the second situational variable.

[0026] Furthermore, the step of performing spatiotemporal alignment and correlation analysis between the multimodal response data and the stimulus timeline corresponding to the psychological stimulus scenario includes:

[0027] Obtain stimulus anchor points on the stimulus timeline that correspond to the context variable. The stimulus anchor points are used to identify the time points when the context variable is triggered or presented.

[0028] Based on the stimulus anchor, the multimodal response data is segmented in the time dimension to obtain response data slices corresponding to the stimulus anchor.

[0029] Within each response data slice, the user's procedural behavioral features under the influence of the contextual variables are extracted, wherein the procedural behavioral features include at least one of response delay, intensity of facial expression changes, voice tone fluctuation patterns, text sentiment tendencies, and conflict or coordination between multimodal response data.

[0030] Based on time-series information, the procedural behavioral features within different reaction slices are serialized and analyzed for correlation, resulting in correlation analysis results that reflect the evolution of the procedural behavioral features over time.

[0031] Furthermore, the calculation of the user's mental schema parameters based on the association analysis results and the inverse mapping model includes:

[0032] Obtain the procedural behavioral characteristics corresponding to each stimulus anchor point, and determine the type and preset intensity of the situational variable corresponding to the stimulus anchor point that induces the procedural behavioral characteristics;

[0033] The process-oriented behavioral characteristics, along with the corresponding situational variable types and preset intensities, are used as a set of input data.

[0034] The estimated value of the target mental construct is calculated based on multiple sets of input data and the inverse mapping model, and the estimated value is used as the user's mental schema parameter;

[0035] The inverse mapping model is a conditional probability generation model. Given the situational variables, the estimated value is determined by maximizing the posterior probability of the existence of a mental schema within the user, based on the observed procedural behavioral characteristics.

[0036] Furthermore, the construction of the inverse mapping model includes:

[0037] Obtain a sample set containing known mental schema labels, wherein each sample in the sample set has a pre-defined set of situational variables corresponding to the mental schema label;

[0038] Acquire multimodal response data for each sample under corresponding contextual variable stimuli, and extract procedural behavioral features from the multimodal response data;

[0039] The inverse mapping model is trained by taking the combination of the situational variable set and the procedural behavioral features corresponding to each sample as input and the corresponding mental schema label as the training target.

[0040] In the training process of the inverse mapping model, constraints based on psychological prior knowledge are adopted. These constraints are used to characterize the causal relationship between mental schemas and procedural behavioral features, so that the trained inverse mapping model can inversely infer the corresponding mental schema parameters based on the input set of situational variables and the observed procedural behavioral features.

[0041] Furthermore, the mental schema parameters are input into the trained report big language model to obtain the mental state assessment report;

[0042] The psychological state assessment report includes at least: a description of the user's psychological characteristics determined based on the psychological schema parameters; a psychodynamic correlation explanation of the user's behavior patterns observed in the psychological stimulus scenario and the psychological schema parameters; and developmental recommendations based on the correlation explanation and matching the user's behavior patterns.

[0043] Furthermore, after generating the psychological state assessment report based on the mental schema parameters, the method further includes:

[0044] The psychological schema parameters, correlation analysis results, and corresponding psychological stimulus scenarios are stored in the user's temporal psychological profile;

[0045] The psychological response pattern schema of the user is updated based on the time-series psychological profile. The psychological response pattern schema is used to characterize the user's stable response tendency, state fluctuation range and change sensitivity to different types of situational variables in the dimension of preset mental health assessment goals.

[0046] The psychological response pattern is used to screen situational variables and scenario logic that induce target psychological responses during the generation of psychological stimulus scenarios; it is also used to perform personal benchmark calibration on the psychological schema parameters output by the inverse mapping model.

[0047] Furthermore, after updating the user's psychological response pattern schema based on the time-series psychological profile, the method further includes:

[0048] The psychological schema parameters are compared with the historical psychological schema parameters in the psychological response pattern schema to generate evolutionary analysis conclusions of user psychological characteristics;

[0049] Based on the evolutionary analysis conclusions and the correlation analysis results, at least one item in the psychological state assessment report will be enhanced in depth or its presentation priority will be rearranged. The content of the assessment report includes descriptions of psychological characteristics, psychodynamic correlation explanations, and developmental suggestions.

[0050] On the other hand, embodiments of this application provide a mental health analysis system based on a large model and multimodal input, the system comprising:

[0051] The first acquisition module is used to acquire the user's data to be analyzed and preset mental health assessment goals, wherein the data to be analyzed includes at least one of facial images, voice data or text data;

[0052] The analysis module is used to analyze the data to be analyzed based on the preset mental health assessment goals to obtain initial psychological data;

[0053] A generation module is used to generate multimodal interactive psychological stimulation scenarios based on the initial psychological data and the multimodal model;

[0054] The second acquisition module is used to acquire multimodal response data during the interaction between the user and the psychological stimulation scenario;

[0055] The alignment module is used to perform spatiotemporal alignment and correlation analysis on the multimodal response data and the stimulus timeline corresponding to the psychological stimulus scenario to obtain the correlation analysis results. The stimulus timeline includes the stimulus event types, temporal relationships and logical associations used to induce the target psychological response.

[0056] The calculation module is used to calculate the user's mental schema parameters based on the correlation analysis results and the inverse mapping model. The mental schema parameters are used to characterize the user's potential psychological patterns.

[0057] The second generation module is used to generate a psychological state assessment report based on the psychological schema parameters.

[0058] This application discloses a mental health analysis method and system based on a large model and multimodal input. It acquires user multimodal data to be analyzed and preset mental health assessment goals, and generates multimodal interactive psychological stimulation scenarios based on initial psychological data to actively induce user responses, thus overcoming the limitations of traditional technologies that passively rely on user spontaneous expression. By using multimodal response data during multimodal interaction and performing spatiotemporal alignment and correlation analysis with the stimulus timeline, it achieves in-depth analysis of user responses at the temporal and logical levels. Furthermore, it utilizes an inverse mapping model to calculate psychological schema parameters, revealing users' potential cognitive patterns and behavioral motivations, breaking through the bottleneck of existing technologies that perform superficial and isolated analysis of emotions or topics. Based on the psychological schema parameters, it generates an assessment report, enabling mental state assessment to cover deep psychological structures and adapt to complex cognitive behaviors in dynamic interactive scenarios, improving the comprehensiveness, objectivity, and accuracy of the assessment. Attached Figure Description

[0059] Figure 1 This is a flowchart illustrating a mental health analysis method based on a large model and multimodal input, as shown in the embodiments of this application.

[0060] Figure 2 This is a structural block diagram illustrating a mental health analysis system based on a large model and multimodal input, as shown in the embodiments of this application. Detailed Implementation

[0061] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0062] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0063] Traditional psychological assessment methods suffer from high subjectivity, low standardization, and insufficient ecological validity when analyzing mental health. Existing AI-assisted psychological assessment technologies primarily rely on sentiment and thematic analysis of users' spontaneously expressed text content. This approach is passive and limited by user expression, failing to proactively explore or access the user's unspoken, unconscious, or difficult-to-describe deeper psychological structures. Furthermore, its analytical dimensions are typically superficial and isolated, focusing on emotional labels or thematic classifications of expressed content, lacking the ability to uncover the underlying cognitive logic and behavioral motivations behind psychological states, thus failing to overcome the limitations of traditional assessment methods.

[0064] To address the problems of existing technologies, embodiments of this application provide a method and system for mental health analysis based on large models and multimodal inputs. The following section first introduces the mental health analysis method based on large models and multimodal inputs provided by embodiments of this application.

[0065] Figure 1 This illustration shows a flowchart of a mental health analysis method based on a large model and multimodal input, according to an embodiment of this application. Figure 1 As shown in the embodiments of this application, a mental health analysis method based on large models and multimodal inputs includes the following steps:

[0066] S101. Obtain the user's data to be analyzed and preset mental health assessment goals. The data to be analyzed includes at least one of facial images, voice data, or text data.

[0067] Among them, facial images are used to capture changes in the user's facial expressions, voice data is used to reflect the user's tone, speaking speed and language content, and text data is used to reflect the user's language content; the preset mental health assessment goals are the assessment directions determined before conducting mental health analysis, and the dimensions of mental health assessment include, but are not limited to, anxiety level, depressive tendency, stress coping ability and interpersonal relationship patterns.

[0068] In this embodiment, the user can upload facial images, audio files, or text descriptions to the electronic device via an input device. The electronic device uses the acquired facial images, audio files, or text descriptions as data to be analyzed. The input device includes, but is not limited to, a mouse and a keyboard. The preset mental health assessment goals can be determined by professionals based on the user's initial needs or clinical diagnosis. For example, the preset mental health assessment goals determined for the user may be anxiety level or depressive tendency. In another optional embodiment, the data to be analyzed can also be collected when the user interacts with the electronic device simply, such as capturing facial expressions through a camera, collecting speech through a microphone, or inputting text through a keyboard. The preset mental health assessment goals can also be initially determined based on the results of a preset assessment questionnaire or scale.

[0069] S102. Analyze the data to be analyzed based on the preset mental health assessment goals to obtain initial psychological data.

[0070] Initial psychological data is used to describe descriptive information that characterizes a user’s current psychological state or tendency.

[0071] In this embodiment, a sentiment analysis algorithm is used to process text data to identify the emotional polarity of the text data, acoustic feature extraction and natural language analysis are performed on speech data to obtain speech rate, tone and language expression, facial expression recognition is performed on facial images to determine joy, anger, sorrow and happiness, and the analysis results of the above data to be analyzed are integrated to obtain a preliminary description of the user's current psychological state, i.e., initial psychological data.

[0072] For example, a preliminary description could be "stable mood with a slightly positive tendency", "speaks at a relatively fast pace with mild anxiety", or "natural facial expressions and a strong willingness to interact".

[0073] S103. Generate multimodal interactive psychological stimulation scenarios based on initial psychological data and multimodal models.

[0074] Specifically, at least one target psychological construct that matches the preset mental health assessment goal is selected from the psychological schema library. The target psychological construct is a psychological model constructed based on psychological theory. Personalized situational elements corresponding to the target psychological construct are generated based on the initial psychological data. A scene logic containing at least one situational variable is constructed based on the target psychological construct. The situational variable is used to actively induce external observational behavioral patterns corresponding to the target psychological construct during multimodal interaction. The personalized situational elements and scene logic are fused based on the multimodal model to obtain a dynamic scene of multimodal interaction between the personalized situational elements and scene logic. The dynamic scene is used as a psychological stimulus scene.

[0075] Among them, the multimodal model is an artificial intelligence model that can process and fuse multiple types of data. The multimodal model can be implemented by combining a large language model with an image generation model and a speech synthesis model. The multimodal model is used to generate psychological stimulation scenarios for multimodal interaction. The psychological stimulation scenario is a multimodal interactive environment that actively induces specific psychological reactions in users. The psychological stimulation scenario includes multiple stimulation elements such as visual, auditory, and textual, thereby simulating or constructing specific situations related to the user's psychological characteristics in order to observe the user's real reactions.

[0076] The psychological schema library is a structured knowledge base containing a large number of psychological constructs, cognitive patterns, and emotional response mechanisms validated by psychological theories. For example, it includes databases of various cognitive distortions, attachment styles, and core beliefs. Cognitive distortions include, but are not limited to, catastrophizing and all-or-nothing thinking; attachment styles include, but are not limited to, anxious attachment and avoidant attachment. Each construct is associated with diagnostic criteria and potentially induced behavioral patterns. In other optional implementations, the psychological schema library is an ontology-based knowledge graph, with psychological concepts as nodes and relationships between nodes as edges. These relationships include causality, inclusion, and correlation. The target construct is a specific psychological model selected from the psychological schema library that is directly related to the preset mental health assessment goal. For example, when the preset mental health assessment goal is to assess social anxiety, the target construct is a negative evaluation fear or social avoidance behavior pattern.

[0077] The role of personalized contextual elements is to make the psychological stimulation scenario more closely resemble the user's personal experiences and characteristics, thereby improving the ecological validity and user immersion of the psychological stimulation scenario and making it easier to induce genuine psychological reactions. For example, by analyzing the user's text or voice data, information such as the user's interests, professional background, interpersonal relationship characteristics, and commonly used vocabulary style can be obtained and integrated into the narrative, character settings, or dialogue content of the psychological stimulation scenario. In other optional embodiments, visual or auditory elements in the scenario can also be determined based on the user's age, gender, cultural background, and other information, combined with the differences in responses to specific stimuli among different groups in psychological research. Visual elements include, but are not limited to, scene layout and character images, while auditory elements include, but are not limited to, background music and voice tone.

[0078] The process of constructing scenario logic based on target psychological constructs, including at least one situational variable, includes: analyzing the target psychological constructs to obtain a core cognitive-emotional conflict pattern; constructing a pair of related variables consisting of a first situational variable and a second situational variable based on the conflict pattern; and generating scenario logic based on the related variables and preset rules. The first situational variable is used to actively induce intuitive responses corresponding to the conflict pattern in multimodal interaction, and the second situational variable is used to apply cognitive load or present contradictory information to stimulate cognitive adjustment behavior after the intuitive response is induced. The preset rules include the triggering conditions, timing of action, and response relationship of the first and second situational variables.

[0079] Target constructs are psychological models built upon psychological theories, such as cognitive dissonance, learned helplessness, and attachment patterns. Core cognitive-emotional conflict patterns are inconsistencies, contradictions, or tensions between cognition and emotion that exist deep within a user's mind and lead to psychological distress or specific behavioral patterns. For example, a person may firmly believe that they should be perfect, but feel extremely anxious and helpless when faced with setbacks. This gap between cognition and emotion is a conflict pattern. In this embodiment, a large language model can be used to perform deep learning on psychological theoretical texts, enabling the large language model to identify and extract typical conflict patterns under different constructs.

[0080] The first situational variable directly touches upon and induces an unthinking, automatic, intuitive response in the user related to the conflict pattern. For example, when the conflict pattern is an excessive fear of failure, the first situational variable presents a visual or auditory stimulus simulating a failure scenario to trigger immediate anxiety or avoidance. The second situational variable intervenes strategically after the intuitive response is induced, by applying cognitive load or presenting contradictory information to prompt the user to engage in deeper cognitive processing and adjustment. For example, after the failure scenario mentioned above, the second situational variable presents information that encourages reflection, provides different perspectives, or challenges existing beliefs, prompting the user to re-evaluate the meaning of failure or their own coping strategies.

[0081] In this embodiment, the first situational variable and the second situational variable can be constructed in the following way:

[0082] Based on psychological experimental paradigms, validated psychological experimental designs are transformed into operable situational variables. These designs include, but are not limited to, cognitive dissonance experiments and attribution style experiments. Alternatively, generative artificial intelligence models combined with descriptions of conflict patterns can be used to generate situational elements that conform to psychological principles and can induce specific responses.

[0083] The scenario logic is a framework that guides the dynamic evolution of psychological stimulus scenarios. It specifies how and when related variable pairs are presented, and the interaction between them. The preset rules are the triggering conditions, timing of action, and response relationship of the first and second situational variables. The triggering condition is a specific reaction of the user to the first situational variable, such as changes in facial expression or fluctuations in tone of voice reaching a threshold. The timing of action is that the second situational variable is presented after the first situational variable is presented, such as waiting for 5 seconds or until the user reacts. The response relationship is that if the user shows strong negative emotions towards the first situational variable, the second situational variable focuses on emotion regulation; if the user shows avoidance, it focuses on challenging avoidance behavior.

[0084] In this embodiment, the scene logic can be generated in the following way:

[0085] By using expert systems, psychological intervention processes and experimental design rules can be encoded into a series of condition-action rules, forming decision trees or state machines; or, by using reinforcement learning models to simulate user interactions with psychological stimulus scenarios, the triggering conditions and timing of variables can be learned and optimized to maximize the effect of inducing target psychological responses and cognitive adjustment behaviors.

[0086] Multimodal models transform abstract personalized contextual elements and scenario logic into concrete, vivid, and interactive multimodal stimulus scenarios. For example, a multimodal model generates a virtual scenario based on personalized contextual elements and scenario logic. The personalized contextual elements are the user's profession as a teacher, the scenario logic is a challenging problem encountered in the classroom, and the virtual scenario is a dynamic video or virtual reality scene of the classroom environment, student roles, teacher roles, and specific teaching situations. The model also adjusts the visual and auditory elements in the psychological stimulus scenario in real time based on the user's input. The dynamic scenario is the interactive environment that is ultimately presented to the user to induce psychological responses.

[0087] In this embodiment, by selecting target psychological constructs from a psychological schema library that match the preset mental health assessment goals, the generated psychological stimulation scenarios are ensured to have a solid psychological theoretical basis, thereby enabling precise exploration of the user's deep psychological structure. Personalized situational elements are generated based on initial psychological data, ensuring that the psychological stimulation scenarios closely match the user's personal experiences, interests, and characteristics, thus enhancing the immersion and ecological validity of the psychological stimulation scenarios and prompting users to produce more authentic and natural psychological responses. By constructing scenario logic that includes situational variables, external observational behavioral patterns corresponding to the target psychological constructs can be proactively and purposefully induced, rather than passively waiting for user expression, thereby enhancing the initiative and depth of the psychological assessment to a certain extent. A multimodal model is used to integrate personalized situational elements and scenario logic to generate a dynamic multimodal interactive scenario, making the psychological stimulation scenarios not only theoretically deep and personalized but also highly interactive and realistic, effectively capturing the user's multimodal response data in complex situations.

[0088] S104. Acquire multimodal response data during the user's interaction with psychological stimulus scenarios.

[0089] Multimodal response data (MRT) is real-time behavioral data generated by users during their interaction with psychological stimuli and captured by multiple modal sensors. MRT includes facial expressions, voice tone, body movements, and text input. MRT reflects the user's immediate psychological state and behavioral patterns under specific stimuli. Multimodal sensors include, but are not limited to, cameras, microphones, and keyboards.

[0090] In this embodiment, when a user interacts with the generated virtual scene, the camera captures the user's facial expression changes in real time, the microphone records the user's voice response, and the keyboard records the user's text input. In other optional embodiments, when a user interacts with the virtual scene, their body movements, eye movements, and other physiological signals can also be collected by corresponding sensors, and the collected content can be used as part of the multimodal response data.

[0091] S105. Perform spatiotemporal alignment and correlation analysis on the multimodal response data and the stimulus timeline corresponding to the psychological stimulus scenario to obtain the correlation analysis results. The stimulus timeline includes the type of stimulus event used to induce the target psychological response, the temporal relationship and logical association.

[0092] Specifically, stimulus anchors corresponding to contextual variables are obtained on the stimulus timeline. These anchors identify the points in time when the contextual variables are triggered or presented. Multimodal response data is segmented along the time dimension based on these anchors to obtain response data slices corresponding to the anchors. Within each response data slice, procedural behavioral features of users under the influence of contextual variables are extracted. These procedural behavioral features include at least one of the following: response delay, intensity of facial expression changes, vocal tone fluctuation patterns, text sentiment tendency, and conflict or harmony among multimodal response data. Based on temporal information, the procedural behavioral features within different response slices are serialized and subjected to association analysis to obtain association analysis results reflecting the evolution of procedural behavioral features over time.

[0093] In this context, stimulus anchors are time stamps representing key events within a psychological stimulus scenario. For example, during the scenario generation phase, triggering conditions and expected triggering times for each situational variable are pre-defined. During scenario execution, the timestamps of these events are recorded as stimulus anchors. Situational variables include, but are not limited to, the appearance of a specific image, the playback of a specific sound, and the posing of a specific question in a dialogue. Alternatively, stimulus anchors can be determined by real-time monitoring of the psychological stimulus scenario's state or user behavior. For instance, computer vision technology can detect the appearance time of a specific object within the scenario, or natural language processing technology can identify the mention time of a keyword in a dialogue, thereby automatically recognizing and using the triggering times of situational variables as stimulus anchors.

[0094] For response data slices, a preset time period can be taken before and after each stimulus anchor point, such as 2 seconds before the stimulus anchor point to 5 seconds after the stimulus. The multimodal response data within the preset time period can be extracted and used as the response data slice. Alternatively, the preset time period can be dynamically adjusted according to the nature of the situational variables and the expected psychological response delay. For example, a longer preset time period can be set for stimuli that require longer cognitive processing, while a shorter preset time period can be set for instantaneous responses. The start and end points of the response can also be determined by combining the changing trends of physiological signals, thereby adaptively defining the slice boundaries.

[0095] Among them, procedural behavioral features capture the dynamic and immediate responses of users to specific stimuli. For the extraction of procedural behavioral features, the time interval between the stimulus anchor point and the user's first significant response can be calculated as the response delay. Significant responses include, but are not limited to, changes in facial expressions and the onset of voice responses. The intensity of facial expression changes can be determined by analyzing the intensity changes, duration, or frequency of specific expressions in the response data slices using computer vision and deep learning models. Voice intonation fluctuation patterns can be extracted using speech signal processing technology to extract features such as the fundamental frequency, speech rate, and volume of the speech, and analyze their changing trends and fluctuation amplitudes in the response data slices. When text input is present, natural language processing models are used to analyze the emotional polarity and intensity of the text. Conflicts or coordination between multimodal response data can be determined by comprehensively analyzing the synchronicity, consistency, or contradictions between different modal response data.

[0096] For serialization and correlation analysis, time series analysis methods can be used to model the procedural behavioral feature sequences extracted from different response data slices. This analysis can reveal the trends, periodicity, and autocorrelation of the feature sequences to uncover the dynamic evolution of users' psychological responses. Hidden Markov models or recurrent neural networks can be employed in time series analysis. Furthermore, causal inference algorithms can be combined to analyze the causal relationships between different situational variables and subsequent procedural behavioral features. Clustering or classification algorithms can be used to identify typical behavioral patterns or psychological response trajectories exhibited by users under different stimulus sequences. Causal inference algorithms such as Granger causality tests can be used in these cases.

[0097] In this embodiment, stimulus anchors are used as time markers to provide precise reference points for data alignment, thereby avoiding errors caused by time offsets. Multimodal response data is segmented based on stimulus anchors, ensuring that each response data slice focuses on a specific stimulus-affected period, facilitating the isolation and analysis of behavioral responses under contextual variables. Process behavioral features are extracted within each slice to capture subtle behavioral changes and internal coordination. Based on temporal information, the process behavioral features within different response slices are serialized and correlated, integrating these features in chronological order to reveal the evolution of behavioral patterns with stimuli, generating dynamic correlation results that comprehensively reflect the continuity and changing trends of users' psychological responses.

[0098] S106. Calculate the user's mental schema parameters based on the association analysis results and the inverse mapping model. The mental schema parameters are used to characterize the user's potential psychological patterns.

[0099] Specifically, the process behavior characteristics corresponding to each stimulus anchor are obtained, and the type and preset intensity of the situational variable corresponding to the stimulus anchor that induces the process behavior characteristics are determined. The process behavior characteristics and the type and preset intensity of the situational variable corresponding to the process behavior characteristics are used as a set of input data. Based on multiple sets of input data and the inverse mapping model, the estimated value of the target mental construct is calculated, and the estimated value is used as the user's mental schema parameter. Among them, the inverse mapping model is a conditional probability generation model. Under the condition of determining the situational variables, the estimated value is determined by maximizing the posterior probability of the existence of the user's internal mental schema based on the observed process behavior characteristics.

[0100] In this embodiment, a pre-trained feature extraction model is used to quantify the user's behavior before and after a specific stimulus anchor point, thereby obtaining procedural behavioral features. The situational variable type and preset intensity are determined and stored in the stimulus timeline during the generation stage of the psychological stimulus scenario. The situational variable type and preset intensity can be directly obtained from the stimulus timeline. For example, the situational variable type includes, but is not limited to, cognitive conflict, emotional induction, and social pressure. The preset intensity includes low, medium, and high. Through timestamp matching, the extracted procedural behavioral features, stimulus anchor points, and associated situational variable types and intensities are bound together. The feature extraction model includes, but is not limited to, a deep learning-based facial expression recognition model, a voice emotion analysis model, and a text sentiment analysis model.

[0101] In this embodiment, the procedural behavioral features, situational variable types, and preset strengths are concatenated into a unified feature vector as input data; or, a structured data object, such as JSON or a dictionary, containing procedural behavioral features, situational variable types, and preset strengths is constructed as input data.

[0102] When calculating the estimated value of the target mental construct based on multiple sets of input data and an inverse mapping model, and using the estimated value as a parameter of the user's mental schema, the deep psychological patterns of the user are inferred from the input data, and the deep psychological patterns are quantified into actionable parameters. For example, multiple sets of data on "process-oriented behavioral characteristics + situational variable type + preset intensity" generated by the user under different stimulus anchor points are obtained. The above data are the interactions of the same user at different time points, or the data of different stimulus anchor points in the same interaction process.

[0103] When the inverse mapping model receives input data, it outputs the probability distribution or point estimate of the target mental construct through its internal conditional probability generation mechanism. For example, if the target mental construct is "cognitive flexibility", the inverse mapping model outputs a value between 0 and 1, representing the degree of the user's cognitive flexibility. The estimate can be a single value, a multi-dimensional vector, or a probability distribution, without any specific limitations. The estimate is used to characterize the user's specific mental schema and ultimately serves as the user's mental schema parameter.

[0104] In this embodiment, the conditional probability generation model can be a model based on architectures such as Bayesian networks, hidden Markov models, variational autoencoders, or generative adversarial networks. The conditional probability generation model can learn the conditional dependencies between situational variables E, procedural behavioral features O, and mental schemas S. Given situational variables E and observed procedural behavioral features O, the conditional probability generation model calculates the posterior probability P(S|O,E) of different mental schemas S and selects the mental schema that maximizes the posterior probability as the estimate. In this embodiment, the mental schema with the highest probability can be determined by Bayesian inference, maximum a posteriori estimation, or sampling methods.

[0105] In this embodiment, by acquiring the procedural behavioral characteristics corresponding to each stimulus anchor point and clarifying the type and preset intensity of the situational variables that induce the procedural behavioral characteristics, the direct correlation between behavioral data and specific situations is ensured, avoiding one-sided analysis divorced from context. The procedural behavioral characteristics, situational variable types, and preset intensity are used as inputs to the inverse mapping model, enabling the model to comprehensively capture the user's response patterns in specific situations. The psychological schema parameters calculated by the inverse mapping model are used as representations of the user's potential psychological patterns, providing a solid and reliable quantitative basis for subsequent psychological state assessment reports, thus improving the comprehensiveness and accuracy of mental health analysis.

[0106] The construction of the inverse mapping model includes: acquiring a sample set containing known mental schema labels, wherein each sample in the sample set has a pre-defined set of situational variables corresponding to the mental schema label; acquiring multimodal response data of each sample under the corresponding situational variable stimulus, and extracting procedural behavioral features from the multimodal response data; using the combination of the situational variable set and procedural behavioral features corresponding to each sample as input, and the corresponding mental schema label as the training target to train the inverse mapping model; wherein, during the training process of the inverse mapping model, constraints based on psychological prior knowledge are adopted, and the constraints are used to characterize the causal relationship from mental schema to procedural behavioral features, so that the trained inverse mapping model can inversely infer the corresponding mental schema parameters based on the input set of situational variables and the observed procedural behavioral features.

[0107] In this embodiment, mental schema labels are classifications or quantitative representations of users' deep psychological patterns that have been professionally assessed or verified. For example, mental schema labels can be attachment styles, cognitive bias types, or personality trait dimensions. The sample set originates from large-scale psychological experimental data, clinical case data, or data generated after assessing a large number of subjects using standardized psychological assessment tools. Each sample in the sample set has a pre-defined set of situational variables corresponding to its mental schema label, clarifying the structure of the sample data. That is, each user sample with a labeled mental schema is associated with the situational conditions under which its mental schema may be activated or manifested under specific situational variable stimuli. The set of situational variables includes a series of pre-defined stimuli. For example, in interpersonal communication situations, stimuli are specific events such as rejection, misunderstanding, or gaining support; in stress coping situations, stimuli are facing challenges, task failure, or time constraints. The pre-defined situational variables enable the inverse mapping model to learn different behavioral patterns that different mental schemas may trigger under specific stimuli.

[0108] In this embodiment, procedural behavioral features in multimodal response data can be extracted using computer vision, natural language processing, and machine learning techniques.

[0109] Using the combination of situational variables and procedural behavioral features corresponding to each sample as input when training the inverse mapping model, the data received by the inverse mapping model includes not only the user's behavioral performance but also the situational context that triggers these procedural behavioral features. For example, in a specific context, such as being criticized, the user exhibits procedural behavioral features such as a stiff facial expression and a faster speaking speed. All of the above information is used as input to the inverse mapping model. Training the inverse mapping model with the corresponding mental schema labels as training targets refers to training the model using a supervised learning paradigm. The mental schema labels are the output that the inverse mapping model needs to predict, that is, the user's true deep psychological patterns. During the training process, the difference between the predicted output of the inverse mapping model and the true mental schema labels is compared, and optimization algorithms are used to continuously adjust the internal parameters of the inverse mapping model to minimize the difference. This enables the inverse mapping model to learn the inverse mapping relationship from situational variables and procedural behavioral features to mental schemas. The optimization algorithm can be the gradient descent algorithm.

[0110] In this context, prior psychological knowledge includes known associations between mental schemas and specific behavioral patterns, the influence of different situational variables on the activation of mental schemas, and general patterns of psychological responses. Integrating prior psychological knowledge into the training of the inverse mapping model can prevent the model from learning spurious associations or mappings that do not conform to psychological principles. The core function of the constraint is to characterize the causal relationship from mental schemas to procedural behavioral characteristics. For example, if psychological theory indicates that perfectionism schemas lead to excessive anxiety and repetitive checking behavior in situations of failure, then this constraint will strengthen the mapping path from perfectionism schemas to anxiety and checking behavior in the inverse mapping model. The causal relationship can be represented by adding a regularization term to the loss function, constructing a causal graphical model, or using causal inference algorithms.

[0111] S107. Generate a psychological state assessment report based on mental schema parameters.

[0112] Specifically, the mental schema parameters are input into the trained report big language model to obtain a mental state assessment report. The mental state assessment report includes at least: a description of the user's psychological characteristics determined based on the mental schema parameters; a psychodynamic correlation explanation of the user's behavior patterns and mental schema parameters observed in the psychological stimulus scenario; and developmental recommendations based on the correlation explanation and matching the user's behavior patterns.

[0113] Among them, the mental schema parameters are a set of values ​​used to represent the user's potential psychological patterns, and the report large language model is a natural language processing model trained on a large amount of psychological texts, clinical cases, assessment guidelines and professional report data. The natural language processing model can be generated by fine-tuning a pre-trained language model based on the Transformer architecture on a professional psychology corpus.

[0114] In this embodiment, the psychological state assessment report includes at least three components. The first part is a description of the user's psychological characteristics based on mental schema parameters. This description provides an objective and comprehensive overview of the user's personality traits, cognitive style, emotion regulation ability, interpersonal interaction patterns, etc. For example, it can describe the thinking biases, defense mechanisms, or coping strategies that the user may exhibit in specific situations. The second part is a psychodynamic correlation interpretation of the user's behavioral patterns and mental schema parameters observed in the psychological stimulus scenario. User behavioral patterns are multimodal response data captured in the psychological stimulus scenario, such as response delay, intensity of facial expression changes, voice tone fluctuation patterns, text sentiment tendencies, and conflicts or coordinations between multimodal response data. The psychodynamic correlation interpretation aims to deeply analyze the intrinsic connection between these explicit behavioral patterns and the user's deep mental schema, revealing the cognitive-emotional conflicts, underlying motivations, or unconscious processes behind the behavior. For example, when a user exhibits avoidance behavior under specific stimuli, the report's large language model, combined with its mental schema parameters, explains that this may be related to a certain attachment pattern or self-protection mechanism formed in early childhood experiences. The third part is developmental recommendations based on the correlation interpretation and matching the user's behavioral patterns. The development recommendations are highly personalized and actionable, designed to help users understand their own psychological mechanisms and provide specific strategies and methods to promote mental health development or cope with specific challenges. For example, in response to the cognitive biases revealed in the report, users are advised to try thought restructuring techniques in cognitive behavioral therapy, and for difficulties in emotion regulation, mindfulness practice or emotion expression training is recommended.

[0115] In this embodiment, after generating a psychological state assessment report based on mental schema parameters, the method further includes: storing the mental schema parameters, correlation analysis results, and corresponding psychological stimulus scenarios in the user's temporal psychological profile; updating the user's psychological response pattern schema based on the temporal psychological profile, whereby the psychological response pattern schema is used to characterize the user's stable response tendency, state fluctuation range, and sensitivity to changes in different types of situational variables across the dimensions of preset psychological health assessment goals; wherein, the psychological response pattern schema is used to screen situational variables and scenario logic that induce target psychological reactions during the generation of psychological stimulus scenarios; and is also used to perform personal benchmark calibration on the mental schema parameters output by the inverse mapping model.

[0116] In this embodiment, relational databases such as PostgreSQL and MySQL, or non-relational databases such as MongoDB and Cassandra can be used to store mental schema parameters, association analysis results, and corresponding psychological stimulus scenarios. Each data entry is timestamped to facilitate time series querying and analysis.

[0117] Among them, the psychological response pattern schema is a dynamically evolving model. The psychological response pattern schema can capture the stable response tendency, state fluctuation range, and sensitivity to changes of users in different types of situational variables on the preset psychological health assessment target dimensions. The update of the psychological response pattern schema can be achieved through incremental learning or online learning algorithms, such as Bayesian update, Kalman filter, or recurrent neural network models. With the continuous input of new time-series psychological profile data, the model parameters can be continuously adjusted and optimized, thereby reflecting the long-term changes and short-term fluctuations of users' psychological state in real time. In addition, a periodic batch update method can also be adopted, that is, after accumulating a certain amount of new data or after a specific time interval, the psychological response pattern schema can be retrained or fine-tuned to identify and adapt to the evolution trend of users' psychological patterns.

[0118] In this embodiment, stable response tendency refers to the typical psychological or behavioral patterns that a user tends to exhibit when faced with specific situational variables, such as the average level or frequency of anxiety in socially stressful situations; state fluctuation range is used to quantify the typical range of variation in a user's psychological response, such as the fluctuation range of emotional intensity or cognitive performance; change sensitivity is used to measure the degree and speed at which a user's psychological state or response responds to changes in situational variables, such as the intensity of the response to minor criticism or the speed of adapting to a new environment; by quantifying stable response tendency, state fluctuation range, and change sensitivity, the psychological response pattern schema constructs a comprehensive and personalized psychological response profile for the user.

[0119] In this embodiment, by analyzing the user's historical reaction patterns and sensitivities recorded in the psychological reaction pattern schema, the situational variables and scenario logic that best match the preset mental health assessment goal and are most likely to effectively induce the target psychological reaction are selected. For example, based on the matching degree screening, the characteristics of the situational variables are compared with the sensitivities in the user's psychological reaction pattern schema, and those variables that are expected to produce strong reactions are selected first.

[0120] In this embodiment, when there is a deviation between the psychographic parameters output by the inverse mapping model and the user's historical pattern reflected by the psychographic response pattern basic model, the psychographic response pattern basic model can correct the psychographic parameters output by the inverse mapping model. For example, by applying a correction factor to adjust the parameter values ​​so that they are more consistent with the user's long-term psychological characteristics.

[0121] In this embodiment, the psychological response pattern schema is dynamically updated based on the time-series psychological profile, overcoming the problem that traditional static assessment models cannot capture psychological fluctuations and long-term changes, ensuring the real-time nature and accuracy of the assessment. At the same time, the psychological response pattern schema is used to screen the situational variables and scenario logic that induce the target psychological response during the generation of psychological stimulus scenarios, thereby selecting the most effective stimulus based on the user's historical response patterns. The psychological response pattern schema is also used to perform personal benchmark calibration on the psychological schema parameters output by the inverse mapping model. By comparing and adjusting with the user's own historical baseline, the potential bias of the output of the inverse mapping model is eliminated, making the final psychological schema parameters more accurately reflect the user's true psychological patterns, thereby improving the reliability and personalization of the assessment results.

[0122] In this embodiment, after updating the user's psychological response pattern schema based on the time-series psychological profile, the method further includes: comparing the psychological schema parameters with the historical psychological schema parameters in the psychological response pattern schema to generate an evolutionary analysis conclusion on the user's psychological characteristics; and enhancing the content depth or reordering the display priority of at least one item in the psychological state assessment report based on the evolutionary analysis conclusion and the correlation analysis results. The content of the assessment report includes a description of psychological characteristics, a psychodynamic correlation explanation, and development suggestions.

[0123] Historical mental schema parameters are mental schema parameters stored in the psychological response pattern schema, calculated by the user at previous evaluation periods or different time points. Comparing these parameters with historical mental schema parameters in the psychological response pattern schema aims to identify the changing trends, stability, and volatility of the user's psychological patterns over time. For example, quantitative analysis can be performed by calculating statistical indicators such as differences, rates of change, and volatility of mental schema parameters at different time points; alternatively, time series analysis methods, such as autoregressive integral moving average models or long short-term memory networks, can be used to identify long-term evolution patterns, periodic changes, or the impact of sudden events on mental schema parameters. Through the above comparative analysis, evolutionary analysis conclusions of the user's psychological characteristics are generated. These conclusions reveal the dynamic changes, development trends, potential psychological growth or regression patterns, and factors that may influence these changes.

[0124] In this embodiment, the association analysis results reflect the evolution of users' procedural behavioral characteristics over time in psychologically stimulating scenarios, revealing the causal or correlational relationship between specific situational variables and user responses. The psychological state assessment report is the final psychological assessment result presented to the user. Content depth enhancement involves providing more detailed and in-depth explanations of specific content in the report based on evolutionary analysis conclusions and correlation analysis results. For example, if evolutionary analysis shows significant changes in a user's psychological dimension, a detailed explanation of these changes can be added to the psychological characteristic description, or more specific and targeted intervention strategies can be provided in the development recommendations. This can be achieved by using a preset rule engine to trigger corresponding report enhancement templates based on specific evolutionary patterns. Alternatively, a report language model can be used to generate more in-depth and personalized report content by combining evolutionary analysis conclusions and correlation analysis results. Prioritization of presentation involves adjusting the presentation order of report content based on the importance of evolutionary analysis conclusions and correlation analysis results or the user's current focus. For example, if evolutionary analysis indicates an urgent or significant problem for a user in a certain area, the psychological characteristic description, dynamic explanation, and development recommendations related to that problem can be placed first to highlight their importance. This can be achieved through a weighting mechanism that assigns weights to each part of the psychological state assessment report based on the importance of evolutionary analysis conclusions and correlation analysis results, and then sorts them according to their weights. Alternatively, a user behavior preference model can be used, combined with historical interaction data, to predict the content that users are most likely to focus on and prioritize its display.

[0125] In this embodiment, the psychological schema parameters are compared with the historical psychological schema parameters in the psychological response pattern schema to generate evolutionary analysis conclusions of the user's psychological characteristics. This directly captures the dynamic changes of the user's psychological characteristics, making up for the shortcomings of traditional static assessments. This makes the understanding of the user's psychological state more continuous and developmental. Furthermore, based on the evolutionary analysis conclusions and correlation analysis results, at least one item in the psychological state assessment report is enhanced in depth or its display priority is rearranged. This ensures that the psychological state assessment report not only reflects the user's current psychological state but also combines its historical evolution trajectory and behavioral patterns under specific stimuli to provide more targeted and personalized descriptions of psychological characteristics, psychodynamic correlation explanations, and development suggestions.

[0126] Based on the mental health analysis method based on large models and multimodal inputs provided in the above embodiments, this application also provides a specific implementation of a mental health analysis system based on large models and multimodal inputs. Please refer to the following embodiments.

[0127] First see Figure 2 The mental health analysis system 200 based on large models and multimodal input provided in this application includes the following modules:

[0128] The first acquisition module 201 is used to acquire the user's data to be analyzed and preset mental health assessment goals. The data to be analyzed includes at least one of facial images, voice data or text data.

[0129] Analysis module 202 is used to analyze the data to be analyzed based on preset mental health assessment goals to obtain initial psychological data;

[0130] The generation module 203 is used to generate multimodal interactive psychological stimulus scenarios based on initial psychological data and multimodal models;

[0131] The second acquisition module 204 is used to acquire multimodal response data during the user's interaction with the psychological stimulus scenario;

[0132] Alignment module 205 is used to perform spatiotemporal alignment and correlation analysis on the multimodal response data and the stimulus timeline corresponding to the psychological stimulus scenario to obtain the correlation analysis results. The stimulus timeline includes the stimulus event type, temporal relationship and logical association used to induce the target psychological response.

[0133] Calculation module 206 is used to calculate the user's mental schema parameters based on the correlation analysis results and the inverse mapping model. The mental schema parameters are used to characterize the user's potential psychological patterns.

[0134] The second generation module 207 is used to generate a psychological state assessment report based on mental schema parameters.

[0135] As an optional implementation of this embodiment, the generation module 203 includes:

[0136] The construct selection module is used to select at least one target psychological construct from the psychological schema library that matches the preset mental health assessment goal. The target psychological construct is a psychological model built based on psychological theory.

[0137] The element generation module is used to generate personalized contextual elements corresponding to the target psychological construct based on the initial psychological data.

[0138] The scenario building module is used to construct scenario logic based on the target mental construct, which includes at least one situational variable. The situational variable is used to actively induce external observation behavior patterns corresponding to the target mental construct during multimodal interaction.

[0139] The fusion module is used to fuse personalized contextual elements and scene logic based on a multimodal model to obtain a dynamic scene with multimodal interaction between personalized contextual elements and scene logic.

[0140] Scenes are used as modules to treat dynamic scenes as psychological stimulus scenarios.

[0141] In this optional embodiment, the scene construction module is specifically used for:

[0142] The target psychological construct is analyzed to obtain the core cognitive-emotional conflict pattern; based on the conflict pattern, a pair of related variables consisting of a first situational variable and a second situational variable is constructed; based on the related variables and preset rules, scenario logic is generated; wherein, the first situational variable is used to actively induce intuitive responses corresponding to the conflict pattern in multimodal interaction, and the second situational variable is used to apply cognitive load or present contradictory information to stimulate cognitive adjustment behavior after the intuitive response is induced. The preset rules include the triggering conditions, timing of action and response relationship of the first situational variable and the second situational variable.

[0143] As an optional implementation of this embodiment, the alignment module 205 is specifically used for:

[0144] Stimulus anchors corresponding to contextual variables are obtained on the stimulus timeline. These anchors identify the points in time when the contextual variables are triggered or presented. Multimodal response data is segmented along the time dimension based on these anchors to obtain response data slices corresponding to the anchors. Within each response data slice, procedural behavioral features of users under the influence of contextual variables are extracted. These procedural behavioral features include at least one of the following: response delay, intensity of facial expression changes, vocal tone fluctuation patterns, text sentiment tendency, and conflict or harmony among multimodal response data. Based on temporal information, the procedural behavioral features within different response slices are serialized and subjected to association analysis to obtain association analysis results reflecting the evolution of procedural behavioral features over time.

[0145] As an optional implementation of this embodiment, the calculation module 206 is specifically used for:

[0146] The process behavior features corresponding to each stimulus anchor are obtained, and the type and preset intensity of the situational variable corresponding to the stimulus anchor that induces the process behavior features are determined. The process behavior features, their corresponding situational variable types, and preset intensities are used as a set of input data. Based on multiple sets of input data and an inverse mapping model, the estimated value of the target mental construct is calculated, and the estimated value is used as the user's mental schema parameter. The inverse mapping model is a conditional probability generation model; given the situational variables, it determines the estimated value by maximizing the posterior probability of the user's internal mental schema based on the observed process behavior features.

[0147] As an optional implementation of this embodiment, the mental health analysis system 200 based on large models and multimodal inputs further includes:

[0148] The sample acquisition module is used to acquire a sample set containing known mental schema labels. Each sample in the sample set has a pre-set set of situational variables corresponding to the mental schema label.

[0149] The data acquisition module is used to acquire multimodal response data for each sample under the corresponding contextual variable stimulus, and extract procedural behavioral features from the multimodal response data;

[0150] The training module is used to train the inverse mapping model by taking the combination of situational variables and procedural behavioral features corresponding to each sample as input and the corresponding mental schema labels as training targets. During the training process of the inverse mapping model, constraints based on psychological prior knowledge are used to characterize the causal relationship from mental schemas to procedural behavioral features, so that the trained inverse mapping model can inversely infer the corresponding mental schema parameters based on the input situational variable set and the observed procedural behavioral features.

[0151] As an optional implementation of this embodiment, the second generation module 207 is specifically used for:

[0152] The mental schema parameters are input into the trained report big language model to obtain a mental state assessment report. The mental state assessment report includes at least: a description of the user's psychological characteristics determined based on the mental schema parameters; a psychodynamic correlation explanation of the user's behavior patterns and mental schema parameters observed in the psychological stimulus scenario; and developmental recommendations based on the correlation explanation and matching the user's behavior patterns.

[0153] As an optional implementation of this embodiment, the mental health analysis system 200 based on large models and multimodal inputs further includes:

[0154] The storage module is used to store the psychological schema parameters, correlation analysis results, and corresponding psychological stimulus scenarios into the user's time-series psychological profile after generating a psychological state assessment report based on the psychological schema parameters.

[0155] The update module is used to update the user's psychological response pattern schema based on the time-series psychological profile. The psychological response pattern schema is used to characterize the user's stable response tendency, state fluctuation range, and sensitivity to changes in different types of situational variables across the dimensions of preset mental health assessment goals. Among them, the psychological response pattern schema is used to screen the situational variables and scenario logic that induce the target psychological response during the generation of psychological stimulus scenarios. It is also used to perform personal benchmark calibration on the psychological schema parameters output by the inverse mapping model.

[0156] As an optional implementation of this embodiment, the mental health analysis system 200 based on large models and multimodal inputs further includes:

[0157] The conclusion generation module is used to compare the psychological schema parameters with the historical psychological schema parameters in the psychological response pattern schema after updating the user's psychological response pattern schema based on the time-series psychological profile, and generate the evolutionary analysis conclusion of the user's psychological characteristics.

[0158] The enhancement module is used to enhance the content depth or reorder the presentation priority of at least one item in the psychological state assessment report based on the conclusions of evolutionary analysis and the results of correlation analysis. The content of the assessment report includes descriptions of psychological characteristics, psychodynamic correlation explanations, and developmental recommendations.

[0159] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A mental health analysis method based on large models and multimodal inputs, characterized in that, include: The system acquires user data to be analyzed and preset mental health assessment goals, wherein the data to be analyzed includes at least one of facial images, voice data, or text data. The data to be analyzed is analyzed based on the preset mental health assessment goals to obtain initial psychological data; Based on the initial psychological data and the multimodal model, a multimodal interactive psychological stimulation scenario is generated; Acquire multimodal response data during the user's interaction with the psychological stimulus scenario; The multimodal response data and the stimulus timeline corresponding to the psychological stimulus scenario are spatiotemporally aligned and correlated to obtain correlation analysis results. The stimulus timeline includes the stimulus event types, temporal relationships and logical associations used to induce the target psychological response. Based on the correlation analysis results and the inverse mapping model, the user's mental schema parameters are calculated, and the mental schema parameters are used to characterize the user's potential psychological patterns. A psychological state assessment report is generated based on the aforementioned psychological schema parameters.

2. The mental health analysis method based on a large model and multimodal input according to claim 1, characterized in that, The generation of multimodal interactive psychological stimulus scenarios based on the initial psychological data and multimodal model includes: At least one target psychological construct that matches the preset mental health assessment goal is selected from the psychological schema library, and the target psychological construct is a psychological model constructed based on psychological theory; Based on the initial psychological data, personalized situational elements corresponding to the target psychological construct are generated; Based on the target mental construct, a scenario logic containing at least one situational variable is constructed. The situational variable is used to actively induce externally observed behavioral patterns corresponding to the target mental construct during multimodal interaction. Based on the multimodal model, the personalized contextual elements and the scene logic are fused to obtain a dynamic scene of multimodal interaction between the personalized contextual elements and the scene logic; The dynamic scene is used as the psychological stimulus scene.

3. The mental health analysis method based on a large model and multimodal input according to claim 2, characterized in that, The construction of scene logic based on the target mental construct, which includes at least one situational variable, includes: The core cognitive-emotional conflict pattern was obtained by analyzing the target psychological constructs. Based on the conflict pattern, construct a pair of related variables consisting of a first situational variable and a second situational variable; The scenario logic is generated based on the associated variables and preset rules; Wherein, the first situational variable is used to actively induce intuitive responses corresponding to the conflict mode in multimodal interaction, and the second situational variable is used to apply cognitive load or present contradictory information to stimulate cognitive adjustment behavior after the intuitive response is induced. The preset rules include the triggering conditions, timing of action and response relationship of the first situational variable and the second situational variable.

4. The mental health analysis method based on a large model and multimodal input according to claim 2, characterized in that, The step of performing spatiotemporal alignment and correlation analysis between the multimodal response data and the stimulus timeline corresponding to the psychological stimulus scenario includes: Obtain stimulus anchor points on the stimulus timeline that correspond to the context variable. The stimulus anchor points are used to identify the time points when the context variable is triggered or presented. Based on the stimulus anchor, the multimodal response data is segmented in the time dimension to obtain response data slices corresponding to the stimulus anchor. Within each response data slice, the user's procedural behavioral features under the influence of the contextual variables are extracted, wherein the procedural behavioral features include at least one of response delay, intensity of facial expression changes, voice tone fluctuation patterns, text sentiment tendencies, and conflict or coordination between multimodal response data. Based on time-series information, the procedural behavioral features within different reaction slices are serialized and analyzed for correlation, resulting in correlation analysis results that reflect the evolution of the procedural behavioral features over time.

5. The mental health analysis method based on a large model and multimodal input according to claim 4, characterized in that, The calculation of the user's mental schema parameters based on the correlation analysis results and the inverse mapping model includes: Obtain the procedural behavioral characteristics corresponding to each stimulus anchor point, and determine the type and preset intensity of the situational variable corresponding to the stimulus anchor point that induces the procedural behavioral characteristics; The process-oriented behavioral characteristics, along with the corresponding situational variable types and preset intensities, are used as a set of input data. The estimated value of the target mental construct is calculated based on multiple sets of input data and the inverse mapping model, and the estimated value is used as the user's mental schema parameter; The inverse mapping model is a conditional probability generation model. Given the situational variables, the estimated value is determined by maximizing the posterior probability of the existence of a mental schema within the user, based on the observed procedural behavioral characteristics.

6. A mental health analysis method based on a large model and multimodal input as described in claim 1 or 5, characterized in that, The construction of the inverse mapping model includes: Obtain a sample set containing known mental schema labels, wherein each sample in the sample set has a pre-defined set of situational variables corresponding to the mental schema label; Acquire multimodal response data for each sample under corresponding contextual variable stimuli, and extract procedural behavioral features from the multimodal response data; The inverse mapping model is trained by taking the combination of the situational variable set and the procedural behavioral features corresponding to each sample as input and the corresponding mental schema label as the training target. In the training process of the inverse mapping model, constraints based on psychological prior knowledge are adopted. These constraints are used to characterize the causal relationship between mental schemas and procedural behavioral features, so that the trained inverse mapping model can inversely infer the corresponding mental schema parameters based on the input set of situational variables and the observed procedural behavioral features.

7. The mental health analysis method based on a large model and multimodal input according to claim 1, characterized in that, The generation of a psychological state assessment report based on the psychological schema parameters includes: The mental schema parameters are input into the trained report big language model to obtain the mental state assessment report; The psychological state assessment report includes at least: a description of the user's psychological characteristics determined based on the psychological schema parameters; a psychodynamic correlation explanation of the user's behavior patterns observed in the psychological stimulus scenario and the psychological schema parameters; and developmental recommendations based on the correlation explanation and matching the user's behavior patterns.

8. The mental health analysis method based on a large model and multimodal input according to claim 7, characterized in that, After generating the psychological state assessment report based on the mental schema parameters, the method further includes: The psychological schema parameters, correlation analysis results, and corresponding psychological stimulus scenarios are stored in the user's temporal psychological profile; The psychological response pattern schema of the user is updated based on the time-series psychological profile. The psychological response pattern schema is used to characterize the user's stable response tendency, state fluctuation range and change sensitivity to different types of situational variables in the dimension of preset mental health assessment goals. The psychological response pattern is used to screen situational variables and scenario logic that induce target psychological responses during the generation of psychological stimulus scenarios; it is also used to perform personal benchmark calibration on the psychological schema parameters output by the inverse mapping model.

9. A mental health analysis method based on a large model and multimodal input as described in claim 8, characterized in that, After updating the user's psychological response pattern schema based on the time-series psychological profile, the method further includes: The psychological schema parameters are compared with the historical psychological schema parameters in the psychological response pattern schema to generate evolutionary analysis conclusions of user psychological characteristics; Based on the evolutionary analysis conclusions and the correlation analysis results, at least one item in the psychological state assessment report will be enhanced in depth or its presentation priority will be rearranged. The content of the assessment report includes descriptions of psychological characteristics, psychodynamic correlation explanations, and developmental suggestions.

10. A mental health analysis system based on large models and multimodal inputs, characterized in that, The system includes: The first acquisition module is used to acquire the user's data to be analyzed and preset mental health assessment goals, wherein the data to be analyzed includes at least one of facial images, voice data or text data; The analysis module is used to analyze the data to be analyzed based on the preset mental health assessment goals to obtain initial psychological data; A generation module is used to generate multimodal interactive psychological stimulation scenarios based on the initial psychological data and the multimodal model; The second acquisition module is used to acquire multimodal response data during the interaction between the user and the psychological stimulation scenario; The alignment module is used to perform spatiotemporal alignment and correlation analysis on the multimodal response data and the stimulus timeline corresponding to the psychological stimulus scenario to obtain the correlation analysis results. The stimulus timeline includes the stimulus event types, temporal relationships and logical associations used to induce the target psychological response. The calculation module is used to calculate the user's mental schema parameters based on the correlation analysis results and the inverse mapping model. The mental schema parameters are used to characterize the user's potential psychological patterns. The second generation module is used to generate a psychological state assessment report based on the psychological schema parameters.

Images