Psychological state assessment method, system, and readable storage medium based on environmental perception

Abstract

The present invention relates to the field of psychological state assessment technology, and specifically discloses a psychological state assessment method, system, and readable storage medium based on environmental perception. The method comprises the following steps: capturing environmental scenes and behavioral images of an assessed subject through an RGB camera, extracting object categories, spatial layouts, and dynamic events using a semantic segmentation model; capturing environmental soundprints through a microphone, and identifying sound categories using a sound recognition model; synchronously capturing facial micro-expression feature points, speech prosodic features, and head, neck, and shoulder vibration frequencies of the assessed subject; and generating semantic vectors from object / scene labels using a semantic embedding model. The psychological state assessment method based on environmental perception in an embodiment of the present invention overcomes the limitations of traditional psychological assessments that neglect environmental factors through three core innovations: quantitative modeling of environmental semantics, causal reasoning and dynamic attention mechanisms, and personalized adaptive calibration, significantly improving the accuracy, robustness, and practicality of psychological state assessment.

Description

Technical Field

[0001] The present invention relates to the technical field of psychological state assessment, and in particular to a psychological state assessment method, system, and readable storage medium based on environmental perception. Background Art

[0002] Mental state assessment has important application value in areas such as mental health monitoring and social interaction analysis. Traditional assessment methods mainly rely on users actively filling out psychological scales or wearing contact physiological sensors, which have the following significant drawbacks:

[0003] First, user privacy concerns or cognitive biases can distort the scale results, and the scale requires active user participation, making it unsuitable for real-time monitoring in natural settings. Second, existing non-contact technologies (such as text / image-based analysis, application CN111477328B; multimodal micro-expression analysis, application CN117936032A) focus solely on user behavior data and ignore environmental stimuli, such as social situations and noise interference, that directly impact psychological states. A common situation is that enclosed spaces can trigger anxiety, but existing methods fail to incorporate spatial layout into their assessment. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems in the related art to a certain extent. To this end, the present invention aims to provide a psychological state assessment method, system, and readable storage medium based on environmental perception to achieve quantitative analysis of the impact of the environment on the psychological state.

[0005] To achieve the above objectives, a first embodiment of the present invention provides a method for assessing a psychological state based on environmental perception, comprising the following steps:

[0006] S1. Use an RGB camera to capture images of the environment and the behavior of the subject being evaluated, and use a semantic segmentation model to extract object categories, spatial layout, and dynamic events. Use a microphone to capture environmental soundprints, and use a sound recognition model to identify sound categories. Simultaneously, collect facial micro-expression feature points, speech rhythm features, and head, neck, and shoulder vibration frequencies of the subject being evaluated.

[0007] S2. Object / scene labels are generated into semantic vectors through a semantic embedding model. Spatial layout and sound labels are converted into sparse vectors. These vectors are then mapped to the preset environmental emotion dimensions through a multi-layer perceptron to obtain stress values, relaxation levels, and social intensity.

[0008] S3. Construct an environment-psychology causal graph, define the causal relationship nodes and edge weights between the environmental emotional dimension, behavioral characteristics, and psychological state; adjust the attention of environmental characteristics to behavioral characteristics through the causal attention module, and fuse them to generate a multimodal feature vector;

[0009] S4. Input the fused feature vector into a causal Transformer model containing a dynamic causal weight update unit, and output the psychological state dimension assessment result;

[0010] S5. Generate a psychological state explanation report based on environmental semantics and dynamically calibrate the model causal weight based on the historical data of the evaluated object.

[0011] In some embodiments of the present invention, in step S1, the semantic segmentation model is Mask R-CNN, which is used to perform real-time semantic segmentation on images captured by the RGB camera and output a set of object categories in the environment. , spatial layout label and dynamic event collection The sound recognition model is YAMNet, which is used to classify environmental soundprints into a set of predefined sound categories. .

[0012] In some embodiments of the present invention, in step S2, the environmental emotion dimension includes a pressure value , relaxation and social intensity , a quantitative mapping of object, space, and sound labels to emotional dimensions is achieved through a predefined semantic-emotion mapping graph, and the mapping graph is iteratively optimized through an online knowledge graph update algorithm.

[0013] In some embodiments of the present invention, in step S3, the causal graph modeling step includes:

[0014] Define the node set as environmental emotional dimensions, behavioral characteristics, and psychological states;

[0015] Calculating causal effects with Do-Calculus As edge weights, = , representing a node For Node direct causal strength.

[0016] In some embodiments of the present invention, in step S3, the input of the causal attention module is the environmental feature query vector , behavioral feature key vector Sum value vector , the calculation formula is:

[0017]

[0018] in, From the environmental emotional dimension The linear projection of and From behavioral characteristics Linear projection of is the causal weight matrix, element Representing environmental characteristics Behavioral characteristics the causal strength of is the key vector dimension, Represents element-wise multiplication.

[0019] In some embodiments of the present invention, in step S4, the dynamic causal weight updating unit implements adaptive adjustment of the causal weight through a gating mechanism:

[0020]

[0021] in, The fusion feature vector is obtained by concatenating the environmental emotion dimension and the behavioral feature and then encoding it through the Transformer; is the real-time environmental emotion vector; is the learnable gating weight matrix, is the Sigmoid activation function; is the basic causal weight matrix, which is obtained through global data training.

[0022] In some embodiments of the present invention, in step S4, during the process of outputting the psychological state dimension assessment result, the loss function of the psychological state assessment is considered to be the causal mean square error:

[0023]

[0024] in, is the total number of samples, For the The true value vector of the psychological state of samples, is the model prediction value; is the set of directed edges in the causal graph, is the true causal strength, Predicting causal strength for the model; is a regularization parameter used to balance the psychological state prediction error and the causal weight fitting error.

[0025] In some embodiments of the present invention, in step S5, the dynamic calibration of the model causal weights based on the historical data of the evaluated object includes personalized calibration, and the personalized calibration is achieved by:

[0026] For new users, before using Second-rate( is a natural number and )Evaluation results Fine-tune the causal weight matrix:

[0027]

[0028] in: is a learnable personalized bias matrix with the same dimension as Consistent; MLP is a multi-layer perceptron used to map historical evaluation results into weight adjustment vectors; calibrated causal weights The evaluation process is only applied to the current user.

[0029] To achieve the above objectives, a second embodiment of the present invention provides a psychological state assessment system based on environmental perception, the system comprising:

[0030] The multimodal data acquisition module is configured to collect environmental scenes and user behavior images through an RGB camera, collect environmental voiceprints through a microphone, and simultaneously collect user facial micro-expression feature points, speech rhythm features, and head, neck, and shoulder vibration frequencies; the environmental semantic analysis module is configured to use a semantic segmentation model to extract object categories, spatial layouts, and dynamic events, use a sound recognition model to identify sound categories, and generate environmental emotion dimension values ​​through a semantic-emotion mapping map; the causal fusion module is configured to construct an environmental-psychological causal graph, fuse environmental features and behavioral features through a causal attention module, and generate a multimodal feature vector; the psychological state assessment module is configured to use a causal Transformer model containing a dynamic causal weight update unit to output the psychological state dimension assessment results; the feedback and optimization module is configured to generate a psychological state explanation report and dynamically calibrate the model causal weights based on user historical data.

[0031] To achieve the above-mentioned purpose, a third embodiment of the present invention proposes a readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the above-mentioned psychological state assessment method based on environmental perception is implemented.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] The environmental perception-based psychological state assessment method, system, and readable storage medium of the embodiments of the present invention break through the limitations of traditional psychological assessment that ignores environmental factors through three core innovations: quantitative modeling of environmental semantics, causal reasoning and dynamic attention mechanism, and personalized adaptive calibration. They construct an "environment-behavior-psychology" three-in-one assessment framework, significantly improve the accuracy, robustness, and practicality of psychological state assessment, and provide a new technical paradigm for the field of intelligent mental health monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The disclosure of the present invention is described with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. In the drawings, the same reference numerals are used to refer to the same components. Among them:

[0035] Figure 1 is a flow chart of a method for evaluating a psychological state based on environmental perception in one embodiment of the present invention;

[0036] Figure 2 This is a heat diagram of environmental semantic-emotion mapping in one embodiment of the present invention;

[0037] Figure 3 1 is a schematic diagram of a dynamic change curve of the environmental emotion dimension in one embodiment of the present invention;

[0038] Figure 4 is a comparative schematic diagram of personalized calibration in one embodiment of the present invention;

[0039] Figure 5 2 is a schematic diagram of the structure of a psychological state assessment system based on environmental perception according to another embodiment of the present invention. DETAILED DESCRIPTION

[0040] The following describes embodiments of the present invention in detail, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and are not to be construed as limiting the present invention.

[0041] The following describes a method, system, and readable storage medium for assessing a mental state based on environmental perception according to embodiments of the present invention with reference to the accompanying drawings.

[0042] Figure 1 4 is a flow chart of a method for psychological state assessment based on environmental perception according to an embodiment of the present invention.

[0043] like Figure 1 As shown, the mental state assessment method based on environmental perception includes the following steps:

[0044] S1. Collect environmental scenes and behavior images of the evaluated object through an RGB camera, and use a semantic segmentation model (such as Mask R-CNN) to extract object categories, spatial layout, and dynamic events;

[0045] Among them, object categories include "desk" and "green plants"; spatial layouts include "open office" and "closed conference room"; dynamic events include "multiple people talking" and "object movement".

[0046] A microphone is used to collect environmental soundprints, such as keyboard tapping sounds and alarm sounds, and a sound recognition model (such as YAMNet) is used to identify the sound category. At the same time, the facial micro-expression feature points, speech rhythm features, and head, neck, and shoulder vibration frequencies of the evaluated subjects are synchronously collected. Micro-expression feature points include 68 feature points such as raised corners of the mouth and frowning, speech rhythm features include speaking speed and pitch, and head, neck, and shoulder vibration frequencies include slight tremors when nervous.

[0047] Since traditional psychological assessments rely solely on user self-reports or physiological indicators, this step simultaneously collects environmental stimulus data (such as scene layout and sound type) and user behavior data for the first time to construct a complete dataset of "environment-human" interaction.

[0048] As an example, in a driving scenario, stress levels can be comprehensively assessed by identifying “congested traffic conditions” (environment) and the driver’s “frequent frowning” (behavior).

[0049] Moreover, this solution uses computer vision and speech recognition technology to convert vague environmental information into structured data, such as "enclosed space" and "high-frequency noise", laying the foundation for subsequent quantitative analysis of psychological state.

[0050] S2. Generate semantic vectors for object / scene labels (e.g., "green plant," "conference table") using a semantic embedding model (e.g., BERTopic). Also, convert spatial layouts (e.g., "enclosed") and sound labels (e.g., "alarm sound") into sparse vectors (e.g., one-hot encoding).

[0051] Using a multi-layer perceptron (MLP), these vectors are mapped to pre-defined environmental emotion dimensions to generate stress, relaxation, and social intensity. For example, a stress value like "conference table" equals stress +10, a relaxation value like "green plants" equals relaxation +15, and a social intensity like "multi-person conversation" equals social intensity +20.

[0052] Traditional methods cannot quantify the psychological impact of the environment. This step converts abstract environmental features into computable emotional indicators through semantic-emotional mapping. The mapping relationship is iteratively optimized through an online knowledge graph algorithm. For example, when a user feedback states that "green plants do not relieve stress," the weight is automatically adjusted to ensure that the model can adapt to diverse scenarios.

[0053] As an example, “natural scenery” is automatically mapped to a relaxation value of +20, and “alarm sound” is mapped to a stress value of +18, so that environmental factors can be directly involved in the calculation of psychological state.

[0054] S3. Construct an environmental-psychological causal graph, defining the causal nodes and edge weights between environmental affective dimensions, behavioral traits, and psychological states. Traditional models only analyze data correlations, such as "noise and anxiety occur simultaneously." This step uses the causal graph to clearly identify direct influencing paths, such as "noise → physiological arousal → anxiety," to avoid spurious correlations. For example, if "music" and "relaxation" are only indirectly related, the model will weaken their direct weight through the mediation of "pleasant scenes," improving inference accuracy.

[0055] A causal attention module is designed to adjust the weight of environmental features over behavioral features, adjusting the interaction between them and fusing them to generate a multimodal feature vector. For example, if the causal strength of "high-frequency noise" and "voice tremor" is high, the model prioritizes fusing these two features. The causal attention module automatically suppresses irrelevant environmental features, such as the impact of background bird song on office stress, and focuses on feature combinations with high causal strength, such as "enclosed space + rapid speech," improving model efficiency and accuracy.

[0056] S4. The fused feature vector is input into a causal Transformer model that includes a dynamic causal weight update unit. This model has a built-in dynamic weight update unit that adjusts causal weights in real time through a gate mechanism. For example, when a "driving scene + sudden braking sound" is detected, the weight of "stress value" over "micro-expression tension" is automatically increased, and the psychological state dimension assessment results are output, such as an anxiety score of 75 and a relaxation score of 20.

[0057] The loss function is designed to constrain both the prediction error and the causal weight fitting error, ensuring that the model is both accurate and consistent with the true causal relationship.

[0058] S5. Generate a psychological status interpretation report based on environmental semantics. For example, "The current psychological stress value is 68 points, due to the detection of a closed space (stress + 8), a conversation between three people (social intensity + 15), and voice frequency fluctuations (anxiety-related characteristics)." Through environmental semantic interpretation, such as "stress increased due to the meeting scene," the trust of the assessed person is enhanced, making it easier for the counselor or the assessed person to understand the assessment basis.

[0059] In addition, the causal weights of the model should be dynamically calibrated based on the historical data of the evaluated object to avoid the misjudgment of individual differences by the general model.

[0060] As an example, it is applicable to psychological monitoring of different drivers in driving scenarios.

[0061] Through the above steps, this solution has built a complete technical chain of "environmental perception-causal modeling-dynamic evaluation-personalized feedback", significantly improving the accuracy, robustness and interpretability of mental state assessment, and providing a new technical path for the field of intelligent mental health monitoring.

[0062] In some embodiments of the present invention, in step S1, the semantic segmentation model is Mask R-CNN, which is used to perform real-time semantic segmentation on the image captured by the RGB camera and output a set of object categories in the environment. , spatial layout label and dynamic event collection .

[0063] Among them, the RGB camera uses a ≥8-megapixel global shutter camera, supports 1920×1080 resolution @30fps, and is equipped with automatic white balance and low-light enhancement functions to ensure image clarity in complex environments.

[0064] The semantic segmentation model processing process is as follows:

[0065] Input: Real-time RGB image frames are fed into the semantic segmentation model Mask R-CNN (an instance segmentation model based on the Faster R-CNN architecture) (H is the image height, W is the width);

[0066] Then output: object category set ,in For strings, such as desks and green plants, each element is accompanied by a confidence score , represents the detection reliability;

[0067] Space Layout Tags Open, closed, semi-open, outdoor ,output through scene classification branch;

[0068] Dynamic event collection ,in Description of dynamic behaviors, such as "multiple people talking", "object movement", "door opening and closing", etc., with time and space coordinates (x,y are image coordinates, z is the timestamp).

[0069] It should be noted that the input image should be resized to a fixed size of 640*480, normalized to the interval [-1,1], and non-maximum suppression (NMS) should be used to filter redundant detection boxes with a threshold of 0.5.

[0070] During the process of collecting environmental voiceprints, acoustic data collection and voice recognition are based on the following configuration:

[0071] Microphone array: integrated 3-channel MEMS microphone, frequency response 20Hz-20kHz, signal-to-noise ratio ≥65dB, supports sound source localization;

[0072] The sound recognition model is YAMNet (CNN-based environmental sound classification model), which is used to classify environmental soundprints into a set of predefined sound categories. Its processing flow is as follows:

[0073] Input: Real-time voiceprint signal (T is the number of sampling points, sampling rate is 44.1kHz);

[0074] Output: A set of predefined sound categories ,in , with the accompanying class probability vector .

[0075] Acoustic data also needs to be preprocessed during collection. The preprocessing steps include: voiceprint framing: frame length 512ms, frame shift 256ms, and Hamming window addition; feature extraction: calculating the Mel spectrum map and generating 40-dimensional MFCC features.

[0076] As an example, user behavior data also needs to be collected, including:

[0077] Facial micro-expressions: Use the OpenFace algorithm to extract the coordinates of 68 facial landmark points (j=1,…,68), calculate the action unit (AU) strength, such as AU1 (inner eyebrow lift), AU12 (mouth corner lift);

[0078] Speech prosodic features: Extract 23-dimensional features such as fundamental frequency (F0), short-term energy, speech rate, and formant frequency; Head, neck, and shoulder vibration: Collect three-axis vibration data through IMU sensors (accelerometer + gyroscope) , the sampling rate was 100 Hz, and the physiological tremor signal was extracted through a band-pass filter (2-20 Hz).

[0079] The following table shows the technical implementation and examples of some actions in the action unit (AU):

[0080] AU number name Description of facial muscle movements Mental state association The role of technology in this solution AU1 Medial brow lift The medial frontalis muscle contracts, moving the eyebrows upward and toward the middle. Surprise, concern, confusion The eyebrow feature point coordinates are extracted through OpenFace, and the difference between the vertical displacement of the eyebrow and the baseline is calculated and quantified into an intensity value (0-5 levels). AU4 frown The corrugator and depressor supercilii muscles contract, pushing the eyebrows down and bringing them together. Stress, anxiety, thinking Calculate the change in the distance between eyebrows and the eyebrow angle, output the standardized intensity value, and associate it with the pressure value dimension AU12 Upturned corners of mouth The zygomatic major muscle contracts, causing the corners of the mouth to lift to the sides Joy, positive emotions Extract the coordinates of the corners of the mouth, calculate the angle of the corners of the mouth and the baseline angle, and input them into the causal attention module AU17 Chin raised The jaw muscles contract, lifting the chin upward Uncertainty, thinking Monitoring jaw movement for micro-expression analysis in social intensity assessment AU25 Lips parted The orbicularis oris muscle relaxes and the lips open Surprise, expression of emotion Detect changes in the distance between the upper and lower lips to assist in determining the impact of sudden environmental stimuli

[0081] As an example, during the environmental scene collection process described above, traditional semantic segmentation models do not differentiate between dynamic objects, such as moving people and objects, and static scenes. However, dynamic objects often have a more significant and immediate impact on the psychological state of the assessed subject. For example, suddenly moving objects can easily trigger a sudden change in attention and psychological stress reactions. To accurately capture such dynamic environmental stimuli, a dynamic object priority mechanism is introduced to increase the weight of its corresponding semantic features:

[0082]

[0083] in, : The intersection-over-union ratio of the dynamic object mask and the current frame, reflecting the coverage and detection accuracy of the dynamic object in the current frame; : The historical maximum intersection-over-union ratio is used as a reference benchmark to measure the saliency of the currently detected dynamic object; : Weight coefficient, determined through experiments, aims to highlight the dynamic change characteristics of dynamic objects; when near hour, Approaching , indicating that the current dynamic object detection is stable and significant, and its semantic feature weight needs to be improved; on the contrary, if Smaller, Approaching , to avoid excessive attention due to unstable detection.

[0084] Regarding the above dynamic object priority mechanism, two specific examples are shown here to verify the effectiveness of the dynamic object priority mechanism.

[0085] Example 1: Dynamic Interference Assessment in an Office Scenario

[0086] Scenario description: The subject works in an open office and a colleague walks quickly past his desk (dynamic object).

[0087] Traditional processing methods: The semantic segmentation model treats dynamic "colleagues" and static scene elements such as "desks" and "computers" equally, and does not highlight the immediate impact of dynamic objects on psychological state.

[0088] The final psychological state assessment may ignore this dynamic interference, resulting in inaccurate calculation of dimensions such as stress values.

[0089] This solution processes: 1. Data calculation: dynamic objects (colleagues) in the current frame =0.7, the largest in history =0.8, substitute into the formula:

[0090]

[0091] Significantly improve the weight of the semantic features of the dynamic object.

[0092] 2. Evaluation Effect: When generating the dynamic emotional vector of the environment, this dynamic disturbance is captured in particular. Combined with the subject's potential micro-expression changes (such as shifted gaze or frowning), this approach more accurately calculates the stress level increase. Compared to traditional methods, this solution can accurately and in real time reflect the impact of a dynamic environment on psychological state.

[0093] Example 2: Crowd Dynamics Assessment in a Shopping Mall

[0094] Scenario description: The subject being evaluated is in a promotional area of ​​a shopping mall, surrounded by a crowd of people (dynamic objects).

[0095] Traditional processing methods: The semantic segmentation model does not give special weight to the dynamic element of "surging crowds", making it difficult to accurately quantify its stimulation on psychological states (such as excitement or irritability).

[0096] This solution processes: 1. Data calculation: Assuming the current frame =0.6, =0.8, then

[0097]

[0098] The semantic feature weight of dynamic crowds is enhanced.

[0099] 2. Evaluation Effectiveness: This approach combines the emotional polarity (assuming it's positive) of ambient sounds (such as promotional announcements) to generate a more comprehensive environmental dynamic emotion vector. When assessing psychological states, it more accurately reflects the subject's emotional fluctuations (such as increased excitement) caused by crowd dynamics and the promotional atmosphere. Traditional methods tend to overlook the weighting of dynamic objects, leading to evaluation bias.

[0100] From the above examples, we can see that the dynamic object priority mechanism quantifies the detection significance of dynamic objects ( and relationship), dynamically adjust its semantic feature weights , which enables environmental perception to focus more on dynamic elements that have a significant and immediate impact on psychological state. Compared with traditional methods, it can more accurately capture dynamic environmental stimuli, improve the accuracy and timeliness of psychological state assessment, and effectively verify the technical effect of this solution.

[0101] As an example, the microphone captures environmental soundprints and identifies sound categories, such as "alarm" and "conversation." However, category information alone is insufficient to fully reflect the impact of the environment on psychological state. For example, "alarm" has a negative emotional polarity and can easily induce tension, while "birdsong" may have a positive emotional polarity and bring a sense of relaxation.

[0102] Therefore, the output of the sound recognition model YAMNet can be expanded to include the emotional polarity of the ambient sound (positive / negative / neutral). This can be fused with the semantic segmentation results to generate an ambient dynamic emotion vector, which more accurately maps to the ambient emotional dimension. By supplementing the emotional attributes of sound, the emotional modeling of ambient semantics is deepened, allowing the ambient dynamic emotion vector to more comprehensively represent the potential impact of the environment on psychological state, providing richer and more accurate environmental emotional features for causal fusion and psychological state assessment in step S3 above.

[0103] like Figure 2A heat map of environmental semantics and sentiment mapping is shown, where the horizontal axis represents environmental labels, including objects (such as "green plants"), spatial layouts (such as "enclosed space"), sounds (such as "alarm sounds"), and dynamic events (such as "multi-person conversation"); the vertical axis represents environmental sentiment dimensions, corresponding to stress values, relaxation levels, and social intensity in this solution.

[0104] Figure 2 In the data, positive values ​​indicate positive impacts (e.g., “green plants → relaxation +15”), and negative values ​​indicate negative impacts (e.g., “enclosed space → relaxation -3”). By visualizing the quantitative logic of environmental semantic-emotional mapping heat, it is proved that environmental characteristics can be converted into computable emotional indicators.

[0105] In some embodiments of the present invention, in order to accurately achieve the quantitative mapping of environmental semantics to emotional dimensions and solve the problems of traditional methods that lack consideration of the comprehensive effects of environmental factors and have poor dynamic adaptability, the quantification process of environmental emotional dimensions is specifically refined. The specific technical means are as follows:

[0106] In step S2, the environmental emotional dimension includes the stress value , relaxation and social intensity , and through the predefined semantic-emotion mapping graph, the quantitative mapping of object, space, and sound labels to emotional dimensions can be achieved. The mapping graph is iteratively optimized through the online knowledge graph update algorithm.

[0107] However, traditional methods fail to fully consider the combined impact and dynamic characteristics of various environmental factors. For example, stress levels can be affected by the combined effects of enclosed spaces, high-frequency noise, and social density, but traditional methods fail to systematically quantify and integrate these factors. Relaxation levels also fail to clearly define the synergistic effects of natural elements and specific sounds. Therefore, this solution achieves precise mapping through specific formulas, aligning with the aforementioned "online knowledge graph update algorithm to iteratively optimize the mapping graph" to ensure dynamic adaptability.

[0108] As an example, the quantitative formulas for each environmental emotion dimension are defined and explained as follows:

[0109] 1. Pressure value:

[0110]

[0111] in, : The proportion of enclosed space, obtained by analyzing the static scene mask through the semantic segmentation model, reflects the potential impact of the degree of space closure on pressure;

[0112] : High-frequency noise intensity, obtained by the sound recognition model combined with acoustic data collection, quantifies the psychological stimulation of noise;

[0113] : Social density, calculated based on detected dynamic objects (such as crowds) and semantic segmentation results, reflects the density of social interactions;

[0114] : Environmental factor weights, determined through experiments, balance the contribution of each factor to the pressure value.

[0115] This formula is related to the dynamic object priority mechanism in this solution. If a dynamic object (such as a fast-moving crowd) is detected, its corresponding social density The priority mechanism will increase the weight, thus more significantly affecting , ensuring that dynamic environmental stimuli are accurately captured.

[0116] 2. Relaxation:

[0117]

[0118] : The proportion of natural elements (such as green plants and water features) is obtained by analyzing the static scene mask using a semantic segmentation model, reflecting the contribution of natural elements to the sense of relaxation;

[0119] : The intensity of low-frequency white noise, obtained through sound recognition models and acoustic data processing. This type of sound usually has a soothing effect;

[0120] : Weight of environmental factors, highlighting the dominant role of natural elements in relaxation.

[0121] 3. Social intensity:

[0122]

[0123] : The number of faces detected, obtained through a visual module (such as an RGB camera combined with a face recognition algorithm), reflects the number of social participants;

[0124] : Voice interaction frequency, derived from the microphone array and voice processing module, reflects the frequency of social interaction;

[0125] : Social factor coefficient, which adjusts the calculation of social intensity.

[0126] The above three formulas are the specific implementation of "realizing quantitative mapping through a predefined semantic-sentiment mapping graph", and the mapping graph is iteratively optimized through the online knowledge graph update algorithm.

[0127] like Figure 3This graph shows the dynamic changes in the emotional dimension of the environment. The horizontal axis represents the time series, simulating environmental changes in different scenarios (e.g., "open office → introduction of green plants → alarm sound → multiple people talking → enclosed space"). The vertical axis represents the emotional dimension value, reflecting the real-time impact of environmental stimuli.

[0128] Figure 3 In the figure, the pressure value (red curve) in the initial state (open space + keyboard sound) is 10-6=4; it suddenly increases by 18 when the alarm sounds; and increases by another 12 when switching to the closed space (the direct influence of the space layout).

[0129] Relaxation (green dashed line): The introduction of greenery leads to an increase of 15 (positive sentiment of the object label); the alarm sound leads to a decrease of 10 (negative sound sentiment).

[0130] Social intensity (blue dotted line): Increases by 15×1.3=19.5 when multiple people are chatting (the dynamic object priority mechanism in this solution increases the weight by 30%).

[0131] Figure 3 The mechanism of “dynamic environmental stimulation → real-time update of emotional dimension” is intuitively demonstrated, verifying the dynamic adaptability of the semantic-emotional mapping graph, and also echoing the online update algorithm of step S2.

[0132] Through event labeling, the immediate impact of dynamic objects (such as "multiple people talking") and sudden sounds (such as "alarm sounds") on psychological state is also highlighted, proving the limitations of traditional methods that ignore environmental factors.

[0133] As an example, if the dynamic object priority mechanism finds that a certain type of dynamic object (such as a crowd that suddenly appears) has a negative impact on the pressure value, The impact is significant and can be adjusted by updating the algorithm Weight or The calculation method ensures that the mapping map can adapt to diverse scenarios, which is consistent with the direction of "deep modeling of environmental semantics and emotional association" mentioned above, and improves the accuracy and dynamic adaptability of psychological state assessment.

[0134] In some embodiments of the present invention, the core purpose of constructing an environment-psychology causal graph in step S3 is to address the limitations of correlation analysis in traditional psychological assessment techniques. Existing methods, such as those in applications CN111477328B and CN117936032A, only correlate environment and psychological state through statistical correlation, failing to distinguish between causal relationships and spurious correlations. For example, "music" and "relaxation" may be indirectly related through the mediation of "pleasant scenes," rather than directly causally related, resulting in poor model interpretability and susceptibility to interference.

[0135] Therefore, it can be concluded that: traditional methods cannot clearly determine the direct causal strength of environmental factors (such as "enclosed space") on psychological states (such as "anxiety"), and can only judge the association through co-occurrence frequency, which lacks scientific basis; and when environmental factors (such as dynamic objects, sudden sounds) change, traditional models cannot adjust feature interaction weights in real time, resulting in delayed or inaccurate evaluation.

[0136] Here is a solution:

[0137] By introducing causal reasoning theory (Do-Calculus), the emotional dimensions of the environment (such as stress value) are explicitly modeled. ), direct causal relationships between behavioral characteristics (such as micro-expression frequency) and psychological states (such as anxiety levels), and quantify causal effects As edge weights, it ensures that the model focuses on the real impact path and improves the interpretability and dynamic adaptability of the evaluation.

[0138] The causal diagram modeling steps specifically include:

[0139] 1. Node set definition

[0140] Environmental emotional dimension (U): includes stress value , relaxation , social intensity ;

[0141] Behavioral characteristics (B): including facial action unit intensity (such as the frequency of mouth corner raising) and voice fundamental frequency fluctuations , head, neck and shoulder vibration frequency ;

[0142] Mental state (Y): target evaluation dimension, such as anxiety value, pleasure value, and aggressiveness index, output by the causal Transformer model in step S4 above.

[0143] 2. Edge Weight Calculation: Causal Effect Based on Do-Calculus

[0144] formula:

[0145] in, (environmental emotional dimension or behavioral characteristic node), (mental state node);

[0146] :Intervention variables hour, and The covariance of , characterizes the strength of causal dependence;

[0147] : variable The variance of , used to normalize the causal effect.

[0148] Physical meaning: measure when When actively intervened (such as artificially increasing the length of the closed space), The degree of change, excluding the interference of other variables, directly reflects causal strength.

[0149] 3. Correlation with the dynamic object priority mechanism

[0150] In step S1, the dynamic object priority mechanism is implemented by Causal graph modeling enhances the semantic weight of dynamic environmental features (such as fast-moving crowds), and further incorporates this weight into the calculation of causal effects. Causal graph modeling not only addresses the shortcomings of traditional correlation analysis but also builds a scientific and interpretable framework for psychological state assessment through quantification of causal strength, dynamic weight adjustment, and personalized adaptation.

[0151] Example: Social density of dynamic objects As components, their causal effects Will be due to Enhanced and enhanced, the model prioritizes feature interactions with high causal strength in dynamic scenarios.

[0152] In some embodiments of the present invention, the design of the causal attention module addresses the limitations of traditional attention mechanisms in psychological state assessment. Because traditional multi-head attention mechanisms only calculate feature interaction weights based on data correlation, they are unable to distinguish the causal relationship between environmental and behavioral features. For example, "office greenery" and "relaxation" may be incorrectly associated due to scene correlation, while the actual causal path may be indirect, through "natural element cognition → relaxation." Traditional methods cannot explicitly model this causal logic.

[0153] Furthermore, traditional mechanisms are unable to adjust feature weights in real time when the environment changes dynamically (such as sudden noise or crowd movement), resulting in a delay in the model's response to immediate stimuli. For example, in a driving scenario, the strong causal relationship between the sound of a sudden brake and the driver's micro-expressions must be captured in real time. Traditional methods may overlook such critical interactions due to fixed weights.

[0154] Therefore, the key to solving the above problems lies in how to introduce causal constraints in multimodal feature fusion, so that the model focuses on the true causal path and improves the evaluation accuracy and interpretability in dynamic environments.

[0155] In step S3, the input of the causal attention module is the environment feature query vector , behavioral feature key vector Sum value vector , the calculation formula is:

[0156]

[0157] in, From the environmental emotional dimension The linear projection of and From behavioral characteristics Linear projection of is the causal weight matrix, element Representing environmental characteristics Behavioral characteristics The causal strength is calculated by Do-Calculus; is the key vector dimension, Represents element-wise multiplication.

[0158] In some embodiments of the present invention, the design of the dynamic causal weight updating unit stems from the defect that the causal weight in the traditional psychological assessment model is fixed.

[0159] Traditional approaches assume that the causal relationship between environmental features and psychological states is static, making them unable to adapt to dynamically changing environments. For example, in a driving scenario, the weight of a sudden alarm on anxiety should be significantly higher than that of a similar sound in a quiet office, but traditional models cannot adjust in real time. Furthermore, when the semantics of the environment change (for example, from "quiet" to "noisy"), fixed causal weights cannot reflect the true impact strength between features in the current environment. Furthermore, the same environmental feature (such as "human voice") has different effects on psychological states in different scenarios (e.g., home and workplace), and traditional models lack a context-aware weight adjustment mechanism.

[0160] Therefore, the key to solving the problem lies in real-time perception of environmental changes through the gating mechanism, dynamically adjusting the causal weight matrix, making the model adapt to the dynamic environment and individual scenario differences, and improving the accuracy of psychological state assessment.

[0161] In step S4, the dynamic causal weight update unit realizes adaptive adjustment of causal weight through the gating mechanism:

[0162]

[0163]

[0164] in, : Fusion feature vector, composed of environmental emotion dimension ( , , ) is spliced ​​with behavioral features (such as micro-expressions and speech prosody) and encoded by Transformer, which contains deep fusion information of environment and behavior;

[0165] : Real-time environmental emotion vector, generated by semantic-emotion mapping, reflecting the stress value, relaxation and social intensity of the current environment;

[0166] : A learnable gating weight matrix that learns the influence of the joint representation of the environment and fusion features on the gating signal through training;

[0167] : Sigmoid activation function, compresses the gate signal to the [0, 1] interval, and outputs Gate ∈ [0,1], which represents the update degree of dynamic causal weight in the current environment;

[0168] : The basic causal weight matrix is ​​obtained through global data training, which reflects the causal relationship between environmental characteristics and psychological states in general.

[0169] Explain the above gating signal calculation and weight update mechanism:

[0170] Gate signal calculation: Gate= ( ), the fusion feature H is combined with the real-time environmental emotion After splicing, the network is fed into the gating network, which outputs a gating signal between 0 and 1. For example, if a sudden alarm sound (S_{stress} surges) is detected in a driving scenario, the gating signal Gate approaches 1, indicating that a significant update of the causal weight is required.

[0171] Weight update: , when Gate is close to 1, the dynamically adjusted weight Plays a leading role, the model focuses on the causal relationship in the current environment; when Gate is close to 0, the basic weight is retained , ensuring the evaluation consistency of the model in a stable environment.

[0172] Through the above steps, the dynamic causal weight update unit solves the problem of poor adaptability of traditional models to dynamic environments. It adjusts the causal weights in real time through the gating mechanism, enabling the model to accurately capture the key causal relationships in the current environment.

[0173] In some embodiments of the present invention, the loss function of the traditional mental state assessment model (such as mean square error) ) Focus only on the predicted value and the true value The gap between environmental features and psychological states is neglected, ignoring the causal constraints between environmental characteristics and psychological states. This can lead to the model learning superficial statistical correlations (such as the association between "office greenery" and "relaxation" due to their co-occurrence in different scenes) rather than true causal pathways (such as "natural element cognition → neural relaxation → psychological pleasure"), resulting in poor model interpretability and insufficient generalization.

[0174] Therefore, the key to solving the problem lies in introducing the causal mean square error loss function, while constraining the psychological state prediction error and the causal weight fitting error, to ensure that the model can not only accurately predict the psychological state, but also learn the feature interaction relationship that conforms to the real-world causal logic.

[0175] In step S4, when outputting the psychological state dimension evaluation results, the loss function of the psychological state evaluation is considered to be the causal mean square error:

[0176]

[0177] in, : Total number of samples, covering the dataset size of different environmental scenarios and user states;

[0178] : No. The true value vector of the psychological state of samples, such as , obtained through professional psychological assessment tools or prior knowledge;

[0179] :Model for The predicted value of the psychological state of each sample is output by the causal Transformer model;

[0180] : A set of directed edges in a causal graph, such as , reflecting the causal relationship between the emotional dimension of the environment and psychological state;

[0181] : True causal strength, determined by Do-Calculus or prior knowledge, e.g. , indicating the intensity of the direct impact of stress value on anxiety;

[0182] : The causal strength of the model prediction, learned by the causal Transformer model during training;

[0183] : Regularization parameter used to balance the error in psychological state prediction and the error in causal weight fitting. When is larger, the model pays more attention to the accuracy of causal relationship; when When it is small, it focuses on the accuracy of psychological state prediction.

[0184] In the above steps, the psychological state prediction error term is: Ensure model output Close to the real psychological state , is the reserved term of the traditional loss function;

[0185] And the causal weight fitting error term: Constraining the causal strength of model predictions Approximating the true value For example, if the real scene has “high frequency noise The causal strength of anxiety , the parameters are adjusted during model training. Close to 0.8 to prevent the model from incorrectly associating "high-frequency noise" with "relaxation".

[0186] It should be noted that the loss function and Directly linked to the environment-psychology causal graph constructed in step S3, strengthening the causal relationship learning defined in the causal graph. In addition, the dynamic causal weight update unit in step S4 adjusts the , and the causal weight fitting error term in the loss function provides the optimization target for the adjustment, so Not only adapt to dynamic environments (through Gate), but also conform to real cause and effect relationships, through .

[0187] Through the above steps, the causal mean square error loss function solves the problems of lack of causal relationship modeling and insufficient robustness in traditional models, ensuring that the model strikes a balance between the accuracy of psychological state prediction and the consistency of causal logic.

[0188] In some embodiments of the present invention, since the traditional mental state assessment model adopts a unified causal weight matrix (such as ), ignoring the differences in individual responses to environmental stimuli. For example, in the environmental-psychological causal diagram constructed in step S3, “social density The causal strength of anxiety It is a universal value trained based on global data, but in reality, introverted users are more sensitive to social density, while extroverted users are not.

[0189] This "one-size-fits-all" approach leads to a decrease in the evaluation accuracy of the model when facing new users. Therefore, this solution proposes to use the previous ( ) evaluation results, dynamically fine-tune the causal weight matrix to adapt the model to individual differences and improve the accuracy of personalized evaluation.

[0190] In step S5, the causal weights of the model are dynamically calibrated based on the historical data of the evaluated object, including personalized calibration. The personalized calibration is achieved by:

[0191] For new users, before using Second-rate( is a natural number and )Evaluation results Fine-tune the causal weight matrix (calibration is best when t=3):

[0192]

[0193] in: : The personalized calibrated causal weight matrix only acts on the evaluation process of the current user. For example, when calculating the causal strength of "enclosed space → anxiety", use Replace global ;

[0194] : The basic causal weight matrix is ​​obtained through global data training, reflecting the causal relationship between environmental characteristics and psychological states in general, such as the universal causal strength calculated by Do-Calculus in step S3;

[0195] : Learnable personalized bias matrix with dimensions Consistency is used to capture individual and global differences. For example, if a user is more sensitive to "high-frequency noise", The corresponding item in the middle will increase the weight of "high-frequency noise → anxiety";

[0196] MLP: Multi-layer perceptron, which combines historical evaluation results Mapped to a weight adjustment vector. For example, if the user's anxiety value in the social scene is high in the previous t evaluations, the MLP output vector will increase the "social intensity The weight of "anxiety".

[0197] : The psychological state evaluation result vector of the new user in the previous t times, output by the causal Transformer model in step S4.

[0198] It should be noted that the historical evaluation result processing is to convert the previous t evaluation results into Input MLP, MLP learns the mapping relationship between evaluation results and weight adjustment through nonlinear transformation (such as ReLU activation function). For example, if the user’s anxiety value in the “multi-person conversation” scenario for the first two times is Higher, the MLP output vector will indicate an increase in "social strength →The weight of "anxiety";

[0199] Weight fine-tuning refers to Calculate personalized bias and add it to the basic weight Go up and get .For example, The weight of "social intensity → anxiety" is 0.3, and the personalized bias adjusts it to 0.5, which is more in line with the user's sensitivity to social interaction.

[0200] It is also important to note that the mental state interpretation report generated in step S5 in combination with the environmental semantics provides feedback to the user, which can be indirectly used to adjust The evaluation results after This creates a closed loop of "evaluation-calibration-re-evaluation" to continuously optimize the personalized experience, closely integrating with the aforementioned "real-time feedback and model optimization." Furthermore, this personalized calibration solution addresses the issues of traditional models ignoring individual differences and experiencing poor cold-start adaptability. By dynamically fine-tuning causal weights based on historical evaluation results, the model can better adapt to new users.

[0201] After this step, the personalized calibration Under the constraint of the causal mean square error loss function in step S4, not only the psychological state prediction accuracy is optimized, but also the personalized causal weight is ensured. Stay close to the user's real cause and effect relationship and avoid logical deviations caused by excessive personalization.

[0202] like Figure 4 A comparison experiment of personalized calibration is shown, where the top figure shows the curve under the same environmental stimulation (such as continuous noise); the middle figure shows the uncalibrated pressure value evaluation results (user A is sensitive, user B is tolerant); the bottom figure shows the pressure value evaluation results after personalized calibration. Figure 4 The improvement of evaluation accuracy by personalized calibration mechanism was verified.

[0203] like Figure 5 As shown, a psychological state assessment system based on environmental perception includes:

[0204] Multimodal data acquisition module, specifically including:

[0205] Environmental visual data acquisition: A high-resolution RGB camera (≥8 megapixels, global shutter, supports 1920×1080@30fps) equipped with automatic white balance and low-light enhancement functions is used to capture environmental scenes (such as desks and green plants) and user behavior images (such as facial expressions and body movements) in real time.

[0206] Environmental soundprint collection: Integrated 3-channel MEMS microphone array (frequency response 20Hz-20kHz, signal-to-noise ratio ≥65dB), supports sound source localization and directional noise reduction, and captures environmental soundprints (such as keyboard tapping and alarm sounds).

[0207] Synchronous collection of user behavior data: Facial micro-expressions: The OpenFace algorithm is used to extract the coordinates of 68 facial feature points and calculate the action unit (AU) intensity (such as AU12 raising the corner of the mouth and AU4 frowning).

[0208] Speech prosody: Extracts 23-dimensional acoustic features such as fundamental frequency (F0), speech rate, and short-term energy. The sampling rate is 44.1kHz and the frame length is 512ms.

[0209] Head, Neck, and Shoulder Vibration: Using a six-axis IMU sensor (accelerometer + gyroscope, sampling rate 100Hz), the physiological tremor signal is extracted through bandpass filtering (2-20Hz) to quantify the tension or relaxation state.

[0210] Environmental semantic parsing module, specifically including:

[0211] Semantic feature extraction: Visual semantic parsing: Mask R-CNN performs instance segmentation on the image, outputs structured labels (object category, spatial layout, dynamic events), and uses a dynamic object priority mechanism (based on intersection-over-union Calculating weights ) to enhance the semantic weight of dynamic elements (such as moving people).

[0212] Voiceprint classification: YAMNet classifies voiceprint signals into predefined sound categories (such as "natural sound") and expands the output emotional polarity (positive / negative / neutral). For example, "birdsong" is mapped to positive emotion.

[0213] Semantic-emotional mapping: Mapping environmental labels to emotional dimensions (stress values) through predefined maps (e.g., “green plant relaxation +15”) , relaxation , social intensity ), combined with the online knowledge graph update algorithm (such as automatically adjusting the weight when user feedback "green plants do not relieve stress") to achieve dynamic optimization.

[0214] The causal fusion module includes:

[0215] Construction of environmental-psychological causal diagram: Node definition: environmental emotional dimension, behavioral characteristics and psychological state.

[0216] Causal Strength Calculation: Calculating Edge Weights Based on Do Calculus , clarify the direct causal path of "noise → physiological arousal → anxiety" and avoid spurious correlation interference.

[0217] Causal attention mechanism: Input the environment feature query vector Q and the behavior feature key-value pair K / V, through the formula Adjust the interaction weights, where is the causal weight matrix, which prioritizes the integration of high causal strength features (such as "high-frequency noise" and "voice tremor").

[0218] Mental status assessment module, including:

[0219] Dynamic Causal Transformer Model: Encoding layer: The fused feature vector (environmental emotion + behavioral features) is input into the Transformer encoder to generate a context-aware hidden representation H.

[0220] Dynamic weight update unit: through gating mechanism Adjust the causal weight matrix W_c' in real time. For example, when a sudden braking sound is detected in a driving scenario, increase the weight of "stress value → anxiety".

[0221] Loss function design: using causal mean square error , constraining the prediction error and causal weight fitting error to ensure that the model conforms to the true causal logic.

[0222] Feedback and optimization module, specifically including:

[0223] Interpretation report generation: Combines environmental semantics (e.g., "A closed space + 3 people talking" is detected") to generate readable reports, such as "The current stress value is 68 points, mainly due to the closed environment (+8) and social density (+15)."

[0224] Personalized calibration: For new users, use the results of the previous t evaluations , fine-tuning the causal weights through the formula, for example, the “social density → anxiety” weight of introverted users is increased by 20%.

[0225] The interaction process between the above modules includes:

[0226] Data collection: Multimodal sensors synchronously capture environmental and user behavior data;

[0227] Semantic parsing: extracting structured context labels and mapping them to sentiment dimensions;

[0228] Causal fusion: constructing a causal graph and fusing features through an attention mechanism;

[0229] State evaluation: The dynamic Transformer model outputs the psychological state and adjusts the weights in real time;

[0230] Feedback optimization: Generate explanatory reports, combine historical data to calibrate the model, and improve personalization accuracy.

[0231] The system uses the "environment-behavior-psychology" trinity modeling to break through the traditional method's neglect of environmental factors. For example, in driving scenarios, the system accurately identifies the stress association of "congested road conditions + frequent frowning"; and the dynamic weight update and personalized calibration mechanism (such as formula ) significantly improves the robustness and practicality of the model in diverse scenarios (home, office, cockpit).

[0232] Corresponding to the above embodiment, the present invention further proposes a readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned psychological state assessment method based on environmental perception is implemented.

[0233] The readable storage medium of the present invention can be any form of storage medium that can be read by the processor of a computer device, including but not limited to non-volatile memory, volatile memory, ferroelectric memory, etc. The readable storage medium stores a computer program. When the processor of the computer device reads and executes the computer program stored in the memory, the steps of the above-mentioned environmental perception-based psychological state assessment method can be implemented.

[0234] The computer program includes computer program code, which may be in source code form, object code form, an executable file, or some intermediate form. The computer-readable medium may include any entity or device capable of carrying the computer program code, recording media, a USB flash drive, a removable hard drive, a magnetic disk, an optical disk, computer memory, read-only memory, random access memory, an electrical carrier signal, a telecommunications signal, and software distribution media. It should be noted that the content of the computer-readable medium may be appropriately increased or decreased based on legislative and practical requirements. For example, according to legislative and practical requirements, computer-readable media does not include electrical carrier signals or telecommunications signals.

[0235] It should be noted that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "readable storage medium" can be any device that can contain, store, transmit, disseminate, or transfer a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection having one or more wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing it in a suitable manner if necessary, and then stored in a computer memory.

[0236] It should be understood that various components of the present invention may be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof may be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0237] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0238] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0239] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

[0240] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A psychological state assessment method based on environmental perception, characterized in that: The following steps are involved: S1. Use RGB cameras to capture images of environmental scenes and the behavior of the object being evaluated. Use semantic segmentation models to extract object categories, spatial layouts, and dynamic events, and introduce a dynamic object priority mechanism. Use microphones to collect environmental soundprints, and use sound recognition models to identify sound categories. Synchronously collect the subject's facial micro-expression feature points, speech prosody features, and head, neck, and shoulder vibration frequency; S2. Object / scene labels are generated into semantic vectors through a semantic embedding model. Spatial layout and sound labels are converted into sparse vectors. These vectors are then mapped to the preset environmental emotion dimensions through a multi-layer perceptron to obtain stress values, relaxation levels, and social intensity. S3. Construct an environment-psychology causal graph, define the causal relationship nodes and edge weights between the environmental emotional dimension, behavioral characteristics, and psychological state; adjust the attention of environmental characteristics to behavioral characteristics through the causal attention module, and fuse them to generate a multimodal feature vector; The causal diagram modeling step includes: Define the node set as the environmental emotion dimension , behavioral characteristics and psychological state ; Calculating causal effects with Do-Calculus As edge weights, = , representing a node For Node the direct causal strength; , ; Indicates intervention hour, and The covariance of , characterizes the strength of causal dependence; express The variance of , used to normalize the causal effect; S4. Inputting the fused feature vector into a causal Transformer model including a dynamic causal weight update unit to output the psychological state dimension assessment result. The dynamic causal weight update unit realizes adaptive adjustment of causal weights through a gating mechanism. S5. Generate a psychological state explanation report based on environmental semantics and dynamically calibrate the model causal weights based on the historical data of the evaluated object, including personalized calibration.

2. The evaluation method according to claim 1, wherein: In step S1, the semantic segmentation model is MaskR-CNN, which is used to perform real-time semantic segmentation on the images collected by the RGB camera and output the set of object categories in the environment. , spatial layout label and dynamic event collection ; The sound recognition model is YAMNet, which is used to classify environmental soundprints into a set of predefined sound categories. .

3. The evaluation method according to claim 1, wherein: In step S2, the environmental emotional dimension includes the pressure value , relaxation and social intensity , a quantitative mapping of object, space, and sound labels to emotional dimensions is achieved through a predefined semantic-emotion mapping graph, and the mapping graph is iteratively optimized through an online knowledge graph update algorithm.

4. The evaluation method according to claim 1, wherein: In step S3, the input of the causal attention module is the environmental feature query vector , behavioral feature key vector Sum value vector , the calculation formula is: in, From the environmental emotional dimension The linear projection of and From behavioral characteristics Linear projection of is the causal weight matrix, element Representing environmental characteristics Behavioral characteristics the causal strength of is the key vector dimension, Represents element-wise multiplication.

5. The evaluation method according to claim 3, wherein: In step S4, the adaptive adjustment of causal weights achieved through the gating mechanism is expressed as follows: in, The fusion feature vector is obtained by concatenating the environmental emotion dimension and the behavioral feature and then encoding it through the Transformer; is the real-time environmental emotion vector; is the learnable gating weight matrix, is the Sigmoid activation function; is the basic causal weight matrix, which is obtained through global data training.

6. The evaluation method according to claim 1, wherein: In step S4, during the process of outputting the psychological state dimension evaluation results, the loss function of the psychological state evaluation is considered to be the causal mean square error: in, is the total number of samples, For the The true value vector of the psychological state of samples, is the model prediction value; is the set of directed edges in the causal graph, is the true causal strength, Predicting causal strength for the model; is a regularization parameter used to balance the psychological state prediction error and the causal weight fitting error.

7. The evaluation method according to claim 1, wherein: In step S5, the personalized calibration is achieved by: For new users, before using Assessment results Fine-tune the causal weight matrix, where is a natural number and ; in: is a learnable personalized bias matrix with the same dimension as Consistent; MLP is a multi-layer perceptron used to map historical evaluation results into weight adjustment vectors; calibrated causal weights The evaluation process is only applied to the current user.

8. A psychological state assessment system based on environmental perception, characterized in that: The method for evaluating a mental state based on environmental perception according to any one of claims 1 to 7 comprises: The multimodal data acquisition module is configured to collect environmental scenes and user behavior images through an RGB camera, collect environmental soundprints through a microphone, and simultaneously collect user facial micro-expression feature points, speech rhythm features, and head, neck, and shoulder vibration frequencies; An environmental semantic parsing module is configured to extract object categories, spatial layouts, and dynamic events using a semantic segmentation model, identify sound categories using a sound recognition model, and generate environmental emotional dimension values ​​through a semantic-emotional mapping graph; The causal fusion module is configured to build an environment-psychology causal graph and fuse the environmental features with the behavioral features through the causal attention module to generate a multimodal feature vector; A mental state assessment module configured to output mental state dimension assessment results using a causal Transformer model including a dynamic causal weight update unit; A feedback and optimization module is configured to generate a psychological state explanation report and dynamically calibrate the model causal weights based on user historical data.

9. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for evaluating a mental state based on environmental perception according to any one of claims 1 to 7 is implemented.

Images