A psychological vulnerability recognition method and system based on multi-instance learning

Abstract

A psychological vulnerability recognition method and system based on multi-instance learning, in the method, first, long-time video containing multiple individuals is processed, face images are detected and cropped from video frames, and individual-level data sets are constructed by merging according to identities; then, emotion polarity features and facial action unit features are extracted frame by frame, and are aggregated according to time segments to construct daily statistical features; then, all daily features of each individual are organized as a 'bag-instance' structure, and a bidirectional long short-term memory network is used to encode the instance sequence in time sequence; finally, the mean and variance of the encoded hidden state are calculated and spliced, and the psychological vulnerability recognition result is output by a classifier. The present application realizes non-invasive monitoring in a natural scene under weak supervision, can analyze multiple individuals at a time, reduces the detection cost, and improves the discrimination ability by using the time sequence modeling and mean-variance joint aggregation mechanism.

Description

Technical Field

[0001] This invention relates to computer vision and mental health analysis technology, and in particular to a method and system for identifying psychological vulnerability based on multi-instance learning. Background Technology

[0002] Currently, the most commonly used methods for screening mental health issues include traditional methods such as questionnaires, interviews, or professional psychological counseling. However, these methods have certain limitations in large-scale screening on campuses. First, the results of questionnaires and interviews are somewhat subjective. Investigators or respondents may be unwilling to reveal their true feelings due to trust issues and privacy concerns, such as deliberately concealing their true psychological state out of concern for evaluation or worries about relationships with teachers or classmates. Second, due to limitations in the number of professional psychological teachers on campus, the high cost of one-on-one interviews and assessments, and insufficient mental health resources on campus, most schools use periodic questionnaires for mental health screening. However, repeated administration can easily lead to the practice effect, resulting in perfunctory answers or adjustments to answers, reducing the validity of the questionnaire. Moreover, these traditional assessment methods are more inclined to post-event identification and are difficult to monitor in real time. For example, schools usually conduct phased screenings at the beginning or middle of the semester, which cannot reflect the dynamic changes in students' mental state in a timely manner. This lag leads to intervention only when problems are significant, reducing the effectiveness of intervention. Therefore, how to develop a low-interference, sustainable mental health monitoring system that can be deployed in a campus setting is an urgent research problem that needs to be solved.

[0003] However, most existing research uses data collected in laboratory environments under strictly controlled conditions. This results in discrepancies between participants' behavior and their natural performance in real-life situations, lacking natural data from real-life scenarios and long-term longitudinal data across time dimensions. Current visual information-based mental health analyses primarily rely on static image analysis, paying insufficient attention to temporal changes in emotions. Most studies extract facial features from single-frame images, failing to reflect emotional changes over longer time scales.

[0004] The main drawbacks of current computer-based mental health screening methods are: 1. Scenario limitations: Most data are collected in laboratory environments and acquired under strictly controlled conditions. Therefore, existing methods must go through rigorous laboratory procedures to evaluate subjects.

[0005] 2. High cost per test: Testing requires repeated, lengthy experiments, resulting in high costs.

[0006] 3. Insufficient attention to time sequence: Existing psychoanalytic methods only use data collected in real time as the basis for judgment, and lack attention to long-term changes.

[0007] It should be noted that the information disclosed in the background section above is only for understanding the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0008] The main objective of this invention is to overcome the deficiencies in the aforementioned background technology and provide a psychological vulnerability identification method and system based on multi-instance learning.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: A method for identifying psychological vulnerability based on multi-instance learning includes the following steps: S1. Constructing an individual-level dataset: Process long-duration videos containing multiple individuals, detect and crop individual facial images from video frames, merge samples according to individual identity, and construct an individual-level dataset based on individuals; S2. Construct daily statistical features: For the facial images in the individual-level dataset, extract the emotion polarity features and facial action unit features frame by frame, and perform statistical aggregation according to time segments to construct daily statistical features corresponding to each time segment. The daily statistical features include emotion statistical features and facial action unit statistical features. S3. Temporal Coding and Multi-Instance Learning: The daily statistical features corresponding to all time segments of each individual are organized into a package, and the features of each time segment are treated as an instance. A bidirectional long short-term memory network is used to perform temporal modeling on the instance sequence, and the hidden state of each instance is extracted as its temporal coding feature representation. S4. Aggregation and Classification: Statistically aggregate the hidden states of all instances after temporal encoding, calculate their mean and variance, and concatenate them to form an individual-level feature vector. Then, input the individual-level feature vector into the classifier and output the psychological vulnerability identification result of the individual.

[0010] A psychological vulnerability identification system based on multi-instance learning includes: The feature extraction module, connected to the data acquisition and sample construction module, is used to extract emotion polarity features and facial action unit features frame by frame, and to perform statistical aggregation by time segment to construct daily statistical features for each time segment. The temporal coding module, connected to the feature extraction module, is used to organize the daily statistical feature sequence of each individual into a bag-instance structure, and to use a bidirectional long short-term memory network to temporally encode the instance sequence, outputting the hidden state of each instance as a temporal coding feature. The aggregation and classification module, connected to the temporal coding module, is used to calculate the mean and variance of the hidden states of all instances and concatenate them to form an individual-level feature vector. Then, the classifier outputs the psychological vulnerability identification result of the individual.

[0011] The present invention has the following beneficial effects: This invention proposes a method and system for identifying psychological vulnerability based on multi-instance learning. Its core innovation lies in the fact that, under conditions of relying solely on long-duration video data and lacking fine annotation, it constructs a modeling framework based on multi-instance learning and combines emotional statistical features with facial action unit features to achieve effective modeling and identification of individual psychological vulnerability states. Specifically, this invention transforms long-duration, non-continuous video data into a multi-instance structure that can be used for learning. It combines long-duration statistical features with temporal modeling using a bidirectional long short-term memory network (BiLSTM) and captures the overall level and dynamic changes of individual behavior based on mean-variance pooling operations, thereby achieving effective identification of psychological vulnerability states.

[0012] Compared with existing technologies, this invention achieves a significant breakthrough first in terms of application scenarios. Existing methods often rely on controlled environments or short-term data collection, while this invention uses long-term videos taken in natural settings such as classrooms for analysis, achieving non-intrusive monitoring. Since no active participation or additional cooperation from subjects is required, data can be continuously acquired without interfering with individual behavior, thus significantly improving the feasibility and scalability of the method in real-world school environments. Secondly, regarding detection costs, traditional psychological screening methods typically require individual questionnaires or interviews, which are costly and difficult to implement on a large scale. This invention achieves unified processing and analysis of multiple individuals in the same video, enabling single-collection, multi-target identification. It can analyze the mental health of all individuals appearing in the video, greatly reducing the detection cost per individual and providing an efficient and low-cost solution for mental health screening in group settings such as schools.

[0013] In terms of modeling capabilities, existing techniques based on single-frame images or simple statistical methods struggle to reflect the dynamic changes in psychological states and pay insufficient attention to temporal information. This invention introduces BiLSTM to encode behavioral features across time. This network can simultaneously capture forward and backward dependencies in time series, thus more accurately reflecting the emotional and behavioral trends of individuals over long periods and predicting overall psychological vulnerability. Furthermore, by combining a multi-instance learning framework with statistical feature modeling methods, this invention achieves effective modeling of discontinuous, multi-temporal data under weak supervision with only individual-level labels. Building upon this, the invention also introduces a joint aggregation mechanism of mean and variance. The mean characterizes the overall psychological risk level of individual behavior, while the variance reflects the volatility and instability of behavioral patterns. This mechanism enables the model to not only reflect the average level of individual behavior but also capture its fluctuation characteristics, significantly enhancing its ability to discriminate psychological vulnerability.

[0014] In summary, this invention provides a systematic solution to key problems in existing technologies, such as limited application scenarios, high detection costs, and insufficient utilization of temporal information. Existing research typically focuses on single problems such as emotion recognition, temporal modeling, or multi-sample fusion, while this invention organically integrates long-term video data acquisition, weakly supervised learning, statistical feature modeling, and bidirectional temporal networks, and is designed holistically to meet the needs of real-world scenarios. This organic integration is a significant innovation that has not been widely proposed before, enabling this invention to achieve non-intrusive, low-cost, and highly timely identification of psychological vulnerability in natural environments, providing reliable technical support for large-scale mental health screening and dynamic monitoring.

[0015] Other beneficial effects of the embodiments of the present invention will be further described below. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the overall psychological vulnerability identification method based on multi-instance learning according to the present invention.

[0017] Figure 2 This is a diagram of the overall architecture of BiLSTM according to an embodiment of the present invention.

[0018] Figure 3 This is a flowchart of the psychological vulnerability management visualization system according to an embodiment of the present invention. Detailed Implementation

[0019] The embodiments of the present invention will be described in detail below. It should be emphasized that the following description is merely exemplary and not intended to limit the scope and application of the present invention.

[0020] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0021] This invention aims to address the limitations of existing psychological vulnerability identification methods in real-world scenarios, their high detection costs, and their lack of ability to model long-term behavioral dynamics. It proposes a psychological vulnerability identification method and system based on multi-instance learning. This method organizes long-term video data from natural classroom scenarios into a "package-instance" structure, jointly extracts daily-level emotional statistical features and facial action unit features, captures temporal dependencies through a bidirectional long short-term memory network, and utilizes a mean-variance aggregation mechanism to simultaneously characterize the overall level and fluctuation characteristics of individual psychological risk. Thus, it achieves non-intrusive and low-cost monitoring of group psychological states with only weak individual-level labels, significantly improving the ability to identify psychological vulnerability.

[0022] See Figures 1 to 3 This invention provides a method for identifying psychological vulnerability based on multi-instance learning, comprising the following steps: Step S1. Construct an individual-level dataset: Process long-duration videos containing multiple individuals, detect and crop individual facial images from video frames, merge samples according to individual identity, and construct an individual-level dataset based on individuals.

[0023] In some embodiments, step S1, detecting and cropping individual facial images from video frames, specifically includes: using a face detection and alignment method based on a multi-task convolutional neural network to locate and align key points in the facial regions of the video frames to obtain standardized facial images; the merging of samples according to individual identity specifically includes: extracting facial features from the standardized facial images, performing similarity matching with a pre-built facial feature library, merging facial image samples belonging to the same individual identity, and removing individuals with fewer than a preset threshold of valid facial images. This preset threshold can be determined based on pre-experiments to ensure that each individual has sufficient data points (days) for reliable time-series modeling. In practical applications, this threshold can be adjusted according to the overall distribution of the dataset.

[0024] Step S2. Construct daily statistical features: For the facial images in the individual-level dataset, extract the emotion polarity features and facial action unit features frame by frame, and perform statistical aggregation according to time segments to construct daily statistical features corresponding to each time segment. The daily statistical features include emotion statistical features and facial action unit statistical features.

[0025] In some embodiments, step S2, the step of extracting emotion polarity features frame by frame, specifically includes: using an emotion recognition model based on residual networks, taking a single frame of facial image as input, and outputting the emotion polarity category of the frame, wherein the emotion polarity category includes at least negative and non-negative; the step of statistical aggregation by time segment to construct emotion statistical features specifically includes: taking a natural day as a time segment, calculating the total number of facial frames in the time segment, the number of facial frames judged as negative emotions, the proportion of negative emotion frames, and the longest duration of negative emotions calculated from the timestamp difference within the same day.

[0026] In some embodiments, step S2, extracting facial action unit features, specifically includes: using a facial behavior analysis tool to extract continuous activation intensity values ​​and binary detection identifier values ​​of multiple facial action units in each frame image; and performing statistical aggregation by time segment to construct facial action unit statistical features, specifically includes: taking a natural day as a time segment, calculating the average value of the activation intensity values ​​of each facial action unit in all valid face frames within that time segment, and constructing a daily-level facial action unit feature vector.

[0027] Step S3. Temporal Coding and Multi-Instance Learning: Organize the daily statistical features corresponding to all time segments of each individual into a package, and treat the features of each time segment as an instance. Use a bidirectional long short-term memory network to perform temporal modeling on the instance sequence, and extract the hidden state of each instance as its temporal coding feature representation.

[0028] In some embodiments, step S3, which involves using a bidirectional long short-term memory network to perform time-series modeling of the instance sequence, specifically includes: taking the daily statistical feature vector of each individual arranged in date order as the input sequence, encoding the sequence through a forward long short-term memory network and a backward long short-term memory network respectively, and concatenating the forward hidden state and the backward hidden state corresponding to each time step as the time-series encoded feature representation, so as to simultaneously capture the forward and backward dependencies of the time series.

[0029] Step S4. Aggregation and Classification: Statistically aggregate the hidden states of all instances after temporal encoding, calculate their mean and variance, and concatenate them to form an individual-level feature vector. Then, input the individual-level feature vector into the classifier and output the psychological vulnerability identification result of the individual.

[0030] In some embodiments, step S4, which involves calculating and concatenating the mean and variance, specifically includes: calculating the average value of the hidden states at all effective time steps of an individual in each dimension to form a mean vector; calculating the average value of the squared differences between the hidden state values ​​and the corresponding means in each dimension to form a variance vector; concatenating the mean vector and the variance vector end-to-end to form an individual-level feature vector; performing a dropout operation on the individual-level feature vector to reduce the risk of overfitting; and then inputting the dropout-processed feature vector into a multilayer perceptron classifier for linear mapping to output a binary classification result for the psychologically vulnerable group or the protected group.

[0031] In some embodiments, after step S4, a psychological vulnerability management visualization step is also included: the algorithm is deployed as a psychological risk visualization system, taking the class as the analysis unit, based on the facial image sequence of all students in the classroom video, the emotional distribution characteristics and their time change trends are displayed in a visualization manner, the psychological vulnerability identification results and individual suggestions of each individual are output, and the overall psychological risk overview of the class is generated.

[0032] In some embodiments, during the model training phase, the method uses binary cross-entropy as the loss function and utilizes the Adam optimizer to iteratively optimize the network parameters, wherein the network parameters include at least the weight parameters of the bidirectional long short-term memory network and the weight parameters of the classifier.

[0033] In some embodiments, the bidirectional long short-term memory network captures the trend of an individual's emotional and behavioral changes over a long period of time by simultaneously utilizing the forward and backward dependencies of time series; the joint aggregation mechanism of mean and variance uses the mean to characterize the overall psychological risk level of an individual's behavior and the variance to characterize the volatility and instability of an individual's behavioral patterns; the longest duration of negative emotions in the daily statistical features is used to reflect the persistence characteristics of emotions in the time dimension.

[0034] See Figure 2 and Figure 3This invention also provides a psychological vulnerability identification system based on multi-instance learning, including a feature extraction module, a temporal coding module, and an aggregation and classification module. The feature extraction module, connected to the data acquisition and sample construction module, is used to extract emotional polarity features and facial action unit features frame by frame, and statistically aggregate them according to time segments to construct daily statistical features for each time segment. The temporal coding module, connected to the feature extraction module, is used to organize the daily statistical feature sequence of each individual into a package-instance structure, and uses a bidirectional long short-term memory network to temporally encode the instance sequence, outputting the hidden state of each instance as a temporal coding feature. The aggregation and classification module, connected to the temporal coding module, is used to calculate the mean and variance of the hidden states of all instances and concatenate them to form an individual-level feature vector, and then output the individual's psychological vulnerability identification result through a classifier.

[0035] In some embodiments, the system further includes a data acquisition and sample construction module, used to process long-term videos containing multiple individuals, detect and crop individual facial images, merge samples according to individual identity, and construct an individual-level dataset; In some embodiments, the system further includes a psychological vulnerability management visualization module connected to the aggregation and classification module, used to display an overall overview of psychological vulnerability, individual suggestions, and class risk profiles based on the identification results, with groups as the analysis unit.

[0036] The psychological vulnerability identification method and system proposed in this invention based on multi-instance learning has the following main technical advantages: It achieves non-intrusive and low-cost monitoring of group psychological states by collecting long-term video footage from natural classroom scenarios, enabling unified analysis of multiple individuals simultaneously. Furthermore, it organizes discontinuous daily statistical features (emotional statistical features and facial action unit features) into a "package-instance" structure, employs a bidirectional long short-term memory network to model forward and backward dependencies of temporal behavior, and combines a mean-variance joint aggregation mechanism to simultaneously characterize the overall level and fluctuation characteristics of individual psychological risk. This method effectively solves the problems of limited application scenarios, high detection costs, and insufficient utilization of temporal information in existing technologies, even with only weak individual-level labels, significantly improving the ability to identify psychological vulnerability states.

[0037] The following further describes the implementation methods and experimental verification of specific embodiments of the present invention.

[0038] A method for identifying psychological vulnerability based on multi-instance learning includes the following steps: 1. Data Input and Preprocessing First, long-term video data containing multiple individuals is acquired. This video is then parsed frame by frame or sampled at set time intervals to obtain an image frame sequence. For each frame, a face detection and alignment method based on a multi-task convolutional neural network is used to locate the face regions in the image and perform keypoint alignment processing on the detected faces to eliminate the effects of pose, scale, and illumination variations, thereby obtaining a standardized facial image.

[0039] 2. Extraction of emotional statistical features and facial action unit features

[0040] The cropped facial frames are used to determine emotions and extract facial action unit features to construct daily-level features. Specifically, an emotion polarity determination model based on residual networks (such as ResNet-18) is used to classify facial images as negative or non-negative. The training data used for the emotion recognition model mainly comes from publicly available datasets. The datasets are collected in real, uncontrolled scenes, covering facial images with complex lighting, partial occlusion, multiple poses, and diverse shooting conditions. Their data features show high consistency with facial images captured by high-position cameras in classroom environments, effectively supporting the model's generalization ability in real classroom scenarios. This dataset contains 28,709 training images, covering seven basic emotion categories: anger, disgust, fear, sadness, neutrality, happiness, and surprise. Classified by emotion polarity, there are 13,357 negative emotion images (anger, disgust, fear, sadness) and 15,352 positive and neutral emotion images (happiness, neutrality, surprise), with a relatively balanced sample distribution. Studies have shown that the frequent occurrence of negative emotions and their processing biases are significantly correlated with mental disorders such as depression and anxiety (core dimensions of the Psychological Vulnerability Scale), and can serve as important behavioral indicators. Therefore, modeling the distribution and change patterns of emotions can provide a reliable basis for identifying mental states. Based on this, statistical features such as the proportion of negative emotions and the longest duration of negative emotions are constructed according to the natural day; simultaneously, the Openface tool is used to extract facial motion unit features from each frame, and the average value is calculated according to the date to construct daily features.

[0041] 3. Timing coding

[0042] Daily features constructed according to date are treated as instances in multi-instance learning, with each individual representing a bag. Temporal modeling is performed on instance-level sentiment statistics and facial action unit features. A bidirectional long short-term memory (BiLSTM) network is employed, modeling each daily feature as an instance based on date order. The output daily hidden layer states serve as the temporally encoded feature representation. The hidden layer dimension of the BiLSTM can be selected from preset candidate values ​​(e.g., 32, 64, 128, 256) through cross-validation to choose the dimension that performs best on the validation set.

[0043] 4. Feature aggregation and classification

[0044] Mean-variance aggregation is performed on the hidden states of each instance output by BiLSTM. First, the mean and variance of the hidden states at all effective time steps are calculated and concatenated. Then, based on the aggregated feature vector, the result is output by a classifier to obtain the psychological vulnerability identification result of the individual.

[0045] A psychological vulnerability identification system based on multi-instance learning includes the following modules: The feature extraction module constructs a negative emotion recognition model based on ResNet-18, performs frame-by-frame emotion polarity determination on the captured face, and uses the Openface tool to extract facial action unit features for each frame. Finally, it constructs emotion statistical features and facial action unit statistical features according to the date dimension. The temporal coding module treats the features constructed by date as instances in multi-instance learning, with individuals as bags, and performs temporal modeling on instance-level emotion statistics features and facial action unit features. It mainly uses BiLSTM to output the hidden state of each instance as the feature representation of temporal coding. The aggregation and classification module performs statistical aggregation on the instance-level features after time-series encoding, calculates their mean and variance and concatenates them, and then classifies them based on the aggregated feature vectors to output the psychological vulnerability identification results of individuals.

[0046] Building upon this foundation, the system can further include a data acquisition and sample construction module, as well as a psychological vulnerability management visualization system. The data acquisition and sample construction module processes long-duration videos containing multiple individuals, employing a face detection and alignment method based on a multi-task convolutional neural network to crop individual facial images. These images are then matched against a pre-built facial feature library for similarity, merging the samples to construct an individual-level dataset. The psychological vulnerability management visualization system deploys the algorithm as a psychological risk visualization system. By reading pre-processed facial frames from a designated interface, it displays the psychological vulnerability status and overall overview of all individuals appearing in the entire video.

[0047] In this invention, structured modeling of individual behavioral data from long-duration videos, combined with statistical feature extraction and temporal learning methods, enables effective identification of psychological vulnerability under weak supervision. Its working mechanism can be explained from the following aspects: (1) Data accumulation effect based on long-term video: This invention first uses long-term video data as input, and continuously samples the behavior of individuals over a long period of time, enabling the model to acquire more comprehensive behavioral information. Compared with single-frame or short-term data, long-term data can cover the emotional changes of individuals in different situations, thereby improving the stability and representativeness of features. Therefore, by accumulating information in the time dimension, the impact of instantaneous noise on the recognition results can be effectively reduced, providing a reliable foundation for subsequent modeling.

[0048] (2) Behavioral patterns based on statistical features have psychological significance: In the feature construction stage, this invention extracts statistical quantities, including the proportion of emotions and the duration of emotions, by performing time statistics on emotional features and facial action unit features. These statistical features can reflect the behavioral distribution patterns of individuals over a long period of time. For example, the proportion of negative emotions can reflect the overall emotional tendency, while the frequency of emotion changes can reflect emotional stability. Therefore, through statistical modeling, the original time-series data can be transformed into feature representations with clear psychological significance.

[0049] (3) Dynamic feature extraction based on time-series modeling: Based on the multi-instance structure, this invention further employs a time-series modeling method to encode the instance feature sequence, and simultaneously captures the forward and backward dependencies of the time series through a bidirectional modeling mechanism. This process can effectively extract the changing trend of individual behavior in the time dimension, enabling the model to not only identify static features, but also perceive the evolution process of behavior, thereby more accurately reflecting the dynamic changes in psychological state.

[0050] (4) Feature aggregation based on mean-variance joint modeling: In the instance-level feature fusion process, this invention calculates the mean and variance of all instance features and concatenates them to form an individual-level representation. The mean is used to characterize the overall level of individual behavior, while the variance reflects the degree of behavioral fluctuation. Since psychological vulnerability is reflected not only in average emotional level but also in emotional volatility, introducing variance information can enhance the model's sensitivity to abnormal behavioral patterns, thereby improving the recognition effect.

[0051] In terms of technical conception, this invention, relying solely on long-duration video data and lacking fine annotation, constructs a modeling framework based on multi-instance learning and combines emotional statistical features with facial action unit features to achieve effective modeling and identification of individual psychological vulnerability. Specifically, it transforms long-duration, non-continuous video data into a multi-instance structure that can be used for learning, combines long-duration statistical features with temporal modeling using bidirectional long short-term memory networks, and captures the overall level and dynamic changes of individual behavior based on mean-variance pooling operations, thereby completing the identification of psychological vulnerability.

[0052] The technical solutions of this invention mainly include the following aspects: First, an individual behavior modeling method based on long-term classroom videos, which analyzes video data collected in natural teaching scenarios to achieve non-intrusive monitoring of individual psychological states; Second, a joint feature construction method based on emotional statistical features and facial action unit features, which statistically models the distribution, frequency, and persistence of emotions over time to obtain stable feature representations that reflect individual behavioral patterns; Third, a psychological state modeling method based on multi-instance learning, which organizes individual data into a "package-instance" structure to adapt to weakly supervised learning scenarios with only individual labels, performs temporal modeling based on bidirectional long short-term memory networks, and uses an instance-level feature aggregation method based on mean and variance to jointly characterize the overall level and fluctuation characteristics of individual behavior, thereby improving the ability to identify psychologically vulnerable states.

[0053] In summary, the technical solution of this invention has the following important features: it models individual behavior based on long-term video data, rather than relying solely on single-frame or short-term data; it performs individual-level merging of cross-time samples and ensures that the number of samples reaches a certain scale, thereby forming a dataset that can reflect the overall behavior of individuals; it constructs a multi-instance data structure based on time segments to adapt to weakly supervised learning scenarios; and it comprehensively represents individual behavior by combining statistical features with temporal modeling, and thereby completes psychological state prediction.

[0054] The following section will further explain the implementation details of each step in the above solution using specific examples of classroom video acquisition and processing.

[0055] I. Data Input and Preprocessing

[0056] Classroom videos were captured by a fixed camera installed directly in front of the classroom. The camera had a resolution of at least 8 megapixels, capable of covering the student area and meeting basic facial clarity requirements. Data collection was conducted over two consecutive weeks of regular classroom teaching time, strictly limiting the content to typical teaching scenarios, including teacher lectures and classroom interactions, to avoid interference from self-study periods, exams, or other non-teaching situations that might affect students' emotional and behavioral expressions. No additional tasks or stimuli were introduced during the collection process to ensure the naturalness and authenticity of student behavior. Written informed consent was obtained from the school, students, and parents during the data collection process, clearly informing them of the data's purpose, storage period, and privacy protection measures. All video data was immediately anonymized after collection, removing directly identifiable information, and stored using encryption. Data analysis was conducted on a secure internal server; the raw data was not disclosed, and the experiment had passed ethical review.

[0057] The preprocessing workflow mainly includes video frame extraction, face detection and recognition, and sample merging, aiming to construct a set of student facial samples from the original classroom video. First, the classroom video undergoes uniform frame extraction. The average extraction frequency is approximately 30 frames per minute. After frame extraction, face detection is performed on each frame to locate multiple student facial regions in the classroom scene. Given the small face scale, significant pose variations, and partial occlusion in the classroom environment, only samples whose clarity, integrity, and pose angle meet the requirements are retained. Then, individual identification is performed on the detected face samples. Specifically, a face detection and alignment method based on Multi-Task Convolutional Neural Network (MTCNN) is used to automatically locate and crop student facial regions from the classroom video frames; subsequently, facial features are extracted and matched with a pre-collected and constructed student facial feature database to determine the individual student corresponding to each face sample. The similarity threshold can be set according to the specific application scenario or determined through pre-experimentation to achieve a balance between false and false matches; this invention does not limit the specific threshold value. Based on the above face recognition and matching results, this invention merges facial samples of the same student appearing in different time frames to form a facial sample set for each student, and deletes individual data with too few samples. For each student's sample, it is further categorized according to the collection date.

[0058] II. Feature Construction

[0059] This invention employs a ResNet-18-based emotion recognition model to perform frame-by-frame emotion assessment on student facial images captured from two weeks of classroom videos. The emotion model takes a single frame as input and outputs the corresponding emotion polarity (negative or non-negative). Due to the discontinuous nature of video frame sampling and the varying number of facial frames captured by different students within a finite collection period, this invention constructs statistical features for individual emotional expression based on the frequency distribution, temporal coverage, and duration of negative emotions. Specifically, this invention constructs global or daily emotion statistical features as listed in Table 1. In addition to the global statistical features at the individual student level, this invention further extracts features according to the natural day to construct sentence-level emotion statistical features, including the total number of facial frames collected on that day, the number of facial frames judged as having negative emotions on that day, the percentage of negative emotion frames on that day, and the longest duration of negative emotions on that day.

[0060] Table 1. Symbols and definitions for statistical characteristics of emotion

[0061] This invention utilizes the facial behavior analysis tool OpenFace to extract features from facial images captured in classroom videos. OpenFace can automatically extract various low-level feature descriptors related to psychological states from single frames or video sequences. The Facial Action Coding System (FACS) provides a quantitative standard for facial expression analysis, where the continuous variable AU_r (0–5) represents the activation intensity of facial action units, and the binary variable AU_c (0 / 1) indicates whether the action unit was successfully detected. This invention first aggregates the original frame-level facial features of each student at the individual level. Specifically, by calculating the statistics of each AU at the individual level for all valid facial frames of each student throughout the entire acquisition period, a global descriptive vector is constructed. Similar to the emotion statistical feature extraction process, this invention further divides the data by natural day, extracting daily-level facial AU features.

[0062] III. Overall Model

[0063] A multi-instance learning method based on bidirectional long short-term memory networks (BiLSTM-MIL) is proposed. This method uses individual students as "bags" in multi-instance learning and the feature set of an individual per day as "instances." The overall framework consists of a feature construction module, a temporal coding module, and an individual classification module. In the feature construction module, frame-level features are first extracted from facial videos. Then, the frame-level features are aggregated by time dimension (one day) to obtain daily-level facial features. Simultaneously, the frame-level output results of the emotion recognition model are combined to calculate daily-level emotion statistical features. The two types of features are concatenated to form the daily-level input features corresponding to each time step. These features are then input into a BiLSTM, and the hidden state at each time step is output to the individual classification module. The individual classification module first calculates the mean and variance of the hidden state representations output at each time step, concatenates them, and outputs individual-level probability scores through an MLP, ultimately determining the psychological vulnerability level, with the output result being either a psychologically vulnerable group or a protected group.

[0064] The temporal coding module enables dynamic modeling of students' daily emotional statistical features and facial behavioral features across different dates. Specifically, this invention treats the features constructed on a daily basis as "instances" in multi-instance learning and uses BiLSTM to encode the sequence of instances to obtain contextual dependency information that varies across days.

[0065] Long Short-Term Memory (LSTM) networks are recurrent neural networks capable of modeling long-term dependencies. By introducing input, forget, and output gates, they alleviate the gradient vanishing problem that traditional recurrent neural networks are prone to in long sequence learning. Compared to unidirectional LSTM, BiLSTM can encode sequences simultaneously from both forward and backward directions, making fuller use of contextual information between different time slices, and is a mature existing technology.

[0066] Combining the temporal behavioral characteristics presented in classroom video frames with the statistical distribution patterns of group psychological states, a mean-variance pooling module was designed. Here, the mean characterizes the overall psychological risk level of students throughout the observation period, while the variance reflects the fluctuation and instability of their emotional and facial behavioral expressions. First, the hidden states (h) output for all effective time steps are... i The mean pooling is performed to represent the student's overall psychological risk level during the observation period:

[0067] Then, the variance of the hidden state sequence in each dimension is calculated to characterize the degree of fluctuation in the student's behavioral pattern over multiple days:

[0068] Finally, the mean and variance are concatenated to obtain the instance-level representation of each student:

[0069] The mean-variance pooling design of this invention is inspired by the theory of affective dynamics. Studies have found that individual emotions exhibit significant dynamic characteristics over time, with variability and instability being important indicators of emotional dynamics. Research has also found that individuals with depression and anxiety exhibit significantly higher variability and instability in their daily lives, and this characteristic can be considered a non-specific manifestation of mood disorders. Emotional variability is typically quantified by the standard deviation or variance of an individual's emotions over time, thus statistically characterizing the degree of emotional fluctuation. Based on the above theoretical and empirical evidence, this invention employs a mean-variance pooling strategy, where the mean is used to characterize the overall level (valence) of an individual's emotions, and the variance is used to characterize the fluctuation characteristics (instability) of emotions over time.

[0070] To reduce the risk of overfitting, dropout is first applied to the individual instance-level representation, and then a fully connected layer is used for linear mapping to obtain the final psychological vulnerability classification result.

[0071] IV. Visualization System

[0072] This system is primarily designed for mental health researchers and campus administrators. Users can use it to quickly collect and analyze trends in facial image emotion data of a group in real-world scenarios, predict their psychological vulnerability, and support psychological and behavioral research. Alternatively, a visual interface can provide an intuitive understanding of recent emotional fluctuations among students in a class, offering a reference for subsequent refined screening and intervention.

[0073] The overall process is as follows: the system takes a single class as an analysis unit, and based on the sequence of facial images of all students taken in the classroom, it displays the characteristics of emotion distribution and its time change trend in a visual way. Finally, it performs statistical analysis and predicts the degree of psychological vulnerability, and generates individual suggestions and an overview of the psychological risk of the class as a whole.

[0074] Experimental verification

[0075] In this experimental example, long-term video data collected in a classroom setting was selected as input data. The video resolution was 1920×1080, the frame rate was 25 frames / second, and the video duration covered at least two consecutive weeks (10 teaching days) of classroom time, including 45 individual students. First, the original video was sampled, with frames extracted at a frequency of 30 frames per minute to obtain an image frame sequence. Subsequently, each frame was processed using a face detection and alignment method based on a multi-task convolutional neural network to detect face regions in the image and extract key point information for alignment. The detected face regions were cropped to a uniform size (e.g., 112×112 pixels) and normalized. In the identity merging stage, a pre-trained face feature extraction model was used to extract a 128-dimensional feature vector from each facial image and matched with a pre-built face feature database. Cosine similarity was used as the matching criterion; when the similarity was greater than 0.8, the sample was assigned to the corresponding individual. This method aggregates facial samples of the same student at different time periods to construct an individual-level dataset, with each person having at least 5,000 valid images.

[0076] In the feature extraction stage, emotion features and facial action unit features are extracted for each frame of facial image. The emotion features are output by the emotion recognition model as emotion polarity (negative or non-negative). Each image needs to be labeled with the specific timestamp information from which it was extracted. Facial action unit features are extracted using the Openface model, containing activation intensity values ​​and 0 / 1 values ​​for 17 action units. During statistical feature construction, the data is divided into one-day time windows, with each time window considered an instance. For each instance, the following features are calculated: total number of face frames, total number of face frames with negative emotions, proportion of negative emotions on that day (percentage of negative emotion frames), longest duration of negative emotions on that day (maximum difference in timestamps of negative emotion frames on that day), and average action unit intensity value, thus obtaining an instance-level feature vector of dimension d (e.g., d=33).

[0077] The group psychological vulnerability dataset encompasses data generated by 45 students over two consecutive weeks (10 teaching days), with a total of over 300,000 facial frames extracted. The statistical characteristics of the dataset are shown in the table below:

[0078] All experiments employed Leave-One-Subject-Out Cross-Validation (LOSO-CV) as the evaluation method, with precision, recall, F1 score, and AUC as the primary evaluation metrics. The dropout rate of the dropout layer added to the classification layer was set to 0.2. During training, the number of training epochs was set to 30, the optimizer was Adam, and the learning rate was set to 0.001. To avoid excessively large gradients leading to training instability, gradient clipping was performed with a clipping threshold of 5.0, and an early stopping mechanism was employed to prevent overfitting.

[0079] In the comparative experiments, this invention used XGBoost and LightGBM as instance-level classifiers under the same experimental settings, because both models possess strong nonlinear modeling capabilities and strong adaptability to small to medium-sized samples. The experimental results are shown in the table below:

[0080] Furthermore, ablation experiments were conducted on different aggregation strategies in this invention. The results show that different aggregation strategies exhibit certain differences in model performance. Mean Pooling alone demonstrated superior discriminative ability, but when using the Mean-Variance Pooling strategy, AUC improved by 2.20%, F1 improved by 5.71%, and although recall did not improve, precision improved by 10.71%, indicating a significant reduction in the model's false positive rate. The Max Pooling strategy, focusing only on the most salient instance-level output, easily overlooks important information and performs the worst in multi-instance learning. Compared to Max Pooling, the Max-Variance Pooling strategy shows a slight improvement, further demonstrating that the variance of instance-level outputs provides effective information. In summary, the Mean-Variance Pooling proposed in this invention achieves optimal performance across multiple evaluation metrics, indicating that in group-level psychological vulnerability identification tasks, considering both average and fluctuation features helps to more accurately predict students' psychological vulnerability. The ablation experiment results are shown in the table below:

[0081] In the feature aggregation stage, the hidden states of all instances are statistically calculated, and the mean and variance are calculated separately. These two values ​​are then concatenated into a 128-dimensional individual-level feature vector. Subsequently, this feature is input into a fully connected classifier for prediction, outputting the probability that an individual belongs to the psychologically vulnerable group. The classification threshold is 0.5; if the output probability is greater than or equal to 0.5, the individual is classified as psychologically vulnerable (label 1); otherwise, it is classified as protected (label 0).

[0082] During model training, binary cross-entropy was used as the loss function, the learning rate was set to 0.001, and the Adam optimizer was used for training, with 50 training rounds.

[0083] In a specific example, classroom video data containing multiple students was selected as the experimental dataset. The videos were processed to construct an individual-level sample set. Emotional statistical features and facial action unit features were extracted using the methods described above, and a multi-instance learning model was constructed for training and testing.

[0084] Experimental results show that, compared with the mean aggregation method alone or the traditional single-frame modeling method, the present invention achieves better classification performance in the psychological vulnerability identification task by introducing temporal modeling and mean-variance joint aggregation mechanism, which verifies the effectiveness of the present method in modeling long-term behavioral data under weak supervision.

[0085] Without departing from the core idea of ​​this invention, there are many other variations of this invention: In terms of data acquisition, input data can be not only classroom videos, but also conference videos, surveillance videos, or other long-term video data containing multiple individuals; in terms of feature extraction, in addition to emotional features and facial action unit features, voice features, posture features, or behavioral trajectory features can be introduced to further enrich the individual behavior representation; in terms of temporal modeling, different types of sequence modeling methods can be used to encode instance sequences.

[0086] Furthermore, regarding the time-slice division method, fixed window division, adaptive division, or event-triggered division can be adopted. All of these variations can be replaced while achieving similar technical effects.

[0087] In summary, this invention proposes a method and system for identifying psychological vulnerability based on multi-instance learning. Its core approach lies in: modeling individual behavior based on long-duration classroom videos; analyzing video data collected in natural teaching scenarios to achieve non-intrusive monitoring of individual psychological states; constructing joint features from emotional statistical features and facial action unit features to statistically model the distribution, frequency, and persistence of emotions over time, obtaining stable feature representations reflecting individual behavioral patterns; and, based on this, employing a multi-instance learning framework to organize individual data into a "package-instance" structure to adapt to weakly supervised learning scenarios with only individual labels. A bidirectional long short-term memory network is used to model temporal features, and through instance-level feature aggregation of mean and variance, the overall level and fluctuation characteristics of individual behavior are jointly characterized, thereby effectively improving the ability to identify psychological vulnerability states.

[0088] Compared with the prior art, the present invention has significant beneficial effects: In terms of application scenarios, existing methods mostly rely on controlled environments or short-term data, while this invention models long-term videos in natural scenarios such as classrooms, realizing non-intrusive monitoring in real environments. It does not require active participation or additional cooperation from individuals and can continuously acquire data without interfering with normal behavior, significantly improving the practical applicability of the method.

[0089] In terms of detection costs, traditional methods usually require evaluating or collecting data on each individual, while this invention achieves multi-target recognition with a single collection by uniformly processing and analyzing multiple individuals in the same video, which greatly reduces the detection cost of a single individual.

[0090] In terms of modeling capabilities, existing techniques based on single frames or simple statistical methods are insufficient to reflect the dynamic changes in psychological states. This invention introduces a BiLSTM time series modeling mechanism to capture both forward and backward dependencies of time series, enabling it to more accurately reflect the changing trends of individuals over long periods. Furthermore, by introducing a joint aggregation mechanism of mean and variance, the model can not only characterize the overall level of behavior but also reflect its fluctuation characteristics, thereby significantly improving the ability to identify psychologically vulnerable states.

[0091] Furthermore, this invention, by combining a multi-instance learning framework with statistical feature modeling methods, achieves effective modeling of discontinuous, multi-temporal-series data under weak supervision with only individual-level labels. Existing research typically focuses on single problems such as emotion recognition, temporal modeling, or multi-sample fusion, while this invention systematically integrates long-term video data acquisition, weakly supervised learning, statistical feature modeling, and bidirectional temporal networks, and is designed holistically to meet the needs of real-world scenarios. This is a significant innovation that has not been widely proposed before.

[0092] application

[0093] The psychological vulnerability identification method proposed in this invention, based on long-term video data, multi-instance learning, and statistical feature modeling, has good versatility and scalability. It is not only applicable to student mental health screening in classroom scenarios, but can also be extended to multiple technical fields involving behavior analysis and state recognition.

[0094] In the field of education, this invention can be used for student classroom behavior analysis, learning engagement assessment, and classroom emotional state monitoring. By modeling changes in students' behavior during long-term learning processes, it enables a comprehensive assessment of learning and psychological states, thereby providing data support for personalized teaching and educational management.

[0095] In the field of healthcare, this invention can be applied to the auxiliary screening and long-term monitoring of psychological problems such as depression and anxiety, and is particularly suitable for non-contact, continuous health management scenarios. For example, it can be combined with hospital waiting areas, rehabilitation environments, or home monitoring systems to analyze individual daily behaviors, thereby assisting doctors in making judgments.

[0096] In the fields of smart security and public management, this invention can be used for analyzing crowd behavior and identifying abnormal behavior in public places. For example, in scenarios such as train stations, airports, and shopping malls, by modeling the long-term behavioral characteristics of individuals, individuals with abnormal emotions or potential risks can be identified, thereby improving security management capabilities.

[0097] In the fields of enterprise management and human resources, this invention can be used for employee emotional state analysis, work stress assessment, and team status monitoring. By analyzing long-term behavioral data of employees in the work environment, it provides a reference for organizational management and employee care.

[0098] In the field of intelligent human-computer interaction, this invention can be embedded in smart terminals, virtual assistants, or service robots, enabling the system to dynamically adjust interaction strategies based on the user's long-term behavior and emotional changes, thereby improving the naturalness and personalization of the interaction.

[0099] In the field of traffic and driving safety, this invention can be used for driver fatigue detection, emotional fluctuation recognition, and risk warning. By performing long-term modeling of facial behavior and emotional changes during driving, it can improve driving safety.

[0100] In the field of internet and user behavior analysis, this invention can be used to analyze the state of users when watching videos, participating in online meetings, or engaging in online learning. By combining video or multimodal data, it can evaluate users' emotional state, attention level, and engagement, thereby optimizing content recommendation and service strategies.

[0101] In the field of judicial and interrogation assistance, this invention can be used to analyze changes in an individual's emotions and behaviors during prolonged interactions, providing a reference for assisting in judging their psychological state, but not as the sole basis for decision-making.

[0102] In the field of sports and training, this invention can be used to monitor the emotions and psychological state of athletes during training. By analyzing their behavioral changes during long-term training, it can provide support for training adjustments and psychological interventions.

[0103] Furthermore, this invention can be extended to multiple fields such as smart elderly care, psychological intervention assistance systems, public opinion analysis, and virtual reality behavior monitoring. The technical solution proposed in this invention can be adopted in any application scenario involving modeling individual behavioral changes over time and requiring state recognition under weak supervision or low annotation cost conditions.

[0104] This invention also provides a storage medium for storing a computer program, which, when executed, performs at least the methods described above.

[0105] This invention also provides a control device, including a processor and a storage medium for storing a computer program; wherein the processor executes the computer program by performing at least the method described above.

[0106] This invention also provides a processor that executes a computer program, at least performing the methods described above.

[0107] The storage medium can be implemented by any type of non-volatile storage device, or a combination thereof. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc or CD-ROM; magnetic surface memory can be disk storage or magnetic tape storage. The storage media described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable types of memory.

[0108] In the several embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0109] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0110] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0111] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0112] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0113] The methods disclosed in the several method embodiments provided by this invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0114] The features disclosed in the several product embodiments provided by this invention can be arbitrarily combined without conflict to obtain new product embodiments.

[0115] The features disclosed in the several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0116] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various equivalent substitutions or obvious modifications can be made without departing from the concept of the present invention, and all such modifications, achieving the same performance or application, should be considered within the scope of protection of the present invention.

Claims

1. A method for identifying psychological vulnerability based on multi-instance learning, characterized in that, Includes the following steps: S1. Constructing an individual-level dataset: Process long-duration videos containing multiple individuals, detect and crop individual facial images from video frames, merge samples according to individual identity, and construct an individual-level dataset based on individuals; S2. Construct daily statistical features: For the facial images in the individual-level dataset, extract the emotion polarity features and facial action unit features frame by frame, and perform statistical aggregation according to time segments to construct daily statistical features corresponding to each time segment. The daily statistical features include emotion statistical features and facial action unit statistical features. S3. Temporal Coding and Multi-Instance Learning: The daily statistical features corresponding to all time segments of each individual are organized into a package, and the features of each time segment are treated as an instance. A bidirectional long short-term memory network is used to perform temporal modeling on the instance sequence, and the hidden state of each instance is extracted as its temporal coding feature representation. S4. Aggregation and Classification: Statistically aggregate the hidden states of all instances after temporal encoding, calculate their mean and variance, and concatenate them to form an individual-level feature vector. Then, input the individual-level feature vector into the classifier and output the psychological vulnerability identification result of the individual.

2. The psychological vulnerability identification method based on multi-instance learning as described in claim 1, characterized in that, In step S1, detecting and cropping individual facial images from video frames specifically includes: using a face detection and alignment method based on a multi-task convolutional neural network to locate and align key points in the face regions of the video frames to obtain standardized facial images; merging samples according to individual identity specifically includes: extracting facial features from the standardized facial images, performing similarity matching with a pre-built facial feature library, merging facial image samples belonging to the same individual identity, and removing individuals with fewer than a preset threshold of valid facial images.

3. The psychological vulnerability identification method based on multi-instance learning as described in claim 1, characterized in that, In step S2, the step of extracting emotion polarity features frame by frame specifically includes: using an emotion recognition model based on residual networks, taking a single frame of facial image as input, and outputting the emotion polarity category of the frame, wherein the emotion polarity category includes at least negative and non-negative; the step of statistical aggregation by time segment to construct emotion statistical features specifically includes: taking a natural day as a time segment, calculating the total number of face frames in the time segment, the number of face frames judged as negative emotions, the proportion of negative emotion frames, and the longest duration of negative emotions calculated from the timestamp difference within the same day.

4. The psychological vulnerability identification method based on multi-instance learning as described in claim 1, characterized in that, In step S2, the extraction of facial action unit features specifically includes: using a facial behavior analysis tool to extract the continuous activation intensity values ​​and binary detection identifier values ​​of multiple facial action units in each frame image; the statistical aggregation by time segment to construct facial action unit statistical features specifically includes: taking a natural day as the time segment, calculating the average value of the activation intensity values ​​of each facial action unit in all valid face frames within the time segment, and constructing a daily-level facial action unit feature vector.

5. The psychological vulnerability identification method based on multi-instance learning as described in claim 1, characterized in that, In step S3, the step of using a bidirectional long short-term memory network to perform time-series modeling of the instance sequence specifically includes: taking the daily statistical feature vector of each individual arranged in date order as the input sequence, encoding the sequence through a forward long short-term memory network and a backward long short-term memory network respectively, and concatenating the forward hidden state and the backward hidden state corresponding to each time step as the time-series encoding feature representation, so as to simultaneously capture the forward and backward dependencies of the time series.

6. The psychological vulnerability identification method based on multi-instance learning as described in claim 1, characterized in that, In step S4, the calculation and concatenation of the mean and variance specifically includes: calculating the average value of the hidden state at all effective time steps of an individual in each dimension to form a mean vector; calculating the average value of the squares of the differences between the hidden state values ​​and the corresponding means in each dimension to form a variance vector; concatenating the mean vector and the variance vector end to end to form an individual-level feature vector; performing a dropout operation on the individual-level feature vector to reduce the risk of overfitting; and then inputting the dropout-processed feature vector into a multilayer perceptron classifier for linear mapping to output a binary classification result for the psychologically vulnerable group or the protected group.

7. The psychological vulnerability identification method based on multi-instance learning as described in claim 1, characterized in that, Following step S4, a visualization step for managing psychological vulnerability is also included: the algorithm is deployed as a psychological risk visualization system, with the class as the analysis unit. Based on the facial image sequence of all students in the classroom video, the system visualizes the characteristics of emotional distribution and their temporal trends, outputs the psychological vulnerability identification results and individual suggestions for each individual, and generates an overall psychological risk overview for the class.

8. The psychological vulnerability identification method based on multi-instance learning as described in claim 1, characterized in that, During the model training phase, the method employs binary cross-entropy as the loss function and uses the Adam optimizer to iteratively optimize the network parameters, wherein the network parameters include at least the weight parameters of the bidirectional long short-term memory network and the weight parameters of the classifier.

9. The psychological vulnerability identification method based on multi-instance learning as described in claim 1, characterized in that, The bidirectional long short-term memory network captures the trend of individual emotional and behavioral changes over a long period of time by simultaneously utilizing forward and backward dependencies of time series; the joint aggregation mechanism of mean and variance uses the mean to characterize the overall psychological risk level of individual behavior and the variance to characterize the volatility and instability of individual behavioral patterns; the longest duration of negative emotions in the daily statistical features is used to reflect the persistence characteristics of emotions in the time dimension.

10. A psychological vulnerability identification system based on multi-instance learning, used to implement the method of claim 1, characterized in that, include: The feature extraction module, connected to the data acquisition and sample construction module, is used to extract emotion polarity features and facial action unit features frame by frame, and to perform statistical aggregation by time segment to construct daily statistical features for each time segment. The temporal coding module, connected to the feature extraction module, is used to organize the daily statistical feature sequence of each individual into a bag-instance structure, and to use a bidirectional long short-term memory network to temporally encode the instance sequence, outputting the hidden state of each instance as a temporal coding feature. The aggregation and classification module, connected to the temporal coding module, is used to calculate the mean and variance of the hidden states of all instances and concatenate them to form an individual-level feature vector. Then, the classifier outputs the psychological vulnerability identification result of the individual.

Images