A deep learning driven mental health risk prediction method and system

Abstract

The present application relates to the technical field of deep learning and mental health risk assessment, and discloses a deep learning driven mental health risk prediction method and system, specifically: an audio stream rhythm feature vector and a facial action unit activation intensity vector sequence are acquired; a micro-expression emotional event sequence with a timestamp is generated based on the activation intensity vector sequence; a contradictory event feature sequence is generated according to the time alignment relationship between the rhythm feature vector and the micro-expression emotional event; a density-based time series clustering is performed on the contradictory event feature sequence to generate a contradictory cluster feature sequence with topic annotation; activation intensity values and activation feature vectors of each disorder node are generated according to the mapping relationship between the counseling topic and the mental disorder dimension; a directed graph attention network is used to output a cascading perception embedding vector of each disorder node based on a time causal comorbidity directed graph; and finally, multi-disorder cascading early warning data is generated based on the cascading perception embedding vector and the activation feature vector.

Description

Technical Field

[0001] This invention relates to the field of deep learning and mental health risk assessment technology, and in particular to a deep learning-driven method and system for predicting mental health risks. Background Technology

[0002] With the continued growth in demand for mental health services, automated risk assessment in video-based psychological counseling scenarios has gradually become an important research direction in computational psychology and clinical decision support. Multimodal behavioral signals include phonological prosody, facial movements, and their cross-modal interactions. Therefore, multimodal behavioral signals are considered to carry objective emotional state information beyond self-report scales, providing a potential data foundation for the early identification of mental health risks.

[0003] However, as the assessment task expands from single obstacle identification to multi-obstacle comorbidity scenarios, existing solutions face multi-layered technical bottlenecks. Firstly, in terms of language-side signal extraction, existing solutions generally rely on text semantic models. For sparsely expressed groups (short and ambiguous sentence content), insufficient text length leads to the failure of semantic feature extraction, failing to provide stable language-side emotional representations for subsequent cross-modal analysis. Secondly, in terms of cross-modal contradiction detection, existing solutions lack a processing flow that projects speech emotional signals and facial micro-expression signals onto a unified emotional polarity space and calculates fine-grained contradiction degrees, resulting in the failure to effectively quantify the emotional inconsistency signals between speech and facial expressions. Thirdly, in terms of multi-obstacle association modeling, existing solutions typically construct static undirected comorbidity graphs using comorbidity probabilities as fixed edge weights, failing to encode the temporal causal relationships and cascading propagation delay characteristics between obstacles, resulting in the inability to predict the risk escalation window of downstream obstacles.

[0004] Current multimodal mental health risk assessment schemes do not take into account the temporal directionality and delayed attributes of inter-disorder cascading propagation, resulting in the inability to perform time-aware message transmission along the causal direction in multi-disorder comorbidity scenarios, and thus the inability to generate multi-disorder cascading early warning data that includes the expected risk increase time window. Summary of the Invention

[0005] The summary section of this application is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description section below. This summary section is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0006] In view of the problems and shortcomings of the existing technology, the purpose of this invention is to provide a deep learning-driven method and system for predicting mental health risks, which solves the technical problems of the existing technology, such as the inability to effectively integrate cross-modal emotional conflict signals of audio and video, the lack of ability to model the temporal causal comorbidity between mental disorders, and the inability to output multi-disorder cascade early warning data.

[0007] To achieve the above objectives, the present invention provides the following technical solution: As a first aspect of this application, the present invention discloses a deep learning-driven method for predicting mental health risks, comprising the following steps: Step S1: Obtain the prosodic feature vector of the audio stream and the sequence of facial motion activation intensity vectors for each frame of the video frame; Step S2: Based on the facial action activation intensity vector sequence, apply sliding window difference calculation and micro-expression classification network to generate a time-stamped micro-expression emotion event sequence; Step S3: Generate a contradictory event feature sequence based on the timestamp alignment relationship between the prosodic feature vector and the micro-expression emotional event sequence; Step S4: Perform density-based temporal clustering on the contradictory event feature sequence to generate a contradictory cluster feature sequence with topic annotations; Step S5: Based on the preset mapping relationship table between consultation topics and psychological obstacle dimensions, the dominant topic labels of each contradiction cluster feature are converted into obstacle dimension activation vectors, and then the contradiction cluster driving activation intensity value and activation feature vector of each obstacle node are generated. Step S6: Based on statistical data from clinical longitudinal cohort studies, obtain the temporal causal comorbidity conditional probability and average cascade delay duration among mental disorders to generate a temporal causal comorbidity directed graph. Step S7: Load the contradiction cluster-driven activation feature vector of each obstacle node into the corresponding node of the temporal causal comorbidity directed graph, and use the directed graph attention network to perform one-way message passing along the causal direction to output the cascaded perception embedding vector of each obstacle node. Step S8: The cascaded sensing embedding vector of each obstacle node and its contradictory clustering driven activation feature vector are concatenated and then input into the deep risk prediction network corresponding to each obstacle to generate multi-obstacle cascaded early warning data. In the directed graph of temporal causal comorbidity, the nodes correspond to the psychological disorder dimension, and the directed edges are generated based on the conditional probability of temporal onset between disorders and carry the cascaded delay duration attribute.

[0008] Preferably, generating a timestamped sequence of micro-expression emotion events based on the activation intensity vector sequence includes: A sliding window differential calculation is applied to the frame-by-frame facial motion unit activation intensity vector sequence to obtain the difference vector for each frame; When the difference vector of a certain frame When the norm exceeds a preset abrupt change threshold, the frame is marked as an abrupt change frame; Extract the frames before and after the sudden change frame. The activation mode difference vector within the frame range is input into the microexpression classification network to output the microexpression emotion label and confidence score; The timestamps, micro-expression emotion tags, confidence scores, and the emotion polarity vector output from the penultimate layer of the micro-expression classification network corresponding to each sudden change frame are combined into a quadruple, which is then arranged in chronological order to form the micro-expression emotion event sequence.

[0009] Preferably, generating the contradictory event feature sequence based on the temporal alignment relationship between the prosodic feature vector and the micro-expression emotional events includes: The prosodic feature vectors of each speech segment are projected onto an emotion polarity space of the same dimension as the emotion polarity vector using a linear projection layer to obtain the prosodic emotion polarity vector. ; For the first The emotional polarity vectors of all micro-expression emotional events within the indexed time range of each speech segment are weighted and averaged using confidence scores to obtain the comprehensive micro-expression emotional vector. ; Calculate cross-modal inconsistency When cross-modal contradiction degree If the preset contradiction threshold is exceeded, the audio segment is marked as a contradiction event; The characteristic record of the contradictory event includes [the following]. The normalized features include the contradiction direction feature vector, contradiction intensity value, contradiction event timestamp, and current consultation topic identifier.

[0010] Preferred, the step of performing density-based temporal clustering on the contradictory event feature sequence to generate a contradictory cluster feature sequence with topic annotations includes: With adaptive time window Density-based temporal clustering is performed on the timestamps of each contradictory event in the contradictory event feature sequence, using the neighborhood radius as the basis; For each cluster interval, the contradiction density value, the mean contradiction intensity within the cluster, and the consistency coefficient of contradiction direction are calculated as cluster feature parameters. The frequency of topic identifiers for each contradiction event within each cluster is counted, and the topic identifiers with a frequency ratio exceeding a preset topic ratio threshold are selected to form the dominant topic identifier set for that cluster.

[0011] Prior to this, the step of generating activation intensity values ​​and activation feature vectors for each obstacle node based on the mapping relationship between the consultation topic and the psychological obstacle dimension includes: Based on the pre-set mapping table between consultation topics and psychological obstacle dimensions, the dominant topic identifiers of each conflict cluster are converted into obstacle dimension activation sets. For the first Let each obstacle dimension be a set of contradictory clusters that activate that dimension. Based on the contradictory density values ​​of each cluster With the mean of contradiction intensity The product of these factors is used to calculate the activation strength value of the obstacle node. ; Calculate the activation feature vector of the obstacle node. It is composed of contradiction density value, contradiction intensity mean value, contradiction direction consistency coefficient and weighted average contradiction direction feature vector with contradiction intensity value as weight.

[0012] Preferably, the directed graph attention network includes a node state initialization unit, a directed attention calculation unit, a cascaded delay modulation unit, and a message aggregation update unit. The step of using the directed graph attention network to output the cascaded perception embedding vectors of each obstacle node includes: The node state initialization unit maps the activation feature vectors of each obstacle node to the initial node state vector through a linear transformation. In each round of message passing, the directed attention calculation unit calculates the attention weight of the upstream node to the downstream node only along the direction of the directed edge. The attention weight is determined by combining the current state vector of the upstream node, the weight of the directed edge, and the activation intensity value of the upstream node. The cascaded delay modulation unit calculates the cascaded delay modulation coefficients based on the cascaded delay duration attribute of the directed edge and the duration of the current consultation session. The message aggregation and update unit aggregates all incoming source node messages for each downstream node after being modulated by the attention weight and the cascaded delay modulation coefficient, concatenates them with the current state vector of the downstream node, and then updates the node state through transformation. After a preset number of message transmissions, it outputs the cascaded perception embedding vector of each node.

[0013] Preferably, the step of generating multi-obstacle cascaded early warning data based on the cascaded sensing embedding vector and the activation feature vector includes: The concatenated sensing embedding vectors and activation feature vectors of each obstacle node are concatenated and then input into the corresponding deep risk prediction network to output the current risk probability of each obstacle. When the activation intensity value of the upstream obstacle node exceeds the preset cascading trigger threshold, all outgoing edges of the node are traversed, and the cascading warning probability of the downstream obstacle is calculated as the product of the current risk probability of the upstream obstacle, the weight of the directed edge, and the output value after applying the sigmoid activation function to the activation intensity value of the upstream obstacle. The expected risk increase time window for each cascading path is determined as the interval from the current time to the sum of the average cascading delay time of the corresponding directed edge. When the same downstream obstacle node is activated by multiple cascading paths, the maximum value of the cascading warning probability among each path is taken as the comprehensive cascading warning probability.

[0014] As a third aspect of this application, the present invention also discloses a deep learning-driven mental health risk prediction system, comprising: A multimodal feature extraction module is used to obtain audio stream prosodic feature vectors and facial action unit activation intensity vector sequences, and generate a time-stamped micro-expression emotion event sequence based on the activation intensity vector sequence; A cross-modal contradiction detection module is used to generate a sequence of contradictory event features based on the temporal alignment relationship between the prosodic feature vector and micro-expression emotional events. The contradiction clustering analysis module is used to perform density-based temporal clustering on the contradiction event feature sequence to generate a contradiction cluster feature sequence with topic annotations; The obstacle dimension mapping module is used to generate activation intensity values ​​and activation feature vectors for each obstacle node based on the mapping relationship between the consultation topic and the psychological obstacle dimension. The directed graph propagation module is used to output the cascaded sensing embedding vectors of each obstacle node based on the directed graph attention network of the temporal causal comorbidity directed graph. The cascaded early warning generation module is used to generate multi-obstacle cascaded early warning data based on the cascaded perception embedding vector and the activation feature vector.

[0015] As a third aspect of this application, the present invention also discloses an electronic device comprising: At least one processor, and a memory communicatively connected to said at least one processor; The memory stores instructions that can be executed by the at least one processor, which enable the at least one processor to perform the steps described above in the deep learning-driven mental health risk prediction method.

[0016] As a fourth aspect of this application, the present invention also discloses a computer storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the steps described above in the deep learning-driven mental health risk prediction method.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention aims to provide a deep learning-driven method and system for predicting mental health risks. By employing a deep prosodic feature extraction network, it extracts prosodic feature vectors from the original audio waveform, including the fundamental frequency mean, fundamental frequency variance, speech rate change rate, energy envelope slope, and pause duration ratio. This allows the acquisition of language-side emotional signals to be unconstrained by sentence length, providing continuous emotional representation input even in sparse expression scenarios, thus providing an effective language-side signal foundation for subsequent cross-modal conflict detection. By mapping the prosodic feature vectors to the same emotional polarity space as the micro-expression emotional polarity vector through a linear projection layer, and calculating the cross-modal conflict degree using cosine distance, the emotional inconsistency between phonological emotion and facial micro-expressions can be quantified and represented within a unified metric space, providing a semantically clear sequence of conflict event features for subsequent conflict cluster analysis. An adaptive time window based on the ratio of the current consultation session duration to the total number of detected conflicting events is introduced as the density-based temporal clustering neighborhood radius. This allows the clustering neighborhood range to be automatically adjusted according to the overall temporal density of conflicting events, thus avoiding the problems of real clusters being misjudged as noise and adjacent independent clusters being merged in sessions with sparse and dense conflicting events, respectively.

[0018] Based on clinical longitudinal cohort statistics, a directed graph of temporal causal comorbidity is constructed, carrying attributes of temporal causal conditional probability and average cascade delay duration. A cascade delay modulation coefficient is introduced into the directed graph attention network to perform time-aware modulation of message transmission intensity. This allows the cascade-aware embedding vector of each obstacle node to encode the time-decayed cascade propagation signal from the upstream obstacle, making it possible to include the expected risk increase time window in the multi-obstacle cascade warning data. In the multi-obstacle cascade warning data generation stage, the cascade warning probability of the downstream obstacle is calculated by comprehensively utilizing the current risk probability of the upstream obstacle, the conditional probability between obstacles, and the activation intensity value driven by contradictory clusters. The expected risk increase time window is determined by the average cascade delay duration of the directed edges. This ensures that the output warning data simultaneously includes probability quantification information and temporal location information, providing a more complete auxiliary reference for subsequent assessments by professionals. Attached Figure Description

[0019] The accompanying drawings, which form part of this application, are used to provide a further understanding of the application and to make other features, objects, and advantages of the application more apparent. The illustrative embodiments and descriptions of this application are used to explain the application and do not constitute an undue limitation of the application. In the drawings: Figure 1 This is a flowchart of the mental health risk data processing method provided in the embodiments of the present invention; Figure 2 This is a schematic diagram comparing the original values ​​and standardized values ​​of the prosodic features of a speech segment according to an embodiment of the present invention; Figure 3This is a schematic diagram of the temporal distribution and confidence level of micro-expression emotional events provided in an embodiment of the present invention; Figure 4 This is a schematic diagram comparing the cross-modal inconsistency degree and inconsistency threshold of each speech segment according to an embodiment of the present invention; Figure 5 This is a schematic diagram showing the comparison of contradictory clustering characteristic parameters provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the time-series causal comorbidity directed graph structure provided in the embodiments of the present invention; Figure 7 This is a schematic diagram of the multimodal feature extraction and contradiction detection processing flow provided in the embodiments of the present invention; Figure 8 This is a schematic diagram of the current risk probability and cascading warning probability of each obstacle node provided in the embodiments of the present invention; Figure 9 This is a schematic diagram illustrating the relationship between the cascaded delay modulation coefficient and the session duration provided in an embodiment of the present invention. Detailed Implementation

[0020] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0021] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. Example

[0022] This embodiment discloses a deep learning-driven method for predicting mental health risks. The following will describe this disclosure in detail with reference to the accompanying drawings and embodiments. (Refer to...) Figures 1 to 2 As shown, the specific steps include: Step S1: Obtain the prosodic feature vector of the audio stream and the sequence of facial motion activation intensity vectors for each frame of the video frame; Step S2: Based on the facial action activation intensity vector sequence, a sliding window difference calculation and micro-expression classification network are applied to generate a time-stamped micro-expression emotion event sequence; Step S3: Generate a contradictory event feature sequence based on the timestamp alignment relationship between the prosodic feature vector and the micro-expression emotional event sequence; Step S4: Perform density-based temporal clustering on the contradictory event feature sequence to generate a contradictory cluster feature sequence with topic labels; Step S5: Based on the preset mapping relationship table between consultation topics and psychological obstacle dimensions, convert the dominant topic labels of each contradictory cluster feature into obstacle dimension activation vectors, and then generate the contradictory cluster driving activation intensity value and activation feature vector of each obstacle node. Step S6: Based on statistical data from clinical longitudinal cohort studies, obtain the temporal causal comorbidity conditional probability and average cascade delay duration among mental disorders to generate a temporal causal comorbidity directed graph. Step S7: Load the contradiction clustering-driven activation feature vectors of each obstacle node into the corresponding nodes of the temporal causal comorbidity directed graph, and use the directed graph attention network to perform one-way message passing along the causal direction to output the cascaded perception embedding vectors of each obstacle node. Step S8: The cascaded sensing embedding vector of each obstacle node and its contradictory clustering-driven activation feature vector are concatenated and then input into the deep risk prediction network corresponding to each obstacle to generate multi-obstacle cascaded early warning data.

[0023] Step S1 involves acquiring the prosodic feature vector of the audio stream and the sequence of facial action activation intensity vectors for each frame of the video. During the video psychological counseling process, the audio stream and the sequence of facial video frames are acquired simultaneously. The audio stream is segmented according to natural pauses between sentences to obtain multiple speech segments. For each speech segment, a deep prosodic feature extraction network is used to extract the prosodic feature vector. For the video frame sequence, a facial action encoding network is used to extract the frame-by-frame facial action activation intensity vector. Finally, the facial action activation intensity vector sequence for all video frames is obtained.

[0024] Specifically, the input to the deep prosodic feature extraction network is the original audio waveform of a speech segment, and the output is a prosodic feature vector corresponding to that speech segment. This prosodic feature vector includes five components: fundamental frequency mean, fundamental frequency variance, speech rate change rate, energy envelope slope, and pause duration ratio. Since the dimensions of these five components are different, Z-score normalization is applied to each component before inputting the prosodic feature vector into the subsequent projection layer to eliminate the influence of these dimensional differences on the computation. The deep prosodic feature extraction network encodes the local acoustic patterns of the original audio waveform using multiple one-dimensional convolutional layers, then captures the temporal prosodic dependencies within the speech segment using bidirectional long short-term memory layers, and finally compresses the sequence representation into a five-dimensional prosodic feature vector through global average pooling layers and fully connected layers. During training, a speech dataset labeled with prosodic attributes is used as supervision, and the network parameters are updated using the Adam optimization algorithm with mean squared error loss as the objective function. The input to the facial action coding network is a single frame of facial image, and the output is the facial action activation intensity vector corresponding to that frame, where each component corresponds to the activation intensity value of a facial action unit. The facial action coding network uses a convolutional neural network as the backbone to extract facial spatial features, and regresses the activation intensity of each action unit through a fully connected layer. During training, manually labeled action unit intensity labels are used as supervision, the mean squared error loss function is used as the objective, and the Adam optimization algorithm is used to update the network parameters.

[0025] It should be noted that this embodiment uses a deep prosodic feature extraction network to replace the traditional text semantic model as the language-side signal source. The technical motivation is that for groups with sparse text expression, the text content of sentences is short and vague, while the acoustic emotional information contained in the prosodic features is not limited by the length of the text. It can still provide continuous and stable emotional signal output under the condition of sparse sentence content, thereby providing an effective language-side emotional representation for subsequent cross-modal contradiction detection.

[0026] Step S2 involves applying sliding window differencing and a micro-expression classification network to the facial action activation intensity vector sequence to generate a timestamped micro-expression emotional event sequence. Sliding window differencing is applied to the facial action activation intensity vector sequence to identify frames with abrupt changes in activation intensity. The difference vectors of action unit activation patterns before and after the abrupt change frames are extracted. The micro-expression classification network is used to classify these difference vectors, outputting micro-expression emotional labels and confidence scores, thus generating a timestamped micro-expression emotional event sequence.

[0027] Specifically, let the dimension of the facial action activation intensity vector be... For the first Frame, with width as Sliding window calculation of difference vector ,in Indicates the first The facial motion activation intensity vector of the frame. Indicates the first The facial motion activation intensity vector of the frame. For frame index, The width of the sliding window; when When the preset sudden change threshold is exceeded, the first... Frames are marked as transition frames. For each transition frame, extract the frames before and after it. The activation mode difference vector within a frame range serves as the input to the micro-expression classification network, which outputs the micro-expression sentiment label and confidence score corresponding to the abrupt change frame. The micro-expression classification network primarily consists of multiple fully connected layers, with a receiver dimension of [missing information]. The difference vector is used to extract sentiment discrimination features through an intermediate hidden layer. The penultimate layer outputs a dimension of... The emotional polarity vector is used to calculate the probability distribution of each micro-expression emotion category in the final classification layer; where... Let be the dimension of the sentiment polarity vector. During training, samples labeled with micro-expression sentiment categories are used as supervision, and the cross-entropy loss function is used as the objective function. The Adam optimization algorithm is employed to update the network parameters. Each micro-expression sentiment event is represented by a four-tuple: timestamp, micro-expression sentiment label, confidence score, and the corresponding sentiment polarity vector (obtained from the output of the penultimate layer of the micro-expression classification network, with a dimension of ). ).

[0028] Step S3 involves generating a contradictory event feature sequence based on the timestamp alignment between the prosodic feature vector and the micro-expression emotion event sequence. Based on the timestamp alignment, the prosodic feature vector of each speech segment is mapped to the emotion polarity space to obtain the prosodic emotion polarity vector. The cosine distance between the prosodic emotion polarity vector and the corresponding micro-expression emotion composite vector within the time range is calculated as the cross-modal contradictory degree. When the cross-modal contradictory degree exceeds a preset threshold, it is marked as a contradictory event, and a contradictory event feature sequence is generated.

[0029] Specifically, the sentiment polarity space mapping is implemented through a trainable linear projection layer that projects a 5-dimensional prosodic feature vector to a 5-dimensional vector. The emotional polarity space is used to obtain the rhythmic emotional polarity vector. ,in For speech segment indexing, The dimension is the same as that of the emotion polarity vector in micro-expression emotional events. For the th For each audio segment, a set of emotional polarity vectors for all micro-expression emotional events within its time range is determined. This set is then weighted and averaged using confidence levels to obtain a comprehensive micro-expression emotional vector for that time range. Cross-modal contradiction degree Calculated based on cosine distance: ;in, For the first The prosodic sentiment polarity vector corresponding to each speech segment Both are comprehensive vectors of micro-expression emotions within the corresponding time range, and their dimensions are [missing information]. .when Exceeding the preset contradiction threshold At that time, the speech segment is marked as a contradictory event. In at least one example of the embodiments of this disclosure, the feature record of each contradictory event includes: a contradictory direction feature vector (composed of...). Normalized acquisition, dimension is The system includes the intensity of the conflict, the timestamp of the conflict event (taken as the center moment of the corresponding audio segment), and the identifier of the current consultation topic (obtained from the topic segmentation annotation during the consultation process). All conflict events are arranged in ascending order of timestamps to form a sequence of conflict event characteristics.

[0030] Step S4 involves performing density-based temporal clustering on the contradictory event feature sequence to generate a contradictory cluster feature sequence with topic annotations. The density-based temporal clustering algorithm is applied to the timestamps of each contradictory event in the contradictory event feature sequence, using an adaptive time window as the neighborhood radius to identify the cluster intervals of contradictory events on the time axis. Feature parameters for each cluster interval are calculated, and the dominant topic identifier set corresponding to each cluster is extracted to generate a contradictory cluster feature sequence with topic annotations.

[0031] Specifically, the input to the density-based temporal clustering algorithm is the timestamp sequence of each contradictory event in the contradictory event feature sequence, and the neighborhood radius parameter is set as an adaptive time window. The minimum sample size parameter is set to The density-based temporal clustering algorithm outputs cluster affiliation labels for each contradictory event. Adaptive time window. Determined based on the ratio of the total duration of the current consultation session to the total number of conflicting events detected: ;in, The duration of the current consultation session. This represents the total number of conflicting events detected. This is a preset window scaling factor. It should be noted that the introduction of the adaptive time window allows the cluster neighborhood radius to automatically adjust with the overall temporal density of contradictory events: expanding the neighborhood range in sessions with sparse contradictory events to avoid misclassifying true clusters as noise, and narrowing the neighborhood range in sessions with dense contradictory events to accurately distinguish adjacent independent clusters. In at least one example of this embodiment, the following feature parameters are calculated for each cluster interval: contradictory density value. (Number of conflicting events within a cluster divided by the cluster's time span, in events per second), mean intensity of conflicts within a cluster. (The arithmetic mean of the intensity values ​​of all contradictory events within the cluster, which is the dimensionless cosine distance), and the consistency coefficient of contradictory directions. (The average cosine similarity of all contradictory eigenvectors within a cluster is a dimensionless value), where This is a clustered index. Because... and , The dimensions are different, so before concatenating the three into a clustered feature vector, the... Mean normalization based on range is used to eliminate the influence of dimensional differences on subsequent calculations; and Since it is dimensionless, no additional normalization is needed. The set of dominant topic identifiers corresponding to each cluster is formed by statistically analyzing the frequency of topic identifiers for each contradictory event within the cluster, selecting topic identifiers whose frequency percentage exceeds a preset topic percentage threshold. The complete characteristics of a contradiction cluster are represented as follows: ,in This is a weighted average vector of all contradictory directional feature vectors within the cluster, weighted by the contradictory intensity value, with dimension [missing information]. ; The overall dimension is .

[0032] Regarding step S5, based on the preset mapping relationship table between consultation topics and psychological obstacle dimensions, the dominant topic label of each contradiction cluster feature is converted into an obstacle dimension activation vector. Multiple contradiction cluster features mapped to the same obstacle dimension are weighted and aggregated to generate the contradiction cluster-driven activation intensity value and activation feature vector of each obstacle node.

[0033] Specifically, the pre-defined mapping table between consultation topics and psychological disorder dimensions records the mapping correspondence between each consultation topic identifier and one or more psychological disorder dimensions. For the first... Each contradiction cluster is represented by a set of dominant topic identifiers. Each topic identifier is converted into a corresponding set of obstacle dimension identifiers according to the mapping relationship table between counseling topics and psychological obstacle dimensions. The union of all obstacle dimension identifiers corresponding to topic identifiers constitutes the obstacle dimension activation set of the cluster. The activation value of each activation dimension is the frequency percentage of the corresponding topic identifier in the cluster.

[0034] In at least one example of the embodiments of this disclosure, for the first Let each obstacle dimension be a set of contradictory clusters that activate that dimension. ,in Each element in the index is used to index the elements in the index. The identifier is the product of the conflict density value and the mean conflict intensity value for each cluster. As the weight, the conflict clustering-driven activation intensity value of the obstacle node. and activation feature vector Calculated separately as follows: ； ; in For the first The activation intensity value driven by the contradictory clustering of individual obstacle nodes. For obstacle dimension indexing, For the first The activation feature vectors of each obstacle node. For the first A complete feature representation of a contradictory cluster, with dimension [ ]. ; dimensionality and Same, for In subsequent steps The use of this dimension is based on its definition.

[0035] Regarding step S6, based on statistical data from clinical longitudinal cohort studies, the conditional probability of the temporal order of onset and the average cascade delay duration among mental disorders are obtained to generate a directed graph of temporal causal comorbidity. Specifically, let the total number of dimensions of mental disorders be... For any two obstacle dimensions and ( ),in and All are obstacle dimension indexes, obtained from clinical longitudinal cohort statistics. Prior to obstacles Conditional probability of disease onset and the corresponding average cascade delay time A directed graph of causal comorbidity in a temporal sequence. In, node set Each node in the diagram corresponds to a dimension of psychological barrier, where They are respectively the 1st to the 1st Each psychological barrier dimension corresponds to a node. The total number of dimensions of psychological disorders; when Exceeding the preset edge generation threshold At that time, generate a node To the node directed edges The directed edge carries two attributes: weight value. and time attributes .

[0036] It should be noted that the direction of the directed edges encodes the temporal causal sequence between obstacles, and the temporal attribute of the directed edges encodes the statistical average delay between the deterioration of the upstream obstacle and the increase in the risk of the downstream obstacle. These two attributes together enable the temporal causal comorbidity directed graph to characterize the cascading propagation directionality and propagation time characteristics between obstacles, which is different from the existing static undirected comorbidity graph that uses the comorbidity probability as the fixed edge weight.

[0037] In step S7, the contradiction clustering-driven activation feature vectors of each obstacle node are loaded into the corresponding nodes of the temporal causal co-occurrence directed graph. A one-way message passing is then performed along the causal direction using the directed graph attention network, outputting the cascaded perceptual embedding vectors of each obstacle node. Specifically, the directed graph attention network includes the following components: a node state initialization unit, a directed attention calculation unit, a cascaded delay modulation unit, and a message aggregation update unit. The data transmission relationships between these components are as follows: The node state initialization unit receives the activation feature vectors of each obstacle node. (Dimension is) and contradictory cluster-driven activation intensity value The activation feature vector is activated through linear transformation. Mapped to the initial node state vector The initial node state vector is then passed to the directed attention computation unit. The input dimension of the linear transformation matrix is... The output dimension is , Let be the dimension of the node state vector. In each round of message passing, the directed attention computation unit calculates the attention weights of the upstream node to the downstream node only along the directed edge direction. For directed edges... Attention weight The calculation comprehensively considers the current state vector of the upstream node. Directed edge weights The conflicting clustering-driven activation intensity value of upstream nodes : ; in, and These are the query transformation matrix and the key transformation matrix, respectively. Let be the dimension of the node state vector. For nodes The set of all incoming edge source nodes, It is the sigmoid activation function. For the current message passing round, Indicates transpose; softmax operation is performed on nodes. All incoming edge source nodes Normalization is performed on the top edge so that the sum of the attention weights corresponding to all incoming edges is 1. The directed attention computation unit then calculates the attention weights. It is transmitted to the cascaded delay modulation unit.

[0038] Cascaded delay modulation units based on the time attributes of directed edges The duration of the current consultation session Calculate the cascaded delay modulation coefficients : ;in, Represents an exponential function. For directed edges Average cascading delay time, The duration of the current consultation session is considered. When the duration is close to or exceeds the average cascade delay duration, the cascade delay modulation coefficient approaches 1, indicating that the cascaded propagation signal should be fully transmitted. When the session duration is much shorter than the average cascade delay duration, the cascade delay modulation coefficient decreases, indicating that the contribution of cascaded propagation on the current timescale is suppressed. The cascade delay modulation unit modulates the propagation intensity... The message is passed to the message aggregation and update unit. The message aggregation and update unit updates each downstream node. Aggregate messages from all incoming source nodes and update the node status: ;in To update the transformation matrix, let be the value transformation matrix. This represents a vector concatenation operation, resulting in a vector with dimension . , This represents the preset total number of message passing rounds. (After...) After round-robin directed message passing, the final state vector of each obstacle node. This is the cascaded sensing embedding vector of the node, with a dimension of . The directed graph attention network uses a clinical multimodal consultation dataset with obstacle risk labels as training data, where the supervision signal of each node is the clinical diagnostic label of each obstacle dimension; during training, the final state vector of each obstacle node is... The parameters of each unit in the directed graph attention network are jointly updated using the Adam optimization algorithm, with the risk probability mapped to the linear classification layer and the binary cross-entropy loss function as the objective.

[0039] In at least one example of the embodiments of this disclosure, for nodes in the temporal causal co-occurrence directed graph that have no incoming edges (i.e., nodes without upstream obstacles), their node states are maintained only through self-updates in each round of message passing. ,in For dimension is The zero vector of the node, the cascaded sensing embedding vector of the node only encodes its own contradictory clustering-driven activation information and does not contain cascaded propagation signals from upstream.

[0040] In step S8, the cascaded sensing embedding vector of each obstacle node is concatenated with its contradictory clustering-driven activation feature vector and input into the deep risk prediction network corresponding to each obstacle, outputting the current risk probability of each obstacle; when the contradictory clustering-driven activation intensity value of the upstream obstacle exceeds a preset threshold, the cascaded warning probability of the downstream obstacle and the expected risk increase time window are calculated along the directed edge, generating multi-obstacle cascaded warning data. Specifically, for the first... Each obstacle node is used to embed its cascaded sensing vector. (Dimension is) ) and activation feature vector (Dimension is) splicing to obtain (Dimension is) As input to the corresponding deep risk prediction network, the deep risk prediction network outputs the current risk probability of the obstacle. ,in For the first The current risk probability of each obstacle node. This is an index for obstacle dimensions. Each obstacle dimension corresponds to an independent deep risk prediction network. Each deep risk prediction network mainly consists of multiple fully connected layers, with a receiving dimension of [missing information]. The concatenated vectors are progressively compressed through hidden layers to represent features, and finally output as scalar risk probability values ​​through a sigmoid activation function. During training, clinical diagnostic labels corresponding to the obstacle dimension are used as supervision, and the binary cross-entropy loss function is used as the target. The Adam optimization algorithm is used to update the network parameters.

[0041] In at least one example of the embodiments of this disclosure, the process of generating multi-obstacle cascade early warning data is as follows: for each upstream obstacle node in the temporal causal comorbidity directed graph When contradiction clustering drives activation intensity value Exceeding the preset cascading trigger threshold When traversing nodes All outgoing edges Calculate downstream barriers The probability of cascading early warning is ,in upstream obstacle node The current risk probability, For obstacles Prior to obstacles The conditional probability of the onset of the disease. To determine the activation intensity value driven by contradiction clusters The output value after applying the sigmoid activation function; the expected time window for increased risk is... ,in For the current moment, For directed edges The average cascading delay time. When a downstream obstacle node is activated by multiple cascading paths simultaneously, the maximum cascading warning probability among all paths is taken as the comprehensive cascading warning probability of that downstream obstacle.

[0042] Multi-obstacle cascade early warning data includes: the independent risk probability of each obstacle. (in For obstacle dimension indexing, The output of this method includes the total number of psychological disorder dimensions, the activation disorder sequence on the cascading propagation path (obtained by traversing the nodes that meet the triggering conditions along the outgoing edges in the temporal causal comorbidity directed graph), and the triggering topic interval identifiers for each cascading path (determined by the set of dominant topic identifiers corresponding to the contradictory clusters that activate the starting node of the path). It should be noted that the output of this method is psychological health risk probability data and multi-disorder cascading early warning data. This data serves as supplementary reference information for professionals to use in subsequent assessment processes. The execution process of this method involves mathematical transformations and probability calculations of the multimodal behavioral data.

[0043] The mental health risk data processing system provided in at least one embodiment of this disclosure includes a multimodal feature extraction module, a cross-modal contradiction detection module, a contradiction clustering analysis module, a barrier dimension mapping module, a directed graph propagation module, and a cascaded early warning generation module. The multimodal feature extraction module is used to acquire audio stream prosodic feature vectors and facial action activation intensity vector sequences, and to generate a timestamped micro-expression emotional event sequence based on the facial action activation intensity vector sequence. The cross-modal contradiction detection module is used to generate a contradiction event feature sequence based on the temporal alignment relationship between the prosodic feature vectors and the micro-expression emotional events. The contradiction clustering analysis module is used to perform density-based temporal clustering on the contradiction event feature sequence to generate a contradiction cluster feature sequence with topic annotations. The barrier dimension mapping module is used to generate the contradiction cluster-driven activation intensity value and activation feature vector of each barrier node based on the mapping relationship between the counseling topic and the psychological barrier dimension. The directed graph propagation module is used to output the cascaded perception embedding vector of each barrier node using a directed graph attention network based on a temporal causal comorbidity directed graph. The cascaded early warning generation module is used to generate multi-barrier cascaded early warning data.

[0044] To verify the performance of the gas monitoring and early warning method described in Example 1, Example 2 provides specific experimental verification data. Example 2 adopts the method steps and parameter settings from Example 1, such as... Figures 3 to 9 As shown.

[0045] Multimodal feature extraction is performed in step S1. During video psychological counseling, audio streams and facial video frame sequences are acquired simultaneously. The audio stream is segmented according to natural pauses between sentences to obtain multiple speech segments. For each speech segment, a deep prosodic feature extraction network is used to extract prosodic feature vectors. For the video frame sequence, a facial action coding network is used to extract frame-by-frame facial action unit activation intensity vectors. The prosodic feature vector includes five components: fundamental frequency mean, fundamental frequency variance, speech rate change rate, energy envelope slope, and pause duration ratio. After extraction, each of the five components is Z-score standardized to eliminate dimensional differences.

[0046] During an online video consultation session in C month of 20XX, a psychological support platform simultaneously collected audio and video data from a client (ID V-0312). The consultation session lasted for [duration missing]. The system segments the speech into several segments based on natural pauses, and extracts the dimension of each facial image frame. The facial action unit activation intensity vectors are shown. The table below shows the original values ​​of the prosodic features of four speech segments and the results after Z-score standardization (standardized mean and standard deviation are from training set statistics). As shown in Table 1 below, Table 1 shows the prosodic feature extraction results of speech segments.

[0047]

[0048] Table 1 Step S2 generates a micro-expression emotion event sequence. A sliding window difference calculation is applied to the frame-by-frame facial action unit activation intensity vector sequence to identify frames with abrupt changes in activation intensity. The difference vectors of action unit activation patterns before and after the abrupt change frames are extracted. A micro-expression classification network is used to classify these difference vectors, outputting micro-expression emotion labels and confidence scores, generating a timestamped micro-expression emotion event sequence. Each micro-expression emotion event in the sequence consists of a timestamp, a micro-expression emotion label, a confidence score, and a dimension of [missing information]. The emotional polarity vectors form a quadruple.

[0049] In the consultation session of V-0312, the system uses a sliding window width The frame performs a difference operation on the activation intensity vector sequence of all facial motion units, when Frames exceeding a sudden change threshold of 0.85 are marked as sudden change frames. The dimension of the sentiment polarity vector output by the micro-expression classification network is set to [value missing]. The table below shows four typical micro-expression emotion events detected in this session. Table 2 below shows the sequence of micro-expression emotion events.

[0050]

[0051] Table 2 Step S3 generates the contradictory event feature sequence. Based on the timestamp alignment, the prosodic feature vector of each speech segment is mapped to a dimension of [dimensional value missing] through a trainable linear projection layer. Obtaining the rhythmic emotional polarity vector from the emotional polarity space The comprehensive micro-expression emotion vector is obtained by weighting the emotion polarity vectors of micro-expression emotion events within the corresponding time range with confidence level as the weight. Calculate cross-modal inconsistency degree ,when Exceeding the preset contradiction threshold The time markers are considered contradictory events. Taking speech segment S-02 as an example, its time range includes events E-02 (confidence 0.79) and E-03 (confidence 0.83), and the micro-expression emotion vector... The cross-modal inconsistency is obtained by weighting the two values ​​with confidence levels as the weight. The cross-modal inconsistency score of speech segment S-02 is 1.34, exceeding the threshold of 0.55, and is therefore marked as an inconsistency event. The table below shows the inconsistency detection results for all four speech segments, where the inconsistency direction feature vector of the inconsistency event is... Normalization was used to obtain the timestamps of contradictory events, which were obtained from the center time of the corresponding speech segment. Table 3 below shows the feature sequence of contradictory events.

[0052]

[0053] Table 3 Step S4 generates a conflict cluster feature sequence with topic annotations. A density-based temporal clustering algorithm is applied to the timestamps of each conflicting event in the conflict event feature sequence, using an adaptive time window. Using the neighborhood radius, identify clusters of contradictory events on the time axis and calculate the contradictory density value for each cluster. Mean of contradiction intensity within clusters Coefficient of consistency in the direction of contradiction The dominant topic identifier set corresponding to each cluster is extracted to generate a contradictory cluster feature sequence with topic annotations. Three contradictory events were detected in this session, with timestamps of 203.0 seconds, 438.0 seconds, and 911.0 seconds. ， seconds, preset window scaling factor The adaptive time window is calculated as follows: ;by Seconds represent the neighborhood radius and the minimum number of samples. Density-based temporal clustering was performed. The time interval between events 203.0 seconds and 438.0 seconds was 235 seconds, which is less than the neighborhood radius of 440 seconds, so they were assigned to the same cluster (cluster C-01). The time interval between event 911.0 seconds and its nearest neighbor event 438.0 seconds was 473 seconds, which is more than 440 seconds, so it was assigned to an independent cluster (cluster C-02). For the contradictory density values ​​after normalization, range normalization was used to eliminate their discrepancies. , The dimensional differences between them are shown in Table 4 below, which illustrates the characteristic parameters of the two clusters.

[0054]

[0055] Table 4 Step S5 generates the activation intensity value and activation feature vector of the obstacle node. Based on a preset mapping table between consultation topics and psychological obstacle dimensions, the dominant topic identifier of each conflict cluster is converted into an obstacle dimension activation vector. Multiple conflict cluster features mapped to the same obstacle dimension are weighted and aggregated to generate the conflict cluster-driven activation intensity value for each obstacle node. and activation feature vector This session covered three dimensions of psychological disorders: D-01 (risk of mood dysregulation), D-02 (risk of sleep disorders), and D-03 (risk of anxiety disorders). The mapping relationships are as follows: T-emotion regulation maps to D-01 and D-03, and T-sleep quality maps to D-02 and D-03. Therefore: the activation cluster set of D-01... Activated cluster set of D-02 Activated cluster set of D-03 Taking obstacle dimension D-01 as an example, the weights... For C-01 For C-02 Activation strength value: As shown in Table 5 below, Table 5 presents the calculation results of the activation intensity values ​​for each obstacle node.

[0056]

[0057] Table 5 Step S6 generates a directed graph of temporal causal comorbidity. Based on statistical data from clinical longitudinal cohort studies, the conditional probability of the temporal onset of mental disorders and the average cascade delay duration are obtained. Exceeding the preset edge generation threshold Generate corresponding directed edges in time to construct a time-series causal comorbidity directed graph. The three obstacle dimension node sets involved in this session are: These correspond to D-01, D-02, and D-03, respectively. The table below shows the conditional probabilities and average cascade delay times of the obstacle pairs obtained from clinical longitudinal cohort statistics, as well as the directed edges generated accordingly. As shown in Table 6, Table 6 presents the edge attributes of the temporal causal comorbidity directed graph.

[0058]

[0059] Table 6 The resulting temporal causal comorbidity directed graph contains nodes (D-01) (D-02) (D-03), directed edge (Weight 0.61, delay 42 days) and (Weight 0.47, delay 28 days); Node (D-03) has two incoming edges, and the node... (D-01) and (D-02) has no incoming edges and maintains its node state solely through self-updates in each round of message passing. Cascaded sensing embedding vector generation is performed in step S7. The activation feature vectors of each obstacle node are then... Loaded to the corresponding node in the temporal causal comorbidity directed graph, and mapped to the initial node state vector through a linear transformation by the node state initialization unit. (Dimension) This involves using a directed graph attention network to perform unidirectional message passing along the causal direction, outputting cascaded perceptual embedding vectors for each obstacle node. In this session... seconds, for directed edges ( sky (seconds) and ( sky Calculate the cascaded delay modulation coefficients separately (seconds): ; The above results indicate that the current session duration (1320 seconds, approximately 22 minutes) is much shorter than the average cascading delay duration of the two directed edges (42 days and 28 days, respectively), and the cascading delay modulation coefficients both decay to approximately 0.37, corresponding to the suppression of the cascading propagation signal on the current timescale. After round-robin directed message passing, the node (D-03) Aggregation from and The modulated propagation intensity, and the final state vector of each node. That is, the cascaded sensing embedding vectors of each obstacle node (dimensionality) Step S8 generates multi-obstacle cascade early warning data. The cascade sensing embedding vectors of each obstacle node are then used. (Dimension 16) and activation feature vector (Dimension 11) concatenates to form a vector of dimension 27, inputs it into the corresponding deep risk prediction network, and outputs the current risk probability of each obstacle. When the conflict clustering of upstream obstacle nodes drives the activation intensity value Exceeding the preset cascading trigger threshold At that time, the cascading warning probability of downstream obstacles and the expected time window of risk escalation are calculated along the directed edges. In this session... ， All of these trigger cascading propagation calculations. (Based on nodes) (D-01) along For nodes Taking the cascaded early warning probability of (D-03) as an example, let the output of the deep risk prediction network be... ， ,but: ;node (D-03) was simultaneously and Two paths are activated, and the maximum cascading warning probability between the two paths is taken as the overall cascading warning probability. The table below shows the multi-obstacle cascading warning data finally generated in this session. As shown in Table 7, Table 7 contains the multi-obstacle cascading warning data.

[0060]

[0061] Table 7 The comprehensive cascading warning probability of node D-03 is the maximum value of the two paths, i.e., 0.22. The multi-obstacle cascading warning data finally output by this session includes: the independent risk probability of each obstacle (0.73 for D-01, 0.68 for D-02, and 0.58 for D-03), the activation obstacle sequence on the cascading propagation path (D-01→D-03 and D-02→D-03), and the triggering topic interval identifiers for each path (T-emotion regulation and T-sleep quality). This data is provided as supplementary reference information for professionals to use in subsequent evaluation processes.

[0062] To implement the above embodiments, this application also discloses an electronic device. The electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage device into a random access memory (RAM). Various programs and data required for the operation of the electronic device are also stored in the RAM. The processing unit, ROM, and RAM are interconnected via a bus. An input / output (I / O) interface is also connected to the bus. Typically, the following devices can be connected to the I / O interface: input devices including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices including, for example, magnetic tape, hard disk, etc.; and communication devices. The communication device allows the electronic device to communicate wirelessly or wiredly with other devices to exchange data. Although electronic devices with various devices are shown, it should be understood that it is not required to implement or possess all of the shown devices. More or fewer devices may be implemented or possessed alternatively.

[0063] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer storage medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, it performs the functions defined above in the methods of some embodiments of this disclosure.

[0064] It should be noted that the computer storage medium in some embodiments of this disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0065] In some embodiments of this disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer storage medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer storage medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0066] In other implementations, clients and servers may communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol), and may interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0067] The aforementioned computer storage medium may be included within the aforementioned electronic device, or it may exist independently and not assembled into the electronic device. The aforementioned computer storage medium carries one or more programs that, when executed by the electronic device, enable the electronic device to implement a deep learning-driven method for predicting mental health risks.

[0068] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0069] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions. Units described in some embodiments of the present disclosure may be implemented in software or hardware. The described units may also be located in a processor, and the names of these units do not necessarily constitute a limitation on the unit itself.

[0070] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0071] All technologies not described in detail in this invention are existing technologies. The above descriptions are merely some preferred embodiments of this disclosure and explanations of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalent features without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A deep learning-driven method for predicting mental health risks, characterized in that, Includes the following steps: Step S1: Obtain the prosodic feature vector of the audio stream and the sequence of facial motion activation intensity vectors for each frame of the video frame; Step S2: Based on the facial action activation intensity vector sequence, apply sliding window difference calculation and micro-expression classification network to generate a time-stamped micro-expression emotion event sequence; Step S3: Generate a contradictory event feature sequence based on the timestamp alignment relationship between the prosodic feature vector and the micro-expression emotional event sequence; Step S4: Perform density-based temporal clustering on the contradictory event feature sequence to generate a contradictory cluster feature sequence with topic annotations; Step S5: Based on the preset mapping relationship table between consultation topics and psychological obstacle dimensions, the dominant topic labels of each contradiction cluster feature are converted into obstacle dimension activation vectors, and then the contradiction cluster driving activation intensity value and activation feature vector of each obstacle node are generated. Step S6: Based on statistical data from clinical longitudinal cohort studies, obtain the temporal causal comorbidity conditional probability and average cascade delay duration among mental disorders to generate a temporal causal comorbidity directed graph. Step S7: Load the contradiction cluster-driven activation feature vector of each obstacle node into the corresponding node of the temporal causal comorbidity directed graph, and use the directed graph attention network to perform one-way message passing along the causal direction to output the cascaded perception embedding vector of each obstacle node. Step S8: The cascaded sensing embedding vector of each obstacle node and its contradictory clustering driven activation feature vector are concatenated and then input into the deep risk prediction network corresponding to each obstacle to generate multi-obstacle cascaded early warning data. In the directed graph of temporal causal comorbidity, the nodes correspond to the psychological disorder dimension, and the directed edges are generated based on the conditional probability of temporal onset between disorders and carry the cascaded delay duration attribute.

2. The deep learning-driven method for predicting mental health risks according to claim 1, characterized in that, The generation of a timestamped sequence of micro-expression emotion events based on the activation intensity vector sequence includes: A sliding window differential calculation is applied to the frame-by-frame facial motion unit activation intensity vector sequence to obtain the difference vector for each frame; When the difference vector of a certain frame When the norm exceeds a preset abrupt change threshold, the frame is marked as an abrupt change frame; Extract the frames before and after the sudden change frame. The activation mode difference vector within the frame range is input into the microexpression classification network to output the microexpression emotion label and confidence score; The timestamps, micro-expression emotion tags, confidence scores, and the emotion polarity vector output from the penultimate layer of the micro-expression classification network corresponding to each sudden change frame are combined into a quadruple, which is then arranged in chronological order to form the micro-expression emotion event sequence.

3. The deep learning-driven method for predicting mental health risks according to claim 2, characterized in that, The step of generating a contradictory event feature sequence based on the temporal alignment relationship between the prosodic feature vector and the micro-expression emotional events includes: The prosodic feature vectors of each speech segment are projected onto an emotion polarity space of the same dimension as the emotion polarity vector using a linear projection layer to obtain the prosodic emotion polarity vector. ; For the first The emotional polarity vectors of all micro-expression emotional events within the indexed time range of each speech segment are weighted and averaged using confidence scores to obtain the comprehensive micro-expression emotional vector. ; Calculate cross-modal inconsistency When cross-modal contradiction degree If the preset contradiction threshold is exceeded, the audio segment is marked as a contradiction event; The characteristic record of the contradictory event includes [the following]. The normalized features include the contradiction direction feature vector, contradiction intensity value, contradiction event timestamp, and current consultation topic identifier.

4. The deep learning-driven method for predicting mental health risks according to claim 1, characterized in that, The step of performing density-based temporal clustering on the contradictory event feature sequence to generate a contradictory cluster feature sequence with topic annotations includes: With adaptive time window Density-based temporal clustering is performed on the timestamps of each contradictory event in the contradictory event feature sequence, using the neighborhood radius as the basis; For each cluster interval, the contradiction density value, the mean contradiction intensity within the cluster, and the consistency coefficient of contradiction direction are calculated as cluster feature parameters. The frequency of topic identifiers for each contradiction event within each cluster is counted, and the topic identifiers with a frequency ratio exceeding a preset topic ratio threshold are selected to form the dominant topic identifier set for that cluster.

5. The deep learning-driven method for predicting mental health risks according to claim 4, characterized in that: The process of generating activation intensity values ​​and activation feature vectors for each obstacle node based on the mapping relationship between the consultation topic and the psychological obstacle dimension includes: Based on the pre-set mapping table between consultation topics and psychological obstacle dimensions, the dominant topic identifiers of each conflict cluster are converted into obstacle dimension activation sets. For the first Let each obstacle dimension be a set of contradictory clusters that activate that dimension. Based on the contradictory density values ​​of each cluster With the mean of contradiction intensity The product of these factors is used to calculate the activation strength value of the obstacle node. ; The activation feature vector of the obstacle node is calculated by concatenating the contradiction density value, the mean contradiction intensity value, the contradiction direction consistency coefficient, and the weighted average contradiction direction feature vector with the contradiction intensity value as the weight.

6. The deep learning-driven method for predicting mental health risks according to claim 1, characterized in that, The directed graph attention network includes a node state initialization unit, a directed attention calculation unit, a cascaded delay modulation unit, and a message aggregation and update unit. The step of using the directed graph attention network to output the cascaded perception embedding vectors of each obstacle node includes: The node state initialization unit maps the activation feature vectors of each obstacle node to the initial node state vector through a linear transformation. In each round of message passing, the directed attention calculation unit calculates the attention weight of the upstream node to the downstream node only along the direction of the directed edge. The attention weight is determined by combining the current state vector of the upstream node, the weight of the directed edge, and the activation intensity value of the upstream node. The cascaded delay modulation unit calculates the cascaded delay modulation coefficients based on the cascaded delay duration attribute of the directed edge and the duration of the current consultation session. The message aggregation and update unit aggregates all incoming source node messages for each downstream node after being modulated by the attention weight and the cascaded delay modulation coefficient. After concatenating the current state vector of the downstream node, the node state is updated by transformation. After a preset number of message transmissions, the cascaded perception embedding vector of each node is output.

7. The deep learning-driven method for predicting mental health risks according to claim 6, characterized in that, The generation of multi-obstacle cascaded early warning data based on the cascaded sensing embedding vector and the activation feature vector includes: The concatenated sensing embedding vectors and activation feature vectors of each obstacle node are concatenated and then input into the corresponding deep risk prediction network to output the current risk probability of each obstacle. When the activation intensity value of the upstream obstacle node exceeds the preset cascading trigger threshold, all outgoing edges of the node are traversed, and the cascading warning probability of the downstream obstacle is calculated as the product of the current risk probability of the upstream obstacle, the weight of the directed edge, and the output value after applying the sigmoid activation function to the activation intensity value of the upstream obstacle. The expected risk increase time window for each cascading path is determined as the interval from the current time to the sum of the average cascading delay time of the corresponding directed edge. When the same downstream obstacle node is activated by multiple cascading paths, the maximum value of the cascading warning probability among each path is taken as the comprehensive cascading warning probability.

8. A deep learning-driven mental health risk prediction system, used to perform the method according to any one of claims 1 to 7, characterized in that, include: A multimodal feature extraction module is used to obtain audio stream prosodic feature vectors and facial action unit activation intensity vector sequences, and generate a time-stamped micro-expression emotion event sequence based on the activation intensity vector sequence; A cross-modal contradiction detection module is used to generate a sequence of contradictory event features based on the temporal alignment relationship between the prosodic feature vector and micro-expression emotional events. The contradiction clustering analysis module is used to perform density-based temporal clustering on the contradiction event feature sequence to generate a contradiction cluster feature sequence with topic annotations; The obstacle dimension mapping module is used to generate activation intensity values ​​and activation feature vectors for each obstacle node based on the mapping relationship between the consultation topic and the psychological obstacle dimension. The directed graph propagation module is used to output the cascaded sensing embedding vectors of each obstacle node based on the directed graph attention network of the temporal causal comorbidity directed graph. The cascaded early warning generation module is used to generate multi-obstacle cascaded early warning data based on the cascaded perception embedding vector and the activation feature vector.

9. An electronic device, characterized in that, include: At least one memory for storing computer-executable instructions; At least one processor for running the computer-executable instructions. The computer-executable instructions are executed by the processor to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions that, when executed by at least one processor, implement the method according to any one of claims 1 to 7.

Images