Psychiatric patient dangerous behavior identification system

Abstract

The invention relates to the field of intelligent behavior recognition, in particular to a psychiatric patient dangerous behavior recognition system which comprises a data acquisition module, an intelligent analysis module, a dangerous behavior recognition module, an early warning prompt module and a user control module. According to the method, a pre-trained multi-scale action self-supervision recognition method based on multi-head attention is creatively adopted, better context semantics and cross-level features are obtained through multi-head attention, cross-frame and cross-node modes, and accurate and rapid recognition of patient body features is achieved; according to the method, a fusion multi-head attention model based on optical flow estimation is creatively adopted, more valuable facial features are extracted through multi-convolution integral branching of an optical flow graph, extraction of multi-scale facial micro-features and embedding of a channel attention SE module and a multi-head self-attention processing module, and finally, a facial-body-behavior combined recognition method is adopted, so that the multi-head self-attention recognition method is realized. And comprehensive and accurate identification of dangerous behaviors of mental patients is realized.

Description

Technical Field

[0001] The invention belongs to the field of intelligent behavior recognition, in particular to a dangerous behavior recognition system for psychiatric patients. Background Art

[0002] Psychiatric patients may exhibit various dangerous behaviors, such as self-harm, suicide, and aggressive behaviors. These behaviors not only harm the patients themselves, but may also affect the safety of medical staff and other patients. Timely identification and intervention of patients' dangerous behaviors can effectively reduce the risk of injury to patients and avoid adverse consequences. The existing dangerous behavior recognition system has a certain lag in real-time and response speed, and cannot accurately identify patients' dangerous behaviors in a timely manner. At the same time, the behavior of psychiatric patients is often affected by multiple factors, including individual differences, environmental factors, etc. A single model cannot fully and accurately identify all dangerous behaviors. Summary of the invention

[0003] In view of the above situation, in order to overcome the defects of the existing technology, the existing dangerous behavior recognition system has a certain lag in real-time and response speed, and cannot timely and accurately identify the patient's dangerous behavior. This paper creatively adopts a multi-scale action self-supervised recognition method based on multi-head attention, and integrates the motion characteristics of the multi-scale skeleton into the multi-head attention mechanism. Through cross-frame and cross-node methods, better contextual semantics and cross-level features are obtained to achieve accurate recognition of the patient's physical characteristics. The patient behavior recognition model is pre-trained with historical data, and the historical patient behavior is feature-recognized and analyzed, and compared with the real-time behavior characteristics to increase the response speed. The behavior of psychiatric patients is often affected by multiple factors. Including individual differences, environmental factors, etc., a single model cannot comprehensively and accurately identify all dangerous behaviors. The present invention creatively adopts a face-body-behavior joint recognition method. While identifying the patient's physical features, its facial expression features are introduced for joint behavior analysis. At the same time, the present invention creatively adopts a fusion multi-head attention model based on optical flow estimation. On the basis of the facial optical flow map, multi-convolution branches are introduced to extract multi-scale facial micro-features, and a channel attention SE module and a multi-head self-attention processing module are embedded to extract more valuable facial features and realize accurate recognition of facial features. Finally, through accurate facial and body features, the accuracy of the face-body-behavior joint recognition method is improved.

[0004] The technical solution provided by the present invention is as follows: a system for identifying dangerous behaviors of psychiatric patients, comprising a data acquisition module, an intelligent analysis module, a dangerous behavior identification module, an early warning prompt module and a user control module;

[0005] The data acquisition module collects historical and real-time facial time-series RGB images and whole-body time-series RGB images of psychiatric patients through a high-definition camera, and sends the collected images to the intelligent analysis module;

[0006] The intelligent analysis module includes a facial feature extraction module, a body feature extraction module and a comprehensive intelligent analysis module;

[0007] The facial feature extraction module extracts features from the facial time series RGB image through a multi-head attention model based on optical flow estimation to obtain facial features;

[0008] The body feature extraction module extracts features from the whole-body time-series RGB image through a multi-scale action self-supervised recognition method based on multi-head attention to obtain body features;

[0009] The comprehensive intelligent analysis module matches the historical facial features and body features with the historical real behaviors of the psychiatric patient to obtain the face-body-behavior analysis results, and sends the analysis results to the dangerous behavior identification module;

[0010] The dangerous behavior recognition module extracts features from the real-time collected facial and whole-body RGB images of the psychiatric patient, and compares them with the analysis results to obtain a comprehensive recognition result and send it to the early warning prompt module;

[0011] The early warning module issues early warning information in a timely manner when the comprehensive identification results show that the patient has dangerous behavior;

[0012] The user control module directly controls all modules, and performs information viewing and data storage.

[0013] Furthermore, in the facial feature extraction module, the facial time series RGB image is subjected to feature extraction through a fusion multi-head attention model based on optical flow estimation to obtain facial features, which specifically includes the following steps:

[0014] Step S1: Image preprocessing: cropping the facial time series RGB image to a fixed size and performing cubic difference normalization to obtain a facial time series grayscale image;

[0015] Step S2: constructing an optical flow image, using an optical flow estimation function to construct an optical flow map based on the grayscale image;

[0016] Step S3: Channel attention module processing, so that the 3*3 convolutional neural network with the channel attention SE module extracts features from the optical flow map to obtain a channel attention feature map;

[0017] Step S4: Multi-head self-attention processing, after performing maximum pooling on the channel attention feature map, use the multi-head self-attention network to process it and obtain a 3*3 channel-level global feature: ; ; ; ;

[0018] In the formula, represents the position encoding function, represents the position variable, Represents the dimension of the feature map, Represents the preset adjustment parameters, The pixel position representing the feature map is the horizontal and vertical coordinates. represents the positional encoding of the feature map, represents a multi-head self-attention network, Represents query, key and value processing respectively. Represents the transpose of a matrix;

[0019] Step S5: multi-branch processing, repeating steps S3 and S4, changing the convolution kernel of the convolutional neural network from 3*3 to 5*5 and 7*7 respectively, to obtain 5*5 channel-level global features and 7*7 channel-level global features;

[0020] Step S6: feature fusion, the 3*3 channel-level global features, the 5*5 channel-level global features and the 7*7 channel-level global features are processed by maximum pooling respectively, and then feature connection is performed to obtain facial features.

[0021] Furthermore, in the body feature extraction module, a multi-scale action self-supervised recognition method based on multi-head attention is used to extract features from the whole body time series RGB image to obtain body features, which specifically includes the following steps:

[0022] Step Q1: Extract joint points. Use the KeyPoint-RCNN model to extract joint points from the whole-body time-series RGB image of the mental patient to obtain n joint points.

[0023] Step Q2: constructing a human joint model, connecting the joint points according to the human body structure to obtain a human joint model;

[0024] Step Q3: multi-head attention processing, extracting query encoder and key encoder features of the human joint model to obtain joint query features and joint key features, where the dimension of the joint query feature is n;

[0025] Step Q4: Enhanced feature extraction, multi-scale enhanced feature extraction is performed on the human joint model to obtain body features, which specifically includes the following steps:

[0026] Step Q41: compressing the joint query features in different dimensions by averaging the joint query features to obtain part-level features and human-level features;

[0027] Step Q42: convolution operation, convolving the joint query feature to obtain the joint convolution feature;

[0028] Step Q43: Calculate the inter-frame difference feature by subtracting the joint convolution feature of the current moment of the whole body time series RGB image from the joint convolution feature of the previous moment to obtain the inter-frame difference feature: ;

[0029] In the formula, represents the inter-frame difference feature, Represents the joint convolution feature at the current moment, Represents the joint convolution feature of the previous moment, Represents the sampling frequency of the whole body time series RGB image;

[0030] Step Q44: Calculate the difference feature between nodes, obtain the joint convolution feature with dimension n-1 by reducing the dimension of the joint convolution feature, subtract the joint convolution feature with dimension n from the joint convolution feature with dimension n-1, and obtain the difference feature between nodes;

[0031] Step Q45: difference feature fusion, adding the inter-frame difference feature and the inter-node difference feature to obtain the average value and performing self-attention mechanism processing, then adding and activating the joint convolution feature to obtain the joint activation feature;

[0032] Step Q46: multiply the joint activation feature by the joint query feature and then add the resultant result to the joint query feature to obtain a joint multi-scale enhanced feature;

[0033] Step Q47: Repeat steps Q42 to Q46, taking the part-level features and the human-level features as inputs, respectively, to obtain part-level multi-scale enhanced features and human-level multi-scale enhanced features;

[0034] Step Q48: Multi-scale feature fusion, fusing joint multi-scale enhanced features, part-level multi-scale enhanced features, human-level multi-scale enhanced features, joint query features and joint key features to obtain the physical features of the mental patient.

[0035] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0036] (1) In view of the problem that the existing dangerous behavior recognition system has a certain lag in real-time and response speed, and cannot timely and accurately identify the patient's dangerous behavior, this paper creatively adopts a multi-scale action self-supervised recognition method based on multi-head attention, integrates the motion features of the multi-scale skeleton into the multi-head attention mechanism, obtains better contextual semantics and cross-level features through cross-frame and cross-node methods, realizes accurate recognition of the patient's physical features, and uses historical data to pre-train the patient behavior recognition model, performs feature recognition analysis on historical patient behaviors, and compares them with real-time behavior features to increase the response speed;

[0037] (2) The behavior of psychiatric patients is often affected by multiple factors, including individual differences, environmental factors, etc. A single model cannot fully and accurately identify all dangerous behaviors. The present invention creatively adopts a face-body-behavior joint recognition method. While identifying the patient's physical features, it introduces their facial expression features for joint behavior analysis. At the same time, the present invention creatively adopts a fusion multi-head attention model based on optical flow estimation. On the basis of the facial optical flow map, it introduces multiple convolution branches to extract multi-scale facial micro-features, and embeds a channel attention SE module and a multi-head self-attention processing module to extract more valuable facial features and achieve accurate recognition of facial features. Finally, through accurate facial and body features, the accuracy of the face-body-behavior joint recognition method is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A module diagram of a dangerous behavior identification system for psychiatric patients provided by the present invention;

[0039] Figure 2 Schematic diagram of the facial feature extraction process.

[0040] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0041] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0042] Example 1, see Figure 1, a dangerous behavior identification system for psychiatric patients provided by the present invention includes a data acquisition module, an intelligent analysis module, a dangerous behavior identification module, an early warning prompt module and a user control module;

[0043] The data acquisition module collects historical and real-time facial time-series RGB images and whole-body time-series RGB images of psychiatric patients through a high-definition camera, and sends the collected images to the intelligent analysis module;

[0044] The intelligent analysis module includes a facial feature extraction module, a body feature extraction module and a comprehensive intelligent analysis module;

[0045] The facial feature extraction module extracts features from the facial time series RGB image through a multi-head attention model based on optical flow estimation to obtain facial features;

[0046] The body feature extraction module extracts features from the whole-body time-series RGB image through a multi-scale action self-supervised recognition method based on multi-head attention to obtain body features;

[0047] The comprehensive intelligent analysis module matches the historical facial features and body features with the historical real behaviors of the psychiatric patient to obtain the face-body-behavior analysis results, and sends the analysis results to the dangerous behavior identification module;

[0048] The dangerous behavior recognition module extracts features from the real-time collected facial and whole-body RGB images of the psychiatric patient, and compares them with the analysis results to obtain a comprehensive recognition result and send it to the early warning prompt module;

[0049] The early warning module issues early warning information in a timely manner when the comprehensive identification results show that the patient has dangerous behavior;

[0050] The user control module directly controls all modules, and performs information viewing and data storage.

[0051] By performing the above operations, in order to address the problem that the existing dangerous behavior recognition system has a certain lag in real-time and response speed and cannot timely and accurately identify the patient's dangerous behavior, this paper creatively adopts a multi-scale action self-supervised recognition method based on multi-head attention, integrates the motion features of the multi-scale skeleton into the multi-head attention mechanism, and obtains better contextual semantics and cross-level features through cross-frame and cross-node methods, so as to achieve accurate recognition of the patient's physical features, and uses historical data to pre-train the patient behavior recognition model, perform feature recognition and analysis on historical patient behaviors, and compare them with real-time behavior features to increase the response speed.

[0052] Example 2, see Figure 2Based on the above embodiment, in the facial feature extraction module, this embodiment extracts features from the facial time series RGB image through a fusion multi-head attention model based on optical flow estimation to obtain facial features, which specifically includes the following steps:

[0053] Step S1: Image preprocessing: cropping the facial time series RGB image to a fixed size and performing cubic difference normalization to obtain a facial time series grayscale image;

[0054] Step S2: constructing an optical flow image, using an optical flow estimation function to construct an optical flow map based on the grayscale image;

[0055] Step S3: Channel attention module processing, so that the 3*3 convolutional neural network with the channel attention SE module extracts features from the optical flow map to obtain a channel attention feature map;

[0056] Step S4: Multi-head self-attention processing, after performing maximum pooling on the channel attention feature map, use the multi-head self-attention network to process it and obtain a 3*3 channel-level global feature: ; ; ; ;

[0057] In the formula, represents the position encoding function, represents the position variable, Represents the dimension of the feature map, Represents the preset adjustment parameters, The pixel position representing the feature map is the horizontal and vertical coordinates. represents the positional encoding of the feature map, represents a multi-head self-attention network, Represents query, key and value processing respectively. Represents the transpose of a matrix;

[0058] Step S5: multi-branch processing, repeating steps S3 and S4, changing the convolution kernel of the convolutional neural network from 3*3 to 5*5 and 7*7 respectively, to obtain 5*5 channel-level global features and 7*7 channel-level global features;

[0059] Step S6: feature fusion, the 3*3 channel-level global features, the 5*5 channel-level global features and the 7*7 channel-level global features are processed by maximum pooling respectively, and then feature connection is performed to obtain facial features.

[0060] Embodiment 3: Based on the above embodiment, in the body feature extraction module, this embodiment extracts features from the whole body time series RGB image through a multi-scale action self-supervised recognition method based on multi-head attention to obtain body features, which specifically includes the following steps:

[0061] Step Q1: Extract joint points. Use the KeyPoint-RCNN model to extract joint points from the full-body temporal RGB images of mental patients and obtain 17 joint points.

[0062] Step Q2: constructing a human joint model, connecting the joint points according to the human body structure to obtain a human joint model;

[0063] Step Q3: Multi-head attention processing, query encoder and key encoder feature extraction for the human joint model, to obtain joint query features and joint key features, where the joint query feature dimension is 17;

[0064] Step Q4: Enhanced feature extraction, multi-scale enhanced feature extraction of the human joint model to obtain body features

[0065] Embodiment 4: This embodiment is based on the above embodiment, and step Q4 specifically includes the following steps:

[0066] Step Q41: compressing the joint query features in different dimensions by averaging the joint query features to obtain part-level features and human-level features;

[0067] Step Q42: convolution operation, convolving the joint query feature to obtain the joint convolution feature;

[0068] Step Q43: Calculate the inter-frame difference feature by subtracting the joint convolution feature of the current moment of the whole body time series RGB image from the joint convolution feature of the previous moment to obtain the inter-frame difference feature: ;

[0069] In the formula, represents the inter-frame difference feature, Represents the joint convolution feature at the current moment, Represents the joint convolution feature of the previous moment, Represents the sampling frequency of the whole body time series RGB image;

[0070] Step Q44: Calculate the difference feature between nodes, obtain the joint convolution feature with a dimension of 16 by reducing the dimension of the joint convolution feature, subtract the joint convolution feature with a dimension of 17 from the joint convolution feature with a dimension of 16, and obtain the difference feature between nodes;

[0071] Step Q45: difference feature fusion, adding the inter-frame difference feature and the inter-node difference feature to obtain the average value and performing self-attention mechanism processing, then adding and activating the joint convolution feature to obtain the joint activation feature;

[0072] Step Q46: multiply the joint activation feature by the joint query feature and then add the resultant result to the joint query feature to obtain a joint multi-scale enhanced feature;

[0073] Step Q47: Repeat steps Q42 to Q46, taking the part-level features and the human-level features as inputs, respectively, to obtain part-level multi-scale enhanced features and human-level multi-scale enhanced features;

[0074] Step Q48: Multi-scale feature fusion, fusing joint multi-scale enhanced features, part-level multi-scale enhanced features, human-level multi-scale enhanced features, joint query features and joint key features to obtain the physical features of the mental patient.

[0075] By performing the above operations, the behavior of psychiatric patients is often affected by multiple factors, including individual differences, environmental factors, etc., and a single model cannot comprehensively and accurately identify all dangerous behaviors. The present invention creatively adopts a face-body-behavior joint recognition method. While identifying the patient's physical features, its facial expression features are introduced for joint behavior analysis. At the same time, the present invention creatively adopts a fusion multi-head attention model based on optical flow estimation. On the basis of the facial optical flow map, multi-convolution branches are introduced to extract multi-scale facial micro-features, and a channel attention SE module and a multi-head self-attention processing module are embedded to extract more valuable facial features and realize accurate recognition of facial features. Finally, the accuracy of the face-body-behavior joint recognition method is improved through accurate facial and body features.

[0076] Embodiment 5: This embodiment is based on the above embodiment, and the SE module specifically includes the following processing steps:

[0077] In the Squeeze stage, the input data is a feature map. Each feature map consists of multiple channels. Through the global average pooling operation, the feature map of each channel is compressed and the feature map values ​​on each channel are averaged to generate the global description information of each channel.

[0078] In the Excitation phase, the weight relationship between channels is learned through two fully connected layers. The role of these two fully connected layers is to learn how to assign importance weights to different channels. At the same time, a RELU activation function is added after the first fully connected layer to increase nonlinearity.

[0079] In the Scale stage, each channel in the feature map is reweighted through multiplication operations, and the learned channel weights are applied to the original feature map to obtain the reweighted feature map.

[0080] Embodiment 6, this embodiment is based on the above embodiment, the KeyPoint-RCNN model, the steps of extracting human joint points:

[0081] Step Q11: Object detection, using Faster R-CNN as the object detection branch to detect human objects in the image and generate their bounding boxes;

[0082] Step Q12: key point detection. Based on the detected human target, the key point detection branch is applied to locate the key points of the human body. Specifically, the following steps are included:

[0083] Step Q121: Feature extraction, using a convolutional neural network (CNN) to extract discriminative feature representations from the bounding box of the human target;

[0084] Step Q122: Keypoint prediction, apply a series of convolution and upsampling operations on the feature map to predict the location of each keypoint and generate a keypoint heatmap where each keypoint corresponds to a high response area.

[0085] Step Q123: keypoint decoding, decoding the keypoint heatmap into actual keypoint coordinates by applying thresholding and non-maximum suppression (NMS) on the heatmap;

[0086] Step Q13: key point post-processing, connecting key points to form poses and filtering out unreasonable key point predictions;

[0087] Step Q14: Output results, the KeyPoint-RCNN model will generate 17 joint point locations for each detected human target.

[0088] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0089] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0090] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A system for identifying dangerous behaviors of psychiatric patients, characterized by: It includes data collection module, intelligent analysis module, dangerous behavior identification module, early warning module and user control module; The data acquisition module collects historical and real-time facial time-series RGB images and whole-body time-series RGB images of psychiatric patients through a high-definition camera, and sends the collected images to the intelligent analysis module; The intelligent analysis module includes a facial feature extraction module, a body feature extraction module and a comprehensive intelligent analysis module; The facial feature extraction module extracts features from the facial time series RGB image through a multi-head attention model based on optical flow estimation to obtain facial features; The body feature extraction module extracts features from the whole-body time-series RGB image through a multi-scale action self-supervised recognition method based on multi-head attention to obtain body features; The comprehensive intelligent analysis module matches the historical facial features and body features with the historical real behaviors of the psychiatric patient to obtain the face-body-behavior analysis results, and sends the analysis results to the dangerous behavior identification module; The dangerous behavior recognition module extracts features from the real-time collected facial and whole-body RGB images of the psychiatric patient, and compares them with the analysis results to obtain a comprehensive recognition result and send it to the early warning prompt module; The early warning module issues early warning information in a timely manner when the comprehensive identification results show that the patient has dangerous behavior; The user control module directly controls all modules, and performs information viewing and data storage.

2. A system for identifying dangerous behaviors of psychiatric patients according to claim 1, characterized in that: In the facial feature extraction module, the facial time series RGB image is subjected to feature extraction through a fusion multi-head attention model based on optical flow estimation to obtain facial features, which specifically includes the following steps: Step S1: Image preprocessing: cropping the facial time series RGB image to a fixed size and performing cubic difference normalization to obtain a facial time series grayscale image; Step S2: constructing an optical flow image, using an optical flow estimation function to construct an optical flow map based on the grayscale image; Step S3: Channel attention module processing, so that the 3*3 convolutional neural network with the channel attention SE module extracts features from the optical flow map to obtain a channel attention feature map; Step S4: Multi-head self-attention processing, after performing maximum pooling on the channel attention feature map, use the multi-head self-attention network to process it and obtain a 3*3 channel-level global feature: ； ； ； ； In the formula, represents the position encoding function, represents the position variable, Represents the dimension of the feature map, Represents the preset adjustment parameters, The pixel position representing the feature map is the horizontal and vertical coordinates. represents the positional encoding of the feature map, represents a multi-head self-attention network, Represents query, key and value processing respectively. Represents the transpose of a matrix; Step S5: multi-branch processing, repeating steps S3 and S4, changing the convolution kernel of the convolutional neural network from 3*3 to 5*5 and 7*7 respectively, to obtain 5*5 channel-level global features and 7*7 channel-level global features; Step S6: feature fusion, the 3*3 channel-level global features, the 5*5 channel-level global features and the 7*7 channel-level global features are processed by maximum pooling respectively, and then feature connection is performed to obtain facial features.

3. A system for identifying dangerous behaviors of psychiatric patients according to claim 2, characterized in that: In the body feature extraction module, the feature extraction of the whole body time series RGB image is performed through the multi-scale action self-supervised recognition method based on multi-head attention to obtain the body features, which specifically includes the following steps: Step Q1: Extract joint points. Use the KeyPoint-RCNN model to extract joint points from the whole-body time-series RGB image of the mental patient to obtain n joint points. Step Q2: constructing a human joint model, connecting the joint points according to the human body structure to obtain a human joint model; Step Q3: multi-head attention processing, extracting query encoder and key encoder features of the human joint model to obtain joint query features and joint key features, where the dimension of the joint query feature is n; Step Q4: Enhanced feature extraction: multi-scale enhanced feature extraction is performed on the human joint model to obtain body features.

4. A system for identifying dangerous behaviors of psychiatric patients according to claim 3, characterized in that: Step Q4 specifically includes the following steps: Step Q41: compressing the joint query features in different dimensions by averaging the joint query features to obtain part-level features and human-level features; Step Q42: convolution operation, convolving the joint query feature to obtain the joint convolution feature; Step Q43: Calculate the inter-frame difference feature by subtracting the joint convolution feature of the current moment of the whole body time series RGB image from the joint convolution feature of the previous moment to obtain the inter-frame difference feature: ； In the formula, represents the inter-frame difference feature, Represents the joint convolution feature at the current moment, Represents the joint convolution feature of the previous moment, Represents the sampling frequency of the whole body time series RGB image; Step Q44: Calculate the difference feature between nodes, obtain the joint convolution feature with dimension n-1 by reducing the dimension of the joint convolution feature, subtract the joint convolution feature with dimension n from the joint convolution feature with dimension n-1, and obtain the difference feature between nodes; Step Q45: difference feature fusion, adding the inter-frame difference feature and the inter-node difference feature to obtain the average value and performing self-attention mechanism processing, then adding and activating the joint convolution feature to obtain the joint activation feature; Step Q46: multiply the joint activation feature by the joint query feature and then add the resultant result to the joint query feature to obtain a joint multi-scale enhanced feature; Step Q47: Repeat steps Q42 to Q46, taking the part-level features and the human-level features as inputs, respectively, to obtain part-level multi-scale enhanced features and human-level multi-scale enhanced features; Step Q48: Multi-scale feature fusion, fusing joint multi-scale enhanced features, part-level multi-scale enhanced features, human-level multi-scale enhanced features, joint query features and joint key features to obtain the physical features of the mental patient.

Images