An intelligent care human body abnormal behavior recognition system, method, computer and storage medium

Abstract

The application discloses a human abnormal behavior recognition system and method for intelligent care, a computer and a storage medium, and relates to the field of intelligent care.The application solves the problem of certain limitations of the human abnormal behavior recognition technology in the prior art in video classification.The system comprises a human skeleton information acquisition unit, a human skeleton data modeling unit and an abnormal behavior recognition unit.The human skeleton information acquisition unit is used for acquiring human skeleton joints.The human skeleton data modeling unit is used for constructing a human skeleton data model according to the acquired human skeleton joints and a double-flow space-time self-attention network.The abnormal behavior recognition unit is used for performing human abnormal behavior recognition according to the human skeleton data model.The application can realize real-time monitoring of patients by using the abnormal behavior recognition technology.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent care, and particularly to a human abnormal behavior recognition system for intelligent care. BACKGROUND

[0002] With the wide use of virtual reality technology, human-computer interaction, intelligent security and other fields in real life, the research on behavior recognition is increasingly developed. The purpose is to get the ongoing behavior action in the video or image sequence, and judge the behavior category. Behavior recognition, as a very active research object in the field of visual understanding, has a very wide application prospect. For example, video content retrieval can analyze the input original video content, and then extract key information for retrieval, query and storage, to cope with the rapid increase of data information including images and videos in the big data era. In the aspect of human-computer interaction, when the computer can understand human posture and action information, a more efficient human-computer interaction environment can be built. In the aspect of motion auxiliary analysis, after the human behavior analysis technology, the body motion of the detected target in the video can be tracked and analyzed, and the motion trajectory, angle, speed and other information of each key part can be extracted, so as to provide quantitative motion index for people, and play the role of auxiliary training and auxiliary rehabilitation.

[0003] Human abnormal behavior recognition is one of the indispensable key technologies of intelligent video monitoring system, and also belongs to the category of advanced image processing technology. It has corresponding application in many fields of life. In the field of intelligent monitoring, the abnormal behavior recognition technology can be used to realize real-time monitoring of patients, timely find the abnormality of patients, and give doctors valuable time for rescue; in the field of intelligent care, it can monitor whether the elderly living alone falls or has other accidents in real time, and timely notify family members and doctors to reduce the occurrence of tragedies. In the traditional human abnormal behavior recognition system, the video classification and detection method generally uses manually selected feature descriptors to represent video content, and then uses a classifier to classify to complete the classification and detection of the video. Common feature description methods include scale-invariant feature transform (SIFT) descriptor, histogram of oriented optical flow (HOF), etc. The static information and motion information of the video are extracted using such feature description methods, and then the bag-of-words model is used for feature coding. Finally, the feature code is sent into the classifier for video classification and detection. Common classifiers include support vector machine (SVM). The IDT (improved dense trajectories) algorithm can be regarded as a representative of the traditional method. This method was proposed by Wang et al. in 2013 and published in the ICCV conference with the title "Action Recognition with Improved Trajectories". The algorithm densely samples feature points on the video frame in the multi-scale space, tracks the feature points and extracts trajectories, HOF and other four features, and at the same time eliminates background interference trajectories and optical flow by estimating the camera motion. Then, Fisher vector is used to encode the four features to obtain IDT features. The IDT algorithm has achieved good results in traditional video classification methods and has good robustness, but the algorithm speed is slow, and the manual features are difficult to express the high-level semantic information of the video content. In the video classification of large-scale video data and a large number of semantic categories, it shows certain limitations and cannot accurately recognize human abnormal behavior. SUMMARY

[0004] The present application solves the problem of certain limitations of the prior art human abnormal behavior recognition technology in video classification.

[0005] The present application provides a human abnormal behavior recognition system for intelligent care, which comprises:

[0006] A human skeleton information acquisition unit, a human skeleton data modeling unit and an abnormal behavior recognition unit.

[0007] The human skeleton information acquisition unit is configured to acquire human skeleton joints;

[0008] The human skeleton data modeling unit is configured to construct a human skeleton data model according to the acquired human skeleton joints and a dual-flow spatio-temporal self-attention network;

[0009] The abnormal behavior recognition unit is configured to recognize human abnormal behaviors according to the human skeleton data model.

[0010] Further, a preferred embodiment is provided, wherein the human skeleton information acquisition unit comprises:

[0011] A human 33-pose-point acquisition module is configured to acquire human image information and acquire data information of 33 pose points in the human image information according to a machine learning pipeline framework Mediapipe;

[0012] A human 25-joint acquisition module is configured to merge the 33 pose points and acquire three-dimensional coordinates of 25 human joints.

[0013] Further, a preferred embodiment is provided, wherein the 33 pose points are merged by using a method of removing redundant parameters.

[0014] Further, a preferred embodiment is provided, wherein the human skeleton data modeling unit comprises:

[0015] ,

[0016] wherein, is a skeleton attention map of the t-th frame, denotes matrix transposition, and are two embedding functions, is human skeleton data of the t-th frame.

[0017] Based on the same inventive concept, the present application also provides an intelligent care human abnormal behavior recognition method, which comprises:

[0018] acquiring human skeleton joints;

[0019] constructing a human skeleton data model according to the acquired human skeleton joints and a dual-flow spatio-temporal self-attention network;

[0020] recognizing human abnormal behaviors according to the human skeleton data model.

[0021] Further, a preferred embodiment is provided, wherein the constructing a human skeleton data model according to the acquired human skeleton joints and a dual-flow spatio-temporal self-attention network comprises:

[0022] Collect human image information, and obtain data information of 33 pose points in the human image information according to a framework Mediapipe of a machine learning pipeline;

[0023] Merge the 33 pose points to obtain three-dimensional coordinates of 25 joints of a human body.

[0024] Further, a preferred embodiment is also provided, which adopts a method of removing redundant parameters to merge the 33 pose points.

[0025] Further, a preferred embodiment is also provided, which adopts a method of removing redundant parameters to merge the 33 pose points.

[0026] ,

[0027] wherein, is a skeleton attention map of the t-th frame, denotes matrix transposition, and are two embedding functions, is skeleton data of the t-th frame.

[0028] Based on the same inventive concept, the present application also provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the intelligent care human abnormal behavior recognition method according to any one of the above.

[0029] Based on the same inventive concept, the present application also provides a computer readable storage medium for storing a computer program, wherein the computer program executes the intelligent care human abnormal behavior recognition method according to any one of the above.

[0030] The present application has the advantages of:

[0031] The present application solves the problem of certain limitations of the prior art human abnormal behavior recognition technology in video classification.

[0032] The intelligent care human abnormal behavior recognition system according to the present application designs a double-flow spatio-temporal self-attention network, and models human skeleton data on a spatial flow and a time flow. On the NTURGB+D 60 skeleton data set, the recognition accuracy reaches 87.9% under the X-SUB evaluation standard, and is not affected by the limitations of video classification.

[0033] The human body abnormal behavior recognition system for intelligent care has small behavior recognition calculation resource consumption and is less affected by light, adopts skeleton information as the theme of body abnormal behavior recognition, is more stable compared with unprocessed limb information, contains higher level semantic information in small amount of data, and has strong adaptability to dynamic environment and is not disturbed by environment and background.

[0034] The application is applied to the field of intelligent care and can realize real-time monitoring of patients by using abnormal behavior recognition technology. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 A principle diagram of a human body abnormal behavior recognition system for intelligent care according to the first embodiment is shown in the figure;

[0036] Figure 2 A 33-pose point schematic diagram according to the second embodiment is shown in the figure; wherein, S0 represents a nose, S1 represents a left eye inner side, S2 represents a left eye, S3 represents a left eye outer side, S4 represents a right eye inner side, S5 represents a right eye, S6 represents a right eye outer side, S7 represents a left ear, S8 represents a right ear, S9 represents a mouth left side, S10 represents a mouth right side, S11 represents a left shoulder, S12 represents a right shoulder, S13 represents a left elbow, S14 represents a right elbow, S15 represents a left wrist, S16 represents a right wrist, S17 represents a left hand little finger, S18 represents a right hand little finger, S19 represents a left hand finger tip, S20 represents a right hand finger tip, S21 represents a left hand thumb, S22 represents a right hand thumb, S23 represents a left hip, S24 represents a right hip, S25 represents a left knee, S26 represents a right knee, S27 represents a left ankle, S28 represents a right ankle, S29 represents a left heel, S30 represents a right heel, S31 represents a left toe, and S32 represents a right toe.

[0037] Figure 3 A 25-skeleton joint point schematic diagram according to the second embodiment is shown in the figure; wherein, 1 represents a spine bottom, 2 represents a spine middle, 3 represents a neck, 4 represents a head, 5 represents a left shoulder, 6 represents a left elbow, 7 represents a left wrist, 8 represents a left hand, 9 represents a right shoulder, 10 represents a right elbow, 11 represents a right wrist, 12 represents a right hand, 13 represents a left hip, 14 represents a left knee, 15 represents a left ankle, 16 represents a left foot, 17 represents a right hip, 18 represents a right knee, 19 represents a right ankle, 20 represents a right foot, 21 represents a spine, 22 represents a left hand tip, 23 represents a left hand thumb, 24 represents a right hand tip, and 25 represents a right hand thumb.

[0038] Figure 4 A space position coding schematic diagram according to the third embodiment is shown in the figure;

[0039] Figure 5 A time position coding schematic diagram according to the third embodiment is shown in the figure;

[0040] Figure 6 A spatial attention map calculation schematic diagram according to the third embodiment;

[0041] Figure 7 A spatial attention module schematic diagram according to the third embodiment;

[0042] Figure 8 A tanh activation function schematic diagram according to the third embodiment;

[0043] Figure 9 A leaky ReLU function schematic diagram according to the third embodiment;

[0044] Figure 10 A human skeleton data model schematic diagram according to the third embodiment;

[0045] Figure 11 A network training accuracy change process schematic diagram according to the eleventh embodiment;

[0046] Figure 12 A network training loss change process schematic diagram according to the eleventh embodiment. DETAILED DESCRIPTION

[0047] In order to make the technical solutions and advantages of the present application clearer, several embodiments of the present application will be further described in detail in conjunction with the accompanying drawings. However, the following described embodiments are only several preferred embodiments of the present application, and are not used to limit the present application.

[0048] Embodiment one, see Figure 1 This embodiment describes an intelligent care human abnormal behavior recognition system. The system comprises:

[0049] a human skeleton information acquisition unit, a human skeleton data modeling unit, and an abnormal behavior recognition unit.

[0050] The human skeleton information acquisition unit is used to acquire human skeleton joints.

[0051] The human skeleton data modeling unit is used to construct a human skeleton data model according to the acquired human skeleton joints and a double-flow spatio-temporal self-attention network.

[0052] The abnormal behavior recognition unit is used to perform human abnormal behavior recognition according to the human skeleton data model.

[0053] In practical application, a USB driver-free camera is used as an external device of the human abnormal behavior recognition system for smart care to obtain video data of human behavior. Compared with the method of using radar and infrared, it saves computing resources and reduces hardware cost. OpenCV is used to call the camera to display the processed video. OpenCV, as a powerful and stable open source computer vision library, provides a convenient interface for calling the camera.

[0054] Embodiment two, see Figure 2 and Figure 3 This embodiment is a further limitation of the human abnormal behavior recognition system for smart care described in embodiment one. The human skeleton information acquisition unit comprises:

[0055] A human 33 pose point acquisition module is configured to collect human image information and acquire data information of 33 pose points in the human image information according to a machine learning pipeline framework Mediapipe.

[0056] A human 25 joint point acquisition module is configured to merge the 33 pose points and acquire three-dimensional coordinates of 25 joint points of the human body.

[0057] Specifically, the embodiment uses the MediaPipe open source library to acquire human skeleton joint point coordinates from RGB frames, and uses the human pose tracking algorithm BlazePose to infer 33 3D coordinates and background segmentation masks from RGB video frames, including: MediaPipe Pose uses a two-step detector-tracker ML process. The detector first locates the human / pose region of interest (ROI) within the frame. Then the tracker uses the ROI cropped frame as input to predict the pose landmarks and segmentation mask within the ROI. When processing RGB frames, the detector is only called at the first frame and when the tracker can no longer recognize the presence of the body pose in the previous frame. For other frames, the ROI is derived only from the pose landmarks of the previous frame. The lightweight BlazeFace model in MediaPipe Pose predicts the positions of 33 pose points. As shown in Figure 2 , it is a schematic diagram of 33 pose points, which clearly predicts two additional virtual key points, firmly describes the human center, rotation and scale as a circle.

[0058] Since the returned joint coordinates need to be modeled in the behavior recognition field to further realize the behavior recognition function, the 33 key points returned by MediaPipe need to be processed into 25 skeleton joints, which is the most mainstream in the field of behavior recognition. The 25 skeleton joints retain relatively robust human skeleton information while discarding redundant pose point information. It saves the modeling calculation cost, ensures relatively small calculation amount in the recognition process, and promotes the real-time performance of the recognition stage. The 25 human skeleton joints are as shown in Figure 3 .

[0059] Embodiment three, see Figure 4 , Figure 5 , Figure 6 , Figure 7 , Figure 8 , Figure 9 and Figure 10 explain the present embodiment. The present embodiment is a further limitation of the intelligent care human abnormal behavior recognition system described in embodiment two, which adopts a method of removing redundant parameters to combine 33 pose points. Embodiment four, the present embodiment is a further limitation of the intelligent care human abnormal behavior recognition system described in embodiment one, wherein the human skeleton data modeling unit comprises:

[0060] ,

[0061] wherein, is the skeleton attention map of the t-th frame, denotes matrix transposition, and are two embedding functions, is the skeleton data of the t-th frame.

[0062] Specifically, the original skeleton data collected is human dynamic behavior semantics composed of a series of frames, each frame containing 25 joints. Each point represents a joint of the human body in the form of 3D coordinates. The previous data-driven method of skeleton-based action recognition relies on traversal rules or manual design of graph topology to convert the original skeleton data into meaningful forms such as point sequences, pseudo images or graphs, so as to input them into deep networks (such as RNN, CNN and GCN) for feature extraction. However, it cannot be guaranteed that the manually programmed rules are the best choice for modeling the global correlation of joints. The present embodiment adopts the transformer basic module, i.e. self-attention mechanism, to learn the global dependency between input elements with lower computational complexity and better parallelism.

[0063] The skeleton data is analogous to the words in a long sentence. The relationship between a frame and all other frames in a series of frames is analogous to the relationship between a word and all other words in a long sentence. The embodiment uses a fully self-attention mechanism to model the skeleton data. The advantage of using a self-attention mechanism for skeleton data is that it does not need to know the intrinsic relationship between elements, so it has greater flexibility in constructing the intrinsic pattern relationship of dynamic skeleton data. At the same time, since the number of human joints is limited, the additional cost of using a self-attention mechanism is relatively small.

[0064] The input of the original self-attention mechanism is sequential data, while skeleton data exists in spatial and temporal dimensions, so the embodiment designs a dual-flow spatio-temporal self-attention network to decompose the self-attention mechanism into spatial attention and temporal attention, respectively, to model the spatial variation relationship of the skeleton data in the spatial flow and the time series variation relationship of the skeleton data in the temporal flow.

[0065] The input of the self-attention mechanism consists of a set of queries Q, keys K and values V of dimension C, which are packaged in matrix form for fast calculation. First, the dot product of all the keys of the query is calculated, and each key is divided by Then, the softmax function is used to obtain the weight of the value, and the calculation method is shown in the following formula:

[0066] ,

[0067] Where, represents the self-attention mechanism.

[0068] In the transformer, since convolution and recurrent neural networks are not used, position information needs to be added for each word to preserve the input sequence of the word. The sine and cosine functions are used for word position encoding, as shown in the following formula, which adds position encoding to the input at the bottom of the encoder and decoder stack of transform:

[0069]

[0070]

[0071] Where pos is the position of the word, and i is the dimension.

[0072] The skeleton joints are organized into a tensor that is input into the neural network. Because each element of the tensor has no predefined order or structure to show its identity (e.g., joint index or frame index), a position encoding module is needed to provide a unique marker for each joint. Different frequency sine and cosine functions are used as encoding functions, as shown in formulas 3-4 and 3-5.

[0073]

[0074]

[0075] pos represents the position of the element, i represents the dimension of the position encoding vector. However, the input of the skeleton data has two dimensions, namely space and time. The skeleton data input in the spatial stream of the dual-flow spatio-temporal self-attention network is spatially position encoded, and the skeleton input data is temporally position encoded in the time stream. Taking spatial position encoding as an example, the joints in the same frame are sequentially encoded, and the same joints in different frames have the same encoding. That is, for the first frame, the positions are 1, 2, 3, and for the second frame, they are also 1, 2, 3.

[0076] For temporal position encoding, it is a reverse analogy, that is, the joints in the same frame have the same encoding, and the same joints in different frames are sequentially encoded, and finally the position feature is added to the input data. As shown in Figure 4 and Figure 5 are spatial and temporal position encoding diagrams, respectively. In this way, each element is aligned with a unique marker, which better learns the mutual relationship between the joints and also better expresses the differences in space and time.

[0077] For dynamic skeleton data, the input network is a three-order tensor , where T represents the number of frames, N represents the number of elements, and C represents the number of channels. The attention map calculation strategy of human skeleton data is as shown in Figure 6 .

[0078] Only the joints in the same frame are used to calculate the attention map, and then the obtained attention maps of all frames are averaged and shared. It is equivalent to adding a time consistency constraint to the attention map calculation, which can to some extent reduce the overfitting problem caused by the point product calculation modeling.

[0079] By connecting T frames into an N x TC matrix, the sum of matrix multiplication can be effectively realized through a large matrix multiplication operation. The specific calculation of the spatial attention map is shown in the following formula:

[0080]

[0081] , where is the skeleton attention map of the t-th frame, represents the matrix transpose, and are two embedding functions, is the skeleton data of the t-th frame. The calculation of the temporal attention map is to calculate the attention map of the same joint node with all other frame joint nodes.

[0082] The spatial attention module is constructed according to the spatial attention map, and a schematic diagram of the constructed spatial attention module is as shown in Figure 7 The input of the spatial stream is a tensor of After the spatial attention module, the shape of the output matrix is First, the spatial position encoding is added to the input. Then the input is input to two linear mapping functions of The tensor output from the linear layer is similar to K and V in the transformer. Unlike the transformer, a Tanh activation function is used. Because the output of Tanh is not limited to positive values, negative relationships can be generated to provide greater flexibility, and finally the attention map is multiplied with the original input to obtain the output feature. The Tanh activation function is as shown in Figure 8 . is greater than to remove feature redundancy and reduce the amount of calculation.

[0083] The calculation process of the single-head self-attention module before the multi-head attention operation, a total of 3 single-head self-attention modules are spliced together through the concat operation to allow the model to jointly focus on information from different representation subspaces. The result of splicing all the single-head self-attention maps is mapped to the output space with a linear layer. The linear module is composed of a leakyReLU nonlinear function, a linear layer, and a Batch Norm layer. The leaky ReLU nonlinear function gives all negative values a non-zero slope, as shown in Figure 9 The linear layer further expresses and extracts the features of the attention map. The normalization used in the transformer is LN, which is performed for a single training sample and does not depend on other data. It considers the average input value and input variance of all dimensions of a layer, and then uses the same normalization operation to transform the input of each dimension, which is suitable for natural language processing. The normalization used here is BN, which normalizes for a single neuron, and uses the data of a mini-batch during network training to calculate the mean and variance of the neuron .

[0084] There is a residual connection before and after the input of the linear module, which is used to stabilize the training of the network and introduce different features, and saves the features of the low-level network.

[0085] The input of the time attention module in the time stream is a tensor of After the spatial attention module, the shape of the output matrix is The calculation processes of the remaining modules are consistent with the calculation process of the spatial stream.

[0086] In summary, the human skeleton data model as shown in Figure 10 includes 25 joints, T frames, and a skeleton sequence of 3 channels (x, y, z three-dimensional coordinates). In each layer of the spatial stream, the input is first regarded as a 25 x TC matrix, i.e., N elements with TC channels, and is fed into the spatial attention module to model the spatial relationship between the joints. In each layer of the temporal stream, the input is first regarded as a T x 25C matrix, i.e., T elements with NC channels, and is fed into the temporal attention module to model the temporal relationship between the frames. Each stream is stacked with 8 layers to update the features. The output features of the two streams are spliced, the features of the two streams are fused using a linear layer, and the final fused features are globally averaged pooled and fed into a fully connected layer to obtain the behavior classification result.

[0087] The temporal stream and the spatial stream of the dual-stream spatio-temporal self-attention network have exactly the same number of channels in the same stacked layer, as shown in Table 1:

[0088] Table 1 Channel parameters of the dual-stream spatio-temporal self-attention network

[0089]

[0090] Embodiment five, the human abnormal behavior recognition method for intelligent care according to the embodiment includes:

[0091] obtaining human skeleton joint nodes;

[0092] constructing a human skeleton data model according to the obtained human skeleton joint nodes and the dual-stream spatio-temporal self-attention network;

[0093] recognizing human abnormal behaviors according to the human skeleton data model.

[0094] Embodiment six, the embodiment is a further limitation of the human abnormal behavior recognition method for intelligent care according to embodiment five, and constructing a human skeleton data model according to the obtained human skeleton joint nodes and the dual-stream spatio-temporal self-attention network includes:

[0095] collecting human image information, and obtaining data information of 33 pose points in the human image information according to the framework Mediapipe of the machine learning pipeline;

[0096] merging the 33 pose points to obtain three-dimensional coordinates of 25 human joint nodes.

[0097] Embodiment seven, the embodiment is a further limitation of the human abnormal behavior recognition method for intelligent care according to embodiment six, and the 33 pose points are merged by using the method of removing redundant parameters.

[0098] Embodiment eight, the embodiment is further limited to the method for identifying abnormal behavior of human body of intelligent care described in embodiment five, the human body skeleton data modeling unit comprises:

[0099] ,

[0100] wherein, is the skeleton attention map of the t-th frame, denotes the matrix transpose, and are two embedding functions, is the skeleton data of the t-th frame.

[0101] Embodiment nine, the computer device comprises a memory and a processor, the memory stores a computer program, when the processor runs the computer program stored in the memory, the processor executes the method for identifying abnormal behavior of human body of intelligent care according to any one of embodiments five to seven.

[0102] Embodiment ten, the computer readable storage medium stores a computer program, the computer program executes the method for identifying abnormal behavior of human body of intelligent care according to any one of embodiments five to seven.

[0103] Embodiment eleven, see Figure 11 and Figure 12 This embodiment is described. This embodiment provides a specific embodiment for the system for identifying abnormal behavior of human body of intelligent care described in embodiment one, and also serves to explain embodiments two to four, specifically:

[0104] 1. Establishing human abnormal behavior dataset

[0105] The established dataset contains 8 kinds of human abnormal behaviors of headache, chest pain, nausea and vomiting, neck pain, hitting, kicking, falling down and pushing, and one normal behavior of sitting. Each behavior has 180 training samples and 60 verification samples.

[0106] 2. Accuracy test of human abnormal behavior recognition system

[0107] The experimental environment is Pytorch1.10 framework, RTX3090 graphics card and Pycharm community edition software. The batch size is 32, the learning rate is 0.01, and the learning rate is reduced to 0.001 at the 35th epoch. The Adam optimizer and the Cross Entropy loss function are trained for 50 epochs. The accuracy of the network on the training set and the test set during the training process is as follows: Figure 11As shown, Loss changes as Figure 12 As shown.

[0108] From Figure 11 It can be seen that with the increase of the number of iteration steps, the accuracy on the training set gradually rises to 1, and the accuracy on the test set also gradually rises, with the highest accuracy of 98.52%.

[0109] From Figure 12 It can be seen that the Loss on the test set gradually decreases to 0.066 with the increase of the number of iteration steps.

[0110] Table 2 Human behavior recognition accuracy confusion matrix

[0111]

[0112] From Table 2, it can be seen that the network performs well on the data set, and even for behaviors with low recognition degree such as 'punching', the accuracy can reach 95%.

[0113] Five volunteers were selected to test the real-time human abnormal behavior recognition function, each volunteer performed 10 different behaviors from 3 angles.

[0114] Table 3 Human abnormal behavior recognition real-time test results

[0115]

[0116] From the table, it can be seen that the recognition degree of punching is lower and it is the most difficult to recognize.

[0117] In summary, the human abnormal behavior recognition system of the intelligent care provided by the embodiment has high recognition accuracy and is suitable for a wide range of scenarios.

[0118] The application is described in detail through specific embodiments, but the above description is only a preferred embodiment of the application

[0119] and does not limit the application. Any modification, implementation

[0120] combination, equivalent replacement and improvement within the spirit and principle of the application should be included in the protection scope of the application.

Claims

1. A smart monitoring system for recognizing abnormal human behavior, characterized in that, The system includes: Human skeleton information acquisition unit, human skeleton data modeling unit, and abnormal behavior recognition unit; The human skeleton information acquisition unit is used to acquire human skeletal joint points; the human skeleton information acquisition unit includes: The module for acquiring 33 human pose points is used to collect human image information and obtain the data information of 33 pose points in the human image information according to the Mediapipe framework of the machine learning pipeline. The human body 25 joint point acquisition module is used to merge 33 posture points to obtain the three-dimensional coordinates of 25 human body joint points; The human skeleton data modeling unit is used to construct a human skeleton data model based on the acquired human skeletal joints and the dual-stream spatiotemporal self-attention network. The abnormal behavior recognition unit is used to recognize abnormal human behavior based on the human skeleton data model. The dual-stream spatiotemporal self-attention network decomposes the self-attention mechanism into spatial attention and temporal attention, respectively modeling the spatial variation relationship of skeletal data in the spatial stream and modeling the temporal variation relationship of skeletal data in the temporal stream. In each layer of the spatial flow, the input is first treated as a 25×TC matrix, i.e., N elements with TC channels, and fed into the spatial attention module to model the spatial relationships between joints. In each layer of the temporal flow, the input is first treated as a T×25C matrix, i.e., T elements with NC channels, and fed into the temporal attention module to model the temporal relationships between frames. Each flow is stacked with 8 layers to update features. The output features of the two flows are concatenated, and the features of the two flows are fused using a linear layer. The final fused features are then global average pooled and fed into a fully connected layer to obtain the behavior classification results. In the spatial stream of the dual-stream spatiotemporal self-attention network, the skeletal data input is spatially encoded, and in the temporal stream, the skeletal input data is temporally encoded. For spatial encoding, joints in the same frame are encoded sequentially, and the same joints in different frames have the same encoding. For temporal encoding, joints in the same frame have the same encoding, and the same joints in different frames are encoded sequentially. Finally, the positional features are added to the input data. The human skeleton data modeling unit includes: ， in, Here is the skeleton attention map for frame t. Indicates matrix transpose. and These are two embedded functions. This represents the skeleton data for frame t.

2. The intelligent care system for recognizing abnormal human behavior according to claim 1, characterized in that, The 33 attitude points were merged by removing redundant parameters.

3. A method for recognizing abnormal human behavior in intelligent care, characterized in that, The method includes: Obtain the joint points of the human skeleton; A human skeleton data model is constructed based on the obtained human skeletal joints and the dual-stream spatiotemporal self-attention network; Human abnormal behavior is identified based on the aforementioned human skeleton data model; The dual-stream spatiotemporal self-attention network decomposes the self-attention mechanism into spatial attention and temporal attention, respectively modeling the spatial variation relationship of skeletal data in the spatial stream and modeling the temporal variation relationship of skeletal data in the temporal stream. In each layer of the spatial flow, the input is first treated as a 25×TC matrix, i.e., N elements with TC channels, and fed into the spatial attention module to model the spatial relationships between joints. In each layer of the temporal flow, the input is first treated as a T×25C matrix, i.e., T elements with NC channels, and fed into the temporal attention module to model the temporal relationships between frames. Each flow is stacked with 8 layers to update features. The output features of the two flows are concatenated, and the features of the two flows are fused using a linear layer. The final fused features are then global average pooled and fed into a fully connected layer to obtain the behavior classification results. In a dual-stream spatiotemporal self-attention network, spatial position encoding is performed on the skeletal input data in the spatial stream, and temporal position encoding is performed on the skeletal input data in the temporal stream. For spatial position encoding, joints in the same frame are encoded sequentially, and the same joints in different frames have the same encoding. For temporal position encoding, joints in the same frame have the same encoding, and the same joints in different frames are encoded sequentially. Finally, position features are added to the input data. The construction of the human skeleton data model based on the acquired human skeletal joints and the dual-stream spatiotemporal self-attention network includes: Collect human image information and obtain data information of 33 pose points in the human image information according to the Mediapipe framework of the machine learning pipeline; Merge 33 attitude points to obtain the three-dimensional coordinates of 25 joints of the human body; The construction of the human skeleton data model based on the acquired human skeletal joints and the dual-stream spatiotemporal self-attention network includes: ， in, Here is the skeleton attention map for frame t. Indicates matrix transpose. and These are two embedded functions. This represents the skeleton data for frame t.

4. The method for identifying abnormal human behavior in intelligent care according to claim 3, characterized in that, The 33 attitude points were merged by removing redundant parameters.

5. A computer device, characterized in that: It includes a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes a method for identifying abnormal human behavior in intelligent care according to any one of claims 3-4.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program that executes a method for identifying abnormal human behavior in intelligent care as described in any one of claims 3-4.

Images