A method for identifying and alarming violence based on space-time information

Abstract

The application discloses a violence behavior recognition and alarm method based on space-time information, comprising the following steps: constructing a physical layer: collecting action videos and performing data transmission; constructing a data processing layer: performing division processing on the data obtained in the physical layer and generating an action feature map; calculating space-time feature information in human motion according to the action feature map obtained in the data processing layer; constructing a behavior recognition layer: first, using a composite neural network to construct a recognition model; then, training the recognition model; finally, inputting to-be-recognized data into the trained recognition model to obtain an output result; constructing a control layer: establishing a matching relationship of an action behavior and an alarm association library to obtain an automatic recognition instruction, and triggering the automatic recognition instruction according to the output result of the behavior recognition layer. The alarm recognition method provides context information for behavior recognition, effectively distinguishes similar actions, solves the lag problem of violence behavior recognition, and improves the reliability and real-time performance of an intelligent alarm system.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of behavior recognition, and particularly relates to a violence behavior recognition and alarm method based on space-time information. BACKGROUND

[0002] Fast recognition of violence behavior is a powerful guarantee for avoiding personnel casualties that may cause social, economic and ecological damage. The method based on manual features mainly focuses on making local or global feature descriptors to extract space-time features, wherein the spatial features are mainly motion features related to speed and acceleration. In the basic process of recognition, features are extracted through feature descriptors, and then redundant information in the features is removed through bag-of-words or principal component analysis, and finally sent to a support vector machine or a random forest model for classification.

[0003] With the continuous maturity of deep learning technology and the proposal of large violence data sets, the deep convolution neural network (DCNN) has made remarkable achievements in the violence behavior recognition task, and has become the main research direction in this field. Unlike manual features, deep learning technology automatically learns space-time features through data sets and neural networks, and then classifies violence through fully connected layers.

[0004] Recently, people realize that CNN and RNN usually represent the skeleton sequence as a vector of joint coordinates or a pseudo image applying convolution operation, which cannot capture the spatial structure of human bones. Due to the rapid development of human behavior recognition technology in the field of behavior recognition, the accuracy of behavior recognition based on human bone information is also continuously improved. In the field of public security, there is no standard for the monitoring and alarm system based on real-time recognition of human bone information, which is a problem to be developed and solved. Existing human behavior methods mainly focus on capturing the space-time information of joints and bones during human motion from the result, and ignore the influence of joints on bone information during training, thereby ignoring many potential motion features. The violence behavior alarm system based on behavior recognition often triggers false alarm instructions due to the misjudgment of similar behaviors by the system. SUMMARY

[0005] The purpose of the present application is to provide a violence behavior recognition and alarm method based on space-time information, which solves the lag problem of violence behavior recognition through calculation of space-time information with high correlation, and improves the reliability and real-time performance of the intelligent alarm system.

[0006] In order to achieve the above purpose, the solution of the present application is as follows:

[0007] A violence behavior recognition and alarm method based on space-time information, comprising the following steps:

[0008] Step 1, constructing a physical layer: collecting action videos and transmitting data;

[0009] Step 2, constructing a data processing layer: receiving data transmitted by the physical layer and dividing the data obtained in the physical layer to obtain an action feature map;

[0010] Step 3, calculating spatiotemporal feature information in human motion according to the action feature map obtained by the data processing layer;

[0011] Step 4, constructing a behavior recognition layer: first, using a composite neural network to construct a recognition model; then, training the recognition model; finally, inputting the data to be recognized into the trained recognition model to obtain an output result;

[0012] Step 5, constructing a control layer: establishing a matching relationship of the action behavior and the alarm association library to obtain an automatic recognition instruction, and triggering the automatic recognition instruction according to the output result of the behavior recognition layer.

[0013] In the above step 1, the physical layer includes a camera, a communication device and a basic hardware facility based on a depth camera; the action video includes but is not limited to the actions of waving hands, lifting hands, punching and kicking legs.

[0014] The specific content of the above step 2 is to define the input global feature matrix:

[0015] X∈R C×V×T

[0016] Wherein V∈{v1,v2…v V} represents a vertex set, C is the number of channels, T is the number of frames, and R is the number of samples;

[0017] The human skeleton is divided into 9 parts, including head, upper limb trunk, left arm, right arm, left hand, right hand, lower limb, left leg and right leg, to obtain a joint feature matrix:

[0018] X∈R N×C×T

[0019] Wherein X p represents a feature matrix corresponding to the divided part, and N represents the number of joints corresponding to the part;

[0020] The feature information of each part is represented as:

[0021]

[0022] Wherein, i=1,2,…,V, f(.) represents an aggregation function, P(i) represents a set of joint nodes of each part, V represents the number of nodes, represents the feature of each part, and X k represents the initial feature matrix of each part.

[0023] In step 3 above, a feature extraction network is first constructed: features are extracted from the action feature map obtained in step 2, and then a skeletal joint fusion information neural network and a channel sharing neural network are constructed to calculate the spatiotemporal feature information in human movement.

[0024] The specific content of constructing the above-mentioned skeletal joint fusion information neural network is as follows: Define the function between joint motion and bone motion, as shown in the following formula:

[0025] Δx′ j =f j (Δx j ,x b );

[0026] Δx′ b =f b (Δx b ,x j );

[0027] Where, Δx j Information representing a joint, x b This represents the bone vector connecting this joint, Δx′. j This indicates that the skeletal vector x b and the corresponding join function f j x updated by (.) j Motion information of adjacent joints, Δx b Represents information about a skeleton; x j This represents the joint vector connecting this bone, Δx′. b This indicates that the joint vector x j and the corresponding join function f b x updated by (.) b Motion information of adjacent bones;

[0028] New motion features are generated by using information from joints and updated bones, as shown in the following formula:

[0029]

[0030] Among them, A k′ Let A represent the adjacency matrix of the joints. k′ =A k +B k +C k ∈R n×n B k Let C represent the adaptive adjacency matrix. k V represents a normalized adjacency matrix of feature correlations. k′ It is the adjacency matrix between bones, V k′ =Vk +Y k +Z k ∈R n ×n , f(.) represents a spatial transfer function, W k represents a learnable weight, Y k represents a skeleton adaptive adjacency matrix, Z k represents a normalized skeleton feature related adjacency matrix;

[0031] The fused joint information is input to the bidirectional long short-term memory network at a fixed frame interval, a double-layer structure is adopted to extract the time information of the human body motion, a 1*1 convolution is used for down-sampling to reduce the time dimension of the input and output, and then the joint and bone motion information in time is obtained through the bidirectional long short-term memory network, and the formula is as follows:

[0032] x' = f BL-2 (f BL-1 (x k , W1), W2) k = 1, 2, …, K

[0033] Wherein, f BL-2 (.) and f BL-1 (.) represent two identical bidirectional long short-term memory networks, W1 and W2 represent the weights of the two functions respectively.

[0034] The specific content of constructing the above channel sharing neural network is: the joints and bones of the 9 parts divided in step 2 are classified, which are represented as joints themselves, centripetal set and centrifugal set; bone itself, centripetal set and centrifugal set, convolution operation is performed on the 9 parts respectively, P1, P2 and P3 matrices are obtained respectively, the matrix features of each channel are extracted and spliced to obtain comprehensive human joint and bone features, and the formula is as follows:

[0035]

[0036] Wherein, a ij represents a manually defined or set trainable parameter, W1 represents the weight of the channel part in human body division, W2 represents the global human body weight, A1 represents the adjacency matrix of the global human body, β represents an adjustable parameter, F(.) represents a connection function, and a l represents a trainable parameter;

[0037] According to the fused information, the global information of human motion is adjusted in real time, the global human motion information A1W2 and an adjustable parameter β are used to weigh each part of information, and finally the total feature of human joints and bones is obtained.

[0038] In the step 4, the composite neural network includes a skeleton joint fusion information neural network and a channel sharing neural network, and the skeleton joint fusion information neural network and the channel sharing neural network each include a space modeling module, a space-time modeling module and a residual connection; the specific content of constructing the recognition model is that the architecture layer number of the composite neural network is defined as 10, first entering a BN layer, and then the main body framework is arranged in the order of MP-TGC, CDJBF*3, MP-TGC, CDJBF*2, MP-TGC*3, followed by a global average pooling layer, and then performing a normalization operation.

[0039] In the step 4, the specific content of training the recognition model is that the space part of the composite neural network is trained, the space feature map is taken as the input, the corresponding feature matrix representation is obtained through the feature extraction function and taken as the input, the skeleton features and the joint features are trained by using the multi-channel convolutional neural network, the three single-channel neural networks are used in the channel sharing neural network to extract the correlation of the human joint features and the human skeleton features, and the correlation matrix and the feature transmission function are used in the skeleton joint fusion information neural network to fuse the motion information of the skeleton into the joint flow to obtain the multi-feature joint flow information.

[0040] The time part of the composite neural network is trained, the information of the skeleton and the joint is collected at a fixed frame interval as the input of the time flow in the two networks, the bidirectional long short-term memory recurrent neural network is used in the channel sharing neural network to train and obtain the multi-channel time information, and the time convolutional neural network is used to extract the information in the time aspect.

[0041] The residual mechanism between the MP-TGC and the CDJBF is defined, the semi-supervised training method is introduced to train the recognition model, and the corresponding residual mechanism is introduced in the middle of the fourth layer CDJBF and the fifth layer MP-TGC and the middle of the seventh layer CDJBF and the eighth layer MP-TGC to extract the feature information of the skeleton and the joint in the space-time.

[0042] In the step 4, before the to-be-recognized data is input, the action capture is performed through the camera, and the type of the input is judged by using the trained composite neural network.

[0043] The specific content of the step 5 is that first, the matching relationship of the action behavior and the alarm association library is established, the corresponding action triggered instruction is determined, and the automatic alarm is realized; the control layer triggers the instruction according to the result transmitted by the behavior recognition layer and controls through the network.

[0044] The application provides a violent behavior recognition and alarm method based on space-time information, which calculates the space-time information with high correlation of human behaviors, wherein the fusion of skeleton flow and joint flow training, the learning of channels according to subset division in the two flows and the fusion of global human motion features provide context information for behavior recognition, thereby effectively distinguishing similar actions, solving the lag problem of violent behavior recognition and improving the reliability and real-time performance of the intelligent alarm system. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 is a framework diagram of the violent behavior recognition and alarm method based on space-time information of the application.

[0046] Figure 2 is a characteristic skeleton (BCCS-9) division processing example of the violent behavior in the embodiment.

[0047] Figure 3 is a structure diagram of a channel sharing neural network (MP-TGC) in the application.

[0048] Figure 4 is a structure diagram of a skeleton joint fusion information neural network (CDJBF) in the application.

[0049] Figure 5 is a structure diagram of a composite neural network structure (MP-JBF) and a training process in the application. DETAILED DESCRIPTION

[0050] The technical solutions and beneficial effects of the application will be described in detail below with reference to the drawings.

[0051] As shown in Figure 1 , it is a framework diagram of the violent behavior recognition and alarm method based on space-time information of the application, which includes the following steps:

[0052] Step 1, constructing a physical layer: collecting action videos and performing data transmission; the physical layer includes a camera, a communication device and other basic hardware facilities, wherein the camera in the embodiment is a Kinect camera, the physical layer collects action videos of different individuals in a public scene including waving, lifting hands, punching and kicking, etc., and then transmits them to a data processing layer.

[0053] Step 2, constructing a data processing layer: receiving the data transmitted by the physical layer, dividing and processing the data obtained in the physical layer to obtain an action feature map; as shown in Figure 2 , the characteristic skeleton (BCCS-9) based on violent behavior is used for division processing and generating an action feature map in the embodiment; first, the input global feature matrix is defined as X∈R C×V×T , wherein V∈{v1,v2…v V} represents a set of vertices, before model construction, special bone part division (BCCS-9) is performed for violence, and the body parts are divided into 9 parts according to common violent behaviors such as beating, pushing down, etc. The body parts are divided into 9 parts, which are head (3, 4), upper limb trunk (9, 21, 2, 5), left arm (12, 11, 10), right arm (6, 7, 8), left hand (24, 25), right hand (23, 22), lower limb (17, 1, 13), left leg (18, 19, 20), and right leg (14, 15, 16), so the overall joint feature matrix corresponds to X [in R N×C×T , wherein X p refers to the feature matrix corresponding to the divided part, therefore, the feature information of each part is as follows:

[0054]

[0055] wherein i = 1, 2, …, V, f (.) is an aggregation function, P (i) is a set of joints of each divided part, and V represents the number of nodes, represents the feature of each part, and the features of each part of the body are input as input for subsequent space-time convolution.

[0056] Step 3, calculating the space-time feature information in human motion according to the action feature map obtained by the data processing layer; in the present application, two convolutional neural networks, a skeletal joint fusion information neural network and a channel sharing neural network, are used to calculate the space-time feature information in human motion; wherein the skeletal joint fusion information neural network is as shown in Figure 4 , the function of the convolutional neural network is to transmit information between the joint stream and the skeletal stream in motion, so as to promote the two streams to learn more discriminative feature representations, and the function between the joint motion and the skeletal motion is defined as follows:

[0057] Δx′ j =f j (Δx j ,x b );

[0058] Δx′ b =f b (Δx b ,x j );

[0059] In the double-stream network, Δx j is the information x b of a joint, Δx′ j is the x b updated by the bone vector x j and the corresponding connection function f jThe motion information of adjacent joints generates new motion features by joint and updating bone information, and the calculation formula is as follows:

[0060]

[0061] Wherein A k′ is the adjacency matrix of the joint, A k′ =A k +B k +C k ∈R n×n ,B k is the adaptive adjacency matrix, C k is a normalized feature-related adjacency matrix, V k′ is the adjacency matrix between bones, the relationship is defined similar to the joint, V k′ =V k +Y k +Z k ∈R n×n , f(.) is a spatial transfer function, and a series function contact(.) is used here for information transfer, W k is a learnable weight, and the related matrix V determines the motion transfer direction between the joint and the bone.

[0062] The fused joint information is input to the bidirectional long short memory network at a fixed frame interval, a double-layer structure is adopted here to extract the time information of human motion, and a 1x1 convolution is used for down-sampling to reduce the time dimension of the input and output, and then the motion information of the joint and the bone in time is obtained through the bidirectional long short memory network, and the calculation formula is as follows:

[0063] x′=f BL-2 (f BL-1 (x k ,W1),W2)k=1,2,…,K

[0064] Wherein f BL-2 (.) and f BL-1 (.) are two identical bidirectional long short memory networks, and W1 and W2 are the weights of the two functions respectively.

[0065] The channel sharing neural network (MP-TGC) is shown in Figure 3 According to the graph convolutional neural network combined with human joint position information, the joint and bone classification is performed in the divided 9 parts, taking the joint as an example, which is the joint itself, the centripetal set and the centrifugal set. Through the three sets, the 9 parts of the human body are subjected to corresponding convolution operation to obtain P1, P2 and P3 matrices. By extracting the matrix features of each channel and splicing, more rich human joint and bone features are obtained, and the overall calculation formula is as follows:

[0066]

[0067] where a ij is manually defined or set as a trainable parameter, W1 is the weight of the channel part in the human body division, W2 is the global human body weight, A1 is the adjacency matrix of the global human body, β is an adjustable parameter, F(.) is a connection function, and α l is a trainable parameter.

[0068] The global information of human body movement is adjusted in real time according to the fused information, the global human body movement information A1W2 and an adjustable parameter β are used to weigh each part of information, and finally the total feature of the human body joints and bones is obtained.

[0069] Step 4, constructing a behavior recognition layer: first, using a composite neural network to construct a recognition model; then training the recognition model; finally inputting the data to be recognized into the trained recognition model to obtain an output result; as Figure 5 shown, a composite neural network is used to construct the overall model, the number of network overall architecture layers is defined as 10, for the bone flow and joint flow, first entering the BN layer, and then the main framework arrangement order is MP-TGC, CDJBF*3, MP-TGC, CDJBF*2, MP-TGC*3, followed by a global average pooling layer, and then normalization operation is performed, wherein the collected features B i The dimension of the input model is B i (C in ,C out ,S) respectively represent the number of input channels, the number of output channels and the stride. Here, the output of each layer of the bone flow and joint flow in the double-flow composite framework is 64, 64, 64, 64, 128, 128, 128, 256, 256, 256, respectively, and finally global average pooling and normalization classification are performed to obtain the behavior prediction result; both neural networks in the composite structure are composed of a spatial modeling module, a space-time modeling module and a residual connection, and in the residual connection, since the joint flow extraction information is fused with the original bone flow in the network layer, the dimension will change here a residual connection function is proposed, and the formula is as follows:

[0070] x l+1 =x l +F(x l ,W l ),

[0071] x l+1 =h(x l )+F(x l ,W l )

[0072] The last formula is the residual connection formula proposed for the case where the dimension is not changed, where x l represents the input, x l+1 represents the output, W l represents the weight parameter, F(.) is a defined linear function, and h(.) in the formula proposed for the following dimension change is a self-defined linear function.

[0073] The training scheme for the proposed model is based on the characteristic division of the body parts of violent behavior (BCCS-9) scheme and the layer order of the composite network framework and the model selection of the temporal convolution part and the spatial convolution part of the two graph convolutional neural networks in the framework.

[0074] There are many ways to divide the characteristics of the body parts based on the graph convolutional neural network. Here, the most basic division of the upper and lower limbs is combined with actions such as punching and pushing in violent behavior. Considering the large amplitude of hand and foot movements in violent behavior, the training scheme is proposed to start from the upper and lower limbs, with both hands and feet as the center of the two blocks to obtain 9 special parts, which is defined as (BCCS-9A). Similarly, from the perspective of violent behavior, the center point of the human body trunk (2) is taken as the center. The basis for this is that violent behaviors such as punching often involve falling operations, and the displacement of the trunk center (2) is more obvious. Taking the trunk as the center and the limbs as the secondary part, 9 special parts are obtained, which is defined as (BCCS-9B). The two are compared and trained.

[0075] According to the above characteristic division of the body parts of violent behavior, the divided parts and the global are sequentially ordered, defined as P i where P0 is the global feature map of the human body. By training the centripetal set, centrifugal set and self three feature maps of the joints and bones in each part at the same time, the corresponding features are obtained. At the same time, the features in the feature map are fused to obtain the feature information of the corresponding part, and then introduced into the global part for learning to obtain new feature maps for global pooling and normalization operation for classification. Among them, for

[0076] The behavior video collected in real scenes is converted into the input of the spatio-temporal joint network through the data processing layer, and then the classification result is obtained through the trained network model. The input is first processed by the high-priority spatial autoencoder, which can determine the type of input through the gating mechanism.

[0077] Step 5, constructing a control layer: establishing a matching relationship of the action behavior and the alarm association library, obtaining an automatic identification instruction, and triggering the automatic identification instruction according to the output result of the behavior recognition layer. First, the association is designed, and the matching relationship of the behavior and the corresponding alarm association library is established to determine that the corresponding action may trigger the instruction and realize automatic alarm and other controls. The control layer triggers the instruction according to the result transmitted by the behavior recognition layer and controls through the network; the corresponding association of the embodiment is as follows: the actions such as punching and kicking belong to the behavior of beating, and after recognition, they enter the corresponding association library. For the behavior of beating, the highest alarm measure is taken, that is, the surrounding patrol officers are notified of the alarm; the actions such as pushing and falling belong to dangerous behavior, and after recognition, they enter the corresponding association library. For the dangerous behavior, the secondary alarm measure is taken, that is, the surrounding patrol officers are notified.

[0078] In summary, the present application provides a violence behavior recognition and alarm method based on space-time information. The method more reliably and efficiently recognizes and alarms violence behavior based on space-time information, which includes the following steps: constructing a physical layer: collecting action videos and performing data transmission; constructing a data processing layer: dividing and processing the data obtained in the physical layer and generating an action feature map; calculating the space-time feature information in human motion according to the action feature map obtained in the data processing layer; constructing a behavior recognition layer: first, using a composite neural network to construct a recognition model; then, training the recognition model; finally, inputting the data to be recognized into the trained recognition model to obtain an output result; constructing a control layer: establishing a matching relationship of the action behavior and the alarm association library, obtaining an automatic identification instruction, and triggering the automatic identification instruction according to the output result of the behavior recognition layer. The alarm recognition method of the present application calculates the highly correlated space-time information of human behavior, including the fusion of bone flow and joint flow during training, and the fusion of subset division channel learning and global human motion features in the two flows, which provides context information for behavior recognition to effectively distinguish similar actions, solves the problem of lagging violence behavior recognition, and improves the reliability and real-time performance of the intelligent alarm system.

[0079] The above embodiments only illustrate the technical idea of the present application, and cannot limit the protection scope of the present application. Any modification made according to the technical idea of the present application on the basis of the technical solution falls within the protection scope of the present application.

Claims

1. A method for identifying and alarming violent behavior based on spatiotemporal information, characterized in that, Includes the following steps: Step 1, Construct the physical layer: Acquire motion video and transmit it; Step 2, construct the data processing layer: receive the data transmitted from the physical layer, and divide and process the data obtained in the physical layer to obtain the action feature map; Step 3: Calculate the spatiotemporal feature information of human movement based on the motion feature map obtained from the data processing layer; Step 4, Construct the behavior recognition layer: First, use a composite neural network to build a recognition model; then train the recognition model; finally, input the data to be recognized into the trained recognition model to obtain the output result; Step 5, Construct the control layer: Establish a matching relationship between actions and alarm association databases to obtain automatic identification commands, and trigger automatic identification commands based on the output results of the behavior identification layer; In step 3, a feature extraction network is first constructed: features are extracted from the action feature map obtained in step 2, and then a skeletal joint fusion information neural network and a channel sharing neural network are constructed to calculate the spatiotemporal feature information in human movement. The specific content of constructing the skeletal joint fusion information neural network is as follows: Define the function between joint motion and bone motion, as shown in the following formula: ； ； in, This represents information about a joint. This represents the bone vector connecting this joint. Indicates the use of bone vectors and the corresponding join function The updated Motion information of adjacent joints This represents information about a skeleton; This represents the joint vector connecting this bone. Indicates through joint vectors and the corresponding join function The updated Motion information of adjacent bones; New motion features are generated by using information from joints and updated bones, as shown in the following formula: ， in, The adjacency matrix represents the joints. Represents an adaptive adjacency matrix. This represents a normalized adjacency matrix related to features. It is an adjacency matrix between bones. , Represents the spatial transfer function. Represents the learnable weights. Represents the adjacency matrix for skeletal adaptation. This represents a normalized adjacency matrix related to skeletal features; The fused joint point information is input into a bidirectional long short-term memory network at fixed frame intervals. A two-layer structure is used to extract the temporal information of human movements. Convolutional downsampling reduces the temporal dimension of the input and output, and then a bidirectional long short-term memory network is used to obtain the temporal motion information of the joints and bones, as shown in the following formula: ， in, and This represents two identical bidirectional long short-term memory networks. and These represent the weights of the two functions, respectively.

2. The identification alarm method as described in claim 1, characterized in that: In step 1, the physical layer includes cameras, communication devices, and basic hardware infrastructure based on depth cameras; Action videos include, but are not limited to, actions such as waving, raising hands, punching, and kicking.

3. The identification alarm method as described in claim 1, characterized in that, The specific content of step 2 is: defining the input global feature matrix: ， in Represents a set of vertices. It is the number of channels. It's the frame rate. It is the number of samples; The human skeleton is divided into 9 parts: head, upper limbs and trunk, left arm, right arm, left hand, right hand, lower limbs, left leg, and right leg, resulting in a joint feature matrix. ， in , This represents the feature matrix of the corresponding partition. Indicates the number of joints in the corresponding part; The feature information of each part is represented as follows: ， in, , Represents the spatial transfer function. Let V represent the set of joints for each part, and let V represent the number of nodes. Indicate the characteristics of each part, The initial feature matrix represents each part.

4. The identification alarm method as described in claim 1, characterized in that, The specific content of constructing the channel sharing neural network is as follows: The nine parts of the human skeleton divided in step 2 are classified into joints and bones, represented as the joint itself, the centripetal set, and the eccentric set; the bones themselves, the centripetal set, and the eccentric set are then convolved with the nine parts respectively to obtain... , , The matrix is ​​used to extract the matrix features of each channel and then concatenate them to obtain comprehensive human joint and bone features, as shown in the formula below: ， in, This indicates that the parameters are manually defined or set to trainable parameters. This represents the weight of the channel component in the human body segmentation. Represents the global human body weight. This represents the adjacency matrix of the global human body. This indicates an adjustable parameter. Represents the join function. Indicates trainable parameters; Based on the fused information, the global information during human movement is adjusted in real time, and the global human movement information is adjusted accordingly. And an adjustable parameter By weighing the information from each part, we can finally obtain the overall characteristics of human joints and bones.

5. The identification alarm method as described in claim 1, characterized in that, In step 4, the composite neural network includes a skeletal joint fusion information neural network and a channel sharing neural network. Both the skeletal joint fusion information neural network and the channel sharing neural network include a spatial modeling module, a spatiotemporal modeling module, and residual connections. The specific content of constructing the recognition model is as follows: the composite neural network architecture is defined with 10 layers. It first enters the Batch Normalization (BN) layer, and the subsequent main framework arrangement is as follows... , , , , Then a global average pooling layer is applied, followed by a normalization operation.

6. The identification alarm method as described in claim 5, characterized in that, In step 4, the specific content of training the recognition model is as follows: training the spatial part of the composite neural network, taking the spatial feature map as input, obtaining the corresponding feature matrix representation through the feature extraction function as input, training the skeletal features and joint features using a multi-channel convolutional neural network, using three single-channel neural networks in the channel sharing neural network to extract the correlation between the joint features of various parts of the human body and the human skeletal features, and using the correlation matrix and feature transfer function in the skeletal joint fusion information neural network to fuse the motion information of the skeleton into the joint flow to obtain multi-feature joint flow information; The temporal portion of the composite neural network is trained by collecting bone and joint information at fixed frame intervals as inputs to the temporal streams of the two networks. A bidirectional long short-term memory recurrent neural network is used to train the channel-sharing neural network to obtain multi-channel temporal information, and a temporal convolutional neural network is used to extract temporal information. definition and The residual mechanism between layers is used to introduce a semi-supervised training method to train the recognition model; for the fourth layer... and the fifth floor The middle and seventh floors and the eighth floor The corresponding residual mechanism is introduced in the middle to extract the feature information of skeletons and joints in space-time.

7. The identification alarm method as described in claim 1, characterized in that, In step 4, before inputting the data to be identified, motion capture is first performed using a camera, and the type of input is determined by a trained composite neural network.

8. The identification alarm method as described in claim 1, characterized in that, The specific content of step 5 is as follows: First, establish a matching relationship between action behavior and alarm association database, determine the corresponding action triggering command and realize automatic alarm; the control layer triggers the command according to the result passed by the behavior recognition layer and controls it through the network.

Images