A multi-view based behavior recognition method and application thereof

Abstract

The application discloses a multi-view-based behavior recognition method and application thereof, and relates to the technical field of behavior recognition, and specifically discloses the method which comprises the following steps: S1, performing data set division and data preprocessing operation on driving synchronous videos of a driver under multi-view; S2, training feature extraction weights corresponding to single-view, and integrating the feature extraction weights into a feature extraction part of a multi-view behavior recognition network; S3, training the behavior recognition network under multi-view, and introducing a reweighted loss calculation method in the training process; and finally, the obtained network can effectively improve the accuracy of behavior recognition classification.

Description

TECHNICAL FIELD

[0001] The present application relates to a video behavior classification method, in particular to a multi-view driving behavior classification method based on a 3D convolution network, and belongs to the technical field of deep learning and artificial intelligence. BACKGROUND

[0002] The behavior classification algorithm can be roughly divided into two categories: one is an image-based algorithm, and the other is a video-based algorithm. The image-based algorithm classifies each frame image in the input frame sequence using a model. Due to technical limitations, the image-based algorithm is more sensitive to the interaction between people and objects, but it is difficult to effectively identify the communication between people or the spontaneous behavior of the actor. The video-based algorithm classifies the behavior of the entire input frame sequence. Due to the characteristics of the video, the dynamic characteristics of the behavior itself can be well grasped, and compared with images, the more frames of the video itself provide more information for the model, so it has better classification accuracy.

[0003] Although the video-based algorithm is getting better and better, there are still some problems to be solved. First, the current model only uses a single angle of input to identify actions, but due to the nature of light propagating along a straight line, a single view can easily cause occlusion and other problems, resulting in a blind area of view and causing some interference to the accuracy of classification. SUMMARY

[0004] The present application is to solve the above-mentioned deficiencies in the prior art, and proposes a multi-view behavior recognition method and its application to overcome the disadvantages caused by the blind area of a single view and effectively improve the accuracy of driving behavior classification.

[0005] The present application adopts the following technical solutions to solve the technical problems:

[0006] The multi-view behavior recognition method of the present application has the following steps:

[0007] Step 1: collection and preprocessing of behavior video data;

[0008] In a real scene, video data of different views for the same behavior category is collected synchronously and normalized, thereby obtaining a multi-view video set, denoted as V = {V 1 ,V 2 ,...,V i ,..,V I}, wherein V i represents the preprocessed video sequence under any ith view, and denotes the k-th frame image after preprocessing under the i-th view, I denotes the number of views, c denotes the number of image frames, and let the real category label corresponding to the multi-view video set V be T;

[0009] Step 2: The multi-view behavior classification model comprises a primary feature extraction module, an intermediate feature extraction module and a classification module.

[0010] Step 2.1: The primary feature extraction module comprises N-level UMP modules, wherein the n-th level UMP module comprises an n-th level Unit3D module and an n-th level max-pooling module.

[0011] The n-th level Unit3D module comprises X Unit3D blocks, each of which comprises a three-dimensional convolution layer, a batch normalization layer and a RELU function activation layer in sequence, wherein the three-dimensional convolution kernel size of the x-th Unit3D block is k x x = 1, 2,..., X.

[0012] When n = 1, the video sequence V i is input into the n-th level UMP module and output as a feature map F after being processed by the n-th level Unit3D module. After being processed by the n-th level max-pooling module, the feature map F

[0013] When n = 2, 3,..., N, the output of the (n-1)-th level UMP module is F as the input of the n-th level UMP module and output as a feature map F after being processed by the n-th level Unit3D module and the n-th level max-pooling module.

[0014] Step 2.2: The intermediate feature extraction module comprises M-level IBM modules, wherein the m-th level IBM module comprises an m-th level Inception3D module and an m-th level max-pooling module.

[0015] The m-th level Inception3D module comprises four parallel branches, wherein the first branch comprises X1 three-dimensional convolution layers, wherein the convolution kernel size of the x1-th three-dimensional convolution layer is k x1 = 1, 2,..., X1; the second branch comprises X2 three-dimensional convolution layers, wherein the convolution kernel size of the x2-th three-dimensional convolution layer is k x2 = 1, 2,..., X2; the third branch comprises X3 three-dimensional convolution layers, wherein the convolution kernel size of the x3-th three-dimensional convolution layer is k x3 = 1, 2,..., X3; the fourth branch includes X4 three-dimensional convolutional layers and a pooling layer, wherein the convolution kernel size of the x4th three-dimensional convolutional layer is x4 = 1, 2,..., X4;

[0016] When m = 1, the feature map is input into the mth IBM module and processed by the four branches in the mth Inception3D module, respectively, to output the first branch feature map the second branch feature map the third branch feature map the fourth branch pooled feature map The feature maps output by the four branches are tensor-spliced in the channel dimension, thereby obtaining the spliced feature map output by the mth Inception3D module the spliced feature map After being input into the mth maximum pooling module for processing, the output feature map is

[0017] When m = 2, 3,..., M, the feature map output by the (m-1)th IBM module is input into the mth IBM module and processed by the mth Inception3D module and the mth maximum pooling module, thereby outputting the feature map Thus, the feature map output by the Mth IBM module is

[0018] Step 2.3: The classification module includes Y three-dimensional convolutional layers and an average pooling layer, wherein the convolution kernel size of the yth three-dimensional convolutional layer of the classification module is k y_cls ; y = 1, 2,..., Y;

[0019] The feature is input into the classification module, and after being processed by the average pooling layer, the pooled feature is obtained After being processed by the Y three-dimensional convolutional layers in turn, the classification feature map of the video sequence V i under the ith view is obtained

[0020] Step 2.4: The cross-entropy loss function of the behavior classification model under the ith view is constructed using formula (1)

[0021]

[0022] Step 2.5: The behavior classification model is preliminarily trained using the gradient descent method, and the cross-entropy loss function is calculated Until the cross-entropy loss function converges, the optimal weight of the primary feature extraction module corresponding to the i-th view is obtained after preliminary training and the optimal weight of the intermediate feature extraction module

[0023] Step 2.6: After the multi-view video set V is processed by the preliminary trained primary feature extraction module and the intermediate feature extraction module, a behavior feature map set under multi-view is obtained After tensor splicing in the time dimension, the fusion feature FutureMap is obtained fusion , wherein, represents the video V under the i-th view i , and the feature map obtained after feature extraction

[0024] Step 2.7: After the fusion feature FutureMap fusion is processed by the preliminary trained classification module, a classification feature map FutureMap is obtained class ;

[0025] Step 2.8: After dimension compression is performed on the classification feature map FutureMap class , the final classification result T of the multi-view video set V is obtained p ;

[0026] Step 3: The loss function of the behavior classification model under multi-view in the E-th round of training is constructed by using formula (2)

[0027]

[0028] In formula (2), L e represents the easy classification degree loss, and is obtained by formula (3), L h represents the difficult classification degree loss, and is obtained by formula (4), α E represents the importance ratio of the difficult and easy classification loss under the current E-th round of training, and is obtained by formula (5);

[0029] L e = -log(T p ) (3)

[0030] L h = -(T-T p ) 4 10g(T p )-log(T-T p ) (4)

[0031] ​

[0032] In formula (5), f represents a control parameter, E represents the number of rounds of current training, E max represents the number of rounds of total training;

[0033] In the case that the optimal weights of the primary feature extraction module after preliminary training are frozen and the optimal weights of the intermediate feature extraction module are frozen , the behavior recognition classification model after preliminary training is trained by using the gradient descent method, and a loss function is calculated until the cross-entropy loss function converges or reaches the maximum number of rounds E max , so as to obtain the optimal behavior recognition network under multiple perspectives, which is used for recognizing and classifying behaviors under multiple perspectives.

[0034] The electronic device comprises a memory and a processor, wherein the memory is used to store a program supporting the processor to execute the behavior recognition method, and the processor is configured to execute the program stored in the memory.

[0035] The computer readable storage medium stores a computer program, wherein the computer program is executed by the processor to execute the steps of the behavior recognition method.

[0036] Compared with the prior art, the beneficial effects of the present application are as follows:

[0037] 1、The present application extracts the spatio-temporal features under each perspective by using the feature extraction network weight of each perspective after pre-training, and realizes the information interaction between different perspectives by splicing the spatio-temporal features in the time dimension. Meanwhile, a loss function that can change the proportion of difficult classification loss according to the number of training rounds is introduced to realize the dynamic change in the model learning process. Finally, the problem of poor classification effect caused by visual limitation in the single perspective case is solved by the method of multi-perspective fusion, thereby improving the classification accuracy.

[0038] 2、The present application uses the feature extraction weight pre-trained in the single perspective as the feature extraction weight of the fusion network, and freezes these weights and does not participate in the back propagation, thereby avoiding the interference of different perspectives on the feature extraction part in the training process of the multi-perspective fusion network, so that the spatio-temporal features extracted under each perspective are not complete enough, resulting in poor classification effect.

[0039] 3、The application adopts a dynamically changed loss function to perform loss calculation, and by continuously changing the proportion of difficult and easy classification losses, when the algorithm achieves very good classification effect on a certain class and very poor classification effect on other classes, the algorithm focuses on learning the classes with poor classification effect, so that the final algorithm has good sensitivity for each class, and the class with very poor effect does not appear, thereby effectively improving the overall effect of the algorithm. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 A flowchart of the method of the application;

[0041] Figure 2 A schematic diagram of the Inception block structure of the application;

[0042] Figure 3 A schematic diagram of the fusion process of the application;

[0043] Figure 4 A training schematic diagram of the multi-view driving behavior classification method of the application. DETAILED DESCRIPTION

[0044] In this embodiment, to solve the problem that the classification precision is limited due to the visual blind area of the behavior recognition algorithm in a single view, a multi-view based behavior recognition algorithm is used for behavior recognition, and the features extracted from each angle in the time dimension are fused, and the final classification is performed according to the fused features, effectively utilizing the behavior features in each view at the same time, avoiding the problem of limited field of view, thereby effectively improving the precision of behavior recognition classification. Specifically, a multi-view based behavior recognition method, as shown in Figure 1 includes the following steps:

[0045] Step 1: collection and preprocessing of behavior video data;

[0046] In a real scene, video data of different views for the same behavior category is synchronously collected and normalized, thereby obtaining a multi-view video set, denoted as V={V 1 ,V 2 ,...,V i ,..,V I}, wherein V i represents a preprocessed video sequence in any i-th view, and represents the k-th frame image preprocessed in the i-th view, I represents the number of views, c represents the number of image frames, and let the real category label corresponding to the multi-view video set V be T;

[0047] In this embodiment, 3MDAD is used as the data set, and among the 50 actors, the videos of 35 actors are used as training data, 5 actors are used as validation data, and the remaining 10 actors are used as test data. Each actor includes two videos of behavior from two perspectives, and the behavior categories are divided into 16 categories.

[0048] Step 2: The multi-view behavior classification model includes a primary feature extraction module, an intermediate feature extraction module, and a classification module.

[0049] Step 2.1: The primary feature extraction module is composed of N-level UMP modules, wherein the nth-level UMP module is composed of an nth-level Unit3D module and an nth-level max-pooling module.

[0050] The nth-level Unit3D module is composed of X Unit3D blocks, each of which includes a three-dimensional convolution layer, a batch normalization layer, and a RELU function activation layer in sequence, wherein the convolution kernel size of the three-dimensional convolution layer of the xth Unit3D block is k x ; x = 1, 2,... X;

[0051] In this embodiment, X = 2, and the size of the convolution kernel is 1x1x1. This layer is used to preliminarily extract partial features and gradually increase the number of channels to facilitate the extraction of deep features.

[0052] When n = 1, the video sequence V i is input into the nth-level UMP module and processed by the nth-level Unit3D module, and the feature map F is output. After being processed by the nth-level max-pooling module, the maximum pooling is adopted to remove some unimportant feature information while retaining the boundary features, and the data is compressed to reduce the memory pressure during calculation, and the feature map F

[0053] When n = 2, 3,... N, the output of the (n-1)th-level UMP module is as the input of the nth-level UMP module, and after being processed by the nth-level Unit3D module and the nth-level max-pooling module, the feature map F is output.

[0054] Step 2.2: The intermediate feature extraction module is composed of M-level IBM modules, wherein the mth-level IBM module is composed of an mth-level Inception3D module and an mth-level max-pooling module.

[0055] The mth-level Inception3D module includes four parallel branches, such asFigure 2 wherein the first branch includes X1 three-dimensional convolutional layers, wherein the convolution kernel size of the x1th three-dimensional convolutional layer is x1 = 1, 2,..., X1; the second branch includes X2 three-dimensional convolutional layers, wherein the convolution kernel size of the x2th three-dimensional convolutional layer is x2 = 1, 2,..., X2; the third branch includes X3 three-dimensional convolutional layers, wherein the convolution kernel size of the x3th three-dimensional convolutional layer is x3 = 1, 2,..., X3; the fourth branch includes X4 three-dimensional convolutional layers and a pooling layer, wherein the convolution kernel size of the x4th three-dimensional convolutional layer is x4 = 1, 2,..., X4;

[0056] In this embodiment, M = 9, X1 = 1 in the first branch, the convolution kernel size k x_1_1 = 1 x 1 x 1; X2 = 2 in the second branch, wherein the convolution kernel size k x_2_1 = 1 x 1 x 1, and the convolution kernel size k x_2_2 = 3 x 3 x 3; X3 = 2 in the third branch, wherein the convolution kernel size k x_3_1 = 1 x 1 x 1, and the convolution kernel size k x_3_2 = 3 x 3 x 3, and the maximum pooling in the fourth branch uses a pooling window of 3 x 3 x 3, and includes X4 = 1 convolutional layer, and the convolution kernel size k x_4_1 = 1 x 1 x 1. Through the four branches, features of different dimensions are extracted, and these features are fused, so that important features can be effectively strengthened, and more information can be obtained.

[0057] When m = 1, the feature map is input into the mth IBM module, and after being processed by the four branches in the mth Inception3D module, respectively, the first branch feature map , the second branch feature map , the third branch feature map , and the fourth branch pooling feature map are output. The feature maps output by the four branches are tensor spliced in the channel dimension, so as to obtain the spliced feature map output by the mth Inception3D module

[0058] When m = 2, 3,..., M, the feature map output by the (m-1)th IBM module ​As input to the m-th level IBM module, and after processing by the m-th level Inception3D module and the m-th level max-pooling module, the output feature map is obtained. Thus, the feature map is output by the M-th level IBM module.

[0059] Step 2.3: The classification module consists of Y 3D convolutional layers and one average pooling layer. Average pooling is used to enhance the correlation between context information while preserving the background, and it also compresses the data to reduce memory pressure during computation. The kernel size of the y-th 3D convolutional layer in the classification module is k. y_cls ;y = 1, 2, ... Y;

[0060] feature The input is first processed by the average pooling layer in the classification module to obtain the pooled features. After being processed by Y 3D convolutional layers, the video sequence V from the i-th viewpoint is obtained. i Classification feature map

[0061] Step 2.4: Construct the cross-entropy loss function of the behavior classification model from the i-th perspective using equation (1).

[0062]

[0063] Step 2.5: Perform initial training on the behavior classification model using gradient descent and calculate the cross-entropy loss function. Until the cross-entropy loss function The process continues until convergence, thus obtaining the optimal weights of the initial feature extraction module after preliminary training for the i-th viewpoint. And the optimal weights of the intermediate feature extraction module

[0064] By using the pre-trained optimal feature extraction weights for each viewpoint, interference between viewpoints is avoided, resulting in the best performance of the extracted features.

[0065] Step 2.6: After the initial training of the primary and intermediate feature extraction modules, the multi-view video set V is processed to obtain a set of behavioral feature maps from multiple perspectives.

[0066] Then, tensor splicing is performed in the time dimension, such as... Figure 3 As shown, the FutureMap of fused features is obtained. fusion ,in, V represents the video from the i-th perspective. iThe feature map obtained after feature extraction;

[0067] Performing tensor splicing in the time dimension does not increase the number of channels, resulting in a significant increase in the computational load during classification calculations. At the same time, it facilitates the exchange of information from different perspectives, thereby improving the classification effect.

[0068] Step 2.7: Fuse Feature FutureMap fusion After processing by the classification module following initial training, a classification feature map (FutureMap) is obtained. class ;

[0069] Step 2.8: Process the classification feature map FutureMap class After dimensionality compression, the final classification result T of the multi-view video set V is obtained. p ;

[0070] Step 3: Construct the loss function of the behavior classification model trained in the Eth round under multiple perspectives using equation (2).

[0071]

[0072] In equation (2), L e Let L represent the loss due to ease of classification, which is obtained from equation (3). h The loss represents the difficulty in classification, and is obtained from equation (4), α. E This represents the proportion of difficulty classification loss importance under the current E-round training, and is obtained from equation (5);

[0073] L e =-log(T) p (3)

[0074] L h =-(TT) p ) 4 log(T p )-log(TT p (4)

[0075]

[0076] In equation (5), f represents the control parameter, E represents the current training round number, and E max Indicates the total number of training rounds;

[0077] In this embodiment, f = 4, Emax = 100, and ∝ is continuously updated during the training process. E The size of the loss weight is used to control the proportion of easy and difficult classification loss, thereby enabling dynamic changes in the loss weight during training.

[0078] The training process of multi-view behavior recognition algorithms is as follows: Figure 4 As shown, the optimal weights of the primary feature extraction module after initial training are frozen. And the optimal weights of the intermediate feature extraction module In this case, the gradient descent method is used to train the behavior recognition and classification model after the initial training, and the loss function is calculated. Until the cross-entropy loss function Convergence or reaching the maximum number of rounds E max Thus, the optimal behavior recognition network under multiple perspectives is obtained, which is used to recognize and classify behaviors under multiple perspectives.

[0079] In this embodiment, an electronic device includes a memory and a processor. The memory stores a program that supports the processor in executing the behavior recognition method described above. The processor is configured to execute the program stored in the memory.

[0080] In this embodiment, a computer-readable storage medium stores a computer program, which is executed by a processor to perform the steps of the behavior recognition method described above.

[0081] Specifically, this embodiment uses 3MDAD as the dataset. Of the 50 participants, 35 videos are used as training data, 5 as validation data, and the remaining 10 as test data. Each participant includes behavioral videos from two perspectives. The optimal weights in the primary feature extraction module are then determined. And the optimal weights of the intermediate feature extraction module In acquiring the data, videos from a single perspective were used for training. During training, the Stochastic Gradient Descent (SGD) algorithm was used in the PyTorch framework, with the following parameters set: learning rate of 0.1, weight decay of 0.0001, maximum batch size of 1000, and steps of 100 and 300. The server's GPU card can be used for computational acceleration during training.

[0082] In the training process of multi-view behavior recognition algorithms, the optimal weights of the primary feature extraction module are determined. And the optimal weights of the intermediate feature extraction module This algorithm does not participate in training; only the parameters of the classification part are trained. During training, the model is trained using the Stochastic Gradient Descent (SGD) algorithm within the PyTorch framework. The relevant parameters can be set as follows: learning rate of 0.1, weight decay of 0.0001, maximum batch size of 100, and number of steps of 20 and 50. The loss function is calculated using LCASL.

[0083] During training, weight decay occurs in rounds 20 and 50, so the loss function is divided into three phases:

[0084] The first stage, the learning rate is 0.1, Emax is the total number of rounds of the first stage 20, and E is the number of rounds currently trained;

[0085] The second stage, the learning rate is attenuated for the first time, Emax is the total number of rounds of the second stage 30, and E is the number of rounds currently trained minus the total number of rounds before the learning rate is attenuated for the first time;

[0086] The third stage, the learning rate is attenuated for the second time, Emax is the total number of rounds of the third stage 50, and E is the number of rounds currently trained minus the total number of rounds before the learning rate is attenuated for the second time.

[0087] The multi-view behavior classification method uses accuracy as the final evaluation index. Under this method, the behavior recognition accuracy of the test set reaches 83.7%, which is 8.11% higher than the single-view behavior classification algorithm recognition accuracy of 75.59%. At the same time, Lcasl is introduced as the loss calculation, which further improves the effect of the multi-view behavior recognition method.

Claims

1. A multi-view based behavior recognition method, characterized in that, The method comprises the following steps: Step 1: collecting and preprocessing of behavior video data; In real-world scenarios, video data from different perspectives is simultaneously collected for the same behavior category and preprocessed using normalization techniques to obtain a multi-view video set, denoted as […]. ,in, Represents any number of i The preprocessed video sequence from each perspective, and , Indicates the first i The preprocessed first viewpoint k Frame image, I Represents the number of viewpoints. c Let V represent the number of image frames, and let the true category label corresponding to the multi-view video set V be... ; Step 2: a multi-view based behavior classification model, comprising a primary feature extraction module, an intermediate feature extraction module and a classification module; Step 2.1: The primary feature extraction module is composed of N-level UMP modules, wherein the nth-level UMP module is composed of the nth-level module and the nth-level max-pooling module. The nth stage The module consists of X blocks, each block sequentially includes a three-dimensional convolution layer, a batch normalization layer, and a RELU function activation layer, wherein the convolution kernel size of the three-dimensional convolution layer of the xth block is ; x = 1, 2,... X; When n=1, the first i Video sequences from various perspectives Input into the nth level UMP module, and pass through the nth level. After processing by the module, a feature map is output. Feature map After processing by the nth-level max pooling module, the feature map is output. ; When n = 2, 3, …, N, the output of the (n-1)th UMP module as the input of the nth UMP module and after being processed by the nth UMP module After being processed by the module and the nth max-pooling module, the output feature map ; and the feature map is output by the Nth UMP module ; Step 2.2: the intermediate feature extraction module is composed of M IBM modules, wherein the mth IBM module is composed of an mth Inception3D module and an mth max-pooling module; The m-th level Inception3D module includes four parallel branches, wherein the first branch includes The nth three-dimensional convolutional layer, where the nth The kernel size of each 3D convolutional layer is... , =1,2,..., The second branch includes The nth three-dimensional convolutional layer, where the nth The kernel size of each 3D convolutional layer is... , =1,2,..., The third branch includes The nth three-dimensional convolutional layer, where the nth The kernel size of each 3D convolutional layer is... , =1,2,..., The fourth branch includes There is a 3D convolutional layer and a pooling layer, where the first... The kernel size of each 3D convolutional layer is... , =1,2,..., ; When m = 1, the feature map Input into the m-th IBM module, and after being processed by four branches in the m-th Inception3D module respectively, the corresponding outputs are a first branch feature map , a second branch feature map , a third branch feature map , and a fourth branch pooled feature map The feature maps output by the four branches are tensor spliced in the channel dimension, thereby obtaining a spliced feature map output by the m-th Inception3D module , the spliced feature map After being input into the m-th max-pooling module for processing, a feature map is output; When m = 2, 3,..., M, the feature map output by the (m-1)-th IBM module As input to the m-th IBM module, and after processing by the m-th Inception3D module and the m-th max-pooling module, the output feature map , so that the feature map is output by the M-th IBM module ; Step 2.3: the classification module comprises Y three-dimensional convolution layers and one average pooling layer, wherein the convolution kernel size of the yth three-dimensional convolution layer of the classification module is ; y = 1, 2,... Y; The features Input into the classification module, first through the operation of the average pooling layer, obtain the pooled features , and then sequentially through the processing of Y three-dimensional convolution layers, obtain the classification feature atlas of the video sequence under the first i view angle . ; Step 2.4: Constructing the behavior classification model at the cross-entropy loss function under the perspective of formula (1) i :​ = (1) Step 2.5: preliminary training of the behavior classification model using gradient descent method and calculation of cross-entropy loss function , until the cross-entropy loss function converges, thereby obtaining the optimal weight of the preliminary feature extraction module corresponding to the i-th view after preliminary training and the optimal weight of the intermediate feature extraction module ; Step 2.6: the multi-view video set After the processing of the primary feature extraction module and the intermediate feature extraction module after the preliminary training, a set of behavior feature maps under multiple views is obtained = { }After the time dimension is carried out tensor splicing, the fusion feature is obtained , wherein represents the video under the i-th view , the feature map obtained after feature extraction; Step 2.7: the fusion feature After processing by the classification module after preliminary training, a classification feature map is obtained ; Step 2.8: performing dimension compression on the classification feature map After dimension compression, a multi-view video set is obtained Final classification result of the multi-view video set ; Step 3: Constructing the loss function of the behavior classification model of the E-th round of training under multi-view using formula (2) : (2) In formula (2), represents the loss of easy classification degree, and is obtained by formula (3), represents the loss of difficult classification degree, and is obtained by formula (4), represents the proportion of the importance of the loss of easy and difficult classification under the current E-th training, and is obtained by formula (5); (3) (4) (5) In formula (5), f denotes a control parameter, E denotes the current round of training, E max denotes the total number of rounds of training; Freeze the optimal weights of the primary feature extraction module after the preliminary training and the optimal weights of the intermediate feature extraction module , and train the behavior recognition classification model after the preliminary training using the gradient descent method and calculate the loss function until the cross-entropy loss function converges or the maximum number of rounds is reached E max , thereby obtaining the optimal behavior recognition network under multiple perspectives for identifying and classifying behaviors under multiple perspectives.

2. An electronic device comprising a memory and a processor, characterized in that The memory is used for storing a program supporting the processor to execute the behavior recognition method of claim 1, and the processor is configured to execute the program stored in the memory.

3. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to perform the steps of the behavior recognition method of claim 1.

Images