A behavior recognition method, system and related device applied to surveillance video

Abstract

The present application provides a behavior recognition method applied to surveillance videos, comprising: receiving a video to be recognized; determining whether a training model contains all action groups corresponding to the video to be recognized; if so, inputting the video to be recognized into the training model; in the training model, performing action recognition of the corresponding action group on the frame difference of the video to be recognized, and determining the behavior actions contained in the video to be recognized. The present application recognizes corresponding actions in units of action groups, while using frame differences as the main input data, and performs data recognition on the training set based on the frame difference information when the training model is recognized, thereby improving the robustness of the training model, reducing dependence on the background during behavior recognition, and reducing redundant calculations during action recognition. The present application also provides a behavior recognition system, computer-readable storage medium, and server applied to surveillance videos, which have the above-mentioned beneficial effects.

Description

Technical Field

[0001] The present application relates to the field of information monitoring, and in particular to a behavior recognition method, system and related devices applied to monitoring videos. Background Art

[0002] The current recognition process of pedestrian behavior in monitoring scenarios mainly relies on various behavior recognition models. The current behavior recognition model mainly takes video as input and outputs the category of the video sequence. This method requires determining the type of behavior to be monitored during training. If you want to add a behavior category later, you need to retrain the model, which is time-consuming and laborious. At the same time, due to the complexity of real-world scenarios and the limited training set, behaviors are highly dependent on scenarios. It is easy for behaviors to be unrecognizable when migrating from one scenario to another. That is, the same behavior recognition model has differences in the recognition of the same behavior in different scenarios, resulting in low robustness of the behavior recognition model, which is not conducive to practical applications. Summary of the invention

[0003] The purpose of this application is to provide a behavior recognition method, system, computer-readable storage medium and electronic device applied to surveillance videos, which can reduce the robustness of the behavior recognition model and improve the accuracy of behavior recognition.

[0004] In order to solve the above technical problems, this application provides a behavior recognition method applied to surveillance videos. The specific technical solution is as follows:

[0005] Receiving a video to be identified;

[0006] Determine whether the training model contains all action groups corresponding to the video to be identified;

[0007] If yes, input the video to be identified into the training model;

[0008] In the training model, action recognition of the corresponding action group is performed on the frame difference of the video to be recognized to determine the behavioral actions included in the video to be recognized.

[0009] Optionally, before determining whether the training model contains all action groups to be recognized, the following steps are further included:

[0010] Get the training video;

[0011] Dividing the training video into a plurality of video segments, and marking the behavior actions in each of the video segments;

[0012] Extracting preset frame pictures at equal intervals in the video segment, and calculating frame differences between adjacent frame pictures in the extracted pictures;

[0013] Extracting the respective stage features of the image and the frame difference, combining and convolving them to obtain a preset fully connected layer;

[0014] The training model is constructed according to the preset fully connected layer.

[0015] Optionally, if the number of the pictures is N, respectively extracting the stage features of the pictures and the frame differences, combining and convolving to obtain a preset fully connected layer includes:

[0016] If N is an odd number, the (N+1) / 2th picture is used as the target picture; if N is an even number, the N / 2th and N / 2+1th pictures are used as the target pictures; the first stage features of the target picture are extracted using the first residual network;

[0017] Extracting the second stage features of the frame difference using a second residual network;

[0018] The first-stage features and the second-stage features are combined to obtain the preset fully connected layer.

[0019] Optionally, after calculating the frame difference between adjacent frame images, the following steps are further included:

[0020] The target image and the frame difference are normalized within a preset range.

[0021] Optionally, if the training model does not include all action groups to be identified, it also includes:

[0022] Get the new action behavior of the new action group;

[0023] Adding the new action behavior as a new task set to the training set;

[0024] The fully connected layer and classification layer of the existing task set are fixed, a new classification layer corresponding to the new task set is established, and the training model is updated using a preset learning rate.

[0025] Optionally, constructing the training model according to the preset fully connected layer includes:

[0026] Determine a preset number of task sets;

[0027] A preset number of fully connected layers are connected after the preset fully connected layer; each of the fully connected layers is connected to a classification layer corresponding to a category of each task set;

[0028] The task set is used as a model input, and a mixed loss and a preset number of subtask losses are configured; the mixed loss is used to remove the last dimension of the output of each classification layer and combine them to form an output vector.

[0029] Optionally, when configuring a hybrid loss and a preset number of subtask losses, also include:

[0030] The weight values ​​of the subtask loss and the mixed loss are respectively configured according to the number and category of the task set.

[0031] The present application also provides a behavior recognition system applied to surveillance videos, comprising:

[0032] A receiving module, used for receiving a video to be identified;

[0033] A judgment module, used to judge whether the training model contains all action groups to be identified;

[0034] An input module, used for inputting the video to be identified into a training model when the judgment result of the judgment module is yes;

[0035] The recognition module is used to perform action recognition of the corresponding action group on the frame difference of the video to be recognized in the training model to obtain the behavioral actions contained in the video to be recognized.

[0036] The present application also provides a computer-readable storage medium having a computer program stored thereon, and the computer program implements the steps of the above-mentioned method when executed by a processor.

[0037] The present application also provides a server, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the above-mentioned method when calling the computer program in the memory.

[0038] The present application provides a behavior recognition method applied to surveillance videos, including: receiving a video to be recognized; determining whether a training model contains all action groups corresponding to the video to be recognized; if so, inputting the video to be recognized into the training model; in the training model, performing action recognition of the corresponding action group on the frame difference of the video to be recognized, and determining the behavior actions contained in the video to be recognized.

[0039] The present application divides the behavior to be recognized into multiple action groups. When performing action recognition on the video to be recognized, the corresponding action is recognized in units of action groups. At the same time, frame difference is used as the main input data, and data recognition is performed on the training set based on the frame difference information when training the model for recognition, thereby improving the robustness of the training model, reducing the dependence on the background during the behavior recognition process, and reducing redundant calculations in the action recognition process.

[0040] The present application also provides a behavior recognition system, a computer-readable storage medium and a server applied to surveillance videos, which have the above-mentioned beneficial effects and are not described in detail here. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0042] Figure 1 A flowchart of a behavior recognition method applied to surveillance video provided by an embodiment of the present application;

[0043] Figure 2 A flow chart of the generation process of the training model provided in the embodiment of the present application;

[0044] Figure 3 A schematic diagram of the structure of the training model provided in the embodiment of the present application;

[0045] Figure 4 A schematic diagram of the updating process of the training model provided in the embodiment of the present application;

[0046] Figure 5 A schematic diagram of the structure of a behavior recognition system applied to surveillance video provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0048] Please refer to Figure 1 , Figure 1 A flowchart of a behavior recognition method applied to surveillance video provided in an embodiment of the present application, the method comprising:

[0049] S101: receiving a video to be identified;

[0050] S102: Determine whether the training model contains all action groups corresponding to the video to be identified; if so, proceed to S103;

[0051] S103: Inputting the video to be identified into a training model;

[0052] S104: In the training model, action recognition of a corresponding action group is performed on the frame difference of the video to be recognized, and the behavior actions included in the video to be recognized are determined.

[0053] The video to be identified in the embodiment of the present application is mainly a surveillance video, which is used to identify human actions in the surveillance video. There is no specific limitation on how to receive the video to be identified. It can directly obtain the video shot by each surveillance camera, or obtain it from the database of the surveillance camera, or use the specified video input by the user as the video to be identified. In particular, since the video can also be regarded as a continuous playback of pictures, this embodiment can also perform behavior recognition on multiple continuous pictures.

[0054] After that, a judgment is made for the action group that needs to be identified for the video to be identified, that is, whether the existing training model contains all the action groups of the video to be identified. Action group refers to the grouping of user behavior actions, and the grouping method is usually set by the action state and monitoring target that may appear in the application environment of the model. For example, if the training model is deployed at the gate of a subway station to monitor fare evasion and fall events, there may be two action groups in this application scenario, namely, the fare evasion monitoring action group (including three behaviors: normal passage, gate turning, and ticket following) and the fall event detection action group (including normal and fall situations). Of course, in other application scenarios, those skilled in the art can also configure different action groups according to monitoring requirements, which are not limited here one by one.

[0055] In step S102, it is necessary to determine whether the existing training model contains all the action groups of the video to be identified. Once it is confirmed that the training model meets the action recognition requirements of the video to be identified, the video to be identified is input into the training model. After that, the action recognition of the corresponding action group can be performed on the frame difference of the video to be identified to determine the behavior actions contained in the video to be identified.

[0056] It is easy to understand that, in this embodiment, it is assumed that the training model has been acquired or generated before executing step S102, and in this embodiment, no specific limitation is made on how to acquire or generate the training model.

[0057] When performing recognition, it is necessary to perform action recognition of the corresponding action group on the frame difference of the video to be recognized, so as to determine the behavior actions contained in the video to be recognized. Frame difference refers to the frame difference characteristics of the video to be recognized before and after. Obviously, for surveillance videos, the background of the monitored picture is usually fixed and only pedestrians perform behavioral actions. Therefore, using the training model to recognize frame difference can reduce the dependence on background information, pay more attention to the action itself, and reduce the redundant calculations caused by environmental background recognition.

[0058] The embodiment of the present application divides the behavior to be recognized into multiple action groups. When performing action recognition on the video to be recognized, the corresponding action is recognized in units of action groups. At the same time, frame difference is used as the main input data, and data recognition is performed on the training set based on the frame difference information when training the model for recognition, thereby improving the robustness of the training model, reducing the dependence on the background in the behavior recognition process, and reducing redundant calculations in the action recognition process.

[0059] See also Figure 2 , Figure 2 The following is a schematic diagram of the structure of the training model provided in the embodiment of the present application. The following is a preferred method for constructing a training model provided in the present application, and the process is as follows:

[0060] S201: Obtain training video;

[0061] S202: Segment the training video into a plurality of video segments, and mark the behavior actions in each of the video segments;

[0062] S203: extracting preset frame pictures at equal intervals in the video segment, and calculating frame differences between adjacent frame pictures in the extracted pictures;

[0063] S204: extracting the respective stage features of the picture and the frame difference, combining and convolving them to obtain a preset fully connected layer;

[0064] S205: Construct the training model according to the preset fully connected layer.

[0065] In step S201, the training video may be a sample video containing behavioral actions, which is used to construct a training model.

[0066] In step S202, the training video can be divided into several video segments, and the behavior actions in each video segment are marked. The purpose of dividing the training video is to improve the training efficiency and training effect, and to avoid affecting the generation speed of the training model due to the excessive content of the training video. The training video is a sample video, and the behavior actions in each video segment need to be marked so that the training model can learn.

[0067] In step S203, pictures and frame differences between adjacent frames in the pictures need to be extracted respectively. There is no limit on the number of pictures to be extracted, but the time lengths between the pictures should be equal, and after the pictures are extracted, the extracted pictures are arranged in the order of time frames, and the frame differences between adjacent pictures in the extracted pictures are calculated.

[0068] In step S204, the stage features of the picture and the frame difference are extracted respectively, and the preset fully connected layer is obtained after combination and convolution. In this process, the stage features need to be selected according to the number of pictures, and the specific process can be as follows:

[0069] Step 1: If N is an odd number, use the (N+1) / 2th image as the target image; if N is an even number, use the N / 2th and N / 2+1th images as the target images;

[0070] The second step is to use the first residual network to extract the first stage features of the target image;

[0071] The third step is to use the second residual network to extract the second stage features of the frame difference;

[0072] Step 4: Combine the first-stage features and the second-stage features to obtain the preset fully connected layer.

[0073] It can be seen that the target image can be 1 or 2, which is determined by the parity of the number of extracted images. In the process of feature extraction, since the camera position is fixed in the monitoring scene and most of the background information remains unchanged, the frame difference usually only contains the target's motion information. In the TSN network, the frame difference information and RGB image must pass through two independent Resnet networks respectively, and the final classification layer output is merged. The information utilization rate is low, and the network cannot learn the frame difference information and RGB image information. One solution is to input RGB and frame difference into the network at the same time, but the image characteristics of RGB and frame difference are very different, and direct fusion convergence is difficult. A better way is to fuse in high-dimensional space.

[0074] To this end, the present application needs to use the first residual network and the second residual network to extract the stage features of the picture and the frame difference respectively. However, the first residual network and the second residual network do not have to be different residual networks, but can also be the same residual network. For example, two resnet50s can be used as feature extractors.

[0075] Taking N=3 as an example, 3 frames are randomly selected in each video segment (3 frames are selected at equal intervals during the test phase), and recorded as Calculate frame difference and As a more preferred implementation process, after calculating the frame difference between adjacent frame pictures, the target picture and the frame difference can also be normalized to a preset range, for example and Normalized to the interval [-1,1].

[0076] See also Figure 3 , Figure 3 This is a schematic diagram of the structure of the training model provided in the embodiment of the present application. When performing feature extraction, and Use a resnet50 as the first residual network to extract the features of the first two stages. Use another resnet50 as the second residual network to extract the features of the first two stages, and finally combine them through stages 3, 4 and the pooling layer to obtain the fully connected layer features at time n, which are used as the preset fully connected layer to build the training model. By fusing the RGB image features of the target image with the frame difference features of the previous and next moments, the training model can utilize the correlation between the first-stage features of the image and the second-stage features of the frame difference, reducing the training model's dependence on background information, allowing the training model to focus more on the action itself and reducing redundant operations.

[0077] Specifically, in the process of building a training model, a preset number of task sets can be determined first, and a preset number of fully connected layers are connected after the preset fully connected layer, and each fully connected layer is connected to the classification layer of the corresponding category of each task set, and the training model can be obtained. The task set is used as the model input, and a mixed loss and a preset number of subtask losses are configured. The mixed loss is used to remove the last dimension of the output of each classification layer and combine them to form an output vector. When training a multi-task model, the combination of subtask loss and mixed loss is used to allow each task set data to be used as a negative sample of other task sets, thereby enriching the training samples and scenarios of each task set and avoiding model overfitting.

[0078] Typically, the weight values ​​of the mixed loss and the subtask loss can be configured to be 1, but the weight values ​​of the subtask loss and the mixed loss can also be configured separately according to the number and category of the task sets.

[0079] Specifically, assume that there are M task sets initially, each task set has C subclasses (the number of categories in each task set can be different), and M fully connected layers are connected after the fully connected layer as feature conversion, and then each fully connected layer is connected after its own classification layer. The number of categories in each classification layer is C+1. The additional category is used to distinguish different data sets. When data that does not belong to the task is input, the data can be classified into this additional category.

[0080] During the training phase, the datasets of M task sets are mixed and input. The model contains subtask losses and mixed losses. There are M subtask losses in total, which is the same as the number of task sets. They are used to optimize the discrimination within each task. There is only one mixed loss. The output of each task classification layer is combined by removing the last dimension to form a new mixed output vector. The label of each input data can be determined according to the corresponding relationship. The mixed loss merges all task sets to improve the discrimination between classes.

[0081] Assume that a data of the mth task is input, its label in the mth task is 2, its label in each task is {C1+1, C2+1, …, 2}, and the label of the mixed loss is The subtask loss is:

[0082]

[0083] Where i is the i-th sample in the training sample with batch size N, and m is the m-th task in the total task set M. is the label of the jth class of the ith sample in the mth task set. At the same time, the mixed loss is:

[0084]

[0085] Where i is the i-th sample in the training sample with batch size N, y j is the mixed loss label of the i-th sample in the j-th category, aj is the jth value of the mixed output vector.

[0086] The above subtask losses and mixed losses can be added according to the set weights to train the network. The weights can be adjusted and set by technicians in this field according to the data set and are not limited here.

[0087] This embodiment provides a generation process of a training model. When generating a training model, subtask losses and loss combinations are used to allow each task set data to be used as negative samples of other task sets, thereby enriching the training samples and scenarios of each task set and avoiding overfitting of the training model. An additional negative sample category is added to the classification layer corresponding to each action group. When samples of other action groups are input into the network, the classification layer of this action group can classify them into this category, thereby improving the overall indicators of the model. According to this generation process, even if the training model is generated, if a new task set needs to be added later, it is only necessary to fix the fully connected layer and classification layer of the existing task set, add the new task set data to the training set, fix all other parameters and only train the new classification layer, so that the model can adapt to the new task set, so that the training model can be adaptively updated without regeneration.

[0088] Based on the above embodiment, as a preferred embodiment, if the training model does not include all the action groups to be identified, that is, the training model does not meet the recognition requirements of the action groups corresponding to the video to be identified, the training model can be optimized at this time, see Figure 4 , Figure 4 This is a schematic diagram of the updating process of the training model provided in the embodiment of the present application. The specific process may be as follows:

[0089] S301: Obtain new action behavior of a new action group;

[0090] S302: adding the new action behavior as a new task set to the training set;

[0091] S303: Fix the fully connected layer and classification layer of the existing task set, establish a new classification layer corresponding to the new task set, and update the training model using a preset learning rate.

[0092] When the model is trained and a new task set needs to be added later, the fully connected layer and classification layer of the existing task set are fixed, the new task set data is added to the training set, all other parameters are fixed first, only the new classification layer is trained, and then the entire training model is fine-tuned to adapt the training model to the new task set. When fine-tuning the training model, a smaller learning rate can be used to fine-tune the model. The specific learning rate is not limited here and can be set by technicians in this field.

[0093] It can be seen that, based on the previous embodiment, the actions are divided into action groups, and each action group corresponds to a classification layer in the model training process. When it is necessary to support the recognition of new actions, it is only necessary to add a new classification layer in the process of training the model, so that the training model can be flexibly expanded without retraining the entire training model. While ensuring that the indicators of each task remain unchanged, the new task set can be easily expanded.

[0094] At this time, the complete execution process of this embodiment is as follows:

[0095] The first step is to receive the video to be identified;

[0096] Step 2: determine whether the training model contains all action groups corresponding to the video to be identified; if so, proceed to step 3; if not, proceed to step 5;

[0097] Step 3: input the video to be identified into the training model;

[0098] Step 4: In the training model, the frame difference of the video to be identified is used to perform action recognition of the corresponding action group to determine the behavior actions contained in the video to be identified;

[0099] Step 5: Obtain new action behaviors of the new action group;

[0100] Step 6: Add the new action behavior as a new task set to the training set;

[0101] Step 7: Fix the fully connected layer and classification layer of the existing task set, establish a new classification layer corresponding to the new task set, update the training model using the preset learning rate, and return to step 4.

[0102] The following uses the behavior monitoring of pedestrians in the subway as an example to supplement the training model provided by this application. Usually, the behavior of pedestrians in the subway can be divided into three task sets: A{climbing over the gate, tailgating, normal passage}, B{falling, normal}, and C{fighting, normal}, which are trained separately, combined, and trained according to the generation method of the training model of this application, that is, first train AB and then add C, and when training C, only add the training data of C to train the classification layer of C, and then add a small number of AB samples to fine-tune the entire network to obtain the training model. The recall rate is used to measure whether multiple positive examples are classified as positive examples, and is usually used to reflect the accuracy of the machine learning model. At this time, the recall rates under different training methods are shown in Table 1 below:

[0103] Table 1 Different training methods and corresponding recall rates

[0104]

[0105]

[0106] From the results in Table 1, we can see that after the combined training, the indicators of task B have improved, while those of task A and task B have slightly decreased. When AB is trained first and then C is added, the indicators are slightly lower than before, but since only the data of C is trained, the training time is much shorter than retraining ABC from scratch, which saves training costs and facilitates the updating and iteration of the training model to adapt to different behavior recognition scenarios.

[0107] A behavior recognition system applied to surveillance videos provided in an embodiment of the present application is introduced below. The behavior recognition system described below and the behavior recognition method applied to surveillance videos described above can be referenced to each other.

[0108] See also Figure 3 , Figure 3 A schematic diagram of the structure of a behavior recognition system for monitoring video provided in an embodiment of the present application, the system includes:

[0109] A receiving module, used for receiving a video to be identified;

[0110] A judgment module, used to judge whether the training model contains all action groups to be identified;

[0111] An input module, used for inputting the video to be identified into a training model when the judgment result of the judgment module is yes;

[0112] The recognition module is used to perform action recognition of the corresponding action group on the frame difference of the video to be recognized in the training model to obtain the behavioral actions contained in the video to be recognized.

[0113] Based on the above embodiments, as a preferred embodiment, it also includes:

[0114] The model training module is used to obtain a training video; divide the training video into several video segments and mark the behavior actions in each of the video segments; extract preset frame images at equal intervals in the video segments, and calculate the frame difference between adjacent frame images in the extracted images; respectively extract the stage features of the images and the frame differences, combine and convolve to obtain a preset fully connected layer; and construct the training model according to the preset fully connected layer.

[0115] Based on the above embodiment, as a preferred embodiment, if the number of the pictures is N, the model training module includes:

[0116] A fully connected layer configuration unit is used to, if N is an odd number, use the (N+1) / 2th picture as the target picture; if N is an even number, use the N / 2th and N / 2+1th pictures as the target pictures; use a first residual network to extract the first stage features of the target picture; use a second residual network to extract the second stage features of the frame difference; combine the first stage features and the second stage features to obtain the preset fully connected layer.

[0117] Based on the above embodiment, as a preferred embodiment, the model training module may further include:

[0118] A normalization unit is used to normalize the target image and the frame difference to within a preset range.

[0119] Based on the above embodiments, as a preferred embodiment, it also includes:

[0120] The model update module is used to obtain new action behaviors of the new action group; add the new action behaviors to the training set as a new task set; fix the fully connected layer and classification layer of the existing task set, establish a new classification layer corresponding to the new task set, and update the training model using a preset learning rate.

[0121] Based on the above embodiment, as a preferred embodiment, the model updating module includes:

[0122] A new unit is added to the classification layer, which is used to determine a preset number of task sets; a preset number of fully connected layers are connected after the preset fully connected layer; each of the fully connected layers is connected to the classification layer of the corresponding category of each task set; the task set is used as the model input, and a mixed loss and a preset number of sub-task losses are configured; the mixed loss is used to remove the last dimension of the output of each classification layer and combine them to form an output vector.

[0123] Based on the above embodiments, as a preferred embodiment, it also includes:

[0124] A loss weight configuration unit is used to configure the weight values ​​of the subtask loss and the mixed loss respectively according to the number and category of the task set.

[0125] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed, the steps provided in the above embodiment can be implemented. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0126] The present application also provides a server, which may include a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps provided in the above embodiment may be implemented. Of course, the server may also include various network interfaces, power supplies and other components.

[0127] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the system provided in the embodiment, since it corresponds to the method provided in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0128] Specific examples are used herein to illustrate the principles and implementation methods of the present application, and the description of the above embodiments is only used to help understand the method and core ideas of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

[0129] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A behavior recognition method applied to surveillance video, characterized in that: include: Receiving a video to be identified; Determine whether the training model contains all action groups corresponding to the video to be identified; The action group is a grouping of user behavior actions according to action status and monitoring target; If yes, input the video to be identified into the training model; In the training model, the frame difference of the video to be identified is used to perform action recognition of the corresponding action group to determine the behavior actions contained in the video to be identified; The frame difference is a frame difference feature before and after the video to be identified; Before judging whether the training model contains all action groups to be identified, the following steps are also included: Get the training video; Divide the training video into a plurality of video segments, and mark the behavior actions in each of the video segments; Extracting preset frame pictures at equal intervals in the video segment, and calculating frame differences between adjacent frame pictures in the extracted pictures; Extracting the respective stage features of the image and the frame difference, combining and convolving them to obtain a preset fully connected layer; Constructing the training model according to the preset fully connected layer; If the training model does not include all action groups to be identified, it also includes: Get the new action behavior of the new action group; Adding the new action behavior as a new task set to the training set; The fully connected layer and classification layer of the existing task set are fixed, a new classification layer corresponding to the new task set is established, and the training model is updated using a preset learning rate.

2. The behavior recognition method according to claim 1, characterized in that: If the number of the pictures is N, respectively extracting the stage features of the pictures and the frame differences, combining and convolving to obtain a preset fully connected layer includes: If N is an odd number, the (N+1) / 2th picture is used as the target picture; if N is an even number, the N / 2th and N / 2+1th pictures are used as the target pictures; the first stage features of the target picture are extracted using the first residual network; Extracting the second stage features of the frame difference using a second residual network; The first-stage features and the second-stage features are combined to obtain the preset fully connected layer.

3. The behavior recognition method according to claim 1 or 2, characterized in that: After calculating the frame difference between adjacent frame pictures, it also includes: The target image and the frame difference are normalized within a preset range.

4. The behavior recognition method according to claim 1, characterized in that: Constructing the training model according to the preset fully connected layer includes: Determine a preset number of task sets; A preset number of fully connected layers are connected after the preset fully connected layer; each of the fully connected layers is connected to a classification layer corresponding to a category of each task set; The task set is used as a model input, and a mixed loss and a preset number of subtask losses are configured; the mixed loss is used to remove the last dimension of the output of each classification layer and combine them to form an output vector.

5. The behavior recognition method according to claim 4, characterized in that: When configuring a hybrid loss and a preset number of subtask losses, it also includes: The weight values ​​of the subtask loss and the mixed loss are respectively configured according to the number and category of the task set.

6. A behavior recognition system applied to surveillance video, characterized in that: include: A receiving module, used for receiving a video to be identified; A judgment module, used to judge whether the training model contains all action groups to be identified; The action group is a grouping of user behavior actions according to action status and monitoring target; An input module, used for inputting the video to be identified into a training model when the judgment result of the judgment module is yes; A recognition module, used for performing action recognition of a corresponding action group on the frame difference of the video to be recognized in the training model to obtain the behavior actions contained in the video to be recognized; The model training module is used to obtain a training video; divide the training video into a plurality of video segments and mark the behavior actions in each of the video segments; extract preset frame images at equal intervals in the video segments, and calculate the frame difference between adjacent frame images in the extracted images; Extracting the respective stage features of the picture and the frame difference, combining and convolving them to obtain a preset fully connected layer; constructing the training model according to the preset fully connected layer; Model update module, used to obtain new action behaviors of new action groups; Adding the new action behavior as a new task set to the training set; The fully connected layer and classification layer of the existing task set are fixed, a new classification layer corresponding to the new task set is established, and the training model is updated using a preset learning rate.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the behavior recognition method applied to surveillance video as described in any one of claims 1 to 5 are implemented.

8. A server, characterized in that: It comprises a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps of the behavior recognition method applied to surveillance video as described in any one of claims 1 to 5 are implemented.

Images