Motion recognition method and computing device

Abstract

The present application discloses an action recognition method and computing device. The action recognition method includes: sending image data to an action recognition model to obtain an action recognition result, wherein the action recognition model is a new EfficientNet recognition network model constructed based on the basic network structure of EfficientNet, and the new EfficientNet recognition network model modifies at least one of the modules, layers and operation directions in the basic network structure of EfficientNet. The training method selects one of the multiple new EfficientNet recognition network models as the action recognition model, so that in the action recognition process, the operation speed can be improved and the operation accuracy can be guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more specifically, to an action recognition method and a computing device. Background Art

[0002] In recent years, due to the development of educational video live broadcast, there is a need for an educational intelligent recording system. The intelligent recording system uses a high-definition camera to capture the images of teachers and students, and through video image analysis, it can switch between multiple teaching scenes, automatically follow the teachers and students, and identify the movements of the subjects.

[0003] The intelligent recording system needs to detect and track the teacher in the video and recognize the teacher's lecture movements. However, conventional posture recognition methods, such as OpenPose and DensePose, are time-consuming. In actual teacher lecture recognition methods, the processing speed is often too slow to meet the needs of practical applications.

[0004] For example, the OpenPose algorithm is used to detect teachers in images and recognize teaching actions. The OpenPose algorithm is used to extract key points of the human body in the input image. The algorithm adopts a bottom-up strategy. It first extracts the key point positions in the image, and then calculates the human skeleton information through the learned key point relationships. Based on these key point information, the position of the teacher in the image can be calculated, which solves the problem of teacher detection. However, in actual teacher teaching recognition methods, the algorithm's processing speed is often slow and cannot meet the needs of actual applications.

[0005] Therefore, it is expected to improve the action recognition method to increase the operation speed. Summary of the invention

[0006] In view of the above problems, an object of the present invention is to provide a motion recognition method and a computing device to improve the computing speed.

[0007] According to one aspect of the present invention, there is provided an action recognition method, comprising: sending image data to an action recognition model to obtain an action recognition result, wherein the action recognition model is a new EfficientNet recognition network model constructed based on the basic network structure of EfficientNet, and the new EfficientNet recognition network model modifies at least one of the modules, layers and operation directions in the basic network structure of EfficientNet.

[0008] Optionally, the training method of the action recognition module includes: receiving randomly sampled data representing multiple action categories to obtain a training set; constructing multiple new EfficientNet recognition network models based on the basic network structure of EfficientNet; testing the multiple new EfficientNet recognition network models based on the training set to obtain multiple test results; and selecting one of the multiple new EfficientNet recognition network models as the action recognition model based on the multiple test results.

[0009] Optionally, each of the EfficientNet recognition network models includes multiple mobile flip bottleneck convolution modules, 2 convolution layers, 1 global average pooling layer and 1 classification layer, wherein the output of each of the EfficientNet recognition network models matches the number of categories contained in the training set.

[0010] Optionally, in multiple EfficientNet recognition network models, the mobile flip bottleneck convolution module has different activation functions, and the activation function is selected from any one or more of sigmoid, tanh, ReLU, Leaky ReLU, and swish.

[0011] Optionally, in multiple EfficientNet recognition network models, the mobile flip bottleneck convolution module includes a first convolution, a first batch of normalization layers, a first activation function, a second convolution, a second batch of normalization layers, a second activation function, a global average pooling layer, a third convolution, a third activation function, a fourth convolution, a fourth activation function, data fusion, a fifth convolution, and a second batch of normalization layers, wherein each step in the mobile flip bottleneck convolution module is adjustable.

[0012] Optionally, in multiple EfficientNet recognition network models, the method of performing data fusion by the mobile flip bottleneck convolution module includes multiplication, addition or direct splicing into an array.

[0013] Optionally, in multiple EfficientNet recognition network models, the convolution kernel size, convolution step size and output channel of the mobile flip bottleneck convolution module are adjusted.

[0014] Optionally, in multiple EfficientNet recognition network models, the number of times the mobile flip bottleneck convolution module is used is not exactly the same.

[0015] Optionally, one of the multiple EfficientNet recognition network models with the fastest operation speed is selected as the action recognition model.

[0016] According to another aspect of the present invention, there is provided a motion recognition method, comprising: sending image data to a motion recognition model to obtain a motion recognition result, wherein the motion recognition model is obtained by using the motion recognition method as described above.

[0017] Optionally, it also includes smoothing the action recognition results, and the smoothing includes: if the linked list length of the current frame is not greater than a predetermined value, the action recognition result of the current frame is added to the queue and a null value is output; if the linked list length of the current frame is greater than a predetermined value, the action recognition result first stored in the head of the linked list is taken out, the action recognition result of the current frame is added to the queue, and the maximum number of consecutive occurrences of each action category in the queue is counted; when the maximum number of consecutive occurrences of the action category is not greater than a threshold, the action recognition result of the previous frame is output; when the maximum number of consecutive occurrences of the action category is greater than the threshold, the maximum number of consecutive occurrences of the action category is updated and output.

[0018] According to another aspect of the present invention, a computing device is provided, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include instructions for executing any of the action recognition methods described above.

[0019] The motion recognition method and computing device provided by the present invention obtain a motion recognition model by modifying the basic network structure of EfficientNet, which greatly improves the calculation speed compared with the traditional motion recognition model.

[0020] Furthermore, in this technical solution, a variety of different improved EfficientNet recognition network models are constructed, and one of the models with better effects is selected as the action recognition model, which can further improve the operation speed.

[0021] Furthermore, in this technical solution, the model can be compressed while ensuring accuracy according to the actual task and working platform, and operators with fast computing speed on the platform can be found as much as possible. By continuously trying to improve, an action recognition model with fast computing speed and high accuracy can be finally obtained.

[0022] Furthermore, a smoothing step is added to the action recognition method, which can make the intelligent recording and broadcasting system more stable and perform better. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0024] Figure 1A flow chart of a method for motion recognition according to an embodiment of the present invention is shown;

[0025] Figure 2 A schematic diagram of a basic network structure based on EfficientNet according to an embodiment of the present invention is shown;

[0026] Figure 3 A schematic diagram of a mobile flip bottleneck convolution module according to an embodiment of the present invention is shown;

[0027] Figure 4 A flow chart of smoothing processing in a motion recognition method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0028] Various embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. In each of the accompanying drawings, the same elements are represented by the same or similar reference numerals. For the sake of clarity, the various parts in the accompanying drawings are not drawn to scale.

[0029] The specific implementation of the present invention is further described in detail below in conjunction with the drawings and examples.

[0030] Figure 1 A flow chart of a method for motion recognition according to an embodiment of the present invention is shown; Figure 2 A schematic diagram of a basic network structure based on EfficientNet according to an embodiment of the present invention is shown; Figure 3 A schematic diagram of a mobile flip bottleneck convolution module according to an embodiment of the present invention is shown; Figure 4 A flow chart of smoothing processing in a motion recognition method according to an embodiment of the present invention is shown.

[0031] The action recognition method provided in the present application includes: sending image data to an action recognition model to obtain an action recognition result, wherein the action recognition model is a new EfficientNet recognition network model constructed based on the basic network structure of EfficientNet, and the new EfficientNet recognition network model modifies at least one of the modules, layers and operation directions in the basic network structure of EfficientNet.

[0032] Specifically, the action recognition method includes steps S101-S105, and optionally, further includes step S106, wherein steps S101-S104 are specific embodiments of the training method of the action recognition model.

[0033] In step S101, randomly sampled data representing multiple action categories are received to obtain a training set. Taking the teaching scene as an example, data of various teachers in various teaching scenes are collected, and the teacher target is detected by using an image detection algorithm, and the data is saved by random sampling, and at least the data is used as a training set.

[0034] In step S102, multiple new EfficientNet recognition network models are constructed based on the basic network structure of EfficientNet. In this step, each EfficientNet recognition network model includes multiple mobile flip bottleneck convolution modules, 2 convolution layers, 1 global average pooling layer and 1 classification layer, and the output of each EfficientNet recognition network model matches the number of categories contained in the training set. For example, there are 16 mobile flip bottleneck convolution modules.

[0035] Optionally, in multiple EfficientNet recognition network models, the mobile flip bottleneck convolution modules have different activation functions, and the activation function is selected from any one or more of sigmoid, tanh, ReLU, Leaky ReLU, and swish.

[0036] Optionally, in multiple EfficientNet recognition network models, the mobile flip bottleneck convolution module includes the mobile flip bottleneck convolution module including a first convolution, a first batch of normalization layers, a first activation function, a second convolution, a second batch of normalization layers, a second activation function, a global average pooling layer, a third convolution, a third activation function, a fourth convolution, a fourth activation function, data fusion, a fifth convolution, and a second batch of normalization layers, wherein each step in the mobile flip bottleneck convolution module is adjustable.

[0037] Optionally, in multiple EfficientNet recognition network models, methods of performing data fusion by moving the flip bottleneck convolution module include multiplication, addition, or direct concatenation into an array.

[0038] Optionally, in multiple EfficientNet recognition network models, adjust the convolution kernel size, convolution stride, and output channels of the mobile flip bottleneck convolution module.

[0039] Optionally, in multiple EfficientNet recognition network models, the number of times the mobile flip bottleneck convolution module is used is not exactly the same.

[0040] EfficientNet is a standardized convolutional network expansion method that can achieve high accuracy and fully save computing resources. The basic network architecture of EfficientNet is designed by using neural architecture search. Convolutional neural network models are usually trained under the condition of known hardware resources. When better hardware resources are available, better training results can be obtained by scaling up the network model. In order to systematically study model scaling, the researchers proposed a new model scaling method for the basic network model of EfficientNet, which uses a simple and efficient composite coefficient to weigh the network depth, width, and input image resolution. By scaling up the EfficientNet basic model, a series of EfficientNet models were obtained. This series of models has high efficiency and accuracy.

[0041] The mobile flip bottleneck convolution is also obtained through the neural network architecture search. The module structure is similar to the depthwise separable convolution. The mobile flip bottleneck convolution first performs a 1x1 point-by-point convolution on the input and changes the output channel dimension according to the expansion ratio (for example, when the expansion ratio is 3, the channel dimension will be increased by 3 times. But if the expansion ratio is 1, the 1x1 point-by-point convolution and the subsequent batch normalization and activation function are directly omitted). Then a kxk depthwise convolution is performed. If compression and excitation operations are to be introduced, this operation will be performed after the depthwise convolution. The original channel dimension is restored at the end with a 1x1 point-by-point convolution. Finally, we perform drop connect and skip connection on the input. This method gives the model a random depth, shortens the time required for model training, and improves model performance (note that in EfficientNet, only when the same mobile flip bottleneck convolution is repeated, will the connection be dropped and the input skip connection be skipped, and the depth convolution step will be changed to 1). Drop connect is an operation similar to dropout, and an identical skip is added at the beginning and end of the module. Note that batch normalization is performed after each convolution operation in this module, and the activation function uses the Swish activation function.

[0042] Specifically, taking the EfficientNet-b0 model as an example, the basic network structure of EfficientNet of this application is explained.

[0043] In the first stage T1, the input 224x224x3 image is sequentially subjected to convolution, batch normalization (BatchNormalization, BN) and activation function to obtain the result of the first stage.

[0044] In the convolution (the convolution kernel is, for example, 32 kernels 3×3×3, the step size is, for example, 2×2, and the padding is "same", that is, the width and height of the output are reduced by half), the output of the convolution operation is a feature map with a dimension of (112×112×32).

[0045] In the batch normalization layer, the input of this layer is a feature map of (112×112×32), so the total number of parameters in this layer is 128 (32x4).

[0046] In the activation function, the activation function is, for example, a Swish activation function. Optionally, the activation function is selected from any one or more of sigmoid, tanh, ReLU, Leaky ReLU, and swish.

[0047] In the second stage T2, the 112x112x32 feature map output in the previous stage is subjected to a moving flip bottleneck convolution (with an expansion ratio of 1, a depth convolution kernel size of 3x3, a kernel step size of 1x1, including compression and excitation operations, connection deactivation and connection skipping), and the result of the second stage is output.

[0048] Please refer to Figure 3 The mobile flip bottleneck convolution module includes the first convolution, the first batch normalization layer, the first activation function, the second convolution, the second batch normalization layer, the second activation function, the global average pooling layer, the third convolution, the third activation function, the fourth convolution, the fourth activation function, data fusion, the fifth convolution, and the second batch normalization layer. Among them, the first convolution, the first batch normalization layer, and the first activation function can be used as a data stream, the second convolution, the second batch normalization layer, and the second activation function can be used as a data stream, the global average pooling layer, the third convolution, the third activation function, the fourth convolution, and the fourth activation function can be used as a data stream, and the fifth convolution and the second batch normalization layer can be used as a data stream.

[0049] The first convolution is, for example, a point-by-point convolution with an expansion ratio of 6 (the convolution kernel is 96 kernels 1×1×16, the step size is 1×1, and the padding is "same", that is, the width and height of the output remain unchanged). The output of the convolution operation is a feature map with a dimension of (112×112×96).

[0050] In the first batch normalization layer (Batch Normalization, BN), the input of this layer is a feature map of (112×112×96).

[0051] In the first activation function, the activation function is, for example, a Swish activation function. Optionally, the activation function is selected from any one or more of sigmoid, tanh, ReLU, Leaky ReLU, and swish.

[0052] The second convolution is, for example, a depthwise convolution (with 6 convolution kernels (3×3×16), a stride of 2×2, and padding of “same”, i.e., the width and height of the output are halved). The output of the depthwise convolution is a feature map with a dimension of (56×56×96).

[0053] In the first batch normalization layer (Batch Normalization, BN), the input of this layer is a (56×56×96) feature map.

[0054] In the second activation function, the activation function is, for example, a Swish activation function. Optionally, the activation function is selected from any one or more of sigmoid, tanh, ReLU, Leaky ReLU, and swish.

[0055] The global average pooling layer performs global average pooling in the channel dimension and outputs a feature map of (1x1x96).

[0056] In the third convolution (the first convolution in the compression and excitation module, the convolution kernel is 4 kernels 1x1x96, the stride is 1×1, and the padding is "same", that is, the width and height of the output remain unchanged), the output of the convolution operation is a feature map with a dimension of (1×1×4).

[0057] In the third activation function, the activation function is, for example, a Swish activation function. Optionally, the activation function is selected from any one or more of sigmoid, tanh, ReLU, Leaky ReLU, and swish.

[0058] In the fourth convolution (the second convolution in the compression and excitation module, the convolution kernel is 96 kernels 1x1x4, the stride is 1×1, and the padding is "same", that is, the width and height of the output remain unchanged), the output of the convolution operation is a feature map with a dimension of (1×1×96).

[0059] In the fourth activation function, the activation function is, for example, a Sigmoid activation function. Optionally, the activation function is selected from any one or more of sigmoid, tanh, ReLU, Leaky ReLU, and swish.

[0060] In data fusion, the result of the fourth activation function is multiplied by the result of the second activation function to obtain a 56x56x96 feature map. The fusion location is defined as the place where the two data streams are combined. The location of data fusion is optional. Data fusion can be multiplied, added, or directly concatenated into array data.

[0061] The fifth convolution is, for example, a point-by-point convolution (the convolution kernel is 24 kernels 1×1×96, the step size is 1×1, and the padding is "same", that is, the width and height of the output remain unchanged). The output of the convolution operation is a feature map with a dimension of (56×56×24).

[0062] In the second batch normalization layer (Batch Normalization, BN), the input of this layer is a feature map of (56×56×24).

[0063] In the third stage T3, two moving flip bottleneck convolutions are performed on the 112x112x16 feature map output in the previous stage, the first one (expansion ratio is 6, depth convolution kernel size is 3x3, kernel step size is 2x2, including compression and excitation operations, connection deactivation and connection skipping), the second one (expansion ratio is 6, depth convolution kernel size is 3x3, kernel step size is 1x1, including compression and excitation operations, connection deactivation and connection skipping), and the result of the third stage is output.

[0064] In the fourth stage T4, two moving flip bottleneck convolutions are performed on the 56x56x24 feature map output by the previous stage. The first one (expansion ratio is 6, depth convolution kernel size is 5x5, kernel step size is 2x2, including compression and excitation operations, connection deactivation and connection skipping), and the second one (expansion ratio is 6, depth convolution kernel size is 5x5, kernel step size is 1x1, including compression and excitation operations, connection deactivation and connection skipping). The output is a 28x28x40 feature map.

[0065] In the fifth stage T5, the 28x28x40 feature map output by the previous stage is subjected to three moving flip bottleneck convolutions. The first one (expansion ratio is 6, depth convolution kernel size is 3x3, kernel step size is 2x2, including compression and excitation operations, connection deactivation and connection skipping), the second one (expansion ratio is 6, depth convolution kernel size is 3x3, kernel step size is 1x1, including compression and excitation operations, connection deactivation and connection skipping), and the third one (expansion ratio is 6, depth convolution kernel size is 3x3, kernel step size is 1x1, including compression and excitation operations, connection deactivation and connection skipping). The output is a 14x14x80 feature map.

[0066] In the sixth stage T6, the 14x14x80 feature map output by the previous stage is subjected to three moving flip bottleneck convolutions. The first one (expansion ratio is 6, deep convolution kernel size is 5x5, kernel step size is 1x1, including compression and excitation operations, connection deactivation and connection skipping), the second one (expansion ratio is 6, deep convolution kernel size is 5x5, kernel step size is 1x1, including compression and excitation operations, connection deactivation and connection skipping), and the third one (expansion ratio is 6, deep convolution kernel size is 5x5, kernel step size is 1x1, including compression and excitation operations, connection deactivation and connection skipping). The output is a 14x14x112 feature map.

[0067] In the seventh stage T7, four moving flip bottleneck convolutions are performed on the 14x14x112 feature map output by the previous stage. The first one (expansion ratio is 6, depth convolution kernel size is 5x5, kernel step size is 2x2, including compression and excitation operations, connection deactivation and connection skipping), the second one (expansion ratio is 6, depth convolution kernel size is 5x5, kernel step size is 1x1, including compression and excitation operations, connection deactivation and connection skipping), the third one (expansion ratio is 6, depth convolution kernel size is 5x5, kernel step size is 1x1, including compression and excitation operations, connection deactivation and connection skipping), the fourth one (expansion ratio is 6, depth convolution kernel size is 5x5, kernel step size is 1x1, including compression and excitation operations, connection deactivation and connection skipping), the output is a 7x7x192 feature map.

[0068] In the eighth stage T8, a moving flip bottleneck convolution is performed on the 7x7x192 feature map output by the previous stage (the expansion ratio is 6, the depth convolution kernel size is 3x3, the kernel step size is 1x1, including compression and excitation operations, connection deactivation and connection skipping), and the output is a 7x7x320 feature map.

[0069] In the ninth stage T9, the input 7x7x320 image is sequentially subjected to convolution, batch normalization layer, activation function, global average pooling layer, random dropout, and fully connected layer operations to obtain the final result of the model.

[0070] In the convolution (the convolution kernel is 1280, the kernel is 1×1×320, the stride is 1×1, and the padding is "same", that is, the width and height of the output remain unchanged), the output of the convolution operation is a feature map with a dimension of (7×7×1280).

[0071] In the Batch Normalization (BN) layer, the input of this layer is a feature map of (7×7×1280), so the total number of parameters in this layer is 5120 (1280x4), of which 2560 parameters need to be trained and learned.

[0072] In the activation function, the activation function is, for example, a Swish activation function. Optionally, the activation function is selected from any one or more of sigmoid, tanh, ReLU, Leaky ReLU, and swish.

[0073] In the global average pooling layer, the layer performs global average pooling in the channel dimension and outputs a feature map of (1x1x1280).

[0074] In dropout, the (1x1x1280) feature map is randomly dropped.

[0075] In the fully connected layer fc, there are 1000 neurons in this layer.

[0076] Finally, the Softmax activation function is used to output the classification results.

[0077] It should be understood that the various parameters in the above embodiments can be changed. For example, the output of the network needs to match its own recognition task (number of categories, etc.). For another example, the number of times the MBConv module is used in each stage can be changed. For another example, in the MBConv module, the order of data processing in each data stream can be changed.

[0078] The above embodiments are intended to illustrate the scheme of the present application. It should be understood that in addition to the EfficientNet-B0 recognition network, other series of EfficientNet recognition network models can also be used. For example, the EfficientNet series also has 7 other networks (EfficientNet-B1, EfficientNet-B2, EfficientNet-B3, EfficientNet-B4, EfficientNet-B5, EfficientNet-B6, EfficientNet-B7). These networks are obtained by scaling the model based on the parameters of EfficientNet-B0 under different number of operations and running memory constraints through neural network architecture search. It mainly involves three parameters: depth parameter, width parameter and input resolution parameter, which are used to control the scaling of the model. Among them, the depth parameter is multiplied by the number of module repetitions in each stage of EfficientNet-B0 to obtain a deeper network architecture; the width coefficient is multiplied by the number of cores input to each convolution operation in EfficientNet-B0 to obtain a network model with stronger expressive ability; the input resolution parameter controls the length and width of the input image of the network.

[0079] In step S103, multiple new EfficientNet recognition network models are tested based on the training set to obtain multiple test results.

[0080] In step S104, one of the multiple new EfficientNet recognition network models is selected as the action recognition model based on the multiple test results. For example, the one with the fastest operation speed among the multiple EfficientNet recognition network models is selected as the action recognition model.

[0081] In step S105, the image data is sent to the action recognition model to obtain the action recognition result. In this step, the image data is, for example, a data set containing educational scene videos. The educational scene videos in the data set are frame decomposed and the video format files are converted into a continuous image frame sequence to obtain the image data.

[0082] Optionally, the method further includes step S106 of smoothing the action recognition result.

[0083] like Figure 4 As shown, the smoothing process includes steps S1061-S1069.

[0084] In step S1061, it is determined whether the length of the linked list of the current frame is greater than a predetermined value. If the length of the linked list of the current frame is not greater than the predetermined value, steps S1062-S1063 are executed to add the action recognition result of the current frame to the queue and output a null value; if the length of the linked list of the current frame is greater than the predetermined value, steps S1064-S1069 are executed.

[0085] In step S1064, the first action recognition result stored is taken out from the head of the linked list; in step S1065, the action recognition result of the current frame is added to the queue; in step S1066, the maximum number of consecutive occurrences of each action category in the queue is counted; in step S1067, it is determined whether the maximum number of consecutive occurrences of the action category is greater than the threshold. When the maximum number of consecutive occurrences of the action category is not greater than the threshold, step S1068 is executed to output the action recognition result of the previous frame; when the maximum number of consecutive occurrences of the action category is greater than the threshold, step S1069 is executed to update and output the maximum number of consecutive occurrences of the action category.

[0086] According to the embodiments of the present invention as described above, these embodiments do not describe all the details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and changes can be made based on the above description. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can make good use of the present invention and the modified use based on the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for motion recognition in teaching scenarios, characterized in that: include: Send the image data to the action recognition model to obtain the action recognition result; as well as performing smoothing processing on the action recognition result, The action recognition model is a new EfficientNet recognition network model built based on the basic network structure of EfficientNet, and the new EfficientNet recognition network model modifies at least one of the modules, layers and operation directions in the basic network structure of EfficientNet. The training method of the action recognition model includes: Receiving randomly sampled data representing multiple action categories to obtain a training set; Based on the basic network structure of the EfficientNet, a plurality of new EfficientNet recognition network models are respectively constructed; Testing the multiple new EfficientNet recognition network models based on the training set respectively to obtain multiple test results; Selecting one of the multiple new EfficientNet recognition network models as the action recognition model based on the multiple test results, The smoothing process comprises: If the length of the linked list of the current frame is not greater than the predetermined value, the action recognition result of the current frame is added to the queue and a null value is output; If the length of the linked list of the current frame is greater than the preset value, the first action recognition result stored is taken out from the head of the linked list, the action recognition result of the current frame is added to the queue, and the maximum number of consecutive occurrences of each action category in the queue is counted. When the maximum number of consecutive occurrences of an action category is not greater than a threshold, the action recognition result of the previous frame is output; when the maximum number of consecutive occurrences of an action category is greater than a threshold, the maximum number of consecutive occurrences of an action category is updated and output.

2. The action recognition method according to claim 1, characterized in that: Each of the EfficientNet recognition network models includes multiple mobile flip bottleneck convolution modules, 2 convolution layers, 1 global average pooling layer and 1 classification layer. Among them, the output of each of the EfficientNet recognition network models matches the number of categories contained in the training set.

3. The action recognition method according to claim 2, characterized in that: In multiple EfficientNet recognition network models, the mobile flip bottleneck convolution module has different activation functions, and the activation function is selected from any one or more of sigmoid, tanh, ReLU, Leaky ReLU, and swish.

4. The action recognition method according to claim 2, characterized in that: In the multiple EfficientNet recognition network models, the mobile flip bottleneck convolution module includes a first convolution, a first batch of normalization layers, a first activation function, a second convolution, a second batch of normalization layers, a second activation function, a global average pooling layer, a third convolution, a third activation function, a fourth convolution, a fourth activation function, data fusion, a fifth convolution, and a second batch of normalization layers. Among them, each step in the moving flip bottleneck convolution module is adjustable.

5. The action recognition method according to claim 4, characterized in that: In multiple EfficientNet recognition network models, the method of performing data fusion by the mobile flip bottleneck convolution module includes multiplication, addition or direct splicing into an array.

6. The action recognition method according to claim 2, characterized in that: In the multiple EfficientNet recognition network models, the convolution kernel size, convolution step size and output channel of the mobile flip bottleneck convolution module are adjusted.

7. The action recognition method according to claim 2, characterized in that: In a plurality of the EfficientNet recognition network models, the number of times the mobile flip bottleneck convolution module is used is not exactly the same.

8. The action recognition method according to claim 1, characterized in that: The one with the fastest operation speed among the multiple EfficientNet recognition network models is selected as the action recognition model.

9. A computing device comprising: one or more processors; Memory; as well as One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any one of the motion recognition methods described in claims 1-8.

Images