A video recognition method based on three-dimensional convolutional neural network tensor mapping compression

Abstract

The application belongs to the technical field of computer vision processing, and particularly relates to a video recognition method based on three-dimensional convolutional neural network tensor mapping compression, and the core is to process the trained three-dimensional convolutional neural network to obtain a compressed three-dimensional convolutional neural network. In the process, firstly, the to-be-compressed convolution kernel is converted into a matrix, and then the matrix is compressed by using a fine-grained tensor decomposition form; then, the factor matrix obtained by compression is reorganized and expanded into a tensor to form a four-layer continuous lightweight convolution structure; after all the to-be-compressed convolution kernels are compressed, a deeper lightweight three-dimensional convolutional neural network is formed, and the final result can be obtained by training the three-dimensional convolutional neural network. The application reduces the parameter quantity of the three-dimensional convolutional neural network to facilitate deployment in a resource-limited video recognition environment, and improves the calculation efficiency of the three-dimensional convolutional neural network and the processing speed of video recognition.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision processing, and particularly relates to a video recognition method based on three-dimensional convolutional neural network tensor mapping compression. BACKGROUND

[0002] In recent years, video recognition is widely applied in the fields of equipment manufacturing, monitoring security, and unmanned driving. With the development and maturity of deep learning technology, various deep neural network models have made a great leap in video recognition effect by virtue of their excellent feature extraction capability, and gradually replaced traditional methods such as computer vision hand-crafted features and visual geometry analysis. In particular, three-dimensional convolutional neural network (3DCNN) can extract features layer by layer from the high-dimensional tensor angle of the video segment, and has the characteristics of convenient modeling, high recognition accuracy, and strong reliability compared with other methods.

[0003] However, the three-dimensional convolution kernel in 3DCNN has large parameter quantity and high computational complexity, which limits its use in resource-constrained environments. Therefore, it is of great significance to compress the three-dimensional convolution kernel reasonably. Wang et al. first compressed the three-dimensional convolution kernel by tensor decomposition in the literature Compressing 3DCNNs based on tensor train decomposition in 2020, and proved that the three-dimensional convolution kernel has high redundancy, which can maintain the recognition accuracy of 3DCNN while achieving high compression rate. Lee et al. combined tensor decomposition and data quantization in the literature QTTNet: Quantized tensor train neural networks for 3D object and video recognition in 2021, which not only reduces the size of 3DCNN, but also speeds up the calculation of 3DCNN to a certain extent.

[0004] The problem is that the above existing works are based on the estimation ability of tensor decomposition on the original convolution kernel to compress the three-dimensional convolution kernel. In theory, the factor tensor after decomposition must be restored to the convolution kernel before calculation with the input, which brings additional computational complexity. If data quantization is used for acceleration, only floating-point calculations are converted to integer-bit operations, and the overall computational complexity does not decrease. SUMMARY

[0005] (I) Technical problems to be solved

[0006] The technical problem to be solved by the present application is how to solve the problem that the existing video recognition technology uses a three-dimensional convolutional neural network with large scale and slow calculation, and the related compression means cannot meet the fast calculation in a resource-limited environment.

[0007] (II) Technical solutions

[0008] To solve the above technical problems, the present application provides a video recognition method based on three-dimensional convolutional neural network tensor mapping compression, which comprises the following steps:

[0009] Step A: taking the total number of frames T o Collecting video data of sensors such as cameras and cameras, denoted as Wherein, W o represents the width of each frame of video image, H o represents the height of each frame of video image, T o , that is, the total number of frames of video data, that is, the time length, C o represents the number of channels of each frame of video image;

[0010] Step B: using any three-dimensional convolutional neural network with more than 3 layers to identify the video data ;

[0011] Step C: outputting the result after identifying the video data.

[0012] Wherein, the step B comprises:

[0013] Step B1: training the three-dimensional convolutional neural network used for identifying the video data, and the trained three-dimensional convolutional neural network is denoted as Output to step B2;

[0014] Step B2: using a three-dimensional convolutional neural network tensor mapping compression method to process the three-dimensional convolutional neural network trained in step B1 , to obtain a compressed three-dimensional convolutional neural network

[0015] Step B3: using the three-dimensional convolutional neural network compressed in step B2 to identify the video data , to generate the result after identifying the video data to be output.

[0016] Wherein, the step B2 comprises:

[0017] Step 1: taking any three-dimensional convolutional neural network , and taking one layer of three-dimensional convolution as f(·) is denoted as the nonlinear activation function of this layer, denoted as a WxHxT size input feature map of C channels, WxHxT-sized output feature maps denoted as S channels, read a three-dimensional convolution kernel with a window size of w x h x t

[0018] Step 2: Reconstruct the three-dimensional convolution kernel into a matrix Use a fine-grained tensor decomposition method to decompose it, that is,

[0019]

[0020] wherein, and are factor matrices of the kth (k ∈ {1, 2, …, K}) decomposition of the A group, and are factor matrices of the kth decomposition of the B group, is the rank of the kth decomposition of the A group, is the rank of the kth decomposition of the B group, and are unit diagonal matrices, the operator symbol · represents contraction, and the operator symbol represents the Kronecker product;

[0021] Step 3: Ignore the unit diagonal matrices that do not affect the data content and Reorganize the other factor matrices and into four matrices, that is,

[0022]

[0023]

[0024]

[0025]

[0026] wherein,

[0027] Step 4: Expand A1, A2, B1, and B2 into tensors, that is,

[0028]

[0029]

[0030]

[0031]

[0032] Step 5: Let the input feature map be sequentially multiplied by The calculation is performed, and a new nonlinear activation function f(·) is inserted after each calculation to obtain a new output feature map. Right now:

[0033]

[0034] in, Represents a three-dimensional separable convolution;

[0035] After step 5 is completed, if the 3D convolutional neural network... If other layers in the 3D convolutional neural network still have uncompressed 3D convolutional kernels, then proceed to step 1 to compress the next 3D convolutional kernel. All the 3D convolutional kernels to be compressed have been compressed, resulting in a new 3D convolutional neural network. Proceed to step 6;

[0036] Step 6: Apply the backpropagation algorithm to the newly compressed 3D convolutional neural network. Train it to optimize its performance.

[0037] In step 2, the factors of group A After the shrinking operation, all ranks will be eliminated. We obtain a matrix of size wh×C, and the factors of group B. After the shrinking operation, all ranks will be eliminated. We obtain a matrix of size t×S, and these two matrices are then subjected to the Kronecker product operation. The wht×CS matrix of the k-th path is obtained, and the sum of all K-path matrices can be used to recover the matrix. The original 3D convolution kernel can be obtained by reconstruction.

[0038] In step 3, R is required to (B) / R (A) It is an integer.

[0039] In step 4, the matrix is ​​expanded into a tensor by adding an extra dimension with a modulus of 1, while the number of elements and the content remain unchanged.

[0040] In step 5, the input feature map is... sequentially with The specific calculation process is as follows:

[0041] Input feature map First with Perform a 3D convolution with a scanning window of 1×1×1, and transform the number of channels from C to R. (A) The output is activated by f(·); then combined with R (A) Channel 3D separable convolution, the output is activated by f(·);And R (B) / R (A) R (A) Channel 3D separable convolution, the output is activated by f(·);And R (B) Transformed into S, the output is activated by f(·).

[0042] (III) Advantages

[0043] In view of the problems that the three-dimensional convolutional neural network used by the existing video recognition technology is large in size, slow in calculation, and the related compression means cannot meet the fast calculation in the resource limited environment, a video recognition method based on three-dimensional convolutional neural network tensor mapping compression is provided, and a three-dimensional convolutional neural network tensor mapping compression method is used to implement the specific video recognition link. Through fine-grained tensor decomposition, the convolution kernel in the trained three-dimensional convolutional neural network for video recognition is compressed and mapped into four layers of lightweight convolution modules, on the one hand, the parameter amount of the three-dimensional convolutional neural network is reduced so as to be deployed in the resource limited video recognition environment, and on the other hand, the calculation efficiency of the three-dimensional convolutional neural network and the processing speed of the video recognition are improved.

[0044] Specifically, the three-dimensional convolution kernel is compressed into a four-layer continuous lightweight convolution structure in a tensor mapping manner by using fine-grained tensor decomposition, so that the original 3DCNN can reduce the parameter amount and the total calculation amount, and the additional nonlinear activation function can also improve the overall expression ability of the 3DCNN to a certain extent. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 is the flowchart of the technical scheme of the present application;

[0046] Figure 2 is the schematic diagram of fine-grained tensor decomposition;

[0047] Figure 3 is the schematic diagram of converting three-dimensional separable convolution into two-dimensional separable convolution;

[0048] Figure 4 is the schematic diagram of converting three-dimensional separable convolution into one-dimensional separable convolution;

[0049] Figure 5 is the three-dimensional convolutional neural network structure design diagram of the comparative experiment. DETAILED DESCRIPTION

[0050] To make the objectives, contents, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.

[0051] To address the aforementioned technical problems, this invention provides a video recognition method based on three-dimensional convolutional neural network tensor mapping compression, the method comprising:

[0052] Step A: Using the total number of frames T o Video data collected from sensors such as cameras and camcorders is denoted as... Among them, W o H represents the width of each video image frame. o T represents the height of each video frame. o That is, the total number of frames in the video data, which is also the duration, C o This indicates the number of channels in each frame of video image;

[0053] Step B: Use any 3D convolutional neural network with more than 3 layers to process the video data. Perform identification;

[0054] Step C: Output the results of video data recognition.

[0055] Step B includes:

[0056] Step B1: Train the 3D convolutional neural network used for recognizing video data. The trained 3D convolutional neural network is denoted as... Output to step B2;

[0057] Step B2: Apply the 3D convolutional neural network tensor mapping compression method to the 3D convolutional neural network trained in Step B1. After processing, a fully compressed 3D convolutional neural network is obtained.

[0058] Step B3: Complete the compressed 3D convolutional neural network using the methods from step B2. For video data The system performs recognition and generates the results of the recognized video data to be output.

[0059] Step B2 includes:

[0060] Step 1: For any 3D convolutional neural network Let one of the three-dimensional convolutional layers be . f(·) is denoted as the nonlinear activation function of this layer. Let the input feature map of size W×H×T be denoted as C channels. Let the output feature map of size W×H×T have S channels. Read the 3D convolutional kernel with a scanning window of w×h×t for this layer.

[0061] Step 2: Reconstruct the three-dimensional convolution kernel into a matrix Use the fine-grained tensor decomposition method to decompose it, that is,

[0062]

[0063] where, and are the factor matrices of the kth(k∈{1,2,…,K}) decomposition of group A, and are the factor matrices of the kth decomposition of group B, is the rank of the kth decomposition of group A, is the rank of the kth decomposition of group B, and are identity diagonal matrices, the operator symbol·represents contraction, and the operator symbol represents the Kronecker product;

[0064] Step 3: Ignore the identity diagonal matrices that do not affect the data content and Reorganize the other factor matrices and into four matrices, that is,

[0065]

[0066]

[0067]

[0068]

[0069] where,

[0070] Step 4: Extend A1, A2, B1, and B2 into tensors, that is,

[0071]

[0072]

[0073]

[0074]

[0075] Step 5: Let the input feature map be calculated with in turn, and insert a new nonlinear activation function f(·) after each calculation, and finally obtain a new output feature map Right now:

[0076]

[0077] in, Represents a three-dimensional separable convolution;

[0078] After step 5 is completed, if the three-dimensional convolutional neural network... If other layers in the 3D convolutional neural network still have uncompressed 3D convolutional kernels, then proceed to step 1 to compress the next 3D convolutional kernel. All the 3D convolutional kernels to be compressed have been compressed, resulting in a new 3D convolutional neural network. Proceed to step 6;

[0079] Step 6: Apply the backpropagation algorithm to the newly compressed 3D convolutional neural network. Train it to optimize its performance.

[0080] In step 2, the factors of group A After the shrinking operation, all ranks will be eliminated. We obtain a matrix of size wh×C, and the factors of group B. After the shrinking operation, all ranks will be eliminated. We obtain a matrix of size t×S, and these two matrices are then subjected to the Kronecker product operation. The wht×CS matrix of the k-th path is obtained, and the sum of all K-path matrices can be used to recover the matrix. The original 3D convolution kernel can be obtained by reconstruction.

[0081] In step 3, R is required to (B) / R (A) It is an integer.

[0082] In step 4, the matrix is ​​expanded into a tensor by adding an extra dimension with a modulus of 1, while the number of elements and the content remain unchanged.

[0083] In step 5, the input feature map is... sequentially with The specific calculation process is as follows:

[0084] Input feature map First with Perform a 3D convolution with a scanning window of 1×1×1, and transform the number of channels from C to R. (A) The output is activated by f(·); then combined with Perform a scan of R with a scanning window of w×h×1 (A) Channel-wise 3D separable convolution, the output is activated by f(·); then combined with A 3D convolution with a scanning window of 1x1xt and a total of R (B) / R (A) R of the group (A) Channel-wise 3D separable convolution, the output is activated by f(·); finally, a 3D convolution with a scanning window of 1x1x1 is performed, and the number of channels is R A 3D convolution with a scanning window of 1x1xt and a total of R (B) Transformed into S, the output is activated by f(·).

[0085] In the above technical solution, steps A, B1, B3 and C are conventional technical means, and the innovation of the present application is mainly concentrated in step B2. Therefore, specific embodiments are provided for this link as follows.

[0086] Embodiment 1

[0087] This embodiment is directed to step B2, and the technical solution adopted includes the following 6 steps as shown in the figure: Figure 1

[0088] Step 1: For any 3DCNN denoted as , denote one of the three-dimensional convolutions as f(·) is denoted as the nonlinear activation function of this layer, denoted as a WxHxT size input feature map with C channels, denoted as a WxHxT size output feature map with S channels, and a three-dimensional convolution kernel with a scanning window of w x h x t is read for this layer

[0089] The three-dimensional convolution kernel includes CxS scanning windows with a size of w x h x t, and each C scanning window corresponds to the C input channel features Figure 1 After convolution calculation, one WxHxT size feature map is output, and after all scanning windows are calculated, S channel output feature maps are obtained This step generally starts from the first three-dimensional convolution kernel that needs to be compressed in the 3DCNN. If other three-dimensional convolution kernels still need to be compressed, return to this step again.

[0090] Step 2: Reconstruct the three-dimensional convolution kernel into a matrix Use a fine-grained tensor decomposition method to decompose it, that is:

[0091]

[0092] wherein, and are the factor matrices of the kth(k∈{1,2,…,K}) decomposition of group A, and ​is the factor matrix of the kth decomposition of the B group, is the rank of the kth decomposition of the A group, is the rank of the kth decomposition of the B group, and is the identity diagonal matrix, the operator symbol · represents the contraction, and the operator symbol represents the Kronecker product.

[0093] In this step, the three-dimensional convolution kernel is reconstructed into the matrix The purpose is to meet the requirements of the so-called fine-grained tensor decomposition. Generally, taking Figure 2 as an example, let the I-order tensor be able to be compressed in the form of fine-grained tensor decomposition, then there is:

[0094]

[0095] wherein, is the i th (i ∈ {1, 2, …, I}) factor matrix of the k th (k ∈ {1, 2, …, K}) decomposition of the A group, is the i th factor matrix of the k th decomposition of the B group, is the rank of the k th decomposition of the A group, is the rank of the k th decomposition of the B group, and is the identity diagonal tensor. It can be seen that the fine-grained tensor decomposition implies that and have an even number in common, and further requires that the total number of m i and n i must also be even. Therefore, the three-dimensional convolution kernel as a 5-order tensor needs to combine two dimensions as one dimension, that is, there are and 4 factor matrices in each k-way, so as to meet the use conditions of fine-grained tensor decomposition.

[0096] In this step, the factor of the A group is subjected to the contraction operation ·, which eliminates all the ranks to obtain a matrix of size wh×C, and the factor of the B group is subjected to the contraction operation ·, which eliminates all the ranks to obtain a matrix of size t×S. The two matrices are subjected to the Kronecker product operation to obtain a wht×CS matrix of the k th way, and all the K-way matrices are added to restore the matrix Reconstruction of which can obtain the original three-dimensional convolution kernel That is, the factor calculated in this step can be restored to the original three-dimensional convolution kernel, which shows that the form of fine-grained tensor decomposition adopted in this step is correct.

[0097] Step 3: Ignore the unit diagonal matrix which does not affect the data content and Reorganize other factor matrices and into four matrices, that is:

[0098]

[0099]

[0100]

[0101]

[0102] wherein,

[0103] In this step, it is required that R (B) / R (A) is an integer. This condition is to enable the channel number of the separable convolution in the subsequent step to correspond, and it is not required that each pair is an integer, as long as R (B) / R (A) is an integer.

[0104] In this step, the unit diagonal matrices and are ignored because such matrices have no effect on the calculation value and only play a role in dimension matching in fine-grained decomposition. For example, assuming that the matrix and the unit diagonal matrix are given, it is obvious that UEV = UV.

[0105] Step 4: Expand A1, A2, B1 and B2 into tensors, that is:

[0106]

[0107]

[0108]

[0109]

[0110] In this step, the matrix is ​​expanded into a tensor by adding an extra dimension of magnitude 1, without changing the number of elements or the content. Expanding to a tensor is to make it conform to the form of a three-dimensional convolution kernel. Adding an extra dimension of magnitude 1 does not increase the number of elements; for example, a vector of length 10 can be viewed as a 10×1 matrix. From the perspective of the convolution kernel, The scanning window size is w×h×1. The scanning window size is 1×1×1. The scanning window size is 1×1×t. The scanning window size is 1×1×1.

[0111] Step 5: Let the input feature map sequentially with The calculation is performed, and a new nonlinear activation function f(·) is inserted after each calculation to obtain a new output feature map. Right now:

[0112]

[0113] in, This represents a three-dimensional separable convolution.

[0114] In this step, the feature map is input. sequentially with The specific calculation process is as follows:

[0115] enter First with Perform a 3D convolution with a scanning window of 1×1×1, and transform the number of channels from C to R. (A) The output is activated by f(·); then combined with Perform a scan with a window size of w×h×1. (A) Channel-wise 3D separable convolution, the output is activated by f(·); then combined with The scan window is 1×1×t and has a total of R (B) / R (A) Group R (A) Channel-wise 3D separable convolution, the output is activated by f(·); finally, with Perform a 3D convolution with a scanning window of 1×1×1, and the number of channels is determined by R. (B) Transformed to S, the output is activated by f(·). Separable convolution only requires convolving the input feature map of each channel with the corresponding channel's scanning window, unlike ordinary convolution which requires convolving with all scanning windows and then summing the results. The activation function f(·) makes the output... Convolution calculation with the original The results are not equal, meaning the depth of the original 3DCNN is increased.

[0116] In this step, the first 3D separable convolution can be converted into a 2D separable convolution for efficient computation. See details for further information. Figure 3 , w×h×1 of R (A) The channel-separable 3D convolutional kernel does not scan across pixels in the direction of feature map T, which is equivalent to treating the feature map in the direction of T as an independent slice. Thus, the W×H×T 3D feature map of the R(A) channel is equivalently transformed into T·R. (A) The W×H two-dimensional feature map of the channel. The three-dimensional separable convolution kernel is equivalently transformed into R. (A) A w×h 2D convolution kernel for each channel, in order to convert the number of channels R... (A) T·R with feature map (A) Matching can replicate each w×h 2D convolutional window T times using parameter sharing, such as... Figure 3 The dashed box in the figure shows the result. Therefore, we obtain W×H×(T·R). (A) The feature map of ) and w×h×(T·R (A) Two-dimensional separable convolution between convolution kernels.

[0117] In this step, the second 3D separable convolution can be converted into a 1D separable convolution for efficient computation. See details for further information. Figure 4 Note that the second 3D separable convolution includes R. (B) / R (A) Group, Figure 4 Draw only one set of R (A) Diagram of a channel-separable 3D convolution. R1×1×t (A) Since the channel-separable 3D convolution kernel does not scan across pixels in the W×H direction of the feature map, stretching W×H to a single dimension W·H will not affect the calculation results. The corresponding 1×1×t 3D convolution scanning window can be regarded as a 1D scanning window with a modulus of t. Figure 4 The expression 1×t is used for ease of intuitive understanding. Therefore, we can obtain (W·H)×T×R. (A) Feature map and 1×t×R (A) One-dimensional separable convolution between convolution kernels (represented as two-dimensional separable convolution).

[0118] After this step is completed, if If other layers in the convolutional model still have 3D convolutional kernels to be compressed, then proceed to step 1 to compress the next 3D convolutional kernel. All the 3D convolutional kernels to be compressed have been compressed, resulting in a new 3DCNN denoted as . Proceed to step 6.

[0119] Step 6: Use the backpropagation algorithm to process the newly compressed data. Train it to optimize its performance.

[0120] In this step, the original record is: 3DCNN has been converted to a new and Compared to the original It differs in both depth and width, and can be considered a completely new lightweight 3D CNN. Therefore, the new... A certain number of retraining sessions are required to achieve the desired performance.

[0121] To better illustrate the beneficial effects of this embodiment, a 3DCNN recognition comparison experiment on the UCF11 public video dataset is presented.

[0122] The designed 3DCNN network structure is as follows: Figure 5 As shown, each video data set in UCF11 randomly samples 50 frames in length and downsamples each frame to form input frame data of length t×h×w = 50×60×80. The 3DCNN contains 6 convolutional layers, represented as w×h×t,S. The darker-colored layers 2, 3, 4, 5, and 6 use the tensor mapping compression method described in this invention. The two numbers in parentheses represent (R... (A) ,R (B) In contrast, the original convolution kernel was modified using the same parameters A1, A2, B1, and B2 according to... Estimation is performed, and the kernel is restored before normal convolution calculation. During training, a momentum optimizer with a coefficient of 0.9 is used, employing stochastic gradient descent and an initial learning rate of 0.1 for iteration. The dataset is traversed 100 times, and the learning rate is decreased by a factor of 10 every 30 traversals. For validation, the UCF11 dataset is divided into 5 equal parts according to the class distribution, and 5-fold cross-validation is used to test the performance.

[0123] The experimental results are shown in the table below. It is evident that the tensor mapping compression method described in this invention achieves both a high compression ratio and a considerable computational acceleration effect. Furthermore, the recognition accuracy can be further improved due to the use of more nonlinear activation functions.

[0124] Model Accuracy (%) Convolution parameter amount Convolution computation amount (10 6 )]]> Compression ratio Speedup ratio Baseline 93.81±1.18 6,865,344 69,347.16 1.0 1.0 Tensor estimation 91.72±1.67 298,488 69,355.06 23.0 1.0 Tensor mapping 95.94±1.24 298,488 4,646.15 23.0 14.93

[0125] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A video recognition method based on three-dimensional convolutional neural network tensor mapping compression, characterized in that, The method includes: Step A: Based on the total number of frames The video data collected from the sensor is denoted as... ,in, Indicates the width of each video frame. Indicates the height of each video frame. That is, the total number of frames in the video data, which is also the duration. This indicates the number of channels in each frame of video image; Step B: Use any 3D convolutional neural network with more than 3 layers to process the video data. Perform identification; Step C: Output the results of video data recognition; Step B includes: Step B1: Train the 3D convolutional neural network used for recognizing video data. The trained 3D convolutional neural network is denoted as... Output to step B2; Step B2: Apply the 3D convolutional neural network tensor mapping compression method to the 3D convolutional neural network trained in Step B1. After processing, a fully compressed 3D convolutional neural network is obtained. ; Step B3: Complete the compressed 3D convolutional neural network using the methods from step B2. For video data Perform recognition and generate the result of the recognized video data to be output; Step B2 includes: Step 1: For any 3D convolutional neural network Let one of the three-dimensional convolutional layers be denoted as . , Let be the nonlinear activation function of this layer. Recorded as Each channel Size input feature map, Recorded as Each channel Size output feature map, read the scan window of this layer as 3D convolution kernel ; Step 2: Apply 3D convolution kernels Reconstructed into a matrix It is decomposed using a fine-grained tensor decomposition method, namely: in, and yes Group 1 ( The factor matrix of the path decomposition. and yes Group 1 The factor matrix of the path decomposition, for Group 1 The rank of the path decomposition. for Group 1 The rank of the path decomposition. and It is a unit diagonal matrix, and the operators are... Indicates contraction, operator Indicates the Kronecker product; Step 3: Ignore the unit diagonal matrix that does not affect the data content. and Other factor matrices , , and Reorganized into four matrices, namely: in, , ; Step 4: , , and Expanding to a tensor, that is: Step 5: Input feature map sequentially with , , , The calculation is performed, and a new nonlinear activation function is inserted after each calculation. Finally, a new output feature map is obtained. ,Right now: in, Represents a three-dimensional separable convolution; After step 5 is completed, if the 3D convolutional neural network... If other layers in the 3D convolutional neural network still have uncompressed 3D convolutional kernels, then proceed to step 1 to compress the next 3D convolutional kernel. All the 3D convolutional kernels to be compressed have been compressed, resulting in a new 3D convolutional neural network. Proceed to step 6; Step 6: Apply the backpropagation algorithm to the newly compressed 3D convolutional neural network. Train it to optimize its performance.

2. The video recognition method based on three-dimensional convolutional neural network tensor mapping compression as described in claim 1, characterized in that, In step 2, Group factors After shrinking operation It will eliminate all rank The obtained size is The matrix, Group factors After shrinking operation It will eliminate all rank The resulting size is of These two matrices are subjected to the Kronecker product operation. Get the first Road Matrix, all The sum of the path matrices can be restored to a matrix. The original 3D convolution kernel can be obtained by reconstructing it. .

3. The video recognition method based on three-dimensional convolutional neural network tensor mapping compression as described in claim 2, characterized in that, In step 3, it is required that It is an integer.

4. The video recognition method based on three-dimensional convolutional neural network tensor mapping compression as described in claim 3, characterized in that, In step 4, the matrix is ​​expanded into a tensor by adding an extra dimension with a modulus of 1, while the number of elements and the content remain unchanged.

5. The video recognition method based on three-dimensional convolutional neural network tensor mapping compression as described in claim 4, characterized in that, In step 5, the feature map is input. sequentially with , , , The specific calculation process is as follows: Input feature map First with The scanning window is The number of channels in the 3D convolution is determined by... Transform into The output is from Activate; then with The scanning window is of Channel-based 3D separable convolution, output by Activate; then with The scanning window is And there are group Channel-based 3D separable convolution, output by Activate; finally, with The scanning window is The number of channels in the 3D convolution is determined by... Transform into The output is from activation.

6. The video recognition method based on three-dimensional convolutional neural network tensor mapping compression as described in claim 1, characterized in that, In step A, the sensor is a camera.

7. The video recognition method based on three-dimensional convolutional neural network tensor mapping compression as described in claim 1, characterized in that, In step A, the sensor is a camera.

8. The video recognition method based on three-dimensional convolutional neural network tensor mapping compression as described in claim 5, characterized in that, The method employs a three-dimensional convolutional neural network tensor mapping compression approach to implement the specific video recognition process. Through fine-grained tensor decomposition, the convolutional kernels in the trained three-dimensional convolutional neural network for video recognition are compressed and mapped into four lightweight convolutional modules. This reduces the number of parameters in the three-dimensional convolutional neural network for deployment in resource-constrained video recognition environments, while also improving the computational efficiency of the three-dimensional convolutional neural network and the processing speed of video recognition.

Images