Video content intelligent identification and classification method, device and application

Abstract

The invention discloses a video content intelligent identification and classification method, device and application, and the method carries out the identification and classification of video content based on a pre-trained Transform model, and comprises the following steps: segmenting an input video into frames, and dividing each frame of image into a plurality of non-overlapping image blocks; performing feature extraction on each image block and generating a corresponding token; adding a position code for each token to represent the spatial position of the token in the original image; inputting the tokens with the position codes into an improved Transform encoder, and calculating a space-time dependency relationship between the tokens through an improved band-controlled one-way attention mechanism to obtain space-time characteristics of the video; and inputting the spatial-temporal feature sequence output by the Transform encoder into a full connection layer, and outputting the classification probability distribution of the video content through the full connection layer. According to the method, the precision of video content identification and classification can be improved on the premise of greatly reducing the calculation complexity by introducing the Transform model and the improved band-controlled one-way attention mechanism.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device and application for intelligent recognition and classification of video content. Background Art

[0002] The field of machine vision research, combined with machine learning and deep learning algorithms, has already enabled highly accurate identification and classification of individuals in static images. However, the information displayed in an image is not limited to the type of individual; more often, it can convey the individual's specific actions, behaviors, or the type of scene. In-depth work has been conducted on this issue, a prime example being image captioning, which uses natural language processing methods to interpret the actions of individuals in images. Unfortunately, due to the very limited features that can be extracted from a single image, enabling machines to accurately understand the actions in an image requires an extension of the temporal dimension. Therefore, videos with time series provide richer feature information.

[0003] Previously, traditional methods were commonly used for video analysis. However, since the rapid rise of deep networks over a decade ago, they have been widely applied to various machine vision research projects. Convolutional neural networks are the fundamental building blocks of most methods used for tasks such as image segmentation, image classification, and object detection. However, they lack the ability to model long-range dependencies within images. More specifically, in a convolutional neural network, each convolution kernel focuses only on a local subset of pixels in the entire image, forcing the network to focus on local patterns rather than global context. For videos, in addition to analyzing long-range spatial dependencies, temporal relationships are also crucial and have a significant impact on video analysis. Summary of the Invention

[0004] In view of the above-mentioned deficiencies in the prior art, the technical problem to be solved by the present invention is to provide a method, device and application for intelligent video content recognition that can improve recognition efficiency while solving the problem of insufficient accuracy of traditional video recognition.

[0005] To solve the above technical problems, the present invention adopts a technical solution: providing a method for intelligent recognition and classification of video content, which recognizes and classifies video content based on a pre-trained Transformer model, including the following steps:

[0006] The input video is divided into frames, and each frame image is divided into multiple non-overlapping image blocks;

[0007] Extract features from each image block and generate corresponding tokens;

[0008] Add a position code to each token to indicate its spatial position in the original image;

[0009] The tokens with positional encoding are input into the improved Transformer encoder, and the spatiotemporal dependencies between tokens are calculated through the improved controlled unidirectional attention mechanism to obtain the spatiotemporal features of the video.

[0010] The spatiotemporal feature sequence output by the Transformer encoder is input into the fully connected layer, and the classification probability distribution of the video content is output through the fully connected layer.

[0011] Furthermore, in the step of extracting features from each image block and generating corresponding tokens, it specifically includes: using a residual network to extract spatial features from each image block, then reducing the feature dimension through global average pooling, and finally mapping it into a token of fixed dimension through a fully connected layer.

[0012] Furthermore, the improvements to the controlled unidirectional attention mechanism include:

[0013] Decompose global self-attention into two-dimensional calculations: horizontal and vertical;

[0014] Introducing relative position encoding to enhance the Transformer model's ability to perceive spatial positions;

[0015] The influence of position encoding on attention weights is dynamically adjusted through learnable control parameters to optimize feature representation.

[0016] Furthermore, in the step of decomposing the global self-attention into two independent calculations in the horizontal and vertical directions, the calculation formula for the horizontal direction is as follows:

[0017]

[0018] In formula (1), b∈I 1×w(a) Represents all tokens in the same row as the selected a token, represents the correlation between token a and all tokens in the same row, q a represents the query vector of a token, k b The key vector representing the b token, v b A vector of values representing b tokens;

[0019] The calculation formula for the vertical direction is as follows:

[0020]

[0021] In formula (2), represents the correlation between token a and all tokens in the same column, b∈I h(a)×1Represents all tokens in the same column as the selected a token.

[0022] Furthermore, after the introduction of relative position encoding, the calculation formula for the correlation between token a and all tokens in the same row is as follows:

[0023]

[0024] In formula (3), represents the relative position encoding depending on the query vector, represents the relative position encoding depending on the key vector, The representation depends on the relative position encoding of the value vector.

[0025] Furthermore, in the step of dynamically adjusting the influence of position encoding on attention weights through learnable control parameters, the correlation between token a and all tokens in the same row is calculated as follows:

[0026]

[0027] In formula (4), M Q 、M K 、M V1 、M V2 Both represent control parameters, controlling the contribution weight of position encoding.

[0028] Furthermore, the control parameters are updated in real time by training the Transformer model, and the specific steps include:

[0029] Initializing the control parameters;

[0030] Using a public dataset as training data, the video frames and their corresponding category labels are input into the Transformer model, and attention weights are calculated through forward propagation.

[0031] Calculate the cross entropy loss function between the model recognition result and the true label;

[0032] The gradient of the control parameter is calculated by a back propagation algorithm, and the control parameter is iteratively updated until the model converges.

[0033] In order to solve the above technical problems, another technical solution adopted by the present invention is to provide a device for intelligently identifying and classifying video content, comprising:

[0034] A preprocessing module is used to segment the input video into frames and divide each frame image into multiple non-overlapping image blocks;

[0035] Feature extraction module, used to extract features from each image block and generate corresponding tokens;

[0036] The encoding module is used to add position encoding to the token to indicate its spatial position in the original image;

[0037] The encoder is used to calculate the spatiotemporal dependencies between tokens through an improved controlled unidirectional attention mechanism to obtain the spatiotemporal features of the video;

[0038] The classification output module is used to output the classification probability distribution of the video content based on the spatiotemporal feature sequence output by the Transformer encoder.

[0039] In order to solve the above technical problems, another technical solution adopted by the present invention is to provide a method for intelligent management and control of sports venue lighting, comprising the following steps:

[0040] Deploy cameras at sports venues to capture video streams in real time;

[0041] Utilizing the method for intelligent video content recognition and classification to analyze the video stream in real time to detect whether there is a preset motion behavior;

[0042] The lighting is controlled to be turned on and off according to the detection results.

[0043] Furthermore, in the step of controlling the turning on and off of the lighting lamp according to the detection result, it specifically includes: generating a light control signal according to the detection result to control the turning on and off of the lighting lamp, and controlling the lighting lamp to turn on when a preset motion behavior is detected, and vice versa.

[0044] The method, device and application of intelligent recognition and classification of video content of the present invention have at least the following beneficial effects: the present invention effectively solves the problem of insufficient recognition accuracy of traditional convolutional networks in video behavior analysis by introducing the Transformer model and the improved controlled unidirectional attention mechanism, and realizes accurate classification of video content; the controlled unidirectional attention mechanism greatly reduces the computational complexity by decomposing global self-attention into horizontal and vertical calculations, breaking through the traditional Transformer's dependence on high-computing power servers, so that the model can efficiently process large-scale video data while maintaining high accuracy and adapt to resource-limited environments; by introducing relative position encoding and dynamic control parameters, the Transformer model's perception of spatial position is enhanced, while capturing long-range spatial dependencies and fine-grained temporal relationships in the video, and improving the model's ability to understand individual actions and scene types in the video; the method of intelligent recognition and classification of video content of the present invention can be applied to the intelligent management and control of lights in sports venues at night, realizing a complete closed loop from video stream acquisition, behavior recognition to lighting control, with simple operation, applicable to various monitoring scenarios, and wide practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0046] Figure 1 This is a flow chart of an embodiment of a method for intelligently identifying and classifying video content according to the present invention.

[0047] Figure 2 This is the implementation architecture diagram of the controlled unidirectional attention algorithm with controlled position encoding.

[0048] Figure 3 This is the Transformer model training interface diagram of the present invention.

[0049] Figure 4 Page diagram for testing the Transformer model.

[0050] Figure 5 An example diagram for real-time detection of unknown video files.

[0051] Figure 6 This is a structural block diagram of an embodiment of the apparatus for intelligently identifying and classifying video content according to the present invention.

[0052] Figure 7 This is a structural diagram of an implementation of the Transformer model.

[0053] Figure 8 This is a flow chart of an embodiment of a method for intelligent management and control of sports venue lighting according to the present invention.

[0054] Figure 9 The figure is a human-machine interface diagram of a lighting intelligent management and control platform based on the method for intelligent management and control of sports venue lighting. DETAILED DESCRIPTION

[0055] The present invention will be further described below with reference to the accompanying drawings.

[0056] In this embodiment, the Transformer model is used to identify and classify video content. Compared with traditional video analysis methods, the Transformer model has the following advantages: 1. End-to-end learning. The Transformer model can achieve end-to-end learning without the need for manual feature design. It can automatically learn higher-level abstract feature representations from raw data, thereby better adapting to different video analysis tasks. 2. Capturing large-span spatial dependencies. Traditional methods often encounter difficulties in processing large-span video sequences because they find it difficult to capture large spatial dependencies. The Transformer model can capture the dependencies between any two positions in the sequence through the self-attention mechanism, thereby better handling spatial dependencies in video sequences. 3. Parallel processing. The self-attention mechanism in the Transformer model can achieve parallel processing, making the model more computationally efficient during training and inference, which enables the Transformer model to better process large-scale video data and run effectively in resource-limited environments.

[0057] See also Figure 1 , is a flow chart of an embodiment of a method for intelligently identifying and classifying video content according to the present invention. This embodiment specifically includes the following steps:

[0058] S210: Preprocess the input video.

[0059] Specifically, the input video is divided into frames, and each frame image is divided into multiple non-overlapping image blocks.

[0060] S220, feature extraction.

[0061] Specifically, a residual network is used to extract spatial features from each image block, capture the local features of the image block, and generate a preliminary feature map; then, the feature map output by the residual network is pooled through global average pooling to reduce the feature dimension; finally, the pooled feature map is mapped to a token of fixed dimension through a fully connected layer. Each token represents the feature of the image block corresponding to it, and the token corresponds to three feature vectors (q, k, v), where q represents the query vector, k represents the key vector, and v represents the value vector. In view of the fact that traditional convolutional deep networks lack the ability to capture long-distance correlations in data, a residual network is used as the backbone of the Transformer model to extract features. In this embodiment, 3D convolution (the third dimension is time) that relates to temporal information is not considered. This is different from the traditional video convolution structure and is a modification made for the use of Transformer.

[0062] S230, position coding.

[0063] Specifically, a positional encoding is added to each token to indicate its spatial location in the original image. This encoding process is completed before inputting the encoder because the positional encoding needs to be combined with the embedding vector as input to the Transformer encoder. In the Transformer encoder, the self-attention mechanism itself has no positional awareness, so explicit positional encoding is required to enable the model to distinguish tokens at different positions.

[0064] S240: Obtain spatiotemporal features.

[0065] Specifically, the tokens with positional encoding are input into the improved Transformer encoder, and the spatiotemporal dependencies between tokens are calculated through the improved controlled unidirectional attention mechanism to obtain the spatiotemporal features of the video. The Transformer model itself has a self-attention mechanism. The original self-attention mechanism is explained below. Given a feature map Here H and W are the height and width of the feature map, respectively, and C is the number of channels. After the self-attention layer, the dimension of the output attention value is The raw attention can be calculated using the following formula:

[0066]

[0067] Where, query Q = W Q x, key K = W K x and the value V=W V x is mapped from input x, and all mapping matrices W can be learned. k is the dimension of the K vector. This original formula is modeled at the matrix level and can be further formulated at the matrix element level. For any point a=(i,j) in a two-dimensional grid (image or feature map), the input at this point is The output is It can be formulated as:

[0068]

[0069] Here I is the specified grid area (the original attention mechanism refers to the entire image or feature map). Then query q a =W Q x a , key k a =W K x a and the value v a =W V x a , as well as For a given point a (a token), softmax bIndicates that all b points (b tokens) in the area specified by I are correlated with point a. For the case where I specifies a two-dimensional image or feature map area, that is H×W points. In fact, in order to calculate the final output y, for all a points, the mechanism must be based on affine transformation , allowing the capture of global contextual dependencies across the entire feature map, rather than just local dependencies like convolution. However, when the spatial dimension of the input is large, the computational cost of self-attention is very high O(H 2 W 2 ), which limits its use to downsampled feature maps or small-sized images.

[0070] To address the high computational complexity and poor recognition and classification accuracy of the original attention mechanism, this paper proposes a new attention mechanism, namely a controlled unidirectional attention mechanism. The improvements to the controlled unidirectional attention mechanism specifically include the following three aspects:

[0071] ① Decompose global self-attention into calculations in two directions: horizontal and vertical. In order to reduce computational complexity, a unidirectional self-attention algorithm can be used to perform calculations in two independent directions: horizontal and vertical. That is, a single calculation only considers one of the directions. Then the original attention algorithm has to be decomposed into calculations in two directions. In this way, the entire calculation process will become a more efficient algorithm, and its complexity will be reduced to linear. Although the new calculation method lacks some relevant information compared to the original attention mechanism, this small amount of information loss is exchanged for a significant improvement in computing performance. The calculation formula for the horizontal direction is as follows:

[0072]

[0073] Where b∈I 1×w(a) Represents all tokens in the same row as the selected a token, represents the correlation between token a and all tokens in the same row, q a represents the query vector of a token, k b The key vector representing the b token, v b A vector of values representing b tokens.

[0074] The calculation formula for the vertical direction is as follows:

[0075]

[0076] in, represents the correlation between token a and all tokens in the same column, b∈I h(a)×1 Represents all tokens in the same column as the selected a token.

[0077] ② Introduce relative position encoding to enhance the Transformer model's ability to perceive spatial positions. In order to increase the position bias while calculating the affine transformation through the self-attention mechanism, a position bias term is added to make the affine transformation sensitive to position information. This bias term is called relative position encoding. These position encodings can usually be learned through training and have the ability to encode the spatial structure of the image. Specifically, relative position encoding is used for both the query and the key, and a query-dependent bias term is introduced. and key-dependent bias At the same time, output y a In addition to being able to obtain the value v b In addition, you can also get the relative position Therefore, after the introduction of relative position encoding, taking the horizontal direction as an example, the calculation formula for the correlation between the a token and all tokens in the same row is as follows:

[0078]

[0079] in, represents the relative position encoding depending on the query vector, represents the relative position encoding depending on the key vector, Represents a relative position encoding that depends on the value vector. After introducing relative position encoding, the calculation principle for the correlation between token a and all tokens in the same column is the same as above and will not be repeated here. This design ensures that long-range correlations with precise position information are captured at a reasonable computational cost.

[0080] ③The above improved algorithm can calculate the non-local environment with good efficiency and can encode the position bias into the mechanism. In the evaluation of large-scale segmentation datasets, it is easier to learn the position bias of keys, queries, and values; however, for experiments on small-scale datasets, the position bias is difficult to learn, so it is not always accurate in encoding long-range correlations. In the case that the learned relative position encoding is not accurate enough, adding them to the corresponding key, query, and value tensors will lead to performance degradation. Therefore, an improved control block is further proposed, which can control the impact of position bias on non-local context encoding. The influence of position encoding on attention weights is dynamically adjusted through learnable control parameters to optimize feature representation. Specifically, in the step of dynamically adjusting the influence of position encoding on attention weights through learnable control parameters, taking the horizontal direction as an example, the calculation formula of the correlation between token a and all tokens in the same row is as follows:

[0081]

[0082] Among them, M Q 、M K 、M V1 、MV2 Both represent control parameters that control the contribution weight of the positional encoding. These parameters are learnable parameters that together create a control block mechanism that controls the impact of the learned relative positional encoding on the non-local context. Specifically, if the relative positional encoding is learned accurately, then the control mechanism will give it a higher weight function than if the encoding is not learned accurately. Similarly, in the vertical direction, the calculation principle of the correlation between the a token and all tokens in the same column is the same as above, so we will not repeat it here. Please refer to Figure 2 , showing the implementation architecture of the controlled unidirectional attention algorithm with controlled position encoding mentioned above. Figure 2 (a) is the controlled attention layer, which shows in detail the fusion logic of position encoding and control parameters in attention calculation. Figure 2 (b) is the encoder integration scheme, which shows that the controlled unidirectional attention mechanism is applied inside the Transformer encoder.

[0083] The control parameters are updated in real time by training the Transformer model. The specific steps include: initializing the control parameters; using a public dataset as training data, inputting video frames and their corresponding category labels into the Transformer model, calculating attention weights through forward propagation, and performing forward propagation. The public dataset used is UCF101; calculating the cross-entropy loss function between the model recognition result and the true label; calculating the gradient of the control parameters through the backpropagation algorithm, and iteratively updating the control parameters until the model converges.

[0084] S250, classification output.

[0085] Specifically, the spatiotemporal feature sequence output by the Transformer encoder is input into the fully connected layer, and the classification probability distribution of the video content is output through the fully connected layer.

[0086] See also Figure 3To verify the technical superiority of this solution, in this implementation, accuracy calculations were performed on the validation dataset during training and the test set. The top-1 accuracy for the selected UCF101 subset (approximately 50% of the data) remained at around 90%, which is already a good performance for tasks based on video content recognition. Table 1 shows several key metrics. Top-1 accuracy refers to the probability that the highest-probability category in the model's predictions is correct. 89.61% indicates that the model's first-choice answer is correct nearly 90% of the time on the test set. Top-5 accuracy indicates the probability that the correct answer is included in the top five most-probable categories predicted by the model. 97.19% indicates that the model almost always includes the correct answer when given five possible categories. Recall refers to the proportion of correctly identified positive examples among all actual positive examples. 89.67% indicates that the model can effectively identify most relevant instances.

[0087]

[0088] Table 1

[0089] See also Figure 4 and Figure 5 , the present invention uses unknown video files for testing, which shows that the Transformer model in this embodiment can accurately identify the action categories in unknown video files.

[0090] See also Figure 6 , is a structural block diagram of an embodiment of the apparatus for intelligently identifying and classifying video content of the present invention. The apparatus for intelligently identifying and classifying video content of this embodiment is used to implement the method for intelligently identifying and classifying video content as described in the above embodiment. Specifically, the apparatus for intelligently identifying and classifying video content of this embodiment includes a preprocessing module 100, a feature extraction module 200, an encoding module 300, an encoder 400, and a classification output module 500. Among them:

[0091] The pre-processing module 100 is used to segment the input video into frames and divide each frame into a plurality of non-overlapping image blocks.

[0092] The feature extraction module 200 is used to extract features from each image block and generate corresponding tokens. In this embodiment, the feature extraction module 200 uses a residual network, which effectively solves the long-distance dependency modeling problem of traditional convolutional networks while maintaining computational efficiency and modularity.

[0093] The encoding module 300 is used to add position codes to the tokens to indicate their spatial positions in the original image.

[0094] Encoder 400 is used to calculate the spatiotemporal dependencies between tokens using a controlled unidirectional attention mechanism to obtain the spatiotemporal features of the video. In this embodiment, encoder 400 is an improved Transformer encoder. The main feature is that the original self-attention mechanism provided by the encoder is replaced with the controlled unidirectional attention mechanism described in step S240, which solves the problems of high computational complexity and poor recognition and classification accuracy of the original self-attention mechanism.

[0095] The classification output module 500 is used to output the classification probability distribution of the video content based on the spatiotemporal feature sequence output by the Transformer encoder. In this embodiment, the classification output module 500 uses a head network (such as a fully connected network, MLP) to calculate the probability value of each category.

[0096] The feature extraction module 200 (residual network), encoding module 300, encoder 400 and classification output module 500 (fully connected network) together constitute the Transformer model in this embodiment. Figure 7 , Figure 7 (a) is the overall architecture diagram of the Transformer model in this embodiment. Figure 7 (b) is the structure of encoder 400, where the new attention algorithm directly replaces the original attention algorithm.

[0097] The present invention also discloses a specific application of the method for intelligent recognition and classification of video content. In this embodiment, the method for intelligent recognition and classification of video content is used to intelligently control the lighting of sports venues at night. Figure 8 , is a flow chart of an embodiment of a method for intelligently controlling sports field lighting according to the present invention. This embodiment specifically includes the following steps:

[0098] S100: Obtain a video stream.

[0099] Specifically, cameras are deployed in sports venues to capture video streams in real time. In this embodiment, cameras are respectively set up in basketball courts, table tennis courts and football fields.

[0100] S200: Detecting movement behavior.

[0101] Specifically, the video stream is analyzed in real time using the method for intelligent recognition and classification of video content to detect whether there is a preset sports behavior, which includes playing basketball, playing table tennis, and playing football.

[0102] S300: Control the switch of the lighting lamp.

[0103] The lighting is turned on and off according to the detection result. Specifically, a light control signal is generated according to the detection result to control the lighting to be turned on and off. When a preset motion behavior is detected, the lighting is turned on, otherwise, the lighting is turned off.

[0104] See also Figure 9 , is a human-machine interface diagram of a lighting intelligent control platform based on the method of intelligent lighting control for sports venues, which shows in detail the complete closed loop from video stream acquisition, behavior recognition to lighting control.

[0105] The present invention effectively solves the problem of insufficient recognition accuracy of traditional convolutional networks in video behavior analysis by introducing the Transformer model and the improved controlled unidirectional attention mechanism, and realizes accurate classification of video content; the controlled unidirectional attention mechanism greatly reduces the computational complexity by decomposing global self-attention into horizontal and vertical calculations, breaking through the traditional Transformer's dependence on high-computing power servers, so that the model can efficiently process large-scale video data while maintaining high accuracy and adapt to resource-limited environments; by introducing relative position encoding and dynamic control parameters, the Transformer model's perception of spatial position is enhanced, while capturing long-range spatial dependencies and fine-grained temporal relationships in the video, and improving the model's ability to understand individual actions and scene types in the video; the method of intelligent recognition and classification of video content of the present invention can be applied to the intelligent management and control of lights in sports venues at night, realizing a complete closed loop from video stream acquisition, behavior recognition to lighting control, with simple operation, applicable to various monitoring scenarios, and has wide practical value.

[0106] The above description merely expresses the preferred embodiments of the present invention, and its description is relatively specific and detailed, but it should not be construed as limiting the scope of the patent. It should be noted that a person skilled in the art may make a number of variations and improvements without departing from the concept of the present invention, and these variations and improvements fall within the scope of protection of the present invention. Therefore, the scope of protection of the patent of the present invention shall be based on the appended claims.

Claims

1. A method for intelligent identification and classification of video content, characterized in that: Identifying and classifying video content based on a pre-trained Transformer model involves the following steps: The input video is divided into frames, and each frame image is divided into multiple non-overlapping image blocks; Extract features from each image block and generate corresponding tokens; Add a position code to each token to indicate its spatial position in the original image; The tokens with positional encoding are input into the improved Transformer encoder, and the spatiotemporal dependencies between tokens are calculated through the improved controlled unidirectional attention mechanism to obtain the spatiotemporal features of the video. The spatiotemporal feature sequence output by the Transformer encoder is input into the fully connected layer, and the classification probability distribution of the video content is output through the fully connected layer.

2. The method for intelligent identification and classification of video content according to claim 1, wherein: The steps of extracting features from each image block and generating corresponding tokens specifically include: using a residual network to extract spatial features from each image block, then reducing the feature dimension through global average pooling, and finally mapping it into a fixed-dimensional token through a fully connected layer.

3. The method for intelligent identification and classification of video content according to claim 1, characterized in that: Improvements to the controlled unidirectional attention mechanism include: Decompose global self-attention into two-dimensional calculations: horizontal and vertical; Introducing relative position encoding to enhance the Transformer model's ability to perceive spatial positions; The influence of position encoding on attention weights is dynamically adjusted through learnable control parameters to optimize feature representation.

4. The method for intelligent identification and classification of video content according to claim 3, wherein: In the step of decomposing the global self-attention into two independent calculations in the horizontal and vertical directions, the calculation formula for the horizontal direction is as follows: In formula (1), b∈I 1×w(a) Represents all tokens in the same row as the selected a token, represents the correlation between token a and all tokens in the same row, q a represents the query vector of a token, k b The key vector representing the b token, v b A vector of values representing b tokens; The calculation formula for the vertical direction is as follows: In formula (2), represents the correlation between token a and all tokens in the same column, b∈I h ( a ) ×1 Represents all tokens in the same column as the selected a token.

5. The method for intelligent identification and classification of video content according to claim 4, characterized in that: After the introduction of relative position encoding, the calculation formula for the relevance of token a with all tokens in the same row is as follows: In formula (3), represents the relative position encoding depending on the query vector, represents the relative position encoding depending on the key vector, The representation depends on the relative position encoding of the value vector.

6. The method for intelligently identifying and classifying video content as claimed in claim 5, wherein: In the step of dynamically adjusting the influence of position encoding on attention weights through learnable control parameters, the correlation between token a and all tokens in the same row is calculated as follows: In formula (4), M Q 、M K 、M V1 、M V2 Both represent control parameters, controlling the contribution weight of position encoding.

7. The method for intelligently identifying and classifying video content according to claim 3, wherein: The control parameters are updated in real time by training the Transformer model. The specific steps include: Initializing the control parameters; Using a public dataset as training data, the video frames and their corresponding category labels are input into the Transformer model, and attention weights are calculated through forward propagation. Calculate the cross entropy loss function between the model recognition result and the true label; The gradient of the control parameter is calculated by a back propagation algorithm, and the control parameter is iteratively updated until the model converges.

8. A device for intelligent identification and classification of video content, characterized in that: include: A preprocessing module is used to segment the input video into frames and divide each frame image into multiple non-overlapping image blocks; Feature extraction module, used to extract features from each image block and generate corresponding tokens; The encoding module is used to add position encoding to the token to indicate its spatial position in the original image; The encoder is used to calculate the spatiotemporal dependencies between tokens through an improved controlled unidirectional attention mechanism to obtain the spatiotemporal features of the video; The classification output module is used to output the classification probability distribution of the video content based on the spatiotemporal feature sequence output by the Transformer encoder.

9. A method for intelligent management and control of sports venue lighting, characterized in that: The following steps are involved: Deploy cameras at sports venues to capture video streams in real time; Performing real-time analysis on the video stream using the method for intelligent video content recognition and classification according to any one of claims 1 to 7 to detect whether there is a preset motion behavior; The lighting is controlled to be turned on and off according to the detection results.

10. The method for intelligent management and control of sports venue lighting according to claim 9, characterized in that: The step of controlling the lighting on and off according to the detection results specifically includes: generating a light control signal according to the detection results to control the lighting on and off, and controlling the lighting to be turned on when a preset motion behavior is detected, and otherwise turning the lighting off.

Images