Video classification method, device and computer equipment based on reinforcement learning

Abstract

The present application is in the field of reinforcement learning technology of artificial intelligence technology. The present application provides a video classification method, apparatus and computer equipment based on reinforcement learning, wherein the method includes: obtaining the current video frame and visual focus of the video to be classified; inputting the classification scores of the current video frame and the previous video frame into an attribution model to generate a feature map and a first attention map, wherein the loss function of the attribution model includes an area loss subfunction; generating a visual attention map based on the current video frame and the visual focus; inputting the visual attention map into a visual saliency model to generate a second attention map; fusing the second attention map with the first attention map to generate a third attention map; generating a feature vector based on the third attention map and the feature map; and obtaining the classification score of the current video frame and the visual focus of the next video frame based on the feature vector. The above method, apparatus and computer equipment have better interpretability and higher accuracy.

Description

Technical Field

[0001] The present application relates to the technical field of reinforcement learning of artificial intelligence technology. Specifically, the present application relates to a video classification method, apparatus and computer equipment based on reinforcement learning. Background Art

[0002] Today, AI-based video classification technology is widely used in video security, anomaly detection, and other fields. Because reinforcement learning can effectively predict sequential data, it is suitable for video classification tasks. However, with the continuous development of machine learning technology and the increasing demand for it, people are demanding that machine learning models not only have strong recognition and prediction capabilities but also establish communication between users and their internal mechanisms. Some existing video classification reinforcement models combine visual saliency models with visual focus areas to derive video classification predictions. However, due to the gap between image salient areas and decision attribution areas, these solutions lack good interpretability. Furthermore, because the weights of the saliency detection model remain unchanged during training and it lacks its own focus function, if the agent's predicted gaze focus point is incorrect, its focus area will also be incorrect. This will lead to deviations in the feature maps ultimately extracted for recognition, thereby reducing video classification accuracy. Summary of the Invention

[0003] The main purpose of this application is to provide a video classification method, device and computer equipment based on reinforcement learning, which can improve the interpretability of the video classification model in the video classification method and the accuracy of video classification.

[0004] To achieve the above-mentioned object of the invention, the present application provides a video classification method based on reinforcement learning, comprising:

[0005] Obtaining a current video frame and a visual focus point of the video to be classified, wherein the visual focus point is obtained based on a previous video frame;

[0006] Inputting the classification scores of the current video frame and the previous video frame into an attribution model to generate a feature map and a first attention map, wherein the loss function of the attribution model includes an area loss sub-function;

[0007] generating a visual attention map according to the current video frame and the visual attention point;

[0008] Inputting the visual attention map into a visual saliency model to generate a second attention map;

[0009] Fusing the second attention map with the first attention map to generate a third attention map;

[0010] generating a feature vector according to the third attention map and the feature map;

[0011] The classification score of the current video frame and the visual focus point of the next video frame are obtained according to the feature vector.

[0012] In some embodiments, fusing the second attention map with the first attention map to generate a third attention map includes:

[0013] Set the weight value ρ;

[0014] Set ρ as the weight of the second attention map, and (1-ρ) as the weight of the first attention map;

[0015] The third attention map is generated according to the ρ, (1-ρ), the second attention map and the first attention map.

[0016] In some embodiments, inputting the classification scores of the current video frame and the previous video frame into an attribution model to generate a feature map and a first attention map includes:

[0017] The current video frame is input into a convolutional neural network to generate a feature map.

[0018] In some embodiments, inputting the classification scores of the current video frame and the previous video frame into an attribution model to generate a feature map and a first attention map further includes:

[0019] The number of channels of the feature map is changed to C through the first convolutional layer to obtain a first sub-feature map;

[0020] The number of channels of the feature map is changed to 1 through the second convolutional layer to obtain a second sub-feature map;

[0021] Obtaining a first conditional probability distribution according to the first sub-feature map;

[0022] Obtaining a second conditional probability distribution according to the second sub-feature map;

[0023] Combining the first conditional probability distribution and the second conditional probability distribution to obtain a joint distribution probability;

[0024] Substitute the classification score of the previous video frame into the joint distribution probability to generate the first attention map.

[0025] In some embodiments, generating a feature vector according to the third attention map and the feature map includes:

[0026] Aggregating the feature map with the third attention map to generate an aggregated map;

[0027] The aggregation graph and the feature graph are merged to generate the feature vector.

[0028] In some embodiments, the attribution model is trained based on an expectation-maximization algorithm.

[0029] The present application also provides a video classification device, comprising:

[0030] An acquisition module, configured to acquire a current video frame and a video frame focus point of a real-time video, wherein the video frame focus point is obtained based on a previous video frame;

[0031] A first attention map generating module, configured to input the current video frame into an attribution model to generate the first attention map, wherein the loss function of the attribution model includes an area loss sub-function;

[0032] A first feature map generating module, configured to input the current video frame into the attribution model to generate the first feature map;

[0033] A visual attention map generating module, configured to generate a visual attention map according to the current video frame and the value of a preset attention point;

[0034] A second attention map generation module is used to input the visual attention map into a visual saliency model to generate a second attention map;

[0035] A third attention map generating module, configured to fuse the second attention map with the first attention map to generate a third attention map;

[0036] A feature vector generation module, configured to generate a feature vector based on the third attention map and the first feature map;

[0037] The classification score and visual focus point acquisition module is used to obtain the classification score and the visual focus point of the next video frame according to the feature vector.

[0038] The present application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the steps of the reinforcement learning-based interpretability method provided in any of the above embodiments.

[0039] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the reinforcement learning-based interpretability method provided in any of the above embodiments are implemented.

[0040] The present application provides a reinforcement learning-based video classification method, apparatus, computer equipment, and storage medium, which obtain the current video frame and visual focus points of the video to be classified; input the classification scores of the current video frame and the previous video frame into an attribution model to generate a feature map and a first attention map, wherein the loss function of the attribution model includes an area loss sub-function; generate a visual attention map based on the current video frame and the visual focus points; input the visual attention map into a visual saliency model to generate a second attention map; fuse the second attention map with the first attention map to generate a third attention map; generate a feature vector based on the third attention map and the feature map; and obtain the classification score of the current video frame and the visual focus points of the next video frame based on the feature vector. Since the attribution model is naturally interpretable, the third attention map generated by combining it with the visual saliency model can not only explain the salient areas in the video frame, but also the areas that contribute to the prediction results of each step and the line of sight focus areas, thereby improving the interpretability of the video classification model in the video classification method; in addition, since the area loss sub-function is added to the loss function of the attribution model, it has a focusing function. Even if the line of sight focus point obtained based on the previous video frame is wrong, it can still display the line of sight focus area after fusion with the result of the attribution model, and it will not be completely out of focus, so that the final generated feature vector is more accurate, thereby improving the accuracy of video classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a flowchart of a video classification method based on reinforcement learning according to an embodiment of the present application;

[0042] Figure 2 Schematic diagram of the process of step S20 in the video classification method based on reinforcement learning according to an embodiment of the present application;

[0043] Figure 3 Schematic diagram of the process of step S50 in the video classification method based on reinforcement learning according to an embodiment of the present application;

[0044] Figure 4 This is a schematic block diagram of the structure of a video classification device according to an embodiment of the present application;

[0045] Figure 5 This is a schematic block diagram of the structure of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0047] State-of-the-art video classification methods simulate bottom-up and top-down visual attention mechanisms in the observation environment, so that the decisions of the proposed stochastic multi-task agent can be visually explained using attention regions.

[0048] Reference Figure 1 , an embodiment of the present application provides a video classification method, including steps S10-S70, and a detailed description of each step of the video classification method is as follows.

[0049] In one embodiment, the video classification method includes:

[0050] S10, obtaining a current video frame and a visual focus point of the video to be classified, wherein the visual focus point is obtained based on a previous video frame;

[0051] S20, inputting the classification scores of the current video frame and the previous video frame into an attribution model to generate a feature map and a first attention map, wherein the loss function of the attribution model includes an area loss sub-function;

[0052] S30, generating a visual attention map according to the current video frame and the visual attention point;

[0053] S40, inputting the visual attention map into a visual saliency model to generate a second attention map;

[0054] S50: Fusing the second attention map with the first attention map to generate a third attention map;

[0055] S60, generating a feature vector according to the third attention map and the feature map;

[0056] S70: Obtain a classification score of the current video frame and a visual focus point of the next video frame according to the feature vector.

[0057] As described in step S10 above, sparse segment sampling can be performed on the video to be classified to obtain the current video frame. For example, one frame is selected every N video frames as a valid video frame for video recognition, where the value of N is selected based on the actual classification requirements. For example, when used for abnormality recognition in monitoring, the value of N can be selected to be a small value to avoid missing important information. Reasonable sparse segment sampling can remove a large amount of redundancy in the video to be classified, improve recognition efficiency, and avoid losing important information in the video.

[0058] In some video classification tasks (such as anomaly recognition in surveillance videos), the monitored object and its local area typically contain the vast majority of the surveillance utility information. These areas are the Region of Interest (ROI) that the classification model consistently focuses on, and the local visual information generated by these areas is crucial for guiding classification decisions. However, background areas outside the ROI contain almost no surveillance utility information and have minimal impact on classification results. Therefore, ensuring that the ROI visual information of the active video frames in the video to be classified meets the classification requirements is sufficient. Therefore, active video frames can be processed based on foveal vision, known as foveal processing. This approach mimics the information processing mechanisms of the human eye during visual perception to filter surveillance utility information within the active video frames, thereby reducing image data size and improving classification efficiency. The processed image maintains high resolution in the ROI rich in surveillance utility information, meeting the requirements of video classification decisions, while the resolution of background areas further away from the ROI gradually decreases. While reducing the amount of non-surveillance utility information, it still reflects the overall image content, thereby assisting in the proper conduct of surveillance activities.

[0059] To ensure that the display resolution of the image in the effective video frame is consistent with the visual perception resolution, it is first necessary to determine the visual focus point, that is, the corresponding position of the fovea on the image, so as to adjust the spatial resolution distribution of the image to make it consistent with foveal vision. In the embodiment of the present application, the visual focus point of the current video frame is obtained based on the previous video frame. That is, when the classification model outputs the classification result of the previous video frame, it also outputs a visual focus point. This visual focus point is used to preprocess the image of the current video frame.

[0060] As described in step S20 above, in an embodiment of the present application, the current video frame of the video to be classified is input into the attribution model to generate a first attention map and a feature map, wherein the first attention map is used to simulate a bottom-up attention mechanism. Bottom-up attention mechanisms and top-down attention mechanisms are widely used in fields such as image description and visual question answering to improve the performance of the model in deeper image understanding and multi-step reasoning. Therefore, in the video classification method in an embodiment of the present application, a first attention map is generated by introducing an attribution model to make the classification model more interpretable. Since the attribution model is naturally interpretable, the attention map generated by it not only explains the salient areas in the image, but also explains the areas that contribute to the prediction results of each step, as well as the line of sight attention areas, so the attention map can be directly used as a dynamic attribution map of the video classification model.

[0061] In one embodiment, the attribution model can be designed based on a class activation mapping model (CAM). The class activation mapping model uses global average pooling to present in the form of a heat map which pixels the model uses to know that the image belongs to a certain category, making the classification model transparent and explainable.

[0062] In another embodiment, the attribution model can also be designed based on the more interpretable Class Activation Latent Mapping Model (CALM). The CALM adds a latent variable Z to the CALM model to represent the position of each pixel in the image X. The ML algorithm, EM algorithm, etc. are used to learn the probability distribution of position Z given the known classification result Y.

[0063] It should be noted that in order to make the above-mentioned attribution model have a focusing function, an area loss sub-function is added to the loss function during the attribution model training process. Specifically, for example, when the EM algorithm is used to learn the probability distribution of position Z under the premise of known classification result Y, the total loss function of the attribution model is

[0064] L=L EM +λL area

[0065] Among them, L EM is the loss function of the attribution model obtained according to the EM algorithm, L area is the area loss subfunction, and λ is a hyperparameter. The value range of λ is [0,1]. The larger the λ, the more focused the image.

[0066] As described in step S30 above, this embodiment can pre-process the current video frame image according to the current video frame and the visual attention point through the visual fovea module to generate a visual attention map. It should be noted that the visual fovea module is a prior art and will not be described in detail here.

[0067] As described in above-mentioned step S40, the visual saliency model of the embodiment of the present application can be selected from the ITT1 visual saliency model. The ITT1 visual saliency model is a visual attention model designed based on the visual nervous system of early primates. The model first utilizes a Gaussian sampling method to construct a Gaussian pyramid of color, brightness and direction of an image, then utilizes a Gaussian pyramid to calculate the brightness feature map, color feature map and direction feature map of the image, and finally combines the feature maps of different scales to obtain brightness, color and direction saliency maps, which are added together to obtain the final visual saliency map. This method does not require the process of training and learning, and only through pure mathematical methods, the calculation of the saliency map can be completed. In the embodiment of the present application, the second attention map (i.e., visual saliency map) is generated by simulating a top-down attention mechanism with the visual saliency model.

[0068] As described in step S50 above, in this embodiment of the present application, the first attention map generated by the attribution model (i.e., the bottom-up attention mechanism) and the second attention map generated by the visual saliency model (i.e., the top-down attention mechanism) are fused to generate a third attention map, which serves as the total attention map of the visual classification model. By combining the top-down attention mechanism with the bottom-up attention mechanism, video classification has better interpretability.

[0069] As described in steps S60-S70 above, in this embodiment, a feature vector is generated based on the third attention map and the feature map, and the feature vector is input into the video classification agent in the reinforcement learning model to obtain the classification score of the current video frame and the visual focus of the next video frame.

[0070] To simultaneously perform video classification and visual attention point prediction, the video classification agent of this embodiment shares the input feature vector with both tasks. This offers two advantages: first, the sharing of the feature vector establishes a causal relationship between the two tasks, allowing the visual attention mediated by the visual attention point prediction task to more easily explain the video classification results; second, it significantly reduces the communication workload between the environment and the agent, especially when the feature vector is high-dimensional. A dense classification reward and a sparse visual attention reward are set for the video classification agent to encourage accurate and explainable decisions.

[0071] The present application provides a video classification method based on reinforcement learning, which obtains the current video frame and visual focus of the video to be classified; inputs the classification scores of the current video frame and the previous video frame into an attribution model to generate a feature map and a first attention map, wherein the loss function of the attribution model includes an area loss sub-function; generates a visual attention map based on the current video frame and the visual focus; inputs the visual attention map into a visual saliency model to generate a second attention map; fuses the second attention map with the first attention map to generate a third attention map; generates a feature vector based on the third attention map and the feature map; and obtains the classification score of the current video frame and the visual focus of the next video frame based on the feature vector. Since the attribution model is naturally interpretable, the third attention map generated by combining it with the visual saliency model can not only explain the salient areas in the video frame, but also the areas that contribute to the prediction results of each step and the line of sight focus areas, thereby improving the interpretability of the video classification model in the video classification method; in addition, since the area loss sub-function is added to the loss function of the attribution model, it has a focusing function. Even if the line of sight focus point obtained based on the previous video frame is wrong, it can still display the line of sight focus area after fusion with the result of the attribution model, and it will not be completely out of focus, so that the final generated feature vector is more accurate, thereby improving the accuracy of video classification.

[0072] In one embodiment, inputting the classification scores of the current video frame and the previous video frame into an attribution model to generate a feature map and a first attention map specifically includes:

[0073] The current video frame is input into a convolutional neural network to generate a feature map.

[0074] A convolutional neural network is a multi-layered supervised learning neural network. The convolutional layers and pooling layers in the hidden layers are the core modules that implement the convolutional neural network's feature extraction function. This network model uses gradient descent to minimize the loss function by reversely adjusting the network weight parameters layer by layer, improving network accuracy through frequent iterative training. The lower hidden layers of a convolutional neural network are composed of alternating convolutional layers and max-pooling layers. The upper layers are fully connected layers, corresponding to the hidden layers and logistic regression classifiers of traditional multilayer perceptrons. The input of the first fully connected layer is the feature image extracted by the convolutional and subsampling layers. The final output layer is a classifier that can use logistic regression, softmax regression, or even a support vector machine to classify the input image.

[0075] In one embodiment, please refer to Figure 2 , the inputting the classification scores of the current video frame and the previous video frame into the attribution model to generate a feature map and a first attention map further specifically includes the following steps:

[0076] S201, changing the number of channels of the feature map to C through a first convolutional layer to obtain a first sub-feature map;

[0077] S202, changing the number of channels of the feature map to 1 through a second convolutional layer to obtain a second sub-feature map;

[0078] S203: Obtain a first conditional probability distribution according to the first sub-feature graph;

[0079] S204: Obtain a second conditional probability distribution according to the second sub-feature graph;

[0080] S205. Combine the first conditional probability distribution and the second conditional probability distribution to obtain a joint distribution probability;

[0081] S206. Substitute the classification score of the previous video frame into the joint distribution probability to generate the first attention map.

[0082] As described in S201-S206 above, in this embodiment, the number of channels of the feature map extracted by the convolutional neural network is changed to C and 1 respectively through two convolutional layers, where C is the number of categories. The first sub-feature map of C×H×W is softmaxed (normalized) along the channel to obtain the first conditional probability distribution g yz=p(y|x,z); perform L1 normalization on the second sub-feature of 1×H×W to obtain the second conditional probability distribution h z =p(z|x);h z After the broadcast and g yz Multiply element by element to get the joint probability distribution p(y,z|x). Then the attribution graph (i.e. the first attention graph) output by the attribution model is Among them, y is the category of the training image input during attribution model training, is the true category of the input image x (ie, the current video frame) in the embodiment of the present application, H is the height of the input image x, and W is the height of the input image x.

[0083] If p θ′ (z|x,y) is the probability distribution of z given that the classification result of the input image x is y. For example, the loss function of the attribution model obtained by the EM algorithm is

[0084]

[0085] Among them, θ is the parameter of the attribution model, and θ′ is the distribution parameter of z. According to the Bayesian formula,

[0086]

[0087] so

[0088]

[0089] In addition, in order to make the attribution itself have a focusing function, this scheme adds area loss to the loss function:

[0090]

[0091] Therefore, the total loss function of the attribution model is L = L EM +λL area .

[0092] In one embodiment, please refer to Figure 3 The step of fusing the second attention map with the first attention map to generate a third attention map may specifically include the following steps:

[0093] S501, setting a weight value ρ;

[0094] S502: Set ρ as the weight of the second attention map, and (1-ρ) as the weight of the first attention map;

[0095] S503. Generate the third attention map according to the ρ, (1-ρ), the second attention map and the first attention map.

[0096] As described in steps S501-S503 above, in this embodiment of the present application, in order to combine the top-down attention mechanism and the bottom-up attention mechanism, a weight value ρ is set so that the third attention map is the weighted sum of the first attention map and the second attention map. Specifically:

[0097] S=(1-ρ)S bu +ρS td

[0098] Among them, S is the third attention map, S bu is the first attention map, S td is the second attention map. The weights of the first attention map and the second attention map are determined by the classification results Control, where m limits the maximum weight of the second attention map.

[0099] Since both the first and second attention maps depend on the action of the video classification agent, this method can dynamically fuse the top-down attention mechanism and the bottom-up attention mechanism by taking into account the immediate observation of the environment and the previous decision of the video classification agent. Compared with the general static attention fusion method, the video classification method of the present application has more accurate video classification performance than the method of manually setting fixed weight factors; in addition, since the attention mechanism is explicitly formulated for video classification, the final attention map can be used to explain which area of ​​pixels in the current frame contributed to the classification result of the previous frame. , thereby intuitively explaining the final decision (i.e., action) of the video classification agent.

[0100] In one embodiment, generating a feature vector according to the third attention map and the feature map includes:

[0101] Aggregating the feature map with the third attention map to generate an aggregated map;

[0102] The aggregation graph and the feature graph are merged to generate the feature vector.

[0103] Specifically, the third attention map is multiplied by the elements of each channel of the feature map to generate an aggregate map; the aggregate map and the feature map are concatenated using the cat function, and after corresponding mathematical processing, a feature vector is generated.

[0104] In one embodiment, the attribution model is trained based on the Expectation Maximum Algorithm (EM algorithm). In another embodiment, the attribution model may also be trained based on an algorithm that meets the design requirements, such as the Maximum Likelihood Algorithm (ML algorithm), which is not limited here.

[0105] In one embodiment, in order to increase the training samples of the attribution model and thus make the output of the attribution model more accurate, the previous video frame in the video to be classified and the classification score of the previous video frame can be input into the attribution model to train the attribution model.

[0106] Please refer to Figure 4 , an embodiment of the present application further provides a video classification device, comprising:

[0107] An acquisition module 11 is configured to acquire a current video frame and a visual focus point of the video to be classified, wherein the visual focus point is obtained based on a previous video frame;

[0108] a feature map and first attention map generating module 12, configured to input the classification scores of the current video frame and the previous video frame into an attribution model to generate a feature map and a first attention map, wherein the loss function of the attribution model includes an area loss subfunction;

[0109] A visual attention map generating module 13, configured to generate a visual attention map according to the current video frame and the visual attention point;

[0110] A second attention map generating module 14 is configured to input the visual attention map into a visual saliency model to generate a second attention map;

[0111] A third attention map generating module 15 is configured to fuse the second attention map with the first attention map to generate a third attention map;

[0112] A feature vector generating module 16 generates a feature vector according to the third attention map and the feature map;

[0113] The classification score and visual focus point acquisition module 17 is configured to acquire the classification score of the current video frame and the visual focus point of the next video frame according to the feature vector.

[0114] In an embodiment of the present application, sparse segmented sampling can be performed on the video to be classified to obtain the current video frame. For example, one frame is selected every N video frames as a valid video frame for video recognition, where the value of N is selected based on the actual classification requirements. For example, when used for abnormality recognition in monitoring, the value of N can be selected to be a small value to avoid missing important information. Reasonable sparse segmented sampling can remove a large amount of redundancy in the video to be classified, improve recognition efficiency, and avoid losing important information in the video.

[0115] In some video classification tasks (such as anomaly detection in surveillance videos), the monitored object and its local area typically contain the vast majority of the surveillance utility information. These areas are the region of interest (ROI) that the classification model consistently focuses on, and the local visual information generated by these areas is crucial for guiding classification decisions. However, background areas outside the ROI contain almost no surveillance utility information and have minimal impact on classification results. Therefore, ensuring that the ROI visual information of the active video frames in the video to be classified meets the classification requirements is sufficient. Therefore, active video frames can be processed based on foveal vision processing, known as foveal processing. This approach mimics the information processing mechanisms of the human eye during visual perception to filter surveillance utility information within the active video frames, thereby reducing image data size and improving classification efficiency. The processed image maintains high resolution in the ROI rich in surveillance utility information, meeting the requirements of video classification decisions, while the resolution of background areas further away from the ROI gradually decreases. While reducing the amount of non-surveillance utility information, it still reflects the overall image content, thereby assisting in the proper conduct of surveillance activities.

[0116] To ensure that the display resolution of the image in the effective video frame is consistent with the visual perception resolution, it is first necessary to determine the visual focus point, that is, the corresponding position of the fovea on the image, so as to adjust the spatial resolution distribution of the image to make it consistent with foveal vision. In this embodiment of the application, the visual focus point of the current video frame is obtained based on the previous video frame. That is, when the classification model outputs the classification result of the previous video frame, it also outputs a visual focus point. This visual focus point is used to preprocess the image of the current video frame.

[0117] In one embodiment, the current video frame of the video to be classified is input into the attribution model to generate a first attention map and a feature map, wherein the first attention map is used to simulate the bottom-up attention mechanism. Bottom-up attention mechanisms and top-down attention mechanisms are widely used in fields such as image description and visual question answering to improve the performance of the model in deeper image understanding and multi-step reasoning. Therefore, in the video classification method in the embodiment of the present application, the first attention map is generated by introducing the attribution model to make the classification model more interpretable. Since the attribution model is naturally interpretable, the attention map generated by it not only explains the salient areas in the image, but also explains the areas that contribute to the prediction results of each step, as well as the line of sight focus areas, so the attention map can be directly used as a dynamic attribution map of the video classification model.

[0118] In one embodiment, the attribution model can be designed based on a class activation mapping model (CAM). The class activation mapping model uses global average pooling to present in the form of a heat map which pixels the model uses to know that the image belongs to a certain category, making the classification model transparent and explainable.

[0119] In another embodiment, the attribution model can also be designed based on the more interpretable Class Activation Latent Mapping Model (CALM). The CALM adds a latent variable Z to the CALM model to represent the position of each pixel in the image X. The ML algorithm, EM algorithm, etc. are used to learn the probability distribution of position Z given the known classification result Y.

[0120] It should be noted that in order to make the above-mentioned attribution model have a focusing function, an area loss sub-function is added to the loss function during the attribution model training process. Specifically, for example, when the EM algorithm is used to learn the probability distribution of position Z under the premise of known classification result Y, the total loss function of the attribution model is

[0121] L=L EM +λL area

[0122] Among them, L EM is the loss function of the attribution model obtained according to the EM algorithm, L area is the area loss subfunction, and λ is a hyperparameter. The value range of λ is [0,1]. The larger the λ, the more focused the image.

[0123] The embodiment of the present application can pre-process the current video frame image according to the current video frame and the visual attention point through the visual fovea module to generate a visual attention map. It should be noted that the above-mentioned visual fovea module is a prior art and will not be described in detail here.

[0124] In one embodiment, the visual saliency model selected is the ITT1 visual saliency model. The ITT1 visual saliency model is a visual attention model designed based on the visual nervous system of early primates. The model first uses a Gaussian sampling method to construct a Gaussian pyramid of the image's color, brightness, and direction. It then uses the Gaussian pyramid to calculate the image's brightness feature map, color feature map, and direction feature map. Finally, by combining feature maps at different scales, brightness, color, and direction saliency maps are obtained, which are summed to form the final visual saliency map. This method does not require a training or learning process; the calculation of the saliency map is accomplished using purely mathematical methods. In the embodiments of the present application, the visual saliency model is used to simulate a top-down attention mechanism to generate a second attention map (i.e., a visual saliency map).

[0125] In this embodiment of the present application, the first attention map generated by the attribution model (i.e., the bottom-up attention mechanism) and the second attention map generated by the visual saliency model (i.e., the top-down attention mechanism) are fused to generate a third attention map, which serves as the total attention map of the visual classification model. By combining the top-down attention mechanism with the bottom-up attention mechanism, video classification becomes more interpretable.

[0126] A feature vector is generated based on the third attention map and the feature map, and the feature vector is input into the video classification agent in the reinforcement learning model to obtain the classification score of the current video frame and the visual attention point of the next video frame.

[0127] To simultaneously perform video classification and visual attention point prediction, the video classification agent of this embodiment shares the input feature vector with both tasks. This offers two advantages: first, the sharing of the feature vector establishes a causal relationship between the two tasks, so that the visual attention mediated by the visual attention point prediction task can introspectively explain the video classification results; second, it significantly reduces the communication workload between the environment and the agent, especially when the feature vector is high-dimensional. A dense classification reward and a sparse visual attention reward are set for the video classification agent to encourage accurate and explainable decisions.

[0128] It can be understood that the various components of the video classification device proposed in this application can implement the functions of any of the video classification methods provided in any of the above embodiments, and the specific structure will not be repeated here.

[0129] Reference Figure 5 , the embodiment of the present application also provides a computer device, the internal structure of which can be as follows Figure 5 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer design is used to provide computing and control capabilities. The memory of the computer device includes a storage medium and an internal memory. The storage medium stores an operating system, a computer program and a database. The memory provides an environment for the operation of the operating system and the computer program in the storage medium. The database of the computer device is used to store relevant data of the video classification method based on reinforcement learning provided in any of the above embodiments. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, the video classification method based on reinforcement learning provided in any of the above embodiments is implemented.

[0130] A computer-readable storage medium is also provided in an embodiment of the present application. The computer-readable storage medium may be non-volatile or volatile, and stores a computer program thereon. When the computer program is executed by a processor, the reinforcement learning-based video classification method provided in any of the above embodiments is implemented.

[0131] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided in this application and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-speed SDRAM (SSRSDRAM), expanded SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0132] In summary, the present application provides a video classification method, apparatus, computer equipment and storage medium based on reinforcement learning, which obtains the current video frame and visual focus of the video to be classified; inputs the classification scores of the current video frame and the previous video frame into the attribution model to generate a feature map and a first attention map, wherein the loss function of the attribution model includes an area loss sub-function; generates a visual attention map based on the current video frame and the visual focus; inputs the visual attention map into the visual saliency model to generate a second attention map; fuses the second attention map with the first attention map to generate a third attention map; generates a feature vector based on the third attention map and the feature map; and obtains the classification score of the current video frame and the visual focus of the next video frame based on the feature vector. Since the attribution model is naturally interpretable, the third attention map generated by combining it with the visual saliency model can not only explain the salient areas in the video frame, but also the areas that contribute to the prediction results of each step and the line of sight focus areas, thereby improving the interpretability of the video classification model in the video classification method; in addition, since the area loss sub-function is added to the loss function of the attribution model, it has a focusing function. Even if the line of sight focus point obtained based on the previous video frame is wrong, it can still display the line of sight focus area after fusion with the result of the attribution model, and it will not be completely out of focus, so that the final generated feature vector is more accurate, thereby improving the accuracy of video classification.

[0133] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, apparatus, article, or method comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, apparatus, article, or method. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, apparatus, article, or method comprising the element.

[0134] The above description is only a preferred embodiment of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A video classification method based on reinforcement learning, characterized in that: Used for video classification, including: Obtaining a current video frame and a visual focus point of the video to be classified, wherein the visual focus point is obtained based on a previous video frame; Inputting the classification scores of the current video frame and the previous video frame into an attribution model to generate a feature map and a first attention map, wherein the loss function of the attribution model includes an area loss sub-function; generating a visual attention map according to the current video frame and the visual attention point; Inputting the visual attention map into a visual saliency model to generate a second attention map; Fusing the second attention map with the first attention map to generate a third attention map; generating a feature vector according to the third attention map and the feature map; Obtaining a classification score of a current video frame and a visual focus point of a next video frame according to the feature vector; Add area loss to the loss function: The total loss function of the attribution model is: in, is the loss function of the attribution model obtained according to the EM algorithm, is the area loss sub-function, is a hyperparameter, It is the attribution map output by the attribution model, that is, the first attention map.

2. The video classification method based on reinforcement learning according to claim 1, characterized in that The fusing the second attention map with the first attention map to generate a third attention map includes: Set the weight value ρ; Set ρ as the weight of the second attention map, and (1-ρ) as the weight of the first attention map; Generate the third attention map according to the ρ, (1-ρ), the second attention map and the first attention map; Combining the top-down attention mechanism and the bottom-up attention mechanism, the weight value ρ is set so that the third attention map is the weighted sum of the first attention map and the second attention map. Specifically: Among them, S is the third attention map, is the first attention map, is the second attention map.

3. The video classification method based on reinforcement learning according to claim 1, characterized in that The step of inputting the classification scores of the current video frame and the previous video frame into an attribution model to generate a feature map and a first attention map includes: The current video frame is input into a convolutional neural network to generate a feature map.

4. The video classification method based on reinforcement learning according to claim 3, characterized in that The step of inputting the classification scores of the current video frame and the previous video frame into an attribution model to generate a feature map and a first attention map further includes: The number of channels of the feature map is changed to C through the first convolutional layer to obtain a first sub-feature map; The number of channels of the feature map is changed to 1 through the second convolutional layer to obtain a second sub-feature map; Obtaining a first conditional probability distribution according to the first sub-feature map; Obtaining a second conditional probability distribution according to the second sub-feature map; Combining the first conditional probability distribution and the second conditional probability distribution to obtain a joint distribution probability; Substitute the classification score of the previous video frame into the joint distribution probability to generate the first attention map.

5. The video classification method based on reinforcement learning according to claim 1, characterized in that Generating a feature vector according to the third attention map and the feature map includes: Aggregating the feature map with the third attention map to generate an aggregated map; The aggregation graph and the feature graph are merged to generate the feature vector.

6. The video classification method based on reinforcement learning according to claim 1, characterized in that The attribution model is trained based on the maximum expectation algorithm.

7. The video classification method based on reinforcement learning according to claim 6, characterized in that Also includes: The previous video frame and the classification score of the previous video frame are input into the attribution model for training the attribution model.

8. A video classification device, characterized in that: include: An acquisition module, configured to acquire a current video frame and a visual focus point of the video to be classified, wherein the visual focus point is obtained based on a previous video frame; a feature map and first attention map generation module, configured to input the classification scores of the current video frame and the previous video frame into an attribution model to generate a feature map and a first attention map, wherein the loss function of the attribution model includes an area loss subfunction; a visual attention map generating module, configured to generate a visual attention map according to the current video frame and the visual attention point; A second attention map generation module is used to input the visual attention map into a visual saliency model to generate a second attention map; A third attention map generating module, configured to fuse the second attention map with the first attention map to generate a third attention map; A feature vector generation module generates a feature vector according to the third attention map and the feature map; A classification score and visual focus point acquisition module, configured to acquire the classification score of the current video frame and the visual focus point of the next video frame according to the feature vector; Add area loss to the loss function: The total loss function of the attribution model is: in, is the loss function of the attribution model obtained according to the EM algorithm, is the area loss sub-function, is a hyperparameter, It is the attribution map output by the attribution model, that is, the first attention map.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the reinforcement learning-based video classification method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the reinforcement learning-based video classification method according to any one of claims 1 to 7 are implemented.

Images