Video description method and system based on labeling quality and reward deviation, and medium

Abstract

The invention relates to a video description method and system based on annotation quality and reward deviation, and a medium. The method comprises the following steps: extracting video features of a target video; coding the video features by using a hierarchical visual network to obtain multi-level video coding features; decoding the multi-level video coding features through a video description generation model to obtain a video description result; the step of training the video description generation model comprises the substeps of comprehensively evaluating the video annotation quality through a comprehensive scoring module, encouraging the video description generation model to learn towards high-quality annotations according to a comprehensive evaluation score, and inhibiting noise interference introduced by low-quality annotations; and calculating a deviation reward of each video description based on a reinforcement learning algorithm through a self-review module, estimating an expected reward income of each prediction description according to the deviation reward, and guiding the video description generation model to carry out optimization through the expected reward income. Compared with the prior art, the accuracy and semantic richness of video description are improved.

Description

Technical Field

[0001] This invention relates to the field of video description generation technology, and in particular to a video description method, system, and medium based on annotation quality and reward bias. Background Technology

[0002] Video description generation involves abstracting, summarizing, and restating the visual content of a video using natural language or text. It first requires extracting visual semantic features from the video, then translating them into descriptive sentences that conform to human expression habits, providing a coherent expression of objects, people, actions, scenes, and events in the video. Belonging to the category of high-level visual understanding, it requires models to not only accurately detect and represent the static and dynamic information of the video, but also to map each visual semantic object to semantic labels and rearrange and recombine them according to grammatical format to generate readable video description sentences. The entire process involves multiple fields such as computer vision, machine learning, and natural language processing, and the data transformation and processing are quite complex. Existing methods aim to learn effective video representations to generate high-quality language descriptions, but they neglect the impact of video annotation quality on the model's learning performance. Since the goal of the video description task is to adjust parameters by minimizing the loss function to make the predicted language sentences as close as possible to human annotations, low-quality annotations can cause the model to make incorrect adjustments, propagate incorrect gradient signals, and lead the model to learn incorrect patterns or produce inaccurate predictions. Furthermore, existing reinforcement learning bias rewards require complex inference processes, resulting in high training costs. For example, patent application CN111460883A discloses a method for automatic video behavior description based on deep reinforcement learning. This method extracts spatial and temporal features of the video using ResNet and C3D-ResNet, then generates natural language descriptions using an LSTM encoder-decoder framework with attention mechanisms, and optimizes the policy using the CIDEr metric as the reinforcement learning reward. However, this method only minimizes cross-entropy or maximizes the CIDEr score given manually labeled statements, without evaluating, filtering, or weighting the quality of the training labels themselves. Therefore, in low-quality labeled scenarios, it still propagates erroneous gradients and causes the model to learn incorrect patterns. Furthermore, its reinforcement learning stage still relies on costly sampling argmax comparison and CIDEr calculation, failing to address the problems of reward bias and high training overhead. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of low-quality video annotation and complex reinforcement learning bias reward calculation in the existing technology, and to provide a video description method, system and medium based on annotation quality and reward bias, thereby improving the accuracy and semantic richness of video description.

[0004] The objective of this invention can be achieved through the following technical solutions:

[0005] A video description method based on annotation quality and reward bias includes the following steps:

[0006] Extract video features from the target video;

[0007] The video features are encoded using a hierarchical visual network to obtain multi-level video coded features;

[0008] The video description generation model decodes the multi-level video coding features, iterates to obtain the vocabulary output at each time step, and then obtains the video description result.

[0009] The training steps of the video description generation model include: comprehensively evaluating the quality of video annotations through a comprehensive scoring module, encouraging the video description generation model to learn from high-quality annotations based on the comprehensive evaluation score, and suppressing noise interference introduced by low-quality annotations; calculating the deviation reward for each video description based on a reinforcement learning algorithm through a self-censorship module, estimating the expected reward for each predicted description based on the deviation reward, and guiding the video description generation model to optimize based on the expected reward.

[0010] Furthermore, the video features include two-dimensional video features, three-dimensional video features, and target object features.

[0011] Furthermore, the specific steps for comprehensively evaluating the quality of video annotations through the comprehensive scoring module include:

[0012] Extract the sequence of real descriptive statements from the target video and perform preprocessing;

[0013] The annotation quality of the real descriptive sentence sequences in the training data is evaluated based on multiple preset evaluation metrics, and a comprehensive evaluation score is calculated for each annotation. The formula for calculating the comprehensive evaluation score is as follows:

[0014]

[0015] In the formula, M(G) (i) ) represents the overall evaluation score, K represents the types of evaluation indicators, and λ represents the overall evaluation score. k M represents the weighting coefficients for different evaluation matrices. k To evaluate the score calculation function of the matrix, G (i) Label the i-th video.

[0016] Furthermore, the specific steps for encouraging the video description generation model to learn from high-quality annotations and suppressing noise interference introduced by low-quality annotations based on the comprehensive evaluation score include: during the supervised learning phase of model training, using the comprehensive evaluation score as a dynamic weight to adaptively weight and adjust the traditional cross-entropy loss function.

[0017] Furthermore, in the process of training the video description generation model through the comprehensive scoring module, the first-stage loss function used is:

[0018] L1(θ)=L base (θ)+L csm (θ)

[0019]

[0020] In the formula, L1(θ) is the first-stage loss function, L base (θ) represents the overall loss of the hierarchical visual network, L csm (θ) is the loss function of the comprehensive scoring module, T is the total number of video annotations, and M(G) is the loss function of the comprehensive scoring module. (i) (G) is the comprehensive evaluation score. (i) Let p be the probability distribution of the video description predicted by the model, where p is the annotation for the i-th video. Let F be the predicted label for the i-th video, F be the multi-level video coding feature, and θ be the model parameters.

[0021] Furthermore, the formula for calculating the deviation reward in the video description is as follows:

[0022]

[0023] In the formula, b cr The deviation reward is T, where T is the total number of video annotations, and S is the total number of annotations. i The model outputs the corresponding statement for the annotation of the i-th video. The sum of reinforcement learning rewards for the other statements, r(S) i ) is the reinforcement learning reward for the model output statement corresponding to the annotation of the i-th video.

[0024] Furthermore, in the process of training the video description generation model through the self-censorship module, the second-stage loss function used is:

[0025] L2(θ)=L base (θ)+L ssm (θ)

[0026]

[0027] In the formula, L2(θ) is the second-stage loss function, L base (θ) represents the overall loss of the hierarchical visual network. For gradient update size, L ssm (θ) is the loss function of the self-censoring module, r(S) i S is the reinforcement learning reward for the model output statement corresponding to the i-th video annotation. iThe model output statement corresponding to the annotation of the i-th video is b. cr Here, p represents the bias reward, p is the probability distribution of the video description predicted by the model, F is the multi-level video coding feature, and θ is the model parameter.

[0028] Furthermore, the specific steps for optimizing the video description generation model by guiding it with expected reward benefits include:

[0029] The expected reward is estimated based on the deviation reward, and the gradient update size is calculated based on the expected reward. The gradient update size is used as a guiding signal to guide the model to optimize.

[0030] According to another aspect of the present invention, a computer-readable storage medium is provided, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it can implement the video description method based on annotation quality and reward bias as described above.

[0031] According to another aspect of the present invention, a video description system based on annotation quality and reward bias is provided, comprising:

[0032] The video feature extraction module is used to extract video features from the target video;

[0033] The multi-level video coding feature acquisition module uses a hierarchical visual network to encode the video features to obtain multi-level video coding features;

[0034] The video description generation module is used to decode the multi-level video coding features through the video description generation model, iteratively obtain the vocabulary output at each time step, and then obtain the video description result. The training steps of the video description generation model include: comprehensively evaluating the quality of video annotations through a comprehensive scoring module, encouraging the video description generation model to learn from high-quality annotations based on the comprehensive evaluation score, and suppressing noise interference introduced by low-quality annotations; calculating the bias reward for each video description based on a reinforcement learning algorithm through a self-censoring module, estimating the expected reward for each predicted description based on the bias reward, and guiding the video description generation model to optimize based on the expected reward.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] 1. This invention comprehensively evaluates the quality of video annotations through a comprehensive scoring module. Based on the comprehensive evaluation score, it encourages the video description generation model to learn from high-quality annotations and suppresses noise interference introduced by low-quality annotations. In the supervised learning stage of model training, the comprehensive evaluation score is used as a dynamic weight to adaptively adjust the traditional cross-entropy loss function, thereby enhancing the positive impact of high-quality annotations on the model's learning effect and improving the accuracy and semantic richness of the model's video descriptions.

[0037] 2. This invention uses a self-censoring module to calculate the bias reward for each video description based on a reinforcement learning algorithm, and estimates the expected reward for each predicted description based on the bias reward. The gradient update size is then calculated using the expected reward, and used as a guiding signal to guide the model to optimize. Different bias rewards are adjusted for different predicted sentences of the same video, thereby improving the robustness of the model. Attached Figure Description

[0038] Figure 1 This is a flowchart illustrating a video description method based on annotation quality and reward bias proposed in this invention.

[0039] Figure 2 A schematic diagram of the first stage of training for the video description generation model;

[0040] Figure 3 A schematic diagram of the second stage of training for the video description generation model;

[0041] Figure 4 Example sentences for describing a video. Detailed Implementation

[0042] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0043] The following English abbreviations are involved:

[0044] Hierarchical Modular Network (HMN)

[0045] Long Short-Term Memory (LSTM) network

[0046] Example 1

[0047] This embodiment provides a video description method based on annotation quality and reward bias, such as... Figure 1 As shown, it includes the following steps:

[0048] S1. Extract video features from the target video.

[0049] Use a pre-trained network to extract video 2D features, video 3D features, and object features from the video.

[0050] S2. Use a hierarchical visual network to encode video features to obtain multi-level video coding features.

[0051] The final visual representation is generated using a Hierarchical Modular Network (HMN). The loss function of the HMN is used to calculate the cross-entropy loss of the model's output video description sequence and the semantic loss at different levels. The loss of this module is represented as L. base .

[0052] L base =L e +L h +L s +L XE

[0053] In the formula, L XE For cross-entropy loss, L e For entity loss in hierarchical visual networks, L h For predicate loss in hierarchical visual networks, L s The sentence loss is for hierarchical visual networks.

[0054] The physical loss is calculated as follows:

[0055]

[0056] In the formula, N is the number of objects specified in the video. Embed n for entity words i Semantic embeddings of entity words predicted by the model The pairwise matching loss between them.

[0057] The predicate loss is calculated as follows:

[0058]

[0059] In the formula, h represents the features of the predicates contained in the video after pre-trained SBERT encoding. Features are generated after video actions are mapped to the text semantic space.

[0060] The statement loss is calculated as follows:

[0061]

[0062] In the formula, s represents the video description features encoded by pre-trained SBERT. These are global video semantic embedding features.

[0063] S3. The video description generation model decodes the multi-level video coding features and iteratively obtains the vocabulary output at each time step, thereby obtaining the video description result.

[0064] The video caption generation model uses an LSTM network. The training process includes two phases. The first phase involves training the video caption generation model, such as... Figure 2 As shown. The comprehensive scoring module comprehensively evaluates the quality of video annotations, extracts the sequence of true descriptive statements from the target video, and preprocesses it. The sequence of true descriptive statements for the video is represented as follows:

[0065]

[0066] In the formula, G is a sequence of actual description statements. (T) Let T be the Tth true description statement, where T is the number of video annotations.

[0067] The annotation quality of real descriptive sentence sequences in the training data is evaluated based on multiple preset evaluation metrics. A comprehensive evaluation score for each annotation is calculated using different weighted combinations of four evaluation metrics: CIDEr, BLEU4, METEOR, and ROUGE_L. The formula for calculating the comprehensive evaluation score is as follows:

[0068]

[0069] In the formula, M(G) (i) ) represents the overall evaluation score, K represents the types of evaluation indicators, and λ represents the overall evaluation score. k M represents the weighting coefficients for different evaluation matrices. k To evaluate the score calculation function of the matrix, G (i) Label the i-th video.

[0070] The comprehensive evaluation score encourages the video description generation model to learn from high-quality annotations and suppresses noise interference introduced by low-quality annotations: during the supervised learning phase of model training, the comprehensive evaluation score is used as a dynamic weight to adaptively weight and adjust the traditional cross-entropy loss function.

[0071] The first-stage loss function used in training the video description generation model through the comprehensive scoring module is:

[0072] L1(θ)=L base (θ)+L csm (θ)

[0073]

[0074] In the formula, L1(θ) is the first-stage loss function, L base (θ) represents the overall loss of the hierarchical visual network, L csm (θ) is the loss function of the comprehensive scoring module, T is the total number of video annotations, and M(G) is the loss function of the comprehensive scoring module. (i) (G) is the comprehensive evaluation score. (i) Let p be the probability distribution of the video description predicted by the model, where p is the annotation for the i-th video. Let F be the predicted label for the i-th video, F be the multi-level video coding feature, and θ be the model parameters.

[0075] The video description generation model undergoes a second stage of training, such as... Figure 3 As shown. The model's performance is further enhanced by performing reinforcement learning with reward bias using a self-censoring module. First, the sequence of predicted descriptive statements for the video is represented as:

[0076]

[0077] In the formula, To predict the sequence of descriptive statements, S (T) Let T be the T-th predicted description statement, where T is the number of video statement outputs, which is equal to the number of actual description statements in the video.

[0078] The self-censorship module calculates the bias reward for each video description based on a reinforcement learning algorithm. The reinforcement learning bias reward is calculated based on the CIDEr score of each independent predicted sentence in the same video. The expected reward for each predicted description is estimated based on the bias reward. The expected reward is used to guide the optimization of the video description generation model. The gradient update size is calculated based on the expected reward and used as a guiding signal to guide the model optimization.

[0079] The formula for calculating the deviation reward in the video description is:

[0080]

[0081] In the formula, b cr The deviation reward is T, where T is the total number of video annotations, and S is the total number of annotations. i The model outputs the corresponding statement for the annotation of the i-th video. The sum of reinforcement learning rewards for the other statements, r(S) i ) is the reinforcement learning reward for the model output statement corresponding to the annotation of the i-th video.

[0082] The second-stage loss function used in training the video description generation model through the self-censorship module is:

[0083] L2(θ)=Lbase (θ)+L ssm (θ)

[0084]

[0085] In the formula, L2(θ) is the second-stage loss function, L base (θ) represents the overall loss of the hierarchical visual network. For gradient update size, L ssm (θ) is the loss function of the self-censoring module, r(S) i S is the reinforcement learning reward for the model output statement corresponding to the i-th video annotation. i The model output statement corresponding to the annotation of the i-th video is b. cr Here, p represents the bias reward, p is the probability distribution of the video description predicted by the model, F is the multi-level video coding feature, and θ is the model parameter.

[0086] The encoded features are decoded using a two-stage trained video description generation model. An LSTM network is used as the description generator, and the visual features, the word from the previous time step, and the hidden state are input into the LSTM to calculate the new hidden state.

[0087] h t =LSTM([F;Embed(w t-1 )];h t-1 )

[0088] In the formula, h t For the hidden state, F represents the visual features extracted from the video, and Embed(w) represents the hidden state. t-1 ) is the word w t-1 The embedding vector, w t-1 For the word in the previous time step, h t-1 This is the hidden state from the previous time step.

[0089] Based on the current hidden state of the LSTM, a fully connected layer and a softmax function are used to calculate the predicted probability distribution across the entire vocabulary at the current time step. This allows for iterative acquisition of predicted word outputs, which are then used to generate video descriptions word by word in sequence, ultimately resulting in a complete video description. An example of the generated video description sentence is shown below. Figure 4 As shown, Figure 4 In this context, Ground truth refers to the actual situation, Base-model is the video description generated by the baseline model, and STG-DR is the video description generated by the model of this invention.

[0090] To verify the performance of the video description method based on annotation quality and reward bias proposed in this invention, the following experiments were designed.

[0091] (1)M1: This model is a video description generation model. In specific deployment, it is implemented using a hierarchical visual network. This model serves as the benchmark model for fact description generation.

[0092] (2) M2: This model is based on the M1 model. A comprehensive scoring module is added to the M1 model to combine the loss functions of the two parts to optimize the model. This model is designed to verify the effectiveness of the comprehensive scoring module in the model.

[0093] (3)M3: This model is based on the M1 model. A self-censoring module is added to the M1 model to combine the loss functions of the two parts to optimize the model. This model is designed to verify the effectiveness of the self-censoring module in the model.

[0094] The performance comparison results of each model on the MSVD dataset are shown in Table 1.

[0095] Table 1. Performance comparison of each model on the MSVD dataset.

[0096] Model B@4 METEOR ROUGE_L CIDEr M1 58.3 36.8 74.2 102.0 M2 59.6 38.0 75.0 107.9 M3 58.5 37.4 74.3 105.5 This invention (SGT-DR) 61.1 38.1 75.2 108.0

[0097] As shown in Table 1, the model of this invention outperforms other models in all metrics, especially in the CIDEr metric (which measures the similarity between the description and the human annotation), indicating that the video description it generates is more accurate and richer.

[0098] Table 2 shows the performance comparison of each model on the MSR-VTT dataset.

[0099] Table 2 shows the performance comparison of each model on the MSR-VTT dataset.

[0100]

[0101]

[0102] As shown in Table 2, the model of this invention exhibits superior video description generation capabilities on the MSR-VTT dataset, especially in terms of the CIDEr metric, further validating the effectiveness and generalization ability of the model of this invention.

[0103] On the MSVD and MSR-VTT datasets, the CIDEr of the model of this invention reached 108.0 and 53.1 respectively, which is an improvement of 5.9% and 3.0% compared with the benchmark model, verifying that the video description of the model of this invention is more accurate and richer and has good generalization ability.

[0104] Example 2

[0105] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the video description method based on annotation quality and reward bias as proposed in Embodiment 1.

[0106] The rest is the same as in Example 1.

[0107] Example 3

[0108] This embodiment provides a video description system based on annotation quality and reward bias, including:

[0109] The video feature extraction module is used to extract video features from the target video;

[0110] The multi-level video coding feature acquisition module uses a hierarchical visual network to encode video features to obtain multi-level video coding features;

[0111] The video description generation module decodes multi-level video coding features using a video description generation model, iteratively obtaining the vocabulary output at each time step to achieve the video description result. The training steps of the video description generation model include: comprehensively evaluating the quality of video annotations through a comprehensive scoring module, encouraging the video description generation model to learn from high-quality annotations based on the comprehensive evaluation score, and suppressing noise interference introduced by low-quality annotations; calculating the bias reward for each video description using a self-censoring module based on a reinforcement learning algorithm, estimating the expected reward for each predicted description based on the bias reward, and guiding the video description generation model to optimize based on the expected reward.

[0112] The rest is the same as in Example 1.

[0113] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A video description method based on annotation quality and reward bias, characterized in that, Includes the following steps: Extract video features from the target video; The video features are encoded using a hierarchical visual network to obtain multi-level video coded features; The video description generation model decodes the multi-level video coding features, iterates to obtain the vocabulary output at each time step, and then obtains the video description result. The training steps of the video description generation model include: comprehensively evaluating the quality of video annotations through a comprehensive scoring module, encouraging the video description generation model to learn from high-quality annotations based on the comprehensive evaluation score, and suppressing noise interference introduced by low-quality annotations; calculating the deviation reward for each video description based on a reinforcement learning algorithm through a self-censorship module, estimating the expected reward for each predicted description based on the deviation reward, and guiding the video description generation model to optimize based on the expected reward.

2. The video description method based on annotation quality and reward bias according to claim 1, characterized in that, The video features include two-dimensional video features, three-dimensional video features, and target object features.

3. The video description method based on annotation quality and reward bias according to claim 1, characterized in that, The specific steps for comprehensively evaluating the quality of video annotations using the integrated scoring module include: Extract the sequence of real descriptive statements from the target video and perform preprocessing; The annotation quality of the real descriptive sentence sequences in the training data is evaluated based on multiple preset evaluation metrics, and a comprehensive evaluation score is calculated for each annotation. The formula for calculating the comprehensive evaluation score is as follows: In the formula, M(G) (i) ) represents the overall evaluation score, K represents the types of evaluation indicators, and λ represents the overall evaluation score. k M represents the weighting coefficients for different evaluation matrices. k G is a function for calculating the score of the evaluation matrix. (i) Label the i-th video.

4. The video description method based on annotation quality and reward bias according to claim 1, characterized in that, The specific steps for encouraging video description generation models to learn from high-quality annotations and suppressing noise interference introduced by low-quality annotations based on the comprehensive evaluation score include: during the supervised learning phase of model training, using the comprehensive evaluation score as a dynamic weight to adaptively weight and adjust the traditional cross-entropy loss function.

5. The video description method based on annotation quality and reward bias according to claim 1, characterized in that, The first-stage loss function used in training the video description generation model through the comprehensive scoring module is: L1(θ)=L base (θ)+L csm (i) In the formula, L1(θ) is the first-stage loss function, L base (θ) represents the overall loss of the hierarchical visual network, L csm (θ) is the loss function of the comprehensive scoring module, T is the total number of video annotations, and M(G) is the loss function of the comprehensive scoring module. (i) (G) is the comprehensive evaluation score. (i) Let p be the probability distribution of the video description predicted by the model, where p is the annotation for the i-th video. Let F be the predicted label for the i-th video, F be the multi-level video coding feature, and θ be the model parameters.

6. The video description method based on annotation quality and reward bias according to claim 1, characterized in that, The formula for calculating the deviation reward in the video description is as follows: In the formula, b cr The reward is for deviation, where T is the total number of video annotations and S is the total number of annotations. i The model outputs the corresponding statement for the annotation of the i-th video. The sum of reinforcement learning rewards for the other statements, r(S) i ) is the reinforcement learning reward for the model output statement corresponding to the annotation of the i-th video.

7. The video description method based on annotation quality and reward bias according to claim 1, characterized in that, The second-stage loss function used in training the video description generation model through the self-censorship module is: L2(θ)=L base (θ)+L ssm (i) ▽ θ L ssm (θ)≈-(r(S i )-b cr )▽ θ logp(S|F;θ) In the formula, L2(θ) is the second-stage loss function, L base (θ) represents the overall loss of the hierarchical visual network, ▽ θ For gradient update size, L ssm (θ) is the loss function of the self-censoring module, r(S) i S is the reinforcement learning reward for the model output statement corresponding to the i-th video annotation. i The model output statement corresponding to the annotation of the i-th video is b. cr Here, p represents the bias reward, p is the probability distribution of the video description predicted by the model, F is the multi-level video coding feature, and θ is the model parameter.

8. The video description method based on annotation quality and reward bias according to claim 1, characterized in that, The specific steps for optimizing the video description generation model by guiding it with expected reward benefits include: The expected reward is estimated based on the deviation reward, and the gradient update size is calculated based on the expected reward. The gradient update size is used as a guiding signal to guide the model to optimize.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, enables the video description method based on annotation quality and reward bias as described in any one of claims 1 to 8.

10. A video description system based on annotation quality and reward bias, characterized in that, include: The video feature extraction module is used to extract video features from the target video; The multi-level video coding feature acquisition module uses a hierarchical visual network to encode the video features to obtain multi-level video coding features; The video description generation module is used to decode the multi-level video coding features through the video description generation model, iteratively obtain the vocabulary output at each time step, and then obtain the video description result. The training steps of the video description generation model include: comprehensively evaluating the quality of video annotations through a comprehensive scoring module, encouraging the video description generation model to learn from high-quality annotations based on the comprehensive evaluation score, and suppressing noise interference introduced by low-quality annotations; calculating the bias reward for each video description based on a reinforcement learning algorithm through a self-censoring module, estimating the expected reward for each predicted description based on the bias reward, and guiding the video description generation model to optimize based on the expected reward.

Images