Few-sample video abstraction method based on cross-video time sequence invariance

Abstract

The invention relates to a few-sample video abstraction method based on cross-video time sequence invariance. The method comprises the following steps that a time sequence window attention module enhances video sequence key time features; constructing a time sequence invariance enhancement module alignment support set and abstract semantic features thereof, and generating similar features of the same category; extracting multi-scale features of a query set, a support set, a support abstract and similar features of the same category through a multi-scale feature extraction module, and fusing corresponding features of each scale; through a multi-scale fusion module, performing learning attention weight adaptive fusion on the features corresponding to each scale to obtain fusion features; and fusing the features, outputting frame importance scores through a regression network, selecting representative frames, and generating a video abstract. According to the invention, by combining the time sequence invariance of the cross-video time decay, the time sequence window attention of the inter-frame time decay is enhanced, and the cross-video semantic alignment capability and the time continuity of the abstract in the video are improved.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically to a few-sample video summarization method based on cross-video temporal invariance. Background Technology

[0002] With the surge in the number of internet videos, efficient video browsing has become a crucial issue in the field of video processing. Video summarization methods have emerged to address this, and their effectiveness has been widely validated. Video summarization aims to extract representative keyframes or segments from the original video, allowing users to quickly grasp the main content without watching the entire video.

[0003] Existing few-shot methods treat the video to be summarized as the query set and use videos of the same category and their corresponding summaries as the support set and support summaries, thereby achieving cross-video assisted learning. This method significantly simplifies the training process and improves the model's generalization ability while maintaining low annotation requirements. However, few-shot video summarization, while utilizing videos of the same category for assisted learning, also faces the challenge of cross-video semantic alignment. Specifically, videos of the same category often have differences in duration and misalignment of key events, such as inconsistent start and end positions or different rhythm changes. This makes it difficult to directly establish the correspondence between key frames of the same category of videos, thus reducing the quality of the summary. Furthermore, existing methods use attention mechanisms to improve the model's temporal dependencies; however, global attention often introduces long-distance redundant matching, weakening the dynamic continuity between adjacent frames. While local attention can limit the scope of attention, it usually relies on fixed absolute time positions, making it difficult to adaptively capture temporal dependencies in videos with different rhythms and durations. Finally, existing methods rely solely on single-scale features for temporal modeling. However, key events in videos often exhibit diverse time spans and rhythmic changes: some events last only a few frames but show obvious short-term dynamic changes, while others span longer periods and demonstrate slow semantic changes. If the model only models at a single time scale, it is easy to overlook some temporal features, resulting in insufficient feature representation. Summary of the Invention

[0004] The technical problem of this invention is that a few-sample video summarization method based on cross-video temporal invariance can effectively align semantic information across videos, highlight key changes in the video, and enhance video summarization of short-term details and global changes.

[0005] The purpose of this invention is to solve the above problems and propose a few-sample video summarization method based on cross-video temporal invariance, comprising the following steps:

[0006] Step 1: Obtain the query set video sequences, supporting both set video sequences and summary video sequences; Step 2: Align the semantic features of the support set video sequence and the support summary video sequence using the time invariance enhancement module to generate a similar feature sequence of the same category; Step 3: Enhance key temporal features of the query set video sequences, support set video sequences, and support summary video sequences through the temporal window attention module; Step 4: Extract multi-scale features from the query set video sequence, which supports video sequences, summary video sequences, and similar feature sequences of the same category, and fuse the corresponding features of each scale through the multi-scale feature extraction module. Step 5: Through the multi-scale fusion module, the features corresponding to each scale are adaptively fused using learnable attention weights to obtain the model output features; Step 6: Output Features. The prediction model outputs frame importance scores, selects representative frames, and generates a video summary.

[0007] Preferably, step 1 includes the following sub-steps: 1) Input the video title text into the GPT-3.5 model. The GPT-3.5 model classifies the video based on the prompt and title content to obtain the video category; 2) Input the video sequence features into the GoogLeNet network to extract the visual features at the video frame level.

[0008] Furthermore, Step 2 includes the following sub-steps: 1) Calculate the support set video sequences and support for summary video sequences The semantic similarity matrix; 2) Based on the temporal position difference between the support set video sequences and the support summary sequences, a temporal distance matrix is ​​constructed, and then temporal decay weights are generated; 3) Multiply the time decay weights element-wise with the semantic similarity matrix to obtain the time-invariant attention matrix that incorporates time information; 4) The fused attention is transformed by linear transformation to perform feature mapping, resulting in class similarity features with enhanced time invariance. .

[0009] Furthermore, Step 3 includes the following sub-steps: 1) Query set video sequences Supports the collection of video sequences and support for summary video sequences The query matrix Q, key matrix K, and value matrix V are obtained through linear transformations, respectively. 2) Divide the video sequence along the time dimension to obtain local windows. ; 3) For query frames In the window Calculate each keyframe within the window. The degree of correlation is used to obtain an additive attention score. ; 4) Based on the inter-frame time distance Introducing a time decay coefficient Attention score Time decay is performed to obtain ; 5) Set the weights within the window Attention weights are obtained by normalization. ; 6) After weighting the window value matrix, concatenate it with the original input features through residual connections to obtain the temporally enhanced query set video sequence. Supports the collection of video sequences and support for summary video sequences .

[0010] Preferably, step 4 includes the following sub-steps: 1) Query set video sequences Supports the collection of video sequences Supports summary video sequences Similar feature sequences to categories At each time scale The feature sequences are downsampled separately to obtain features of each sequence at different time scales; 2) Concatenate the features of different time scales of each sequence along the channel dimension, and reduce the number of channels to a uniform feature dimension through linear mapping. , obtain fusion features ; 3) Extract temporal local features using one-dimensional convolution at each time scale to obtain multi-scale features. .

[0011] Furthermore, Step 5 includes the following sub-steps: 1) Features for each time scale Importance scores for each scale are calculated through fully connected layers and via a softmax function. ; 2) Multiply each scale feature by its corresponding weight and sum them up to obtain the fused multi-scale features. ; 3) The fused multi-scale features With query features The output features of the model are obtained by summing the residual connections, followed by linear transformation and layer normalization. .

[0012] Preferably, step 6 includes the following sub-steps: 1) Output features Input the prediction model to generate an importance score for each frame; 2) The Kernel Time Segmentation (KTS) algorithm is used to complete the shot detection. Then, the overall importance score of the shot is determined by calculating the average importance score of all frames within a single shot. 3) Using the Knapsack algorithm for the knapsack problem, select the most valuable shot under length constraints and generate a video summary.

[0013] Preferably, in step 6, the prediction model consists of a shared fully connected layer, a tanh activation function, a dropout layer, and a normalization layer, and achieves different outputs through two parallel branches; the first branch outputs the importance score of the frame, the center position of the segment, and the order consistency score; the second branch outputs the segment length offset, which is used to calibrate the boundaries of the video summary.

[0014] Furthermore, the prediction model is trained using a multi-task loss function, which includes classification loss, ranking consistency loss, center loss, and binary cross-entropy loss.

[0015] Preferably, the sorting consistency loss includes the following sub-steps: 1) Calculate the difference between the predicted and true scores for frame pairs constructed using predicted and true scores; 2) Obtain the sorting direction of the actual sequence; 3) Introduce an interval term for each frame pair to construct a target difference with intervals; 4) Calculate the smoothing penalty loss for a single frame pair; 5) Filter indistinguishable frame pairs using a mask; 6) The overall sorting loss is obtained by averaging the effective penalty terms of all masked frame pairs.

[0016] Compared with the prior art, the beneficial effects of the present invention include: 1) The present invention proposes a few-sample video summarization method based on cross-video temporal invariance. By combining the temporal invariance of cross-video temporal decay, it enhances the temporal window attention with inter-frame temporal decay and improves the cross-video semantic alignment capability and the temporal continuity of video summarization.

[0017] 2) The present invention proposes a few-shot video summarization method based on cross-video temporal invariance. By constructing a multi-scale feature extraction and multi-scale fusion module in the cross-video feature fusion process, local and global semantic structures are modeled in parallel from different time scales. Furthermore, learnable multi-scale attention is used to align and fuse multi-scale feature distributions, thereby improving the structural integrity of the summary.

[0018] 3) The present invention proposes a few-sample video summarization method based on cross-video temporal invariance. By introducing ranking consistency loss in the importance score prediction process, the key frame ranking structure inside the video is strengthened by acting on the prediction layer, so that the model can learn a more stable importance distribution that is more in line with the key frame selection logic, thereby improving the consistency of the prediction results. Attached Figure Description

[0019] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0020] Figure 1 This is a flowchart illustrating a few-sample video summarization method based on cross-video temporal invariance according to an embodiment of the present invention.

[0021] Figure 2 This is a schematic diagram of a few-sample video summarization embodiment of the present invention.

[0022] Figure 3 This is a schematic diagram of the prediction model structure according to an embodiment of the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] like Figure 1 As shown, a few-shot video summarization method based on cross-video temporal invariance includes the following steps: Step 1: Obtain the query set video sequences, supporting both set video sequences and summary video sequences.

[0025] Step 1 includes the following sub-steps: 1) Input the video title text into the GPT-3.5 model. The GPT-3.5 model classifies the videos based on the prompt "The following lines are the video titles, generating categories for each video. Classify the videos according to their titles" and the title content, thus obtaining the video categories; 2) Video sequence features The data is input into the GoogLeNet network to extract visual features at the video frame level.

[0026] Specifically, one frame is sampled every 15 frames, and the feature vector of that frame is obtained through the network, with a feature dimension of 1024.

[0027] like Figure 2 As shown, the video sequences are divided into query set video sequences, support set video sequences, and support summary sequences. Query set videos are the videos to be summarized, support set videos are another video selected from the same category, and support summary videos are the actual summaries of the support set videos. The few-shot video summarization method aims to use support set videos to guide the process of generating summaries from query set videos.

[0028] Step 2: Align the semantic features of the support set video sequence and the support summary video sequence using the time invariance enhancement module to generate a similar feature sequence of the same category.

[0029] Step 2 includes the following sub-steps: 1) Calculate the support set video sequences and support for summary video sequences The semantic similarity matrix S is calculated as follows: ; In the formula, This indicates support for video sequences. , Indicates dimension; This indicates support for summary video sequences. S represents the semantic similarity matrix.

[0030] 2) Based on the temporal position difference between the support set video sequence and the support summary sequence, a temporal distance matrix is ​​constructed, and then temporal decay weights are generated.

[0031] Specifically, for each frame in the supported video sequence For each frame in the support summary sequence Constructing a time matrix Introducing a time decay coefficient Based on the time distance matrix Generate time decay weights The expression is: ; In the formula, This indicates exponential calculation.

[0032] 3) Add time decay weights With semantic similarity matrix Element-wise multiplication yields a time-invariant attention matrix that incorporates temporal information. The expression is: ; In the formula, ⊙ represents element-wise multiplication.

[0033] 4) The fused attention is transformed by linear transformation to perform feature mapping, resulting in class similarity features with enhanced time invariance. .

[0034] Step 3: Enhance key temporal features of the query set video sequences, support set video sequences, and support summary video sequences through the temporal window attention module.

[0035] Step 3 includes the following sub-steps: 1) Query set video sequences Supports the collection of video sequences and support for summary video sequences The query matrix Q, key matrix K, and value matrix V are obtained through linear transformations, respectively, as expressed in the following expressions: ； In the formula, and These represent the feature projection layers of the query matrix, key matrix, and value matrix, respectively.

[0036] 2) Divide the video sequence along the time dimension to obtain local windows. The expression is: ; In the formula, m represents the index of the time window. Indicates the length of the time window. This indicates a local window.

[0037] 3) For query frames In the window Calculate each keyframe within the window. The degree of correlation is used to obtain an additive attention score. The expression is: ; In the formula, This indicates the transpose operation.

[0038] 4) Based on the inter-frame time distance Introducing a time decay coefficient Attention score Time decay is performed to obtain The expression is: ； In the formula, Indicates attention score, Indicates the time distance between frames.

[0039] 5) Set the weights within the window Attention weights are obtained by normalization. The expression is: ; In the formula, This indicates a normalization operation.

[0040] 6) After weighting the window value matrix, concatenate it with the original input features through residual connections to obtain the temporally enhanced query set video sequence. Supports the collection of video sequences and support for summary video sequences The expression is: ; ; In the formula, This represents the temporal enhancement feature at position i. This represents the original input feature at position i. Indicates the position of the i-th position and the window The weight coefficient of the j-th position within the matrix. Display window Features of the window value matrix at position j.

[0041] Step 4: Extract multi-scale features from the query set video sequence, which supports video sequences, summary video sequences, and similar feature sequences of the same category, and fuse the corresponding features of each scale through the multi-scale feature extraction module.

[0042] Step 4 includes the following sub-steps: 1) Query set video sequences Supports the collection of video sequences Supports summary video sequences Similar feature sequences to categories At each time scale The feature sequences are downsampled separately to obtain features at different time scales for each sequence, as expressed in the following expression: ; In the formula, Downsample() represents the downsampling operation.

[0043] 2) Concatenate the features of different time scales of each sequence along the channel dimension, and reduce the number of channels to a uniform feature dimension through linear mapping. , obtain fusion features The expression is: ; In the formula, Concat() represents the concatenation operation.

[0044] 3) Extract temporal local features using one-dimensional convolution at each time scale to obtain multi-scale features. The expression is: ; In the formula, Conv1D() represents a one-dimensional convolution operation.

[0045] Step 5: Through the multi-scale fusion module, the features corresponding to each scale are adaptively fused using learnable attention weights to obtain the model output features.

[0046] Step 5 includes the following sub-steps: 1) Features for each time scale Importance scores for each scale are calculated through fully connected layers and via a softmax function. The expression is: ; In the formula, This indicates the softmax function operation.

[0047] 2) Multiply each scale feature by its corresponding weight and sum them up to obtain the fused multi-scale features. The expression is: ; In the formula, This represents the multi-scale features after fusion. This indicates element-wise multiplication.

[0048] 3) The fused multi-scale features With query features The output features of the model are obtained by summing the residual connections, followed by linear transformation and layer normalization. The expression is: ; In the formula, LayerNorm() represents the layer normalization operation.

[0049] Step 6: Output Features. The prediction model outputs frame importance scores, selects representative frames, and generates a video summary.

[0050] Step 6 includes the following sub-steps: 1) Output features Input the prediction model to generate an importance score for each frame; 2) The Kernel Time Segmentation (KTS) algorithm is used to complete the shot detection. Then, the overall importance score of the shot is determined by calculating the average importance score of all frames within a single shot. 3) Using the Knapsack algorithm for the knapsack problem, select the most valuable shot under the length constraint and generate a video summary. The summary length is limited to 15% of the original video.

[0051] Predictive models such as Figure 3 As shown, the prediction model consists of a shared fully connected layer, a tanh activation function, a dropout layer, and a normalization layer. It achieves different outputs through two parallel branches. The first branch outputs the importance score of the frame, the center position of the segment, and the order consistency score. The second branch outputs the segment length offset, which is used to calibrate the boundaries of the video summary.

[0052] The prediction model is trained using a multi-task loss function, which includes classification loss, ranking consistency loss, center loss, and binary cross-entropy loss.

[0053] The order consistency loss includes the following sub-steps: 1) By predicting scores For each frame pair (i,j) constructed with the true score s, the prediction difference between the frame pairs is calculated. The difference between the actual and true values ​​is calculated using the following formula: ; ; In the formula, α represents the temperature coefficient of the scaling prediction difference.

[0054] 2) Obtain the sorting direction of the actual sequence sgn(i,j), the expression is: ; In the formula, sign() represents the sign function.

[0055] 3) Introduce an interval term m for each frame pair to construct the interval-based target difference. The expression is: ; In the formula, This represents the target difference with intervals.

[0056] 4) Use the softplus function to construct a smoothing penalty term. The expression is: ; In the formula, This indicates a smoothing penalty term.

[0057] 5) Filter indistinguishable frame pairs using a binary mask, the expression is: ; 6) The overall ranking loss is obtained by averaging the effective penalty terms of all masked frame pairs. The expression is: ; In the formula, This represents the overall ranking loss.

[0058] To verify the effectiveness of the above embodiments, this invention uses Kendall's τ and Spearman's ρ correlation coefficients as evaluation metrics for comparison with other state-of-the-art methods. Specifically, the benchmark datasets SumMe and TVSum are used as metrics to evaluate the network.

[0059] Kendall's τ and Spearman's ρ correlation coefficients are used to assess the similarity between model-generated summaries and human-generated summaries, and are expressed as: ; ; ; In the formula, C is the number of identical pairs of video frames, D is the number of distinct pairs of video frames, dis represents the difference between two video frames, and n is the number of frames in the video.

[0060] The SumMe dataset consists of 25 video sequences, comprising videos with different content and various camera angles. These videos are either raw or edited public videos, ranging in length from 1 to 6 minutes.

[0061] The TVSum dataset consists of 50 long video sequences divided into 10 categories. The video content is diverse, including documentaries, news, and video blogs. These videos are all post-processed structured videos, ranging in length from 2 to 10 minutes.

[0062] In the experiment, this invention employs a few-shot learning method. Each training sample contains a query video and an auxiliary video of the same category. The auxiliary video is selected from the same category in the training set and includes manually annotated summaries to guide the model in predicting keyframes of the query video. Within each training cycle, an auxiliary video is randomly selected for each query video. To ensure experimental stability and reproducibility, the experiment was independently repeated 5 times, and the final result was the average of all runs. 20% of the videos in the dataset were selected for testing, and the remaining 80% were used for training.

[0063] As shown in Table 1, this invention is compared with state-of-the-art methods on the SumMe and TVSum datasets.

[0064] Table 1

[0065] As shown in Table 1, although existing methods have made some progress in temporal modeling and cross-video feature utilization, they still have several limitations. dppLSTM only uses recurrent LSTM to extract key information from time frames. DSNet only establishes temporal consistency constraints within a single video, without considering structural information between videos. VJMHT fuses cross-video features through a two-layer Transformer, but fails to handle common event misalignments and time length differences between videos, leading to unstable cross-video alignment. A2Summ and DCGM introduce multimodal features to assist learning, but neglect the importance of pure visual temporal relationships for summary prediction. STVT performs well in modeling temporal dependencies, but it is mostly based on single time scale modeling, easily ignoring dynamic changes at different rhythms in the video. GenSumNet combines Bi-LSTM and Transformer architectures to capture short-term and long-term temporal dependencies, but only uses spatial pyramid pooling for feature extraction and fusion, ignoring local details and global structure of the video. STeMI enhances feature representation by combining spatial information with a multi-scale structure, but its self-attention calculates weights independently in each time segment, lacking modeling of continuity between adjacent frames, thus weakening the stability of local dynamics. Furthermore, this method does not consider semantic alignment relationships between different videos, limiting the effective utilization of cross-video information.

[0066] Compared to the methods described above, this invention, through temporal alignment and sorting consistency constraints, ensures that videos of the same category maintain consistent feature change trends, thereby effectively mitigating feature shifts caused by temporal misalignment and rhythm differences. Simultaneously, this invention proposes a temporal window attention mechanism, which enhances local short-term dependencies while maintaining continuous inter-frame changes, achieving more robust temporal modeling. Furthermore, the multi-scale feature extraction and multi-scale fusion module proposed in this invention can capture local and global dynamic patterns at different time scales and achieve adaptive feature fusion through learnable weights. Experimental results further demonstrate that this invention has significant advantages over existing methods in few-shot video summarization tasks.

[0067] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0068] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A few-sample video summarization method based on cross-video temporal invariance, characterized in that, Includes the following steps: Step 1: Obtain the query set video sequences, supporting both set video sequences and summary video sequences; Step 2: Align the semantic features of the support set video sequence and the support summary video sequence using the time invariance enhancement module to generate a similar feature sequence of the same category; Step 3: Enhance key temporal features of the query set video sequences, support set video sequences, and support summary video sequences through the temporal window attention module; Step 4: Extract multi-scale features from the query set video sequence, which supports video sequences, summary video sequences, and similar feature sequences of the same category, and fuse the corresponding features of each scale through the multi-scale feature extraction module. Step 5: Through the multi-scale fusion module, the features corresponding to each scale are adaptively fused using learnable attention weights to obtain the model output features; Step 6: Output Features. The prediction model outputs frame importance scores, selects representative frames, and generates a video summary.

2. The few-sample video summarization method based on cross-video temporal invariance according to claim 1, characterized in that, Step 1 includes the following sub-steps: 1) Input the video title text into the GPT-3.5 model. The GPT-3.5 model classifies the video based on the prompt and title content to obtain the video category; 2) Input the video sequence features into the GoogLeNet network to extract the visual features at the video frame level.

3. The few-sample video summarization method based on cross-video temporal invariance according to claim 1, characterized in that, Step 2 includes the following sub-steps: 1) Calculate the semantic similarity matrix between the support set video sequences and the support summary video sequences; 2) Based on the temporal position difference between the support set video sequences and the support summary sequences, a temporal distance matrix is ​​constructed, and then temporal decay weights are generated; 3) Multiply the time decay weights element-wise with the semantic similarity matrix to obtain the time-invariant attention matrix that incorporates time information; 4) The fused attention is transformed by linear transformation to perform feature mapping, resulting in class similarity features with enhanced time invariance. .

4. The few-sample video summarization method based on cross-video temporal invariance according to claim 1, characterized in that, Step 3 includes the following sub-steps: 1) The query set video sequences, support set video sequences, and support summary video sequences are transformed linearly to obtain the query matrix Q, key matrix K, and value matrix V, respectively; 2) Divide the video sequence along the time dimension to obtain local windows; 3) For the query frame, calculate each frame within the window and each keyframe within the window. The degree of correlation is used to obtain an additive attention score; 4) Based on the inter-frame time distance, a time decay coefficient is introduced to decay the attention score over time to obtain the weight within the window; 5) Normalize the weights within the window to obtain the attention weights; 6) After weighting the window value matrix, the residual connection is used to concatenate it with the original input features to obtain the temporally enhanced query set video sequence, support set video sequence, and support summary video sequence.

5. The few-sample video summarization method based on cross-video temporal invariance according to claim 1, characterized in that, Step 4 includes the following sub-steps: 1) The query set video sequence, support set video sequence, support summary video sequence, and category similarity feature sequence are downsampled at each time scale to obtain the features of each sequence at different time scales; 2) The features of different time scales of each sequence are concatenated along the channel dimension, and the number of channels is reduced to a unified feature dimension through linear mapping to obtain the fused features; 3) Extract temporal local features using one-dimensional convolution at each time scale to obtain multi-scale features. .

6. The few-sample video summarization method based on cross-video temporal invariance according to claim 1, characterized in that, Step 5 includes the following sub-steps: 1) For each feature at each time scale, an importance score for each scale is calculated through a fully connected layer and via a softmax function; 2) Multiply each scale feature by its corresponding weight and sum them up to obtain the fused multi-scale features; 3) The fused multi-scale features and query features are added together through residual connections, and then subjected to linear transformation and layer normalization to obtain the output features of the model.

7. The few-sample video summarization method based on cross-video temporal invariance according to claim 1, characterized in that, Step 6 includes the following sub-steps: 1) Input the output features into the prediction model to generate an importance score for each frame; 2) The kernel time segmentation (KTS) algorithm is used to complete the shot detection. Then, the overall importance score of the shot is determined by calculating the average importance score of all frames in a single shot. 3) Using the Knapsack algorithm for the knapsack problem, select the most valuable shot under length constraints and generate a video summary.

8. The few-sample video summarization method based on cross-video temporal invariance according to claim 1, characterized in that, In Step 6, the prediction model consists of a shared fully connected layer, a tanh activation function, a dropout layer, and a normalization layer. It achieves different outputs through two parallel branches. The first branch outputs the importance score of the frame, the center position of the segment, and the order consistency score. The second branch outputs the segment length offset, which is used to calibrate the boundaries of the video summary.

9. The few-sample video summarization method based on cross-video temporal invariance according to claim 8, characterized in that, The prediction model is trained using a multi-task loss function, which includes classification loss, ranking consistency loss, center loss, and binary cross-entropy loss.

10. The few-sample video summarization method based on cross-video temporal invariance according to claim 9, characterized in that, The order consistency loss includes the following sub-steps: 1) Calculate the difference between the predicted and true scores for frame pairs constructed using predicted and true scores; 2) Obtain the sorting direction of the actual sequence; 3) Introduce an interval term for each frame pair to construct a target difference with intervals; 4) Calculate the smoothing penalty loss for a single frame pair; 5) Filter indistinguishable frame pairs using a mask; 6) The overall sorting loss is obtained by averaging the effective penalty terms of all masked frame pairs.

Images