A Video Summarization Method Based on Two-Layer Routing Sparse Attention and Spatial Pixel Recalibration

Abstract

This invention discloses a video summarization method based on two-layer routed sparse attention and spatial pixel recalibration, belonging to the field of computer vision. The method includes: reading the input video and extracting frame-level feature vectors; constructing a video summarization generation model, performing channel enhancement on the frame-level feature vectors to obtain enhanced features; performing two-layer routed sparse attention and spatial pixel recalibration on the enhanced features, and fusing them to obtain fused features; inputting the fused features into an importance scoring regression network, outputting frame importance scores, and selecting frames to generate video summaries. This invention effectively fuses global dependencies and local details by combining two-layer routed sparse attention and spatial pixel recalibration, accurately identifying key segments in the video while reducing computational complexity. Experimental results based on the SumMe and TVSum benchmark datasets demonstrate that the method of this invention exhibits good performance in video summarization tasks.

Description

Technical Field

[0001] This invention relates to a video summarization method based on two-layer routing sparse attention and spatial pixel recalibration, belonging to the field of computer vision technology. Background Technology

[0002] With the rapid development of multimedia and network technologies, the scale and quantity of video content have experienced explosive growth. As a high-information-density media format, video has been widely used in applications such as security monitoring, intelligent transportation, education and training, and video retrieval and recommendation. However, video data typically features long time sequences, abundant redundant information, and uneven distribution of key information, posing significant challenges to video storage, transmission, and efficient understanding. Therefore, how to automatically extract key segments representing the core content of raw videos and generate concise and representative video summaries has become one of the important research directions in the field of video content analysis.

[0003] Most existing video summarization methods are based on modeling the importance of video frames or segments, assigning importance scores to each frame to select representative key frames or segments. However, in practical applications, video content often has complex spatial structures and long-term temporal dependencies, which places higher demands on the feature modeling capabilities of summarization models. On the one hand, traditional video summarization methods based on convolutional neural networks mainly rely on local receptive fields for feature extraction. While they can effectively capture local visual information, they have limited ability to model long-distance inter-frame dependencies and global semantic consistency, easily overlooking important content that is semantically related but scattered across different time locations. On the other hand, while methods based on self-attention mechanisms have the ability to model global dependencies, their computational complexity typically increases quadratically with video length or feature space size. When processing high-resolution features or long-term videos, the computational and storage costs are high, making it difficult to balance efficiency and performance.

[0004] Furthermore, existing attention mechanisms, when applied to video summarization tasks, often treat all spatial or temporal locations equally, lacking a fine-grained characterization of the differences in importance between different regions and spatial locations. In complex scenes, numerous background or redundant regions can interfere with attention calculations, making it difficult for the model to focus on truly discriminative regions, thus affecting the accuracy of frame-level importance scoring. Simultaneously, some methods only perform weighted modeling in the channel or temporal dimensions, failing to fully integrate spatial dimensions for feature recalibration, thus limiting the model's ability to express key regions.

[0005] In summary, existing video summarization technologies generally suffer from insufficient feature modeling, difficulty in effectively distinguishing key content from redundant information, and a challenge in balancing global semantic understanding with local detail representation when dealing with long-running, highly redundant, and complex video scenes. Furthermore, improving model expressive power often comes at the cost of significantly increased computational complexity and resource consumption, failing to meet the dual demands of efficiency and performance in practical applications. Therefore, there is an urgent need for a video summarization method that can more comprehensively and effectively model video content while maintaining controllable computational overhead, thereby improving the accuracy, representativeness, and practicality of the generated summaries. Summary of the Invention

[0006] The purpose of this invention is to provide a video summarization method based on dual-layer routing sparse attention and spatial pixel recalibration, which aims to overcome the problems of insufficient modeling of feature dependencies and inadequate ability to distinguish key regions from redundant information in the process of complex video content analysis, resulting in low feature expression efficiency and limited accuracy of importance assessment.

[0007] To achieve the above objectives, this invention provides a video summarization method based on two-layer routing sparse attention and spatial pixel recalibration, comprising the following steps:

[0008] S1: Read the input video, perform frame sampling on the input video, and extract the frame-level feature vector corresponding to each video frame;

[0009] S2: Input the frame-level feature vector into the video summarization generation model to predict the frame-level importance score of each video frame. The video summarization generation model includes:

[0010] Channel enhancement module: The channel enhancement module receives frame-level feature vectors, performs convolution operations and non-linear activation processing, and introduces dilated convolution to obtain intermediate features; then, the intermediate features are input into the channel feature recalibration unit for channel-level weighting processing; finally, the enhanced feature representation is output through residual connections.

[0011] Dual-layer routing sparse attention module: The dual-layer routing sparse attention module receives the enhanced feature representation output by the channel enhancement module, first constructs regional routing relationships based on regional feature correlation and filters candidate regions, then performs pixel-level sparse attention calculations within the filtered regions, and finally introduces local position encoding enhancement and output mapping to obtain the output feature representation of the dual-layer routing sparse attention module;

[0012] Spatial pixel recalibration module: The output of the dual-layer routing sparse attention module is used as the input of the spatial pixel recalibration module. Spatial pixel features are obtained through convolution operation. Weight information is generated based on the spatial pixel features. The spatial pixel features are then weighted position by position using the weight information to output the recalibrated feature representation.

[0013] S3: The output of the dual-layer routing sparse attention module and the output of the spatial pixel recalibration module are fused and input into the importance scoring regression network to output frame-level importance scores, select the most representative frames, and generate the final video summary.

[0014] Preferably, the channel enhancement module includes:

[0015] Multi-layer convolution operations are performed on the input frame-level feature vectors to extract features, and a modified linear unit activation function is introduced between adjacent convolution operations, and dilated convolution is introduced in the convolution operations.

[0016] Subsequently, the intermediate features obtained from the convolution operation are input into the channel feature recalibration unit and subjected to channel-level weighting processing;

[0017] Finally, the module input features are fused with the channel-weighted features through residual connections to output an enhanced feature representation.

[0018] Preferably, the channel feature recalibration unit in the channel enhancement module includes:

[0019] A global average pooling operation is performed on the input features to obtain the statistical vectors corresponding to each channel, and a one-dimensional convolution transformation is performed on the statistical vectors to generate channel weights. The input features are then weighted channel by channel using the channel weights.

[0020] Preferably, the dual-layer routing sparse attention module includes three stages: region-level route selection, pixel-level sparse attention calculation, and the introduction of local location encoding enhancement and output mapping.

[0021] In the region-level routing selection stage, the input enhanced feature representation is divided into multiple regions according to a preset region size, and query feature Q, key feature K, and value feature V are generated respectively. Then, a region averaging operation is performed on Q and K in each region to obtain the region-level query feature Q_region and the region-level key feature K_region. Based on the region-level query feature and the region-level key feature, the region correlation is calculated, and a preset number of candidate regions with the highest correlation are selected from the region correlation to obtain the region routing index.

[0022] In the pixel-level sparse attention calculation stage, based on the region routing index, the feature subsets corresponding to the candidate regions are extracted from the original key feature K and value feature V to form aggregated key feature K_g and value feature V_g. Then, sparse attention calculation is performed on the query feature Q and the aggregated key feature K_g to obtain pixel-level attention weights. These weights are then summed with the value feature set after pixel-level secondary filtering of the candidate region value features to obtain pixel-level sparse attention output.

[0023] In the stage of introducing local location coding enhancement and output mapping, a local location coding compensation term is first introduced into the value feature V to generate a local location coding compensation term L. Then, the pixel-level sparse attention output O is fused with the local location coding compensation term L, and the output feature representation of the dual-layer routing sparse attention module is obtained through output mapping.

[0024] Preferably, the spatial pixel recalibration module includes:

[0025] First, depthwise separable convolution is performed on the input features to extract local features and generate spatial pixel feature representations.

[0026] Then, a global average pooling operation is performed on the spatial pixel feature representation, and after normalization, spatial weight information corresponding to the spatial position is generated. The spatial weight information and the spatial pixel feature representation are then subjected to position-by-position weighted operation.

[0027] Finally, a smooth nonlinear transformation is performed, and the output features of the spatial pixel recalibration module are generated by pointwise convolution.

[0028] Preferably, the importance scoring regression network includes: performing regularization, linear mapping, and nonlinear transformation on the fused features in sequence, introducing random deactivation operations between the linear mappings, and finally generating frame-level importance scores for each video frame through the output activation function.

[0029] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0030] 1. This invention introduces a channel enhancement module into the video frame-level feature modeling process to adaptively model and weight the importance of different channels, effectively strengthening key feature channels related to video semantics and suppressing interference from redundant or irrelevant feature channels. This improves the discriminativeness and stability of frame-level feature representation and provides a more reliable feature basis for subsequent importance assessment.

[0031] 2. The dual-layer routing sparse attention module of the present invention filters features at the region level and performs sparse attention calculation at the pixel level, which effectively reduces the scope of unnecessary attention calculations. While ensuring the ability to model global dependencies, it significantly reduces computational complexity, thereby achieving efficient modeling of inter-frame relationships in long videos and improving the accuracy and robustness of frame-level importance score prediction.

[0032] 3. This invention uses a spatial pixel recalibration module to adaptively recalibrate the importance of different spatial locations in the feature map, enabling the model to focus more on key regions and salient content in the video frame, effectively suppressing the interference of background regions and redundant spatial information, thereby improving the video summarization's ability to characterize key visual content and the representativeness of the overall summarization results.

[0033] 4. This invention sets up an importance score regression network to perform regularization, nonlinear mapping, and random deactivation control on the fused feature representation, effectively suppressing feature noise and overfitting problems, making the prediction process of frame-level importance scores smoother and more stable, thereby improving the robustness and consistency of video summarization generation results in different video scenarios. Attached Figure Description

[0034] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0035] Figure 1 The flowchart illustrates the implementation of the video summarization generation method based on two-layer routing sparse attention and spatial pixel recalibration provided by this invention.

[0036] Figure 2 An overall framework diagram of the video summarization method based on two-layer routing sparse attention and spatial pixel recalibration provided in the embodiments of the present invention;

[0037] Figure 3 This is a schematic diagram of the channel enhancement module provided in an embodiment of the present invention;

[0038] Figure 4 This is a schematic diagram of the structure of the dual-layer routing sparse attention module provided in an embodiment of the present invention;

[0039] Figure 5 This is a schematic diagram of the spatial pixel recalibration module provided in an embodiment of the present invention. Detailed Implementation

[0040] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0041] like Figure 1 The diagram shown is a flowchart of the video summarization generation method based on two-layer routing sparse attention and spatial pixel recalibration provided by the present invention, which includes the following steps:

[0042] S1, Read the input video and extract frame-level feature vectors;

[0043] S2, Construct a video summarization generation model, perform channel enhancement on frame-level feature vectors to obtain enhanced features;

[0044] S3, perform two-layer routing sparse attention and spatial pixel recalibration on the enhanced features, and fuse them to obtain fused features;

[0045] S4 inputs the fused features into the importance score regression network, outputs frame importance scores, and selects frames to generate video summaries.

[0046] Example 1:

[0047] This invention provides a preferred embodiment for executing S1, applicable to video data of any duration and type. The specific steps are as follows:

[0048] First, the original video is sampled at a frequency of two frames per second to obtain a sampled frame sequence. ,in Indicates the first Each sampled frame image, The total number of sampled frames is given. Then, a GoogLeNet network pre-trained on the ImageNet dataset is used as the visual feature encoder to extract features from each sampled frame; specifically, the output of the GoogLeNet network's pool5 layer is taken as the representation vector for that frame, with a dimension of 1024. This yields the frame-level feature sequence. ,in For the corresponding sampling frame The 1024-dimensional feature vector is used to characterize the visual content information contained in the frame.

[0049] Example 2:

[0050] This invention provides a preferred embodiment for executing S2. This embodiment aims to construct a video summarization generation model that includes channel enhancement, two-layer routing sparse attention, and spatial pixel recalibration. This model performs hierarchical modeling and adaptive enhancement of frame-level features, balancing global relationship modeling and key region representation while controlling computational overhead. This alleviates the problems of existing technologies being susceptible to redundant information interference and unstable importance assessment during long video processing, and improves the model's overall understanding of video content, thereby achieving more accurate frame-level importance score prediction. Figure 2 The diagram shown is the overall framework of the network model in this embodiment. The network consists of three parts: a channel enhancement module, a two-layer routing sparse attention module, and a spatial pixel recalibration module. The specific construction steps for these three parts are as follows:

[0051] S21, Construct the channel enhancement module. For example... Figure 3 As shown, specifically, this module first receives ,in Indicates the first The frame-level feature vectors corresponding to each sampled frame are used. To facilitate subsequent convolution operations, the frame-level features Z are mapped and reorganized into a two-dimensional feature representation F, which serves as the input features for the channel enhancement module.

[0052] First, a set of directionally separable two-dimensional convolution operations are sequentially performed on the input features F to extract local feature information. The calculation formula is as follows:

[0053] ,

[0054] in, express Convolution operation, express Convolution operation, This represents the modified linear unit activation function; through concatenation and Convolution, achieving equivalent local Receptive field feature extraction.

[0055] In obtaining local features Subsequently, a convolution operation with a dilation rate is introduced to expand the receptive field, thereby enhancing the ability to model long-term or large-scale contextual information. The calculation formula is as follows:

[0056] ,

[0057] in, express The dilated convolution operation, express The dilated convolution operation, in which The void ratio parameter, This indicates a modification to the linear unit activation function. This step effectively improves the context-awareness of features without significantly increasing computational complexity.

[0058] Subsequently, the above convolution output features The input channel feature recalibration unit first performs a global average pooling operation to generate statistical vectors for each channel, and then generates channel weights based on these statistical vectors through a one-dimensional convolution operation. The calculation formula is as follows:

[0059] ,

[0060] in This indicates a global average pooling operation. This represents a one-dimensional convolution transformation operation, and Sigmoid represents a non-linear activation function used to normalize the channel weights.

[0061] Next, the channel enhancement module adjusts the feature tensor according to the channel weight w. Perform channel-wise weighted processing, and then perform a residual concatenation between the weighted feature tensor and the input frame-level feature tensor F to obtain the enhanced feature representation Y. The calculation formula is as follows:

[0062] ,

[0063] in This indicates a channel-by-channel multiplication operation.

[0064] S22, construct a two-layer routing sparse attention module. For example... Figure 4 The diagram shows a schematic of a two-layer routing sparse attention module. The input to this module is the output of the channel enhancement module, i.e., the enhanced feature representation Y. The module efficiently models the input features through three stages: region-level routing selection, pixel-level sparse attention calculation, and the introduction of local location encoding enhancement and output mapping. The output feature is denoted as U.

[0065] The specific implementation method of this module is as follows:

[0066] S221, Regional Routing Stage. First, linear projection is performed on the input feature Y to generate query feature Q, key feature K, and value feature V. Then, Q and K are divided into regions of a preset size, and average pooling is performed in each region to obtain regional query feature Q_region and regional key feature K_region, which are used to represent the overall semantic information of different regions.

[0067] Based on the region-level query feature Q_region and the region-level key feature K_region, a region similarity matrix a_r is constructed, and the specific calculation formula is as follows:

[0068] ,

[0069] Here, a_r describes the relevance between the regions. Then, for each query region, only the regions with the highest relevance are retained. Generate a region-level routing index for each region:

[0070] ,

[0071] in This indicates selecting the top [top] with the highest relevance score. Each region index is used to filter a set of candidate regions that are highly relevant to the current region, thereby enabling region-level sparse routing selection.

[0072] S222, pixel-level sparse attention computation stage. (In the region-level routing index...) Under the constraint, only the feature subsets corresponding to the candidate regions are extracted from the original key features K and value features V to form the aggregated key features K_g and value features V_g:

[0073] ,

[0074] in This represents a feature selection operation based on region indexing, used to extract key features and numerical features corresponding to candidate regions from the original features.

[0075] Subsequently, within the scope defined by the candidate region, a pixel-level relevance score matrix S is calculated for the query feature Q and the aggregated key feature K_g, using the following formula:

[0076] ,

[0077] Where S represents the pixel-level relevance score matrix, which is used to measure the matching relationship between the query location and the pixel locations within the candidate region.

[0078] To further reduce computational complexity, only the highest relevance score is retained. The pixel-level attention weights are obtained by normalizing the pixel positions. The specific calculation formula is as follows:

[0079] ,

[0080] in This represents a pixel-level sparse filtering operation, used to select the top-scoring results. Pixel position; This represents the normalization function used to generate pixel-level attention weights. .

[0081] Based on the pixel-level filtering results, the set of value features involved in the attention calculation is further extracted from V_g, and the calculation formula is as follows:

[0082] ,

[0083] in This represents a feature selection operation based on pixel-level sparse indexing. This represents the set of value features after performing pixel-level secondary filtering on the candidate region value features, used in conjunction with attention weights. Perform a weighted summation.

[0084] Subsequently, pixel-level attention weights are used to apply the value features. Weighted summation is performed to obtain the pixel-level sparse attention output O, calculated as follows:

[0085] ,

[0086] in, This indicates a weighted summation operation performed according to attention weights.

[0087] S223 introduces local position coding enhancement and output mapping.

[0088] To enhance the representation of local structure and location information, a local location encoding compensation term is introduced into the value feature V in the two-layer routing sparse attention module to generate a local location encoding compensation term L, calculated as follows:

[0089] ,

[0090] in This indicates a local location encoding enhancement operation, used to extract and supplement local location information from the value feature V.

[0091] Pixel-level sparse attention output With the local location coding compensation term The components are fused, and the final output of the two-layer routing sparse attention module is obtained through output mapping:

[0092] ,

[0093] in This indicates the output mapping operation, where U is the output feature of the two-layer routing sparse attention module.

[0094] S23, Construct the spatial pixel recalibration module. For example... Figure 5 The diagram shown illustrates the spatial pixel recalibration module. Specifically, it receives a feature tensor U from the preceding module and performs depthwise separable convolution and pointwise convolution operations on the feature tensor to generate a spatial pixel feature representation N. The calculation process is as follows:

[0095] ,

[0096] in, Indicates the kernel size as Depth convolution operation, This represents a pointwise convolution operation;

[0097] Subsequently, a global average pooling operation is performed on the spatial pixel feature representation N, and after normalization by the Softmax function, spatial weight information SW corresponding to the spatial location is generated. The calculation method is as follows:

[0098] ,

[0099] in This indicates a global average pooling operation. This indicates that the normalization process has been performed using the Softmax function.

[0100] Then, the spatial weight information SW and the spatial pixel feature representation N are subjected to position-by-position weighting processing to obtain a weighted feature representation. The calculation process is expressed as follows:

[0101] ,

[0102] in, This indicates position-by-position weighted processing;

[0103] Then, the weighted features are smoothed and nonlinearly transformed using the GELU function, and then the output features M of the spatial pixel recalibration module are generated through pointwise convolution operations. This output feature M is used for subsequent video summarization score prediction. The specific calculation formula for M is as follows:

[0104] ,

[0105] in, This represents a smooth nonlinear transformation operation. This indicates a pointwise convolution operation.

[0106] Example 3:

[0107] This invention provides a preferred embodiment for executing S3. Specifically, the feature representation U output by the dual-layer routing sparse attention module and the feature representation M output by the spatial pixel recalibration module are adaptively fused to obtain a fused feature representation. The fusion can be implemented using a weighted summation method, that is, features from different sources are assigned preset weights and combined to take into account both global dependency modeling information and key local detail information, thereby improving the discriminativeness and stability of the fused feature.

[0108] Subsequently, the fused features are input into an importance score regression network. This network sequentially performs regularization, linear mapping, and nonlinear transformation on the fused features, introducing random deactivation between adjacent linear mappings. Finally, it generates frame-level importance scores for each video frame through an output activation function. This process effectively suppresses overfitting and improves the robustness of the importance scores, resulting in smoother and more stable frame-level importance prediction.

[0109] Finally, the video frames are sorted or filtered according to the frame-level importance score, and the frames that meet the preset conditions are selected as key frames. The final video summary is then generated based on the key frames.

[0110] To verify the effectiveness of the above embodiments, this invention applies the video summarization generation method to a real dataset and uses the F-score (%) as the evaluation metric for comparative experiments with other state-of-the-art methods. The experiments use two benchmark datasets in the video summarization field, SumMe and TVSum, to evaluate the performance of the proposed model. The SumMe dataset contains 25 videos, each approximately 1 to 6 minutes long, covering various scenes and topics; each video is manually summarized and annotated by 15 to 18 users, providing a reference standard for summarization generation. The TVSum dataset contains 50 videos, each approximately 2 to 10 minutes long; each video is annotated with frame-level importance scores by 20 users, providing more granular supervision and comparison for model training and evaluation.

[0111] To ensure the robustness and objectivity of the experimental results, this invention employs a standard 5-fold cross-validation (5FCV) strategy: the dataset is evenly divided into 5 subsets, and each time 4 subsets are selected as the training set for model training and parameter optimization, while the remaining subset is used as the test set for performance evaluation; this process is repeated 5 times, ensuring that each subset serves as a test set once. The average F-score of the 5 experiments is then taken as the final performance result of the method of this invention.

[0112] Table 1 Comparison with advanced methods Comparison results

[0113]

[0114] The method of this invention was validated on the SumMe and TVSum benchmark datasets, and the experimental results are shown in Table 1. The comparative results show that, compared with existing state-of-the-art methods, this invention achieves superior performance in the F-score index, indicating that it can effectively improve the quality of video summarization generation and has good practical value and technical advantages. This invention's method adaptively enhances frame-level features through a channel enhancement module, improving the expressive power of key semantic information at the feature level; further, it achieves region-level filtering and pixel-level sparse modeling within regions through a two-layer routing sparse attention mechanism, reducing attention computation redundancy while enhancing the modeling ability of global dependencies; and it combines spatial pixel recalibration to adaptively reweight salient regions, thus taking into account both global context and key local details, making the generated summaries more representative and accurate. In addition, this invention exhibits stable evaluation results under multi-fold cross-validation settings, demonstrating good robustness and generalization ability, and can be applied to different scenarios and types of video data, providing a reliable technical solution for practical video summarization applications.

Claims

1. A video summarization method based on two-layer routing sparse attention and spatial pixel recalibration, characterized in that, Includes the following steps: S1: Read the input video, perform frame sampling on the input video, and extract the frame-level feature vector corresponding to each video frame; S2: Input the frame-level feature vector into the video summarization generation model to predict the frame-level importance score of each video frame. The video summarization generation model includes: Channel enhancement module: The channel enhancement module receives frame-level feature vectors, performs convolution operations and non-linear activation processing, and introduces dilated convolution to obtain intermediate features; then, the intermediate features are input into the channel feature recalibration unit for channel-level weighting processing; finally, the enhanced feature representation is output through residual connections. Dual-layer routing sparse attention module: The dual-layer routing sparse attention module receives the enhanced feature representation output by the channel enhancement module, first constructs regional routing relationships based on regional feature correlation and filters candidate regions, then performs pixel-level sparse attention calculations within the filtered regions, and finally introduces local position encoding enhancement and output mapping to obtain the output feature representation of the dual-layer routing sparse attention module; Spatial pixel recalibration module: The output of the dual-layer routing sparse attention module is used as the input of the spatial pixel recalibration module. Spatial pixel features are obtained through convolution operation. Weight information is generated based on the spatial pixel features. The spatial pixel features are then weighted position by position using the weight information to output the recalibrated feature representation. S3: The output of the dual-layer routing sparse attention module and the output of the spatial pixel recalibration module are fused and input into the importance scoring regression network to output frame-level importance scores, select the most representative frames, and generate the final video summary.

2. The video summarization method based on dual-layer routing sparse attention and spatial pixel recalibration as described in claim 1, characterized in that, The channel enhancement module performs multi-layer convolution operations on the input frame-level feature vectors to extract features, and introduces modified linear unit activation functions between adjacent convolution operations, and introduces dilated convolutions in the convolution operations; then, the intermediate features obtained from the convolution operations are input into the channel feature recalibration unit for channel-level weighting processing; finally, the module input features are fused with the channel-level weighted features through residual connections to output an enhanced feature representation.

3. The video summarization method based on dual-layer routing sparse attention and spatial pixel recalibration as described in claim 2, characterized in that, The channel feature recalibration unit in the channel enhancement module performs a global average pooling operation on the input features to obtain the statistical vectors corresponding to each channel, and performs a one-dimensional convolution transformation on the statistical vectors to generate channel weights. The input features are then weighted channel by channel using the channel weights.

4. The video summarization method based on dual-layer routing sparse attention and spatial pixel recalibration as described in claim 1, characterized in that, The dual-layer routing sparse attention module includes three stages: region-level route selection, pixel-level sparse attention calculation, and the introduction of local location encoding enhancement and output mapping. In the region-level routing selection stage, the input enhanced feature representation is divided into multiple regions according to a preset region size, and query feature Q, key feature K, and value feature V are generated respectively. Then, a region averaging operation is performed on Q and K in each region to obtain the region-level query feature Q_region and the region-level key feature K_region. Based on the region-level query feature and the region-level key feature, the region correlation is calculated, and a preset number of candidate regions with the highest correlation are selected from the region correlation to obtain the region routing index. In the pixel-level sparse attention calculation stage, based on the region routing index, the feature subsets corresponding to the candidate regions are extracted from the original key feature K and value feature V to form aggregated key feature K_g and value feature V_g. Then, sparse attention calculation is performed on the query feature Q and the aggregated key feature K_g to obtain pixel-level attention weights. These weights are then summed with the value feature set after pixel-level secondary filtering of the candidate region value features to obtain pixel-level sparse attention output. In the stage of introducing local location coding enhancement and output mapping, a local location coding compensation term is first introduced into the value feature V to generate a local location coding compensation term L. Then, the pixel-level sparse attention output O is fused with the local location coding compensation term L, and the output feature representation of the dual-layer routing sparse attention module is obtained through output mapping.

5. The video summarization method based on dual-layer routing sparse attention and spatial pixel recalibration as described in claim 1, characterized in that, The spatial pixel recalibration module includes, First, depthwise separable convolution is performed on the input features to extract local features and generate spatial pixel feature representations. Then, a global average pooling operation is performed on the spatial pixel feature representation, and after normalization, spatial weight information corresponding to the spatial position is generated. The spatial weight information and the spatial pixel feature representation are then subjected to position-by-position weighted operation. Finally, a smooth nonlinear transformation is performed, and the output features of the spatial pixel recalibration module are generated by pointwise convolution.

6. The video summarization method based on dual-layer routing sparse attention and spatial pixel recalibration as described in claim 1, characterized in that, The importance score regression network performs regularization, linear mapping, and nonlinear transformation on the fused features in sequence, and introduces random deactivation operations between the linear mappings. Finally, it generates the frame-level importance score corresponding to each video frame through the output activation function.

Images