Video frame extraction methods, devices, equipment and media

Abstract

This invention relates to the fields of finance, healthcare, and artificial intelligence, and provides a video frame extraction method, apparatus, device, and medium. It enables the generation of a video frame extraction model through self-supervised end-to-end training based on a cross-domain enhanced consistency mechanism. Combining cross-domain enhanced consistency regularization and self-supervised learning improves the adaptability and robustness of keyframe selection under varying conditions. It eliminates the need for target domain labeled data, achieving cross-domain robust keyframe extraction while maintaining spatiotemporal coherence. The preprocessing module in the video frame extraction model obtains a target spatiotemporal cube, introducing temporal context information to facilitate the capture of dynamic relationships between frames. A domain-adaptive feature encoder is used to obtain temporal enhancement features, enabling multi-scale temporal modeling and maintaining inter-frame temporal coherence. Dynamic threshold-based target frame filtering automatically optimizes the filtering granularity according to scene complexity, thereby improving the accuracy and adaptability of keyframe extraction.

Description

Technical Field

[0001] This invention relates to the fields of finance, healthcare and artificial intelligence, and in particular to a method, apparatus, device and medium for extracting video frames. Background Technology

[0002] Keyframe extraction is a fundamental task in video processing. Its purpose is to select the most representative frames from a video sequence to reduce data redundancy and retain core content. For example, in the financial sector, keyframes extracted from ATM surveillance videos can be used to identify abnormal behavior, thereby ensuring fund security; in the healthcare sector, keyframes extracted from operating room surveillance videos can assist in medical teaching.

[0003] In existing technologies, the following methods are mainly used for keyframe extraction:

[0004] (1) Relying on manually designed features (such as color histograms, optical flow, etc.) and heuristic rules (such as lens boundary detection, inter-frame difference threshold, etc.).

[0005] This method has limitations in its ability to generalize across different scenarios, especially when the video content or shooting conditions change significantly, its performance will drop significantly.

[0006] (2) Deep learning technologies, such as frame-level representation learning based on spatial feature extraction using convolutional neural networks, and capturing inter-frame dynamic relationships using time-series models such as long short-term memory networks.

[0007] This method typically requires a large amount of labeled data and is sensitive to domain offsets (such as videos from different lighting, viewpoints, or device sources). Summary of the Invention

[0008] In view of the above, it is necessary to provide a video frame extraction method, apparatus, device and medium, which aims to solve the problems of poor generalization ability, low accuracy and large amount of labeled data required for video frame extraction.

[0009] A video frame extraction method, the video frame extraction method comprising:

[0010] In response to the instruction to extract video frames of the target video within the target domain, the video frame extraction model obtained by self-supervised end-to-end training based on the cross-domain enhanced consistency mechanism is invoked.

[0011] The target video is preprocessed using the preprocessing module in the video frame extraction model to obtain the target spatiotemporal cube;

[0012] The target video and the target spatiotemporal cube are used to extract features by the domain adaptive feature encoder in the video frame extraction model to obtain temporal enhancement features;

[0013] The temporal enhancement features are filtered based on a dynamic threshold to obtain the target frames within the target video.

[0014] A video frame extraction device, the video frame extraction device comprising:

[0015] The retrieval unit is used to retrieve the video frame extraction model obtained by self-supervised end-to-end training based on the cross-domain enhanced consistency mechanism in response to the video frame extraction instruction of the target video in the target domain.

[0016] The preprocessing unit is used to preprocess the target video using the preprocessing module in the video frame extraction model to obtain the target spatiotemporal cube;

[0017] The feature extraction unit is used to extract features from the target video and the target spatiotemporal cube using the domain adaptive feature encoder in the video frame extraction model to obtain temporal enhancement features.

[0018] The filtering unit is used to filter the temporal enhancement features based on a dynamic threshold to obtain the target frames within the target video.

[0019] A computer device, the computer device comprising:

[0020] Memory, storing at least one instruction; and

[0021] The processor executes the instructions stored in the memory to implement the video frame extraction method.

[0022] A computer-readable storage medium storing at least one instruction, which is executed by a processor in a computer device to implement the video frame extraction method.

[0023] As can be seen from the above technical solutions, this invention can obtain a video frame extraction model through self-supervised end-to-end training based on a cross-domain enhanced consistency mechanism. Combining cross-domain enhanced consistency regularization and self-supervised learning, it improves the adaptability and robustness of keyframe selection under varying conditions. It does not require target domain labeled data and achieves cross-domain robust keyframe extraction while maintaining spatiotemporal coherence. The preprocessing module in the video frame extraction model obtains a target spatiotemporal cube, introducing temporal context information to facilitate the capture of dynamic relationships between frames. The domain adaptive feature encoder obtains temporal enhancement features, realizing multi-scale temporal modeling and maintaining temporal coherence between frames. Based on dynamic threshold filtering of target frames, it can automatically optimize the filtering granularity according to scene complexity, thereby improving the accuracy and adaptability of keyframe extraction. Attached Figure Description

[0024] Figure 1This is a flowchart of a preferred embodiment of the video frame extraction method of the present invention.

[0025] Figure 2 This is a functional block diagram of a preferred embodiment of the video frame extraction device of the present invention.

[0026] Figure 3 This is a schematic diagram of the structure of a computer device that implements the video frame extraction method of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] like Figure 1 The diagram shown is a flowchart of a preferred embodiment of the video frame extraction method of the present invention. Depending on different requirements, the order of the steps in this flowchart can be changed, and some steps can be omitted.

[0029] The video frame extraction method is applied to one or more computer devices. The computer device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0030] The computer device can be any electronic product that can interact with the user, such as a personal computer, tablet computer, smartphone, personal digital assistant (PDA), game console, interactive network television (IPTV), smart wearable device, etc.

[0031] The computer equipment may also include network equipment and / or user equipment. The network equipment includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0032] The server can be a standalone server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0033] Artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0034] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0035] The network in which the computer device is located includes, but is not limited to, the Internet, wide area network, metropolitan area network, local area network, and virtual private network (VPN).

[0036] S10, in response to the instruction to extract video frames of the target video within the target domain, invokes the video frame extraction model obtained by self-supervised end-to-end training based on the cross-domain enhanced consistency mechanism.

[0037] In this embodiment, the video frame extraction command can be triggered according to different task requirements in different fields.

[0038] For example, when it is necessary to process surveillance videos of ATMs in the financial sector or operating rooms in the medical and health sector, the video frame extraction instruction can be triggered to extract key frames from the corresponding surveillance videos, reduce data redundancy and retain core content, and provide a data foundation for subsequent tasks such as behavior recognition, content summary generation, and video retrieval.

[0039] In this embodiment, before retrieving the video frame extraction model obtained through self-supervised end-to-end training based on the cross-domain enhanced consistency mechanism, the method further includes:

[0040] Constructing the contrastive loss: in, This represents the contrast loss; The time-series enhancement feature z at time t is represented. tEnhanced features of the positive sample at time t Cosine similarity between them This is obtained by applying a domain-specific enhancement to the spacetime cube; θ represents the temperature coefficient; z - Represents negative sample features; s(z) t ,z - ) represents the temporal enhancement feature z at time t. t With the corresponding negative sample feature z - Cosine similarity between them;

[0041] Constructing the reconstruction loss: in, The reconstruction loss is represented by τ; the true augmentation parameter is represented by MLP. τ (z t ) represents the temporal enhancement feature z at time t. t Multilayer perceptron processing of the corresponding prediction enhancement parameters;

[0042] The sum of the contrast loss and the reconstruction loss is calculated to obtain the enhanced perceptual consistency loss;

[0043] During the training of the video frame extraction model, the adversarial domain discriminator in the domain adaptive feature encoder is used to minimize the feature difference between the source domain and the target domain based on the bulldozer movement distance (Wasserstein distance), and the enhanced perceptual consistency loss is used to jointly optimize the video frame extraction model.

[0044] The domain-specific enhancements can include specific transformation operations such as temporal jitter, spatial deformation, spatial blur, and brightness attenuation. These targeted enhancements can better simulate the complexities of different domains, enabling the model to learn features robust to these changes. For example, in cross-scene video surveillance, when processing video from an outdoor parking lot with rain and fog interference, rain and fog simulation enhancements such as spatial blur and brightness attenuation are applied to the video frames, allowing the model to adapt to these harsh environmental conditions and thus extract keyframes more accurately.

[0045] Based on the contrast loss, positive and negative sample pairs can be constructed in the feature space, and cosine similarity can be calculated. The contrast difficulty can be adjusted by the temperature coefficient.

[0046] Based on the reconstruction loss, augmentation parameters can be predicted using an auxiliary head (such as a multilayer perceptron (MLP)) and the L2 distance between the predicted and true values ​​can be calculated.

[0047] By combining domain-specific augmentation with self-supervised proxy tasks, and jointly optimizing the feature encoder using contrastive loss and adversarial loss, collaborative learning of domain invariance and spatiotemporal consistency is achieved. This helps the model maintain accurate perception of key information when facing data from different domains, avoids errors in keyframe extraction due to domain differences, and improves the adaptability of keyframe extraction under changing conditions.

[0048] The adversarial domain discriminator classifies temporal enhancement features into source and target domains, aligns feature distributions through a gradient inversion layer, minimizes the feature differences between the source and target domains, enables the model to generate domain-invariant representations, improves cross-domain adaptability, and reduces the impact of domain shift.

[0049] In the above embodiments, domain-specific enhancements (such as temporal jitter, spatial deformation, etc.) are applied to the spatiotemporal cube to generate enhanced features, and the model learns enhancement invariance by calculating contrastive loss and reconstruction loss. This improves the model's robustness to input perturbations, ensures the continuity of keyframes in the temporal dimension, and enhances the discriminative power of features and the ability to understand images.

[0050] S11, the target video is preprocessed using the preprocessing module in the video frame extraction model to obtain the target spatiotemporal cube.

[0051] In this embodiment, in order to ensure data consistency, the data needs to be standardized first in order to facilitate the execution of subsequent tasks.

[0052] Specifically, the step of preprocessing the target video using the preprocessing module in the video frame extraction model to obtain the target spatiotemporal cube includes:

[0053] Using the preprocessing module, the target video is normalized and segmented into blocks at a preset size using a sliding window mechanism to obtain the target spatiotemporal cube;

[0054] The preset size is determined based on the temporal context radius, and each sub-cube in the target spatiotemporal cube includes the content of frames within the temporal context radius before and after the current frame.

[0055] For example: when the original video sequence of the target video is V={f1,...,f T When}, we can first set V={f1,...,f T Normalization and block division are performed to generate a fixed-size spacetime cube C. t =[f t-k ,...,f t+k ], where k represents the temporal context radius, T represents the total number of frames in the target video, and t represents the time.

[0056] The temporal context radius can be a hyperparameter determined during training based on task requirements and video characteristics. The temporal context radius determines the number of preceding and following frames contained in each spatiotemporal cube, enabling the model to capture the temporal information of the k frames surrounding the current frame, which helps in understanding the dynamic changes and motion relationships between frames.

[0057] Furthermore, a larger k value means the input spatiotemporal cube contains more frames, resulting in greater computational complexity, but potentially yielding more comprehensive temporal information; a smaller k value results in higher computational efficiency, but may lose some long-term temporal dependencies. In other words, a larger k value can capture longer temporal dependencies, making it suitable for scenarios that require attention to long-term motion trajectories; a smaller k value is more suitable for handling short-term, rapid motion changes.

[0058] The above preprocessing not only provides a standardized input format for subsequent processing, but also introduces temporal context information, making it easier to capture dynamic relationships between frames.

[0059] S12, the domain adaptive feature encoder in the video frame extraction model is used to extract features from the target video and the target spatiotemporal cube to obtain temporal enhancement features.

[0060] In this embodiment, the domain adaptive feature encoder may include a spatial coding subnetwork, a temporal coding subnetwork, and an adversarial domain discriminator.

[0061] In this embodiment, the step of using the domain adaptive feature encoder in the video frame extraction model to extract features from the target video and the target spatiotemporal cube to obtain temporal enhancement features includes:

[0062] In the spatial coding sub-network of the domain adaptive feature encoder, the target video is spatially encoded using the following formula to obtain spatial features: l = 1,...,4; where, represents the spatial features at time t obtained after processing by the l-th level window attention block; Window-MSA represents the multi-head self-attention mechanism of the window; LN represents the layer normalization operation. Let l represent the spatial features at time t obtained after processing by the (l-1)th level window attention block; l represents the level of the window attention block; wherein, the spatial coding sub-network uses a 4-level window attention block of SwinTransformer (Shifted Window Transformer, a visual model based on Transformer) to generate multi-scale spatial features.

[0063] In the temporal coding sub-network of the domain adaptive feature encoder, the target spatiotemporal cube is temporally encoded using the following formula to obtain the temporal enhanced features: z t =ReLU(W d *{h t-k ,...,h t+k}+b d ); where ReLU represents the linear rectified function; W d Represents the learnable weight matrix; {h t-k ,...,h t+k} represents the spatial feature sequence of k frames before and after time t, where k represents the temporal context radius; b d d represents the learnable bias vector; d represents the layer depth.

[0064] The spatial coding subnetwork may include: dilation rate 2^0 convolution, i.e., basic temporal convolutional layer; dilation rate 2^1 convolution, i.e. expanding the temporal receptive field; and dilation rate 2^2 convolution, i.e. capturing long-range dependencies.

[0065] Specifically, the spatial coding sub-network can adopt the hierarchical window attention mechanism of the Swing Transformer, which processes a single frame through four levels of window attention blocks. After each level of processing, layer normalization and residual connections are performed, and finally, frame-level spatial features are obtained through global average pooling.

[0066] Specifically, the temporal coding sub-network can use a TCN (Temporal Convolutional Network) with an increasing dilation rate to perform convolution processing on spatial feature sequences. The dilation rate can be 2d-1, and long temporal dependencies can be captured through causal convolution.

[0067] In the above embodiments, the spatial coding subnetwork can extract spatial features with local-global modeling capabilities and has a certain robustness to changes in illumination, viewpoint, etc.; the temporal coding subnetwork can capture cross-frame dynamic features, realize multi-scale temporal modeling, and maintain inter-frame temporal continuity.

[0068] S13, the temporal enhancement features are filtered based on a dynamic threshold to obtain the target frame within the target video.

[0069] In this embodiment, before filtering the temporal enhancement features based on a dynamic threshold, the method further includes:

[0070] The domain difference score is calculated using the following formula: Where, δ d The domain difference score represents the domain difference score; T represents the total number of frames in the target video; D represents the adversarial domain discriminator. This represents the source domain temporal enhancement features at time t; This represents the target domain temporal enhancement feature at time t;

[0071] The dynamic threshold is calculated based on the domain difference score using the following formula: λ d =λ0(1+αtanh(βδ) d )); where λ d λ represents the dynamic threshold; λ0 represents the base threshold; tanh represents the hyperbolic tangent activation function; α represents the adjustment amplitude control parameter, α = 0.5; β represents the sensitivity control parameter, β = 2.

[0072] Through the above embodiments, the granularity of the filtering can be automatically optimized according to the complexity of the scene, avoiding manual parameter tuning and improving the accuracy and adaptability of keyframe extraction.

[0073] In this embodiment, the step of filtering the temporal enhancement features based on a dynamic threshold to obtain the target frame within the target video includes:

[0074] Obtain the magnitude of the temporal enhancement feature;

[0075] Frames whose size is greater than the dynamic threshold are obtained from the target video and are used as the target frames.

[0076] The core of keyframe extraction is selecting the most representative frames from the video. Temporal enhancement features, as feature vectors that fuse spatiotemporal information, reflect the semantic importance and temporal uniqueness of the frames in terms of their numerical attributes. For example, in video surveillance scenarios, frames where pedestrians enter the frame may have significantly higher feature values ​​for their temporal enhancement features than background frames because they contain new semantic objects. Temporal action transitions (such as a pedestrian changing from walking to standing still) can cause dynamic changes in the features of temporal enhancement features, causing them to exceed a threshold.

[0077] Therefore, unlike traditional methods that rely on manually designed features (such as color histograms) and fixed thresholds, this approach cannot adapt to domain shifts. For example, in outdoor rain and fog scenes, color features become ineffective due to changes in lighting, and fixed thresholds can lead to a large number of missed keyframes. This embodiment combines dynamic thresholds with adaptive features, maintaining screening accuracy even when feature values ​​fluctuate overall due to environmental changes through threshold adjustments.

[0078] This embodiment uses a hybrid visual Transformer (such as VisionTransformer, ViT) and a temporal convolutional network to extract spatial features and inter-frame dynamic features, respectively. The enhanced perceptual consistency loss utilizes domain-specific enhancements (such as temporal jitter and spatial deformation) to constrain feature consistency. The domain-adaptive feature encoder minimizes the feature difference between the source and target domains through adversarial learning, while the enhanced perceptual consistency loss jointly optimizes parameter prediction and feature similarity through contrastive loss and reconstruction loss. The synergistic effect of the domain-adaptive feature encoder and the enhanced perceptual consistency loss achieves dual domain adaptability in feature representation and decision-making processes. Simultaneously, this embodiment dynamically adjusts the keyframe selection threshold to adapt to domain offsets and employs a Swin Transformer and dilated convolutions for efficient computation, eliminating the need for target domain labeled data. This achieves robust cross-domain keyframe extraction while maintaining spatiotemporal coherence.

[0079] In this embodiment, after obtaining the target frame within the target video, the method further includes:

[0080] Get tasks to be processed;

[0081] Retrieve the target model corresponding to the task to be processed;

[0082] The target frame is input into the target model to obtain the processing result of the task to be processed.

[0083] The tasks to be processed may include video processing tasks from different fields.

[0084] For example, in the financial sector, the source domain could be bank surveillance video (1920×1080 resolution, 30fps), and the target domain could be outdoor parking lot surveillance video (including rain and fog interference, 1280×720 resolution). A domain-adaptive feature encoder can generate temporally enhanced features for the rain and fog frames of outdoor video that align with the feature distribution of indoor video. Based on enhanced perceptual consistency loss, feature consistency is constrained through rain and fog simulation enhancement (spatial blurring + brightness attenuation), and the threshold is automatically lowered to adapt to the high dynamic range of outdoor scenes. This ensures that the extracted keyframes fully cover the entire process of pedestrians entering and leaving (F1-score improvement of 42%). The extracted keyframes are then input into a behavior recognition model to identify the presence of abnormal behavior.

[0085] For example, in the healthcare field, the source domain could be hospital operating room surveillance video (1280×720 resolution, 25fps), and the target domain could be ambulance transport surveillance video (including vibration interference, 1080×720 resolution). Using a domain-adaptive feature encoder, temporal enhancement features aligned with the feature distribution of the operating room video can be generated from the vibration frames of the ambulance video. Based on enhanced perceptual consistency loss, feature consistency is constrained through vibration simulation enhancement (spatial distortion + dynamic blur), and the threshold is automatically lowered to adapt to the high dynamic range of the ambulance scene. This ensures that the extracted keyframes fully cover the entire process of emergency medical procedures (F1-score improvement of 38%). The extracted keyframes are then input into a behavior recognition model to identify whether standardized emergency medical procedures are observed.

[0086] As can be seen from the above technical solutions, this invention can obtain a video frame extraction model through self-supervised end-to-end training based on a cross-domain enhanced consistency mechanism. Combining cross-domain enhanced consistency regularization and self-supervised learning, it improves the adaptability and robustness of keyframe selection under varying conditions. It does not require target domain labeled data and achieves cross-domain robust keyframe extraction while maintaining spatiotemporal coherence. The preprocessing module in the video frame extraction model obtains a target spatiotemporal cube, introducing temporal context information to facilitate the capture of dynamic relationships between frames. The domain adaptive feature encoder obtains temporal enhancement features, realizing multi-scale temporal modeling and maintaining temporal coherence between frames. Based on dynamic threshold filtering of target frames, it can automatically optimize the filtering granularity according to scene complexity, thereby improving the accuracy and adaptability of keyframe extraction.

[0087] like Figure 2 The diagram shown is a functional block diagram of a preferred embodiment of the video frame extraction device of the present invention. The video frame extraction device 11 includes a retrieval unit 110, a preprocessing unit 111, a feature extraction unit 112, and a filtering unit 113. The module / unit referred to in this invention is a series of computer program segments that can be executed by a processor and perform a fixed function, and which are stored in memory. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0088] The retrieval unit 110 is used to retrieve the video frame extraction model obtained by self-supervised end-to-end training based on the cross-domain enhanced consistency mechanism in response to the video frame extraction instruction for the target video in the target domain.

[0089] In this embodiment, the video frame extraction command can be triggered according to different task requirements in different fields.

[0090] For example, when it is necessary to process surveillance videos of ATMs in the financial sector or operating rooms in the medical and health sector, the video frame extraction instruction can be triggered to extract key frames from the corresponding surveillance videos, reduce data redundancy and retain core content, and provide a data foundation for subsequent tasks such as behavior recognition, content summary generation, and video retrieval.

[0091] In this embodiment, before retrieving the video frame extraction model obtained through self-supervised end-to-end training based on the cross-domain enhanced consistency mechanism, a contrastive loss is constructed: in, This represents the contrast loss; The time-series enhancement feature z at time t is represented. t Enhanced features of the positive sample at time t Cosine similarity between them This is obtained by applying a domain-specific enhancement to the spacetime cube; θ represents the temperature coefficient; z - Represents negative sample features; s(z) t ,z - ) represents the temporal enhancement feature z at time t. t With the corresponding negative sample feature z - Cosine similarity between them;

[0092] Constructing the reconstruction loss: in, The reconstruction loss is represented by τ; the true augmentation parameter is represented by MLP. τ (z t ) represents the temporal enhancement feature z at time t. t Multilayer perceptron processing of the corresponding prediction enhancement parameters;

[0093] The sum of the contrast loss and the reconstruction loss is calculated to obtain the enhanced perceptual consistency loss;

[0094] During the training of the video frame extraction model, the adversarial domain discriminator in the domain adaptive feature encoder is used to minimize the feature difference between the source domain and the target domain based on the bulldozer movement distance (Wasserstein distance), and the enhanced perceptual consistency loss is used to jointly optimize the video frame extraction model.

[0095] The domain-specific enhancements can include specific transformation operations such as temporal jitter, spatial deformation, spatial blur, and brightness attenuation. These targeted enhancements can better simulate the complexities of different domains, enabling the model to learn features robust to these changes. For example, in cross-scene video surveillance, when processing video from an outdoor parking lot with rain and fog interference, rain and fog simulation enhancements such as spatial blur and brightness attenuation are applied to the video frames, allowing the model to adapt to these harsh environmental conditions and thus extract keyframes more accurately.

[0096] Based on the contrast loss, positive and negative sample pairs can be constructed in the feature space, and cosine similarity can be calculated. The contrast difficulty can be adjusted by the temperature coefficient.

[0097] Based on the reconstruction loss, augmentation parameters can be predicted using an auxiliary head (such as a multilayer perceptron (MLP)) and the L2 distance between the predicted and true values ​​can be calculated.

[0098] By combining domain-specific augmentation with self-supervised proxy tasks, and jointly optimizing the feature encoder using contrastive loss and adversarial loss, collaborative learning of domain invariance and spatiotemporal consistency is achieved. This helps the model maintain accurate perception of key information when facing data from different domains, avoids errors in keyframe extraction due to domain differences, and improves the adaptability of keyframe extraction under changing conditions.

[0099] The adversarial domain discriminator classifies temporal enhancement features into source and target domains, aligns feature distributions through a gradient inversion layer, minimizes the feature differences between the source and target domains, enables the model to generate domain-invariant representations, improves cross-domain adaptability, and reduces the impact of domain shift.

[0100] In the above embodiments, domain-specific enhancements (such as temporal jitter, spatial deformation, etc.) are applied to the spatiotemporal cube to generate enhanced features, and the model learns enhancement invariance by calculating contrastive loss and reconstruction loss. This improves the model's robustness to input perturbations, ensures the continuity of keyframes in the temporal dimension, and enhances the discriminative power of features and the ability to understand images.

[0101] The preprocessing unit 111 is used to preprocess the target video using the preprocessing module in the video frame extraction model to obtain the target spatiotemporal cube.

[0102] In this embodiment, in order to ensure data consistency, the data needs to be standardized first in order to facilitate the execution of subsequent tasks.

[0103] Specifically, the preprocessing unit 111 uses the preprocessing module in the video frame extraction model to preprocess the target video to obtain the target spatiotemporal cube, which includes:

[0104] Using the preprocessing module, the target video is normalized and segmented into blocks at a preset size using a sliding window mechanism to obtain the target spatiotemporal cube;

[0105] The preset size is determined based on the temporal context radius, and each sub-cube in the target spatiotemporal cube includes the content of frames within the temporal context radius before and after the current frame.

[0106] For example: when the original video sequence of the target video is V={f1,...,f T When}, we can first set V={f1,...,f T Normalization and block division are performed to generate a fixed-size spacetime cube C. t =[f t-k ,...,f t+k ], where k represents the temporal context radius, T represents the total number of frames in the target video, and t represents the time.

[0107] The temporal context radius can be a hyperparameter determined during training based on task requirements and video characteristics. The temporal context radius determines the number of preceding and following frames contained in each spatiotemporal cube, enabling the model to capture the temporal information of the k frames surrounding the current frame, which helps in understanding the dynamic changes and motion relationships between frames.

[0108] Furthermore, a larger k value means the input spatiotemporal cube contains more frames, resulting in greater computational complexity, but potentially yielding more comprehensive temporal information; a smaller k value results in higher computational efficiency, but may lose some long-term temporal dependencies. In other words, a larger k value can capture longer temporal dependencies, making it suitable for scenarios that require attention to long-term motion trajectories; a smaller k value is more suitable for handling short-term, rapid motion changes.

[0109] The above preprocessing not only provides a standardized input format for subsequent processing, but also introduces temporal context information, making it easier to capture dynamic relationships between frames.

[0110] The feature extraction unit 112 is used to extract features from the target video and the target spatiotemporal cube using the domain adaptive feature encoder in the video frame extraction model to obtain temporal enhancement features.

[0111] In this embodiment, the domain adaptive feature encoder may include a spatial coding subnetwork, a temporal coding subnetwork, and an adversarial domain discriminator.

[0112] In this embodiment, the feature extraction unit 112 uses the domain adaptive feature encoder in the video frame extraction model to extract features from the target video and the target spatiotemporal cube, obtaining temporal enhancement features including:

[0113] In the spatial coding sub-network of the domain adaptive feature encoder, the target video is spatially encoded using the following formula to obtain spatial features: l = 1,...,4; where, represents the spatial features at time t obtained after processing by the l-th level window attention block; Window-MSA represents the multi-head self-attention mechanism of the window; LN represents the layer normalization operation. Let l represent the spatial features at time t obtained after processing by the (l-1)th level window attention block; l represents the level of the window attention block; wherein, the spatial coding sub-network uses a 4-level window attention block of Swin Transformer (Shifted Window Transformer, a visual model based on Transformer) to generate multi-scale spatial features.

[0114] In the temporal coding sub-network of the domain adaptive feature encoder, the target spatiotemporal cube is temporally encoded using the following formula to obtain the temporal enhanced features: z t =ReLU(W d *{h t-k ,...,h t+k}+b d ); where ReLU represents the linear rectified function; W d Represents the learnable weight matrix; {h t-k ,...,h t+k} represents the spatial feature sequence of k frames before and after time t, where k represents the temporal context radius; b d d represents the learnable bias vector; d represents the layer depth.

[0115] The spatial coding subnetwork may include: dilation rate 2^0 convolution, i.e., basic temporal convolutional layer; dilation rate 2^1 convolution, i.e. expanding the temporal receptive field; and dilation rate 2^2 convolution, i.e. capturing long-range dependencies.

[0116] Specifically, the spatial coding sub-network can adopt the hierarchical window attention mechanism of the Swing Transformer, which processes a single frame through four levels of window attention blocks. After each level of processing, layer normalization and residual connections are performed, and finally, frame-level spatial features are obtained through global average pooling.

[0117] Specifically, the temporal coding sub-network can use a TCN (Temporal Convolutional Network) with an increasing dilation rate to perform convolution processing on spatial feature sequences. The dilation rate can be 2d-1, and long temporal dependencies can be captured through causal convolution.

[0118] In the above embodiments, the spatial coding subnetwork can extract spatial features with local-global modeling capabilities and has a certain robustness to changes in illumination, viewpoint, etc.; the temporal coding subnetwork can capture cross-frame dynamic features, realize multi-scale temporal modeling, and maintain inter-frame temporal continuity.

[0119] The filtering unit 113 is used to filter the temporal enhancement features based on a dynamic threshold to obtain the target frame within the target video.

[0120] In this embodiment, before filtering the temporal enhancement features based on a dynamic threshold, the domain difference score is calculated using the following formula: Where, δ d The domain difference score represents the domain difference score; T represents the total number of frames in the target video; D represents the adversarial domain discriminator. This represents the source domain temporal enhancement features at time t; This represents the target domain temporal enhancement feature at time t;

[0121] The dynamic threshold is calculated based on the domain difference score using the following formula: λ d =λ0(1+αtanh(βδ) d )); where λ d λ represents the dynamic threshold; λ0 represents the base threshold; tanh represents the hyperbolic tangent activation function; α represents the adjustment amplitude control parameter, α = 0.5; β represents the sensitivity control parameter, β = 2.

[0122] Through the above embodiments, the granularity of the filtering can be automatically optimized according to the complexity of the scene, avoiding manual parameter tuning and improving the accuracy and adaptability of keyframe extraction.

[0123] In this embodiment, the filtering unit 113 filters the temporal enhancement features based on a dynamic threshold to obtain target frames within the target video, including:

[0124] Obtain the magnitude of the temporal enhancement feature;

[0125] Frames whose size is greater than the dynamic threshold are obtained from the target video and are used as the target frames.

[0126] The core of keyframe extraction is selecting the most representative frames from the video. Temporal enhancement features, as feature vectors that fuse spatiotemporal information, reflect the semantic importance and temporal uniqueness of the frames in terms of their numerical attributes. For example, in video surveillance scenarios, frames where pedestrians enter the frame may have significantly higher feature values ​​for their temporal enhancement features than background frames because they contain new semantic objects. Temporal action transitions (such as a pedestrian changing from walking to standing still) can cause dynamic changes in the features of temporal enhancement features, causing them to exceed a threshold.

[0127] Therefore, unlike traditional methods that rely on manually designed features (such as color histograms) and fixed thresholds, this approach cannot adapt to domain shifts. For example, in outdoor rain and fog scenes, color features become ineffective due to changes in lighting, and fixed thresholds can lead to a large number of missed keyframes. This embodiment combines dynamic thresholds with adaptive features, maintaining screening accuracy even when feature values ​​fluctuate overall due to environmental changes through threshold adjustments.

[0128] This embodiment uses a hybrid visual Transformer (such as VisionTransformer, ViT) and a temporal convolutional network to extract spatial features and inter-frame dynamic features, respectively. The enhanced perceptual consistency loss utilizes domain-specific enhancements (such as temporal jitter and spatial deformation) to constrain feature consistency. The domain-adaptive feature encoder minimizes the feature difference between the source and target domains through adversarial learning, while the enhanced perceptual consistency loss jointly optimizes parameter prediction and feature similarity through contrastive loss and reconstruction loss. The synergistic effect of the domain-adaptive feature encoder and the enhanced perceptual consistency loss achieves dual domain adaptability in feature representation and decision-making processes. Simultaneously, this embodiment dynamically adjusts the keyframe selection threshold to adapt to domain offsets and employs a Swin Transformer and dilated convolutions for efficient computation, eliminating the need for target domain labeled data. This achieves robust cross-domain keyframe extraction while maintaining spatiotemporal coherence.

[0129] In this embodiment, after obtaining the target frame within the target video, the task to be processed is acquired;

[0130] Retrieve the target model corresponding to the task to be processed;

[0131] The target frame is input into the target model to obtain the processing result of the task to be processed.

[0132] The tasks to be processed may include video processing tasks from different fields.

[0133] For example, in the financial sector, the source domain could be bank surveillance video (1920×1080 resolution, 30fps), and the target domain could be outdoor parking lot surveillance video (including rain and fog interference, 1280×720 resolution). A domain-adaptive feature encoder can generate temporally enhanced features for the rain and fog frames of outdoor video that align with the feature distribution of indoor video. Based on enhanced perceptual consistency loss, feature consistency is constrained through rain and fog simulation enhancement (spatial blurring + brightness attenuation), and the threshold is automatically lowered to adapt to the high dynamic range of outdoor scenes. This ensures that the extracted keyframes fully cover the entire process of pedestrians entering and leaving (F1-score improvement of 42%). The extracted keyframes are then input into a behavior recognition model to identify the presence of abnormal behavior.

[0134] For example, in the healthcare field, the source domain could be hospital operating room surveillance video (1280×720 resolution, 25fps), and the target domain could be ambulance transport surveillance video (including vibration interference, 1080×720 resolution). Using a domain-adaptive feature encoder, temporal enhancement features aligned with the feature distribution of the operating room video can be generated from the vibration frames of the ambulance video. Based on enhanced perceptual consistency loss, feature consistency is constrained through vibration simulation enhancement (spatial distortion + dynamic blur), and the threshold is automatically lowered to adapt to the high dynamic range of the ambulance scene. This ensures that the extracted keyframes fully cover the entire process of emergency medical procedures (F1-score improvement of 38%). The extracted keyframes are then input into a behavior recognition model to identify whether standardized emergency medical procedures are observed.

[0135] As can be seen from the above technical solutions, this invention can obtain a video frame extraction model through self-supervised end-to-end training based on a cross-domain enhanced consistency mechanism. Combining cross-domain enhanced consistency regularization and self-supervised learning, it improves the adaptability and robustness of keyframe selection under varying conditions. It does not require target domain labeled data and achieves cross-domain robust keyframe extraction while maintaining spatiotemporal coherence. The preprocessing module in the video frame extraction model obtains a target spatiotemporal cube, introducing temporal context information to facilitate the capture of dynamic relationships between frames. The domain adaptive feature encoder obtains temporal enhancement features, realizing multi-scale temporal modeling and maintaining temporal coherence between frames. Based on dynamic threshold filtering of target frames, it can automatically optimize the filtering granularity according to scene complexity, thereby improving the accuracy and adaptability of keyframe extraction.

[0136] like Figure 3 The diagram shown is a schematic representation of the structure of a computer device that implements the video frame extraction method of the present invention.

[0137] The computer device 1 may include a memory 12, a processor 13, and a bus (the arrow in the figure represents the bus), and may also include a computer program stored in the memory 12 and executable on the processor 13, such as a video frame extraction program.

[0138] Those skilled in the art will understand that the schematic diagram is merely an example of computer device 1 and does not constitute a limitation on computer device 1. Computer device 1 can be either a bus topology or a star topology. Computer device 1 may also include more or fewer other hardware or software than shown in the diagram, or different component arrangements. For example, computer device 1 may also include input / output devices, network access devices, etc.

[0139] It should be noted that the computer device 1 described is merely an example. Other existing or future electronic products that are adaptable to this invention should also be included within the scope of protection of this invention and are incorporated herein by reference.

[0140] The memory 12 includes at least one type of readable storage medium, such as flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 12 can be an internal storage unit of the computer device 1, such as a portable hard drive of the computer device 1. In other embodiments, the memory 12 can be an external storage device of the computer device 1, such as a plug-in portable hard drive, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., equipped on the computer device 1. Furthermore, the memory 12 can include both internal and external storage units of the computer device 1. The memory 12 can be used not only to store application software and various types of data installed on the computer device 1, such as the code of a video frame extraction program, but also to temporarily store data that has been output or will be output.

[0141] In some embodiments, the processor 13 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits packaged with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 13 is the control unit of the computer device 1, connecting various components of the computer device 1 via various interfaces and lines. It executes programs or modules stored in the memory 12 (e.g., executing video frame extraction programs) and calls data stored in the memory 12 to perform various functions of the computer device 1 and process data.

[0142] The processor 13 executes the operating system of the computer device 1 and various installed applications. The processor 13 executes these applications to implement the steps in the various video frame extraction method embodiments described above, for example... Figure 1 The steps are shown.

[0143] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to complete the present invention. The one or more modules / units may be a series of computer-readable instruction segments capable of performing a specific function, which describe the execution process of the computer program in the computer device 1. For example, the computer program may be divided into a retrieval unit 110, a preprocessing unit 111, a feature extraction unit 112, and a filtering unit 113.

[0144] The integrated unit implemented as a software functional module described above can be stored in a computer-readable storage medium. This software functional module, stored in a storage medium, includes several instructions to cause a computer device (which may be a personal computer, computer equipment, or network device, etc.) or processor to execute portions of the video frame extraction method described in the various embodiments of the present invention.

[0145] If the modules / units integrated in the computer device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware devices. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above.

[0146] The computer program includes computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory, etc.

[0147] Furthermore, the computer-readable storage medium may primarily include a stored program area and a stored data area, wherein the stored program area may store the operating system, an application program required for at least one function, etc.; and the stored data area may store data created based on the use of blockchain nodes, etc.

[0148] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0149] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, in... Figure 3 The bus is represented by only one straight line, but this does not mean that there is only one bus or one type of bus. The bus is configured to enable communication between the memory 12 and at least one processor 13, etc.

[0150] Although not shown, the computer device 1 may also include a power supply (such as a battery) to power various components. Preferably, the power supply can be logically connected to the at least one processor 13 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The computer device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0151] Furthermore, the computer device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish a communication connection between the computer device 1 and other computer devices.

[0152] Optionally, the computer device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), and optionally, a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the computer device 1 and to display a visual user interface.

[0153] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0154] It will be understood by those skilled in the art that Figure 3 The structure shown does not constitute a limitation on the computer device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0155] Combination Figure 1 The memory 12 in the computer device 1 stores multiple instructions to implement a video frame extraction method, and the processor 13 can execute the multiple instructions to achieve the following:

[0156] In response to the instruction to extract video frames of the target video within the target domain, the video frame extraction model obtained by self-supervised end-to-end training based on the cross-domain enhanced consistency mechanism is invoked.

[0157] The target video is preprocessed using the preprocessing module in the video frame extraction model to obtain the target spatiotemporal cube;

[0158] The target video and the target spatiotemporal cube are used to extract features by the domain adaptive feature encoder in the video frame extraction model to obtain temporal enhancement features;

[0159] The temporal enhancement features are filtered based on a dynamic threshold to obtain the target frames within the target video.

[0160] Specifically, the processor 13's implementation method for the above instructions can be found in [reference needed]. Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0161] It should be noted that all data involved in this case was legally obtained. Software tools or components not belonging to this company that appear in the embodiments of this application are merely illustrative examples and do not represent actual use.

[0162] In the several embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0163] This invention can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0164] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0165] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0166] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0167] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0168] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices described in this invention can also be implemented by a single unit or device through software or hardware. Terms such as "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0169] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A video frame extraction method, characterized in that, The video frame extraction method includes: In response to the instruction to extract video frames of the target video within the target domain, the video frame extraction model obtained by self-supervised end-to-end training based on the cross-domain enhanced consistency mechanism is invoked. The target video is preprocessed using the preprocessing module in the video frame extraction model to obtain a target spatiotemporal cube. This includes: using the preprocessing module, the target video is normalized and segmented into blocks using a sliding window mechanism at a preset size to obtain the target spatiotemporal cube; wherein, the preset size is determined based on the temporal context radius, and each sub-cube in the target spatiotemporal cube includes the content of frames within the temporal context radius before and after the current frame. The target video and the target spatiotemporal cube are feature-extracted using the domain-adaptive feature encoder in the video frame extraction model to obtain temporally enhanced features. This includes: in the spatial coding sub-network of the domain-adaptive feature encoder, the target video is spatially encoded using the following formula to obtain spatial features: ;in, Indicates the first Spatial features at time t obtained after processing with a multi-level window attention block; This indicates a multi-head self-attention mechanism for windows; Presentation layer normalization operation; Indicates the first Spatial features at time t obtained after processing with a multi-level window attention block; The level of the window attention block is indicated; wherein, the spatial coding sub-network uses a 4-level window attention block of the Swing Transformer to generate multi-scale spatial features; in the temporal coding sub-network of the domain adaptive feature encoder, the target spatiotemporal cube is temporally encoded using the following formula to obtain the temporal enhanced features: ;in, Represents a linear rectified function; Represents the learnable weight matrix; This represents the spatial feature sequence of k frames before and after time t, where k represents the temporal context radius; Represents a learnable bias vector; Indicates layer depth; The temporal enhancement features are filtered based on a dynamic threshold to obtain the target frames within the target video.

2. The video frame extraction method as described in claim 1, characterized in that, Before retrieving the video frame extraction model obtained through self-supervised end-to-end training based on the cross-domain enhanced consistency mechanism, the method further includes: Constructing the contrastive loss: ;in, This represents the contrast loss; Indicates the temporal enhancement features at time t Enhanced features of the positive sample at time t Cosine similarity between them This is obtained by applying a domain-specific enhancement to the spacetime cube; Indicates the temperature coefficient; Indicates negative sample features; Indicates the temporal enhancement features at time t Features corresponding to negative samples Cosine similarity between them; Constructing the reconstruction loss: ;in, This represents the reconstruction loss; Indicates the actual enhancement parameters; Indicates the temporal enhancement features at time t Multilayer perceptron processing of the corresponding prediction enhancement parameters; The sum of the contrast loss and the reconstruction loss is calculated to obtain the enhanced perceptual consistency loss; During the training of the video frame extraction model, the adversarial domain discriminator in the domain adaptive feature encoder is used to minimize the feature difference between the source domain and the target domain based on the bulldozer's movement distance, and the enhanced perceptual consistency loss is used to jointly optimize the video frame extraction model.

3. The video frame extraction method as described in claim 1, characterized in that, The step of filtering the temporal enhancement features based on a dynamic threshold to obtain the target frames within the target video includes: Obtain the magnitude of the temporal enhancement feature; Frames whose size is greater than the dynamic threshold are obtained from the target video and are used as the target frames.

4. The video frame extraction method as described in claim 2, characterized in that, Before filtering the temporal enhancement features based on a dynamic threshold, the method further includes: The domain difference score is calculated using the following formula: ;in, The domain difference score represents the total number of frames in the target video; T represents the total number of frames in the target video. This refers to the adversarial domain discriminator; This represents the source domain temporal enhancement features at time t; This represents the target domain temporal enhancement features at time t; The dynamic threshold is calculated based on the domain difference score using the following formula: ;in, This refers to the dynamic threshold; Indicates the basic threshold; Represents the hyperbolic tangent activation function; This indicates the adjustment range control parameter. ; This indicates the sensitivity control parameters. .

5. The video frame extraction method as described in claim 1, characterized in that, After obtaining the target frame within the target video, the method further includes: Get tasks to be processed; Retrieve the target model corresponding to the task to be processed; The target frame is input into the target model to obtain the processing result of the task to be processed.

6. A video frame extraction device, characterized in that, The video frame extraction device includes: The retrieval unit is used to retrieve the video frame extraction model obtained by self-supervised end-to-end training based on the cross-domain enhanced consistency mechanism in response to the video frame extraction instruction of the target video in the target domain. A preprocessing unit is used to preprocess the target video using the preprocessing module in the video frame extraction model to obtain a target spatiotemporal cube. The preprocessing unit includes: using the preprocessing module to normalize and segment the target video into blocks at a preset size via a sliding window mechanism to obtain the target spatiotemporal cube; wherein the preset size is determined based on the temporal context radius, and each sub-cube in the target spatiotemporal cube includes the content of frames within the temporal context radius before and after the current frame. The feature extraction unit is used to extract features from the target video and the target spatiotemporal cube using the domain adaptive feature encoder in the video frame extraction model to obtain temporal enhancement features. This includes: in the spatial coding sub-network of the domain adaptive feature encoder, spatially coding the target video using the following formula to obtain spatial features: ;in, Indicates the first Spatial features at time t obtained after processing with a multi-level window attention block; This indicates a multi-head self-attention mechanism for windows; Presentation layer normalization operation; Indicates the first Spatial features at time t obtained after processing with a multi-level window attention block; The level of the window attention block is indicated; wherein, the spatial coding sub-network uses a 4-level window attention block of the Swing Transformer to generate multi-scale spatial features; in the temporal coding sub-network of the domain adaptive feature encoder, the target spatiotemporal cube is temporally encoded using the following formula to obtain the temporal enhanced features: ;in, Represents a linear rectified function; Represents the learnable weight matrix; This represents the spatial feature sequence of k frames before and after time t, where k represents the temporal context radius; Represents a learnable bias vector; Indicates layer depth; The filtering unit is used to filter the temporal enhancement features based on a dynamic threshold to obtain the target frames within the target video.

7. A computer device, characterized in that, The computer device includes: Memory, storing at least one instruction; and The processor executes instructions stored in the memory to implement the video frame extraction method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, which is executed by a processor in a computer device to implement the video frame extraction method as described in any one of claims 1 to 5.

Images