Target Detection Method and System Based on Dynamic Video Frame Sampling

Abstract

This invention discloses a target detection method and system based on dynamic video frame sampling. The method includes: acquiring motion vectors of a compressed video to be processed; clustering the motion vectors of the video frames to obtain the number of clusters; denoising the number of clusters of the motion vectors of the video frames to obtain the number of clusters of the denoised motion vectors of the video frames; using a sliding window, sliding on the number of clusters of the denoised motion vectors of the video frames, and calculating the standard deviation of the number of clusters of motion vectors of the video frames within the window; determining the number of video frames sampled within the window based on the standard deviation; uniformly sampling the video frames within the window based on the number of video frames sampled within the window to obtain sampled video frames; performing video decoding and preprocessing on the sampled video frames; inputting the preprocessed video frames into a target detection network to achieve the recognition of target objects in the video; and finally, obtaining the video frame sequence number of the target object.

Description

Technical Field

[0001] This invention relates to the field of video data analysis technology, and in particular to a target detection method and system based on dynamic video frame sampling. Background Technology

[0002] Video data analysis uses object detection algorithms to retrieve video frames containing specific objects from video data. Video data analysis technology enhances human understanding and exploration of the physical world and is widely used in fields such as surveillance and security, intelligent transportation, and business planning. The development of deep neural network (DNN) technologies, such as object detection and image recognition, has made automated analysis of stored video data possible. Although in practical applications, DNNs have surpassed traditional computer vision algorithms and even human accuracy in terms of accuracy, frame-by-frame inference analysis of video data would result in significant time and economic costs.

[0003] Since video data is usually stored in a compressed format, the video data must be decoded before using DNN to perform object detection on the video data. Then, it is converted into the final RGB format for inference through preprocessing steps such as color channel conversion and size scaling. Experiments have confirmed that data decoding and preprocessing have become the performance bottleneck in DNN-based end-to-end object detection inference [1]. [1] Daniel Kang, Ankit Mathur, Teja Veeramacheneni, Peter Bailis, and Matei Zaharia. 2020. Jointly Optimizing Preprocessing and Inference for DNN-Based Visual Analytics. Proc. VLDB Endow. (2020).

[0004] like Figure 1As shown, DNN-based video data analysis mainly consists of four processes. The first step is video data compression and transfer. Video data is typically compressed and stored on disks or hard drives in H.264 / H.265 format. The analysis program first needs to transfer the compressed video data to the host memory via the file system. The second step is compressed video decoding. Because DNN neural network inference is based on RGB image format, compressed video needs to be decoded using a decoder corresponding to the compression format. The output of the decoded video is generally YUV format video frame data. The third step is video frame preprocessing. Preprocessing typically involves color channel conversion, pixel value normalization, video frame size scaling, and tensor quantization. These steps aim to convert the video frames from the original YUV format into the input format required by the DNN network. The fourth step is inference. DNN inference aims to use the object detection network to identify specific objects in the video frames, ultimately returning the video frame sequence number containing the user-specified object.

[0005] like Figure 2 As shown, to accelerate the DNN-based video data analysis process, existing analysis systems mainly use model cascading or model lightweighting to filter out video frames that do not contain the desired objects, thereby reducing the number of video frames that the backend DNN needs to infer. Since the content of objects in video frames is continuous in time, uniform video frame sampling algorithms are also widely used to accelerate DNN video data analysis. Uniform sampling algorithms ensure consistency and regularity in data selection within time intervals by sampling every fixed number of consecutive frames in the video sequence. This method utilizes redundant information between consecutive video frames, reducing computational burden while maintaining reasonable capture of changes in video content. Experimental statistics show that decoding and preprocessing are becoming performance bottlenecks in the end-to-end processing flow of DNN-based video data analysis. While model cascading or model lightweighting can reduce the inference overhead of the backend DNN, it still requires complete decoding of the compressed video, thus failing to alleviate the performance overhead caused by video decoding and preprocessing. Furthermore, the continuity of video data often exhibits dynamic changes, meaning that there may be periods of continuous video frame stillness or changes in the video frame. Considering that the dynamic and static times of video footage cannot be predicted in advance, although the traditional uniform sampling algorithm can reduce the number of video frames processed by the DNN, its fixed sampling step size cannot fully utilize the dynamic characteristics of the video content. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a target detection method and system based on dynamic video frame sampling. Based on the target detection algorithm processing flow of Deep Neural Network (DNN), a dynamic video frame sampling method based on motion vectors is proposed for target detection applications in compressed videos, which effectively reduces the video decoding cost and DNN inference cost of end-to-end video data analysis based on deep neural networks.

[0007] On the one hand, a target detection method based on dynamic video frame sampling is provided, including:

[0008] Acquire the compressed video to be processed; decode the compressed video and extract the motion vectors of the video frames from the binary bitstream of the compressed video; after obtaining the motion vectors of the video frames, stop decoding the compressed video;

[0009] Clustering is performed on the motion vectors of video frames to obtain the number of clusters of the motion vectors of video frames;

[0010] The number of clusters of motion vectors in video frames is denoised to obtain the number of clusters of motion vectors in the denoised video frames. A sliding window is used to slide on the number of clusters of motion vectors in the denoised video frames and calculate the standard deviation of the number of clusters of motion vectors in the video frames within the window. Based on the standard deviation, the number of video frames sampled within the window is determined.

[0011] Based on the number of video frames sampled within the window, the video frames within the window are uniformly sampled to obtain sampled video frames. The sampled video frames are then decoded to obtain video frames of a set format. The video frames of the set format are then preprocessed and input into the target detection network to achieve the recognition of target objects in the video. Finally, the video frame sequence number of the target object is obtained.

[0012] On the other hand, a target detection system based on dynamic video frame sampling is provided, including:

[0013] The acquisition module is configured to: acquire the compressed video to be processed; decode the compressed video and extract the motion vectors of the video frames from the binary bitstream of the compressed video; and stop decoding the compressed video after obtaining the motion vectors of the video frames.

[0014] The clustering module is configured to cluster the motion vectors of video frames to obtain the number of clusters of the motion vectors of the video frames.

[0015] The denoising module is configured to: denoise the number of clusters of motion vectors in video frames to obtain the number of clusters of motion vectors in the denoised video frames; use a sliding window to slide on the number of clusters of motion vectors in the denoised video frames and calculate the standard deviation of the number of clusters of motion vectors in the video frames within the window; and determine the number of samples of video frames within the window based on the standard deviation.

[0016] The sampling module is configured to: uniformly sample the video frames within the window according to the number of samples of the video frames within the window to obtain sampled video frames; perform video decoding on the sampled video frames to obtain video frames of a set format; preprocess the video frames of the set format; input the preprocessed video frames into the target detection network to realize the recognition of target objects in the video; and finally, obtain the video frame sequence number of the target object.

[0017] Furthermore, an electronic device is also provided, including:

[0018] Memory, used for non-transitory storage of computer-readable instructions; and

[0019] Processor, for executing the computer-readable instructions,

[0020] When the computer-readable instructions are executed by the processor, they perform the method described in the first aspect above.

[0021] In another aspect, a storage medium is also provided for non-transitory storage of computer-readable instructions, wherein when the non-transitory computer-readable instructions are executed by a computer, the instructions of the method described in the first aspect are executed.

[0022] In another aspect, a computer program product is also provided, including a computer program that, when run on one or more processors, is used to implement the method described in the first aspect above.

[0023] The above technical solution has the following advantages or beneficial effects:

[0024] This invention dynamically adjusts the sampling step size of video frames based on the changes in the number of motion vector clusters stored in compressed videos. Motion vectors are metadata stored during video compression encoding. Specifically, video compression encoding is performed in 16×16 macroblocks, and motion vectors characterize the offset relationship between video frame macroblocks and reference frame macroblocks. Inter-frame coding in video encoding achieves data compression by storing only the motion vectors of certain macroblocks and residual data. Since the process of extracting motion vectors only involves the initial stage of complete video decoding, its cost is far less than traditional video data optimization methods based on RGB data. Furthermore, because motion vectors contain spatiotemporal information about moving objects within video frames, adjusting the video frame sampling step size based on changes in the number of motion vector clusters can more fully utilize the dynamic features of video semantics.

[0025] The innovation of this invention is reflected in the following aspects: Addressing the performance bottleneck in decoding and preprocessing faced by RGB-based video data analysis optimization strategies, this invention proposes a dynamic video frame sampling method based on compressed video motion vectors. This process eliminates the need for complete decoding of the video data, reducing the decoding cost of end-to-end video data analysis. Furthermore, by clustering the extracted video frame motion vectors and dynamically adjusting the frame sampling step size based on changes in the number of clusters, this invention samples fewer frames while maintaining the same accuracy compared to traditional uniform video frame sampling algorithms, significantly reducing the DNN inference computation overhead for video data analysis applications. To reduce the additional overhead caused by motion vector clustering and mitigate interference from moving background images, this invention employs a parallelized motion vector clustering strategy to accelerate the clustering process and uses Hample filtering to smooth and denoise the number of video frame motion vector clusters, reducing interference from background distortion, jitter, and other factors. Attached Figure Description

[0026] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0027] Figure 1 It is a traditional video data analysis process based on frame-by-frame inference;

[0028] Figure 2 It is a traditional video data analysis process based on uniform frame sampling;

[0029] Figure 3 This invention demonstrates the video data analysis process based on dynamic sampling of motion vectors. Detailed Implementation

[0030] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0031] In traditional DNN-based frame-by-frame video analysis or video data analysis scenarios employing model cascading or model specialization, applications first need to read the compressed video from storage devices into memory, then fully decode and preprocess the video data to obtain the RGB data format of the video frames. This process involves complete decoding of the compressed video data, inevitably incurring performance overhead. In traditional DNN-based uniform video frame sampling analysis scenarios, video frame sampling is performed at a fixed step size, failing to fully utilize the dynamic changes in content between consecutive video frames to reduce the computational overhead of DNN inference. The implementation strategy of this invention is described below.

[0032] Example 1

[0033] This embodiment provides a target detection method based on dynamic video frame sampling;

[0034] like Figure 3 As shown, the target detection method based on dynamic video frame sampling includes:

[0035] S101: Obtain the compressed video to be processed; decode the compressed video and extract the motion vectors of the video frames from the binary bitstream of the compressed video; after obtaining the motion vectors of the video frames, stop decoding the compressed video;

[0036] S102: Perform clustering processing on the motion vectors of video frames to obtain the number of clusters of motion vectors of video frames;

[0037] S103: Denoise the number of clusters of motion vectors in video frames to obtain the number of clusters of motion vectors in the denoised video frames; use a sliding window to slide on the number of clusters of motion vectors in the denoised video frames and calculate the standard deviation of the number of clusters of motion vectors in the video frames within the window; determine the number of samples of video frames within the window based on the standard deviation.

[0038] S104: Based on the number of video frames sampled within the window, uniformly sample the video frames within the window to obtain sampled video frames; decode the sampled video frames to obtain video frames of a set format; preprocess the video frames of the set format; input the preprocessed video frames into the target detection network to achieve the recognition of target objects in the video; finally, obtain the video frame sequence number of the target object.

[0039] Further, in step S101: Obtain the compressed video to be processed. The compressed video is the original video encoded using the H.264 video standard. During the compression encoding process, the original video generates corresponding motion vectors for macroblocks within each non-keyframe (B-frame or P-frame). The motion vectors are stored in the binary bitstream of the compressed video.

[0040] A B-frame is a video frame that uses image data from the preceding and following frames for predictive coding. A P-frame is a video frame that uses image data from the adjacent preceding frame for predictive coding.

[0041] A macroblock is a basic unit of processing in video image processing, typically a 16x16 pixel region. H.264 encoding divides an image into multiple macroblocks and then encodes each macroblock separately.

[0042] A motion vector is a vector used to describe the positional changes of a macroblock within a reference frame. Simply put, a motion vector represents the movement of a macroblock between the current frame and the reference frame.

[0043] The decoding process of the compressed video includes motion vector prediction, entropy decoding, and macroblock reconstruction.

[0044] Motion vector prediction refers to the process in which each macroblock in H.264 encoding has a motion vector representing its position within a reference frame. During decoding, the decoder first uses motion vector prediction to determine the position of each macroblock in the current frame within the reference frame. Using this motion vector, the decoder can locate the corresponding position in the reference frame and extract pixel data to reconstruct the macroblocks in the current frame.

[0045] Entropy decoding refers to H.264's use of entropy coding (such as CABAC or CAVLC) to compress video data and reduce its size. During decoding, the entropy decoding step converts the compressed data back into the original bitstream. Entropy decoding requires the decoder to process the bitstream in reverse order of encoding, reconstructing various parameters, including motion vectors and residual information. This step decodes the compressed data into basic information usable for image reconstruction.

[0046] Macroblock reconstruction refers to the process of combining decoded data to reconstruct an image frame. The decoder uses motion vectors to obtain prediction blocks from a reference frame, then adds these prediction blocks to the decoded residual data to reconstruct the original macroblock image. This process involves inverse transforms and addition operations on the prediction blocks and residual data to obtain the final pixel values. The reconstructed macroblocks are placed in their correct positions within the current frame until the entire image frame is completely reconstructed.

[0047] Furthermore, the decoding of the compressed video, extracting motion vectors of video frames from the binary bitstream of the compressed video, specifically includes the following extraction process:

[0048] The encoded data of the video frame is obtained by parsing the Network Abstraction Layer (NAL), the original data of the video frame is restored by bitstream entropy decoding, and the motion vector data in the macroblock encoding information is obtained by parsing the macroblock information.

[0049] It should be understood that during the compression encoding of video data, the encoder generates corresponding motion vectors for macroblocks within each non-keyframe (B-frame, P-frame), and these motion vectors are stored in the binary bitstream of the compressed video.

[0050] Taking the H.264 video coding standard, which is widely used in the industry, as an example, its decoding process mainly involves motion vector prediction, entropy decoding, and macroblock reconstruction.

[0051] Motion vector prediction, which involves parsing the motion vectors of video frame macroblocks from the compressed video binary bitstream, accounts for 7%–20% of the overall decoding time, according to experiments. This invention stops decoding after extracting the motion vectors of the video frames.

[0052] Further, in S102: the motion vectors of the video frames are clustered to obtain the number of clusters of the motion vectors of the video frames. Specifically, a density-based clustering algorithm is used to perform the clustering of the motion vectors of the video frames.

[0053] Further, in S102: the motion vectors of the video frames are clustered to obtain the number of clusters of the motion vectors of the video frames. The specific process of the density-based clustering algorithm DBSCAN (Density-Based Spatial Clustering of Application with Noise) includes two stages: core point query and core point promotion; the density is defined by setting the radius threshold eps and the minimum number of points minPts.

[0054] The core point query stage includes: first, determining whether each point is a core point by setting a radius threshold eps and a minimum number of points minPts; then, determining whether the number of points in the neighborhood of each point's set radius threshold eps is greater than or equal to minPts; if it is greater than or equal to, the current point is a core point; if it is less than, the current point is a non-core point.

[0055] The core point promotion phase includes: starting from a core point, adding points (directly density-reachable) within its neighborhood with a set radius threshold eps to the same cluster; continuing to add several newly added points to the cluster as new core points, and expanding the neighborhood of the new core points through parallel computing until the cluster can no longer be expanded.

[0056] For example, after obtaining the motion vectors of the video frames, this invention uses a density-based clustering algorithm to calculate the cluster density of the video frame motion vectors. The clustering algorithm adopts the basic idea of ​​the DBSCAN algorithm, while incorporating parallel optimization.

[0057] This invention employs a parallel execution approach during the core point promotion phase, computing multiple neighboring points of the current core point in parallel to accelerate clustering. The motion vector clustering strategy used in this invention is implemented in parallel, allowing its processing to be pipelined with subsequent keyframe decoding and DNN inference without becoming a performance bottleneck for end-to-end inference.

[0058] Further, in S103: the number of clusters of motion vectors in the video frame is denoised to obtain the number of clusters of motion vectors in the video frame after denoising. Specifically, a filtering algorithm is used to denoise the number of clusters of motion vectors in the video frame.

[0059] Further, step S103: employing a sliding window, sliding on the number of clusters of motion vectors in the denoised video frames, and calculating the standard deviation σ of the number of clusters of motion vectors in the video frames within the window, specifically includes:

[0060]

[0061] Where n is the number of video frames within the sliding window, x i Let be the number of motion vector clusters in the i-th frame within the window, and μ be the average number of motion vector clusters across all frames within the window.

[0062] Further, S103: determining the number of video frames sampled within the window based on the standard deviation, specifically includes:

[0063] sample_n = max(1, α*e) σ )

[0064] Where α is the minimum number of samples within a predefined window, σ is the standard deviation of the number of consecutive frame motion vector clusters within the window, and sample_n is the number of video frames sampled within the window.

[0065] For example, after calculating the number of motion vector clusters in consecutive video frames, the present invention uses a sliding window to calculate the standard deviation of the number of motion vector clusters in the video frames within the window, and dynamically adjusts the step size of the video frame sampling within the window based on this standard deviation.

[0066] Considering that video frames may contain interference factors such as moving backgrounds and image jitter in addition to moving objects, the motion vectors generated by these disturbances will introduce additional errors. Before calculating the standard deviation, the algorithm uses Hample filtering to smooth and denoise the number of motion vector clusters.

[0067] When the standard deviation of the number of motion vector clusters within the window is less than the set pre-set limit, it indicates that the video content remains relatively unchanged. In extreme cases, the minimum number of samples is reached when the standard deviation is 0.

[0068] Conversely, if the content of the video frame within the window changes drastically, the standard deviation σ of the calculated motion vector cluster size increases, and α*e σ The value will also increase accordingly, that is, the number of video frames sampled within the sliding window, sample_n, is positively correlated with the standard deviation σ of the number of motion vector clusters within the window.

[0069] Further, S104: Based on the sampling quantity of video frames within the window, uniformly sample the video frames within the window to obtain sampled video frames, specifically including:

[0070] First, calculate the sampling interval within the window:

[0071]

[0072] Where n is the number of video frames in the window, and sample_n is the number of samples of the video frames in the window;

[0073] Then, starting from the window's initial frame, subsequent frames are selected sequentially at fixed intervals until sample_n sampling frames are reached.

[0074] Further, in S104: the sampled video frames are video decoded to obtain video frames of a set format, such as YUV format.

[0075] YUV format is a color encoding system commonly used in digital video and image processing. It divides an image into luminance (Y) and two chrominance (U, V) components, where Y represents luminance information, and U and V represent chrominance information. With YUV encoding, images can be transmitted or stored with lower bandwidth while maintaining good visual quality, thus it is widely used in video compression and transmission.

[0076] Further, S104: preprocessing the video frames of the set format, including:

[0077] One or more of the following processes: color channel conversion, pixel value normalization, video frame size scaling, or tensor quantization.

[0078] Further, in S104: the preprocessed video frame is input into the target detection network to realize the recognition of target objects in the video, wherein the target detection network is trained on video with known target object location labels.

[0079] Once the number of video frames to be sampled is determined, the video frames within the window will be sampled uniformly. The sampled video frames will be converted into RGB data format for DNN inference through a complete video decoding and preprocessing process. The inference results of the DNN on the sampled frames will be generalized to adjacent unsampled frames. The dynamic sampling method used in this invention does not require pre-extracting motion vectors from all frames in the video file, and the sliding window working method is suitable not only for processing historical video data stored in storage devices but also for online video streams, making it widely applicable.

[0080] Parallel clustering of motion vectors: By clustering motion vectors, an approximate estimate of the number of moving objects within the video frame can be obtained without fully decoding the video frames. Furthermore, a parallel design accelerates the clustering process, reducing additional computational overhead.

[0081] Dynamic Video Frame Sampling Process Based on Motion Vector Clustering: This invention employs a sliding window approach, estimating the variation in the number of moving objects in the video frame by calculating the standard deviation of the number of motion vector clusters within the window. A specific calculation method for the number of video frame samples within the window is also proposed. Due to the sliding window implementation, the three processes of motion vector clustering, video frame sampling, and DNN inference can operate in a pipelined manner, further reducing the computational latency of end-to-end video data analysis.

[0082] Example 2

[0083] This embodiment provides a target detection system based on dynamic video frame sampling, including:

[0084] The acquisition module is configured to: acquire the compressed video to be processed; decode the compressed video and extract the motion vectors of the video frames from the binary bitstream of the compressed video; and stop decoding the compressed video after obtaining the motion vectors of the video frames.

[0085] The clustering module is configured to cluster the motion vectors of video frames to obtain the number of clusters of the motion vectors of the video frames.

[0086] The denoising module is configured to: denoise the number of clusters of motion vectors in video frames to obtain the number of clusters of motion vectors in the denoised video frames; use a sliding window to slide on the number of clusters of motion vectors in the denoised video frames and calculate the standard deviation of the number of clusters of motion vectors in the video frames within the window; and determine the number of samples of video frames within the window based on the standard deviation.

[0087] The sampling module is configured to: uniformly sample the video frames within the window according to the number of samples of the video frames within the window to obtain sampled video frames; perform video decoding on the sampled video frames to obtain video frames of a set format; preprocess the video frames of the set format; input the preprocessed video frames into the target detection network to realize the recognition of target objects in the video; and finally, obtain the video frame sequence number of the target object.

[0088] It should be noted that the acquisition module, clustering module, denoising module, and sampling module described above correspond to steps S101 to S104 in Embodiment 1. The examples and application scenarios implemented by these modules and their corresponding steps are the same, but they are not limited to the content disclosed in Embodiment 1. It should also be noted that these modules, as part of the system, can be executed in a computer system, such as a set of computer-executable instructions.

[0089] The descriptions of each embodiment in the above embodiments have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0090] The proposed system can be implemented in other ways. For example, the system embodiments described above are merely illustrative, and the division of modules described above is only a logical functional division. In actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed.

[0091] Example 3

[0092] This embodiment also provides an electronic device, including: one or more processors, one or more memories, and one or more computer programs; wherein, the processor is connected to the memory, and the one or more computer programs are stored in the memory. When the electronic device is running, the processor executes the one or more computer programs stored in the memory to cause the electronic device to perform the method described in Embodiment 1.

[0093] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0094] Memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of memory may also include non-volatile random access memory. For example, memory may also store information about the device type.

[0095] In the implementation process, each step of the above method can be completed by the integrated logic circuits in the processor hardware or by software instructions.

[0096] The method in Embodiment 1 can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not provided here.

[0097] Those skilled in the art will recognize that the units and algorithm steps described in connection with the various examples of this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention.

[0098] Example 4

[0099] This embodiment also provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the method described in Embodiment 1.

[0100] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A target detection method based on dynamic video frame sampling, characterized in that, include: Obtain the compressed video to be processed; Decode the compressed video and extract the motion vectors of the video frames from the binary bitstream of the compressed video; Once the motion vectors of the video frames are obtained, the decoding of the compressed video is stopped. The motion vectors of video frames are clustered to obtain the number of clusters. Specifically, a density-based clustering algorithm is used to perform the clustering of motion vectors of video frames. The density-based clustering algorithm comprises two stages: core point query and core point promotion. Density is defined by setting a radius threshold eps and a minimum number of points minPts. The core point query stage involves: first, determining whether each point is a core point by setting the radius threshold eps and the minimum number of points minPts; then, checking if the number of points in the neighborhood of each point's set radius threshold eps is greater than or equal to minPts. If it is greater than or equal to minPts, the current point is a core point; otherwise, it is a non-core point. The core point promotion stage involves: starting from a core point, adding points in its neighborhood of the set radius threshold eps to the same cluster; continuing to add several newly added points to the cluster as new core points, and expanding the neighborhood of each new core point through parallel computation until the cluster can no longer be expanded. The number of clusters of motion vectors in video frames is denoised to obtain the number of clusters of motion vectors in the denoised video frames. A sliding window is used to slide on the number of clusters of motion vectors in the denoised video frames and calculate the standard deviation of the number of clusters of motion vectors in the video frames within the window. Based on the standard deviation, the number of video frames sampled within the window is determined. Based on the number of video frames sampled within the window, the video frames within the window are uniformly sampled to obtain sampled video frames. The sampled video frames are then decoded to obtain video frames of a set format. The video frames of the set format are then preprocessed and input into the target detection network to achieve the recognition of target objects in the video. Finally, the video frame sequence number of the target object is obtained.

2. The target detection method based on dynamic video frame sampling as described in claim 1, characterized in that, A sliding window is used, which slides across the number of clusters of motion vectors in the denoised video frames, and the standard deviation of the number of clusters of motion vectors in the video frames within the window is calculated. Specifically, it includes: Where n is the number of video frames within the sliding window. The number of motion vector clusters in the i-th frame within the window. This is the average number of clusters of motion vectors across all frames within the window.

3. The target detection method based on dynamic video frame sampling as described in claim 1, characterized in that, Based on the standard deviation, determine the number of video frames sampled within the window, specifically including: ;in, The minimum number of samples within a predefined window. It is the standard deviation of the number of clusters of motion vectors in consecutive frames within the window. This represents the number of video frames sampled within the window.

4. The target detection method based on dynamic video frame sampling as described in claim 1, characterized in that, Based on the number of video frames sampled within the window, the video frames within the window are uniformly sampled to obtain sampled video frames. Specifically, this includes: First, calculating the sampling interval within the window: Where n is the number of video frames within the window. The number of video frames sampled within the window; then, starting from the first frame of the window, at fixed intervals... Select subsequent frames in sequence until the desired result is reached. One sampled frame.

5. The target detection method based on dynamic video frame sampling as described in claim 1, characterized in that, Preprocessing of video frames in a specified format includes one or more of the following: color channel conversion, pixel value normalization, video frame size scaling, or tensor quantization.

6. A target detection system based on dynamic video frame sampling, characterized in that, include: The acquisition module is configured to acquire the compressed video to be processed. Decode the compressed video by extracting the motion vectors of the video frames from the binary bitstream of the compressed video; stop decoding the compressed video after obtaining the motion vectors of the video frames. The clustering module is configured to perform clustering processing on the motion vectors of video frames to obtain the number of clusters of the motion vectors of video frames. Specifically, a density-based clustering algorithm is used to implement the clustering processing of motion vectors of video frames. The density-based clustering algorithm comprises two stages: core point query and core point promotion. Density is defined by setting a radius threshold eps and a minimum number of points minPts. The core point query stage involves: first, determining whether each point is a core point by setting the radius threshold eps and the minimum number of points minPts; then, checking if the number of points in the neighborhood of each point's set radius threshold eps is greater than or equal to minPts. If it is greater than or equal to minPts, the current point is a core point; otherwise, it is a non-core point. The core point promotion stage involves: starting from a core point, adding points in its neighborhood of the set radius threshold eps to the same cluster; continuing to add several newly added points to the cluster as new core points, and expanding the neighborhood of each new core point through parallel computation until the cluster can no longer be expanded. The denoising module is configured to: denoise the number of clusters of motion vectors in video frames to obtain the number of clusters of motion vectors in the denoised video frames; use a sliding window to slide on the number of clusters of motion vectors in the denoised video frames and calculate the standard deviation of the number of clusters of motion vectors in the video frames within the window; and determine the number of samples of video frames within the window based on the standard deviation. The sampling module is configured to: uniformly sample the video frames within the window according to the number of samples of the video frames within the window to obtain sampled video frames; perform video decoding on the sampled video frames to obtain video frames of a set format; preprocess the video frames of the set format; input the preprocessed video frames into the target detection network to realize the recognition of target objects in the video; and finally, obtain the video frame sequence number of the target object.

7. An electronic device, characterized in that, include: A memory for non-transitory storage of computer-readable instructions; and a processor for executing the computer-readable instructions, wherein the computer-readable instructions, when executed by the processor, perform the method described in any one of claims 1-5.

8. A storage medium characterized by being non-transitory. The system stores computer-readable instructions, wherein, when the non-transitory computer-readable instructions are executed by a computer, the instructions of the method according to any one of claims 1-5 are executed.

9. A computer program product, characterized in that, Includes a computer program, which, when run on one or more processors, is used to implement the method described in any one of claims 1-5.

Images