Video coding method and system for target detection task, terminal and storage medium
By performing background filtering and feature dimensionality reduction on the target video and adaptively updating the feature value range, the problem of low video coding efficiency and accuracy in existing technologies is solved, and efficient coding and analysis in complex scenarios are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-12
- Publication Date
- 2026-04-10
AI Technical Summary
Existing technologies lack coding methods that adaptively update feature value ranges, resulting in low video coding efficiency and low analysis accuracy in complex scenarios.
By filtering out the background of the target video, standardized image features are obtained, dimensionality reduction and interval filtering are performed, feature channels are fused, feature value ranges are calculated and standardized, and the machine vision task network is used for processing to output the task results of the target video.
It effectively improves the coding performance of object detection tasks at the same bit rate, removes prior redundancy of object detection tasks in video images to the maximum extent, and adaptively updates feature standardization to improve coding efficiency and analysis accuracy.
Smart Images
Figure CN121842386A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of video data processing technology, and in particular to a video coding method, system, terminal, and computer-readable storage medium for target detection tasks. Background Technology
[0002] With the rapid development of digital media technology, image data is exploding in size. These images are not only viewed by human users but also used by machine vision algorithms for analysis and decision-making. However, images are often subjected to lossy compression during storage and transmission, leading to a decrease in quality and consequently, a decline in the performance of machine vision algorithms or even their failure. In applications such as surveillance and autonomous driving, it is necessary to run object detection algorithms on compressed images and ensure the effectiveness of the algorithms in analysis and recognition. Therefore, it is essential to optimize the compression process to achieve better object detection algorithm performance at the same compression ratio.
[0003] However, existing solutions involving feature processing typically employ globally fixed quantization strategies or static preprocessing parameters, ignoring the temporal non-stationarity of video data. Furthermore, the numerical range of deep features often fluctuates drastically with the video content. Existing technologies lack an adaptive update mechanism for feature numerical ranges based on the Group of Pictures (GOP) level, leading to improper dynamic range matching during feature quantization. This results in the truncation of key features or the redundant waste of encoded codewords, severely restricting coding efficiency and analysis accuracy in complex scenarios.
[0004] Therefore, existing technologies still need to be improved and developed. Summary of the Invention
[0005] The main objective of this invention is to provide a video coding method, system, terminal, and computer-readable storage medium for target detection tasks, aiming to solve the problem that existing technologies lack coding methods with adaptive updates of feature value ranges, resulting in low video coding efficiency and low analysis accuracy in complex scenarios.
[0006] To achieve the above objectives, the present invention provides a video coding method for target detection tasks, the video coding method for target detection tasks comprising the following steps: Background filtering is performed on the target video to obtain multiple standardized image features for each frame of video data; After dimensionality reduction of all the standardized image features, interval filtering is performed. The filtered features are then fused to obtain single-resolution features. All channels of the single-resolution features are then filtered to obtain dimensionality reduction features for different channels. All the dimensionality reduction features are arranged into a single frame image, the feature value range within the single frame image is calculated, and the feature value range is standardized to obtain multiple pixel feature values. Reconstruct all the pixel feature values to obtain a feature video sequence, perform recovery processing on the feature video sequence to obtain the encoded video of the target video, and use a machine vision task network to process the encoded video and output the task result of the target video.
[0007] Optionally, the video coding method for the target detection task, wherein the background filtering of the target video to obtain multiple standardized image features for each frame of video data specifically includes: The target video input by the user is obtained, and the target video is input frame by frame into the constructed target detection task network, and a target detection box is defined in the target detection task network; If an object to be detected is detected in the target video, the target detection box is used to detect the target video frame by frame in each feature pyramid layer of the target detection task network to obtain the standardized image features output by each feature pyramid layer. If no object to be detected is detected in the target video, the target video is semantically segmented using the semantic segmentation model of the target detection task network to obtain standardized image features.
[0008] Optionally, the video coding method for the target detection task, wherein the step of performing interval filtering after dimensionality reduction processing on all the standardized image features specifically includes: The feature scaling model is used to uniformly scale all feature values of all standardized image features corresponding to each feature pyramid layer, and the feature mean and feature standard deviation of all feature values are calculated. The scaled video features are filtered at intervals using a temporal downsampling model to obtain multiple filtered features. Each video frame features include filtered features of different resolutions output by multiple feature pyramid layers.
[0009] Optionally, the video coding method for the object detection task, wherein fusing the selected features to obtain single-resolution features, and filtering all channels of the single-resolution features to obtain dimensionality-reduced features for different channels, specifically includes: For each frame, all the filtering features at different resolutions are fused to obtain a single resolution feature. Extract all feature channels of the single resolution feature and obtain the channel weights corresponding to all feature channels. Arrange all feature channels according to all channel weights to obtain the channel arrangement order. Based on the channel arrangement order, all static feature channels with activation levels lower than a preset value are selected, and all static feature channels are deleted from the single-resolution feature to obtain the dimensionality reduction feature.
[0010] Optionally, the video coding method for the target detection task further includes, after uniformly scaling all values of the standardized image features, retaining the first and last frame video features of the scaled features when performing interval frame filtering on the scaled features.
[0011] Optionally, the video coding method for the object detection task, wherein arranging all the dimensionality-reduced features into a single-frame image, calculating the feature value range within the single-frame image, and standardizing the feature value range to obtain multiple pixel feature values, specifically includes: All the dimensionality reduction features are arranged into a single frame image according to the raster scan order; The feature normalization module is used to statistically analyze the range of feature values in the single frame image to obtain the maximum and minimum feature values. The feature value range of the single frame image is standardized using the maximum and minimum feature values to obtain multiple pixel feature values: ; in, Represents pixel feature values. Represents the smallest feature value. y represents the maximum feature value, and y represents the current feature value.
[0012] Optionally, the video coding method for the target detection task, wherein reconstructing all the pixel feature values to obtain a feature video sequence, performing recovery processing on the feature video sequence to obtain the encoded video of the target video, and processing the encoded video using a machine vision task network to output the task result of the target video, specifically includes: The pixel feature values are encoded and decoded using a video encoder to output a reconstructed feature video sequence; Using the large feature value and the small feature value, the certificate pixel values in the feature video sequence are converted into floating-point feature values; Perform inverse transformation on all the floating-point feature values to reconstruct all single-frame feature maps in the feature video sequence into corresponding multi-channel feature maps; Using the feature mean and the feature standard deviation, the parameters in all the multi-channel feature maps are fine-tuned to obtain multiple fine-tuned feature maps that meet preset conditions; The fine-tuned feature maps are restored to obtain the encoded video of the target video. The encoded video is then input into a machine vision task network for recognition and analysis, and the task result of the target video is output.
[0013] Furthermore, to achieve the above objectives, the present invention also provides a video coding system for target detection tasks, wherein the video coding system for target detection tasks includes: The image domain processing module is used to filter out the background of the target video and obtain multiple standardized image features for each frame of video data; The feature dimensionality reduction module is used to perform dimensionality reduction processing on all the standardized image features and then perform interval filtering. The filtered features are then fused to obtain a single-resolution feature, and all channels of the single-resolution feature are filtered to obtain dimensionality reduction features for different channels. The feature transformation module is used to arrange all the dimensionality reduction features into a single frame image, calculate the feature value range within the single frame image, and standardize the feature value range to obtain multiple pixel feature values. The feature recovery module is used to reconstruct all the pixel feature values to obtain a feature video sequence, and to perform recovery processing on the feature video sequence to obtain the encoded video of the target video.
[0014] Furthermore, to achieve the above objectives, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a video encoding program for a target detection task stored in the memory and executable on the processor, wherein the video encoding program for a target detection task, when executed by the processor, implements the steps of the video encoding method for a target detection task as described above.
[0015] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a video coding program for an object detection task, and the video coding program for an object detection task, when executed by a processor, implements the steps of the video coding method for an object detection task as described above.
[0016] In this invention, background filtering is performed on the target video to obtain multiple standardized image features for each frame of video data. After dimensionality reduction processing of all the standardized image features, interval filtering is performed, and the filtered features are fused to obtain single-resolution features. All channels of the single-resolution features are then filtered to obtain dimensionality-reduced features for different channels. All the dimensionality-reduced features are arranged into a single-frame image, and the feature value range within the single-frame image is calculated. The feature value range is then standardized to obtain multiple pixel feature values. All the pixel feature values are reconstructed to obtain a feature video sequence. The feature video sequence is then restored to obtain the encoded video of the target video. The encoded video is then processed using a machine vision task network to output the task result of the target video. This invention combines image-domain preprocessing and feature-domain encoding compression to maximize the removal of prior redundancy in target detection tasks in video images. It adaptively updates the numerical extrema used in feature standardization at the image group level, effectively improving the encoding performance of target detection tasks at the same bitrate. Attached Figure Description
[0017] Figure 1 This is a flowchart of a preferred embodiment of the video coding method for target detection tasks of the present invention; Figure 2 This is a flowchart illustrating the encoding framework of a preferred embodiment of the video encoding method for target detection tasks of the present invention. Figure 3 This is a structural diagram of a preferred embodiment of the video coding system for target detection tasks of the present invention; Figure 4 This is a structural diagram of a preferred embodiment of the terminal of the present invention. Detailed Implementation
[0018] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0019] The video coding method for target detection tasks described in the preferred embodiment of the present invention, such as... Figure 1 As shown, the video coding method for the target detection task includes the following steps: Step S10: Filter out the background of the target video to obtain multiple standardized image features for each frame of video data.
[0020] In object detection tasks, the texture details of the background region are often redundant information, while the foreground region and its high-dimensional semantic features, which are crucial to the task, may be smoothed or lost in the process of pursuing overall statistical indicators. Therefore, in the embodiments disclosed in this invention, the background of the target video is first preprocessed to remove the prior redundancy in the video image that is relevant to the object detection task to the greatest extent.
[0021] Specifically, the target video input by the user is obtained, the target video is input frame by frame into the constructed target detection task network, and a target detection box is defined in the target detection task network; If an object to be detected is detected in the target video, the target detection box is used to detect the target video frame by frame in each feature pyramid layer of the target detection task network to obtain the standardized image features output by each feature pyramid layer. If the object to be detected is not detected in the target video, the target video is semantically segmented using the semantic segmentation model of the target detection task network to obtain standardized image features; In this context, the pixel values of image content outside the target detection box are defined as 0.
[0022] Among them, such as Figure 2 As shown, in the embodiments disclosed in this invention, the image domain processing module can be used to remove background regions in the video image that do not affect target detection, and deep features can be extracted through the task network model. Then, deep features suitable for the target detection task can be extracted from the preprocessed video image and feature quantization processing can be performed.
[0023] Specifically, in the embodiments disclosed in this invention, the video image feature extraction method is to fix the algorithm model used in the target detection task, which is based on a feature pyramid network. The video is input frame by frame into this model (the target detection network disclosed in this invention is Faster R-CNN X101 FPN, Faster R-CNN with ResNeXt-101 and Feature Pyramid Network, a two-stage target detection algorithm), and the features of the 2nd, 3rd, 4th, and 5th layers of the feature pyramid are obtained as the extracted video depth features. Among them, the resolution of the 2nd layer feature map is one-quarter of the input video resolution, the 3rd layer is one-eighth, the 4th layer is one-sixteenth, and the 5th layer is one-thirty-second; and the number of channels in each layer feature map is 256.
[0024] By extracting depth features from the video frame by frame, the key information of each frame for the target detection task can be fully preserved. Compared with the method of directly using the video frame itself as the encoding object, the present invention can effectively remove information in the video that is not needed by the task and the model by using the target detection task model to extract features, thereby reducing the amount of data that needs to be processed in subsequent encoding from the source.
[0025] Step S20: After dimensionality reduction processing of all the standardized image features, interval filtering is performed. The filtered features are fused to obtain single-resolution features. All channels of the single-resolution features are then filtered to obtain dimensionality reduction features for different channels.
[0026] Among them, existing video coding methods based on regions of interest, although attempting to introduce attention mechanisms that separate foreground and background, still suffer from the defects of having a single optimization dimension and not being deeply coupled with the feature domain. Such methods usually use saliency detection or pre-analysis to extract regions that are more important for the target detection task, and tilt bit allocation in the image domain by adjusting quantization parameters.
[0027] However, this processing only involves pixel-level weighting and lacks utilization of the intermediate layer features of the deep neural networks used in object detection tasks. Because a joint optimization channel between the image domain and the feature domain is not constructed, the encoder cannot perceive the spatiotemporal distribution patterns of depth features, nor can it use feedback from the feature domain to guide coding decisions. Therefore, this invention discloses further processing of the image domain features and their corresponding features after image domain processing. This process extracts depth features suitable for object detection tasks from the pre-processed video image and performs feature quantization to improve the coding performance of object detection tasks.
[0028] Specifically, a feature scaling model is used to uniformly scale all feature values of all standardized image features corresponding to each feature pyramid layer, and the feature mean and feature standard deviation of all feature values are calculated. The scaled video features are filtered at intervals using a temporal downsampling model to obtain multiple filtered features. Each video frame features include filtered features of different resolutions output by multiple feature pyramid layers.
[0029] Furthermore, when performing interval frame filtering on the scaled features, the first and last frame video features of the scaled features are retained.
[0030] For the standardized image features that have been acquired, the feature values are first uniformly scaled, and then the mean and standard deviation of the feature values are calculated as key parameters to retain in the feature statistical distribution. Through the temporal downsampling module, one frame of video features is discarded every other frame, but the features of the first and last frames of the video are retained. Then, through the feature transformation module, a specific coding neural network is used to fuse the extracted features of different levels and resolutions into features of a single resolution.
[0031] Furthermore, for all the filtering features in each frame, the filtering features at different resolutions are fused to obtain a single resolution feature; Extract all feature channels of the single resolution feature and obtain the channel weights corresponding to all feature channels. Arrange all feature channels according to all channel weights to obtain the channel arrangement order. Based on the channel arrangement order, all static feature channels with activation levels lower than a preset value are selected, and all static feature channels are deleted from the single-resolution feature to obtain the dimensionality reduction feature.
[0032] Existing technologies typically utilize saliency detection or pre-analysis to extract regions that are more important for the target detection task, and then adjust the quantization parameters in the image domain to skew bit allocation. This processing only involves pixel-level weighting and lacks utilization of the intermediate layer features of the deep neural networks used in the target detection task. Therefore, in the content disclosed in this invention, the feature channels are rearranged from important to unimportant to make adjacent channels more similar, and the dynamic range of the channels is used as an indicator of the activation level to delete inactive feature channels, thereby improving the coding accuracy.
[0033] Step S30: Arrange all the dimensionality reduction features into a single frame image, calculate the feature value range within the single frame image, and standardize the feature value range to obtain multiple pixel feature values.
[0034] Specifically, all the dimensionality reduction features are arranged into a single frame image according to the raster scanning order; The feature normalization module is used to statistically analyze the range of feature values in the single frame image to obtain the maximum and minimum feature values. The feature value range of the single frame image is standardized using the maximum and minimum feature values to obtain multiple pixel feature values: ; in, Represents pixel feature values. Represents the smallest feature value. y represents the maximum feature value, and y represents the current feature value.
[0035] In one of the embodiments disclosed in this invention, the features of different channels are arranged into a single frame image according to the raster scanning order through the feature aggregation process. Then, the numerical range of the features is statistically analyzed to obtain the maximum and minimum values. Based on these values, the features are standardized to convert them into integer pixel values that can be processed by the video encoder.
[0036] Step S40: Reconstruct all the pixel feature values to obtain a feature video sequence, perform recovery processing on the feature video sequence to obtain the encoded video of the target video, and use a machine vision task network to process the encoded video and output the task result of the target video.
[0037] Specifically, the pixel feature values are encoded and decoded using a video encoder to output a reconstructed feature video sequence; Using the large feature value and the small feature value, the certificate pixel values in the feature video sequence are converted into floating-point feature values; Perform inverse transformation on all the floating-point feature values to reconstruct all single-frame feature maps in the feature video sequence into corresponding multi-channel feature maps; Using the feature mean and feature standard deviation, the parameters in all the multi-channel feature maps are fine-tuned to obtain multiple fine-tuned feature maps that meet preset conditions; The fine-tuned feature maps are restored to obtain the encoded video of the target video. The encoded video is then input into a machine vision task network for recognition and analysis, and the task result of the target video is output.
[0038] The process involves dimensionality reduction and transformation of the features to obtain images in the form of individual pixel locations. Since the features of each frame of the video undergo dimensionality reduction and transformation, a feature video sequence containing a set of feature images is ultimately formed. This video is then encoded using a standard video encoder. The encoding process includes block partitioning, prediction, transformation, quantization, and entropy coding, resulting in a decodeable bitstream. The decoding end then parses and decodes this bitstream to obtain the reconstructed feature video sequence.
[0039] Furthermore, through the feature dequantization module, the extreme values used in the feature standardization carried in the bitstream are used to restore the integer feature pixel values to approximate floating-point feature values. The reconstructed single-frame feature image is then split into multi-channel feature maps according to the original aggregation rules. Finally, the dimensionality reduction feature fine-tuning module can fine-tune the features output by the previous module using the distribution parameters (mean and standard deviation) of the original features contained in the bitstream, so that the overall features conform to a similar statistical distribution.
[0040] Furthermore, the feature channels deleted by the channel removal module at the encoding end are restored by filling with all zeros; then, the channel order reordered by importance at the encoding end is restored to the channel order required by the original network. Using a decoding neural network that is the counterpart to the encoding neural network used in the feature transformation module at the encoding end, the single-resolution features are restored to features of different levels and resolutions. Then, by copying the most recent frame or linear interpolation, the features of the frames discarded by the encoding end due to temporal downsampling are restored. Finally, while inverse scaling the features, a lightweight post-processing network is used to fine-tune them to compensate for compression loss and improve the final task accuracy.
[0041] This invention provides a method for updating the floating-point numerical range of features at the image group level. This method can adaptively update the extreme values used in feature standardization at the image group level, better adapt to changes in the feature numerical range, effectively improve feature quantization accuracy, and retain feature numerical information as much as possible. Furthermore, the quantized features are encoded and compressed using video coding standards, and then input into the target detection network after decoding to complete the detection task. This effectively improves the coding performance of the target detection task at the same bitrate.
[0042] Furthermore, in another embodiment of the present invention, an image domain processing module and a method for updating the feature floating-point value range at the image group level are disclosed: First, background regions that are not important to the task are removed from the video image, thereby significantly reducing semantic redundancy in the video image before the subsequent feature domain processing module. For the input video, the YOLO26 (You Only Look Once version 26, an object detection model) network is used to perform object detection frame by frame, retaining the image within the detection box and defining all pixel values of the image content outside the detection box as 0. For videos without objects to be detected, the YOLO26-seg (You Only Look Once version 26 - Segmentation, a YOLO26 network for instance segmentation) model can be used for semantic segmentation to obtain pixel-level mask image information, and thus remove pixels in the background region.
[0043] Compared to existing methods that use the entire video sequence as the statistical range for feature values and then standardize them, the image group-level statistical features and standardization disclosed in this invention can more accurately preserve feature information. This is especially true for videos with frequent dramatic movements and large content changes, where the feature value range is usually larger. If the minimum and maximum values of features statistically analyzed from the entire video sequence are used for standardization, some features will have very small floating-point values after standardization. During subsequent quantization, this part of the feature information is easily erased by the quantization process, affecting the preservation of its information. Therefore, the image group-level statistical features and standardization of this invention can reduce the probability of this situation occurring.
[0044] Furthermore, this invention discloses a comparison method to demonstrate the performance advantages of the video coding method for object detection tasks disclosed in this invention: During the testing process, the comparison method was defined as MPEG VCM (MPEG Video Coding for Machines) standard reference software VCM-RS 0.8.1. This method is an image-domain encoding method for object detection tasks. Under low-latency configuration, on the test dataset SFU-HW (Simon Fraser University-Hardware-Aware Workloads, a high-quality labeled dataset for object tracking tasks in efficient video coding scenarios), the encoding performance was tested after encoding 6 bitrate points. The performance improvement was measured by BD-mAP (the average percentage improvement in object detection performance compared to VCM-RS 0.8.1 at the same bitrate) (%), with a performance improvement of 105.68%. The specific performance sequences are shown in the table below. Table 1: Performance Advantages Table
[0045] Among them, BQMall represents the standard test sequence; BasketballDrill represents test material containing complex moving objects and rich scene details; BasketballDrive represents test material with rich details; BasketballPass represents test material containing large-scale global and local motion; BlowingBubbles represents test material containing rich texture details and complex motion; ParkScene represents test material containing rich texture details and complex motion; PartyScene represents test material containing rich texture details and complex motion; RaceHorsesC represents test material containing fast motion and complex dynamic changes; RaceHorsesD represents test material containing fast motion and complex dynamic changes; and Traffic represents test material containing complex dynamic motion and rich scene details. This invention combines image domain preprocessing and feature domain encoding compression to remove prior redundancy in target detection tasks in video images to the greatest extent. The method of updating the floating-point value range of features at the image group level can adaptively update the numerical extrema used in feature standardization at the image group level, which can better adapt to changes in feature value range. The quantized features are encoded and compressed using video coding standards, and then input into the target detection network after decoding to complete the detection task. This effectively improves the coding performance of target detection tasks at the same bitrate.
[0046] Furthermore, such as Figure 3 As shown, based on the above-described video coding method for object detection tasks, this invention also provides a video coding system for object detection tasks, wherein the video coding system for object detection tasks includes: Image domain processing module 51 is used to filter out the background of the target video and obtain multiple standardized image features of each frame of video data; The feature dimensionality reduction module 52 is used to perform dimensionality reduction processing on all the standardized image features and then perform interval filtering, fuse the filtered features to obtain a single resolution feature, and filter all channels of the single resolution feature to obtain dimensionality reduction features of different channels. The feature transformation module 53 is used to arrange all the dimensionality reduction features into a single frame image, calculate the feature value range within the single frame image, and standardize the feature value range to obtain multiple pixel feature values. The feature recovery module 54 is used to reconstruct all the pixel feature values to obtain a feature video sequence, and to perform recovery processing on the feature video sequence to obtain the encoded video of the target video.
[0047] Furthermore, such as Figure 4As shown, based on the above-mentioned video coding method and system for target detection tasks, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 4 Only some of the terminal components are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.
[0048] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory. In other embodiments, the memory 20 may be an external storage device of the terminal, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc. Further, the memory 20 may include both internal and external storage devices. The memory 20 is used to store application software and various types of data installed on the terminal, such as program code installed on the terminal. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores a video encoding program 40 for target detection tasks, which can be executed by the processor 10 to implement the video encoding method for target detection tasks in this application.
[0049] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in the memory 20 or process data, such as executing the video encoding method for the target detection task.
[0050] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the terminal and to display a visual user interface. The components of the terminal communicate with each other via a system bus.
[0051] In one embodiment, when the processor 10 executes the video encoding program 40 for the target detection task in the memory 20, the following steps are performed: Background filtering is performed on the target video to obtain multiple standardized image features for each frame of video data; After dimensionality reduction of all the standardized image features, interval filtering is performed. The filtered features are then fused to obtain single-resolution features. All channels of the single-resolution features are then filtered to obtain dimensionality reduction features for different channels. All the dimensionality reduction features are arranged into a single frame image, the feature value range within the single frame image is calculated, and the feature value range is standardized to obtain multiple pixel feature values. Reconstruct all the pixel feature values to obtain a feature video sequence, perform recovery processing on the feature video sequence to obtain the encoded video of the target video, and use a machine vision task network to process the encoded video and output the task result of the target video.
[0052] Specifically, the process of background filtering of the target video to obtain multiple standardized image features for each frame of video data includes: The target video input by the user is obtained, and the target video is input frame by frame into the constructed target detection task network, and a target detection box is defined in the target detection task network; If an object to be detected is detected in the target video, the target detection box is used to detect the target video frame by frame in each feature pyramid layer of the target detection task network to obtain the standardized image features output by each feature pyramid layer. If no object to be detected is detected in the target video, the target video is semantically segmented using the semantic segmentation model of the target detection task network to obtain standardized image features.
[0053] Specifically, the step of performing dimensionality reduction processing on all the standardized image features followed by interval filtering includes: The feature scaling model is used to uniformly scale all feature values of all standardized image features corresponding to each feature pyramid layer, and the feature mean and feature standard deviation of all feature values are calculated. The scaled video features are filtered at intervals using a temporal downsampling model to obtain multiple filtered features. Each video frame features include filtered features of different resolutions output by multiple feature pyramid layers.
[0054] Specifically, the process of fusing the selected features to obtain single-resolution features, and then filtering all channels of the single-resolution features to obtain dimensionality-reduced features for different channels, includes: For each frame, all the filtering features at different resolutions are fused to obtain a single resolution feature. Extract all feature channels of the single resolution feature and obtain the channel weights corresponding to all feature channels. Arrange all feature channels according to all channel weights to obtain the channel arrangement order. Based on the channel arrangement order, all static feature channels with activation levels lower than a preset value are selected, and all static feature channels are deleted from the single-resolution feature to obtain the dimensionality reduction feature.
[0055] The process of uniformly scaling all values of the standardized image features further includes: when performing interval frame filtering on the scaled features, retaining the first and last frame video features of the scaled features.
[0056] Specifically, the step of arranging all the dimensionality-reduced features into a single-frame image, calculating the feature value range within the single-frame image, and standardizing the feature value range to obtain multiple pixel feature values includes: All the dimensionality reduction features are arranged into a single frame image according to the raster scan order; The feature normalization module is used to statistically analyze the range of feature values in the single frame image to obtain the maximum and minimum feature values. The feature value range of the single frame image is standardized using the maximum and minimum feature values to obtain multiple pixel feature values: ; in, Represents pixel feature values. Represents the smallest feature value. y represents the maximum feature value, and y represents the current feature value.
[0057] Specifically, the process of reconstructing all pixel feature values to obtain a feature video sequence, performing recovery processing on the feature video sequence to obtain the encoded video of the target video, and processing the encoded video using a machine vision task network to output the task result of the target video includes: The pixel feature values are encoded and decoded using a video encoder to output a reconstructed feature video sequence; Using the large feature value and the small feature value, the certificate pixel values in the feature video sequence are converted into floating-point feature values; Perform inverse transformation on all the floating-point feature values to reconstruct all single-frame feature maps in the feature video sequence into corresponding multi-channel feature maps; Using the feature mean and feature standard deviation, the parameters in all the multi-channel feature maps are fine-tuned to obtain multiple fine-tuned feature maps that meet preset conditions; The fine-tuned feature maps are restored to obtain the encoded video of the target video. The encoded video is then input into a machine vision task network for recognition and analysis, and the task result of the target video is output.
[0058] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a video coding program for an object detection task, and the video coding program for an object detection task, when executed by a processor, implements the steps of the video coding method for an object detection task as described above.
[0059] In summary, this invention provides a video coding method and related equipment for target detection tasks. The method includes: background filtering of a target video to obtain multiple standardized image features for each frame of video data; dimensionality reduction processing of all the standardized image features followed by interval filtering; fusion processing of the filtered features to obtain single-resolution features; filtering of all channels of the single-resolution features to obtain dimensionality-reduced features for different channels; arranging all the dimensionality-reduced features into a single-frame image; calculating the feature value range within the single-frame image; standardizing the feature value range to obtain multiple pixel feature values; reconstructing all the pixel feature values to obtain a feature video sequence; performing recovery processing on the feature video sequence to obtain the encoded video of the target video; and processing the encoded video using a machine vision task network to output the task result of the target video. This invention combines image-domain preprocessing and feature-domain encoding compression to maximize the removal of prior redundancy in target detection tasks in video images. It adaptively updates the numerical extrema used in feature standardization at the image group level, effectively improving the coding performance of target detection tasks at the same bitrate.
[0060] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes that element.
[0061] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). The program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The computer-readable storage medium can be a memory, magnetic disk, optical disk, etc.
[0062] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.
Claims
1. A video coding method for target detection tasks, characterized in that, The video coding method for the target detection task includes: Background filtering is performed on the target video to obtain multiple standardized image features for each frame of video data; After dimensionality reduction of all the standardized image features, interval filtering is performed. The filtered features are then fused to obtain single-resolution features. All channels of the single-resolution features are then filtered to obtain dimensionality reduction features for different channels. All the dimensionality reduction features are arranged into a single frame image, the feature value range within the single frame image is calculated, and the feature value range is standardized to obtain multiple pixel feature values. Reconstruct all the pixel feature values to obtain a feature video sequence, perform recovery processing on the feature video sequence to obtain the encoded video of the target video, and use a machine vision task network to process the encoded video and output the task result of the target video.
2. The video coding method for target detection tasks according to claim 1, characterized in that, The background filtering of the target video, which yields multiple standardized image features for each frame of video data, specifically includes: The target video input by the user is obtained, and the target video is input frame by frame into the constructed target detection task network, and a target detection box is defined in the target detection task network; If an object to be detected is detected in the target video, the target detection box is used to detect the target video frame by frame in each feature pyramid layer of the target detection task network to obtain the standardized image features output by each feature pyramid layer. If no object to be detected is detected in the target video, the target video is semantically segmented using the semantic segmentation model of the target detection task network to obtain standardized image features.
3. The video coding method for target detection tasks according to claim 2, characterized in that, The step of performing dimensionality reduction processing on all the standardized image features followed by interval filtering specifically includes: The feature scaling model is used to uniformly scale all feature values of all standardized image features corresponding to each feature pyramid layer, and the feature mean and feature standard deviation of all feature values are calculated. The scaled video features are filtered at intervals using a temporal downsampling model to obtain multiple filtered features. Each video frame features include filtered features of different resolutions output by multiple feature pyramid layers.
4. The video coding method for target detection tasks according to claim 3, characterized in that, The process of fusing the selected features to obtain single-resolution features, and then filtering all channels of the single-resolution features to obtain dimensionality-reduced features for different channels, specifically includes: For each frame, all the filtering features at different resolutions are fused to obtain a single resolution feature. Extract all feature channels of the single resolution feature and obtain the channel weights corresponding to all feature channels. Arrange all feature channels according to all channel weights to obtain the channel arrangement order. Based on the channel arrangement order, all static feature channels with activation levels lower than a preset value are selected, and all static feature channels are deleted from the single-resolution feature to obtain the dimensionality reduction feature.
5. The video coding method for target detection tasks according to claim 4, characterized in that, The process of uniformly scaling all values of the standardized image features further includes: when performing interval frame filtering on the scaled features, retaining the first and last frame video features of the scaled features.
6. The video coding method for target detection tasks according to claim 4, characterized in that, The process of arranging all the dimensionality-reduced features into a single-frame image, calculating the feature value range within the single-frame image, and standardizing the feature value range to obtain multiple pixel feature values specifically includes: All the dimensionality reduction features are arranged into a single frame image according to the raster scan order; The feature normalization module is used to statistically analyze the range of feature values in the single frame image to obtain the maximum and minimum feature values. The feature value range of the single frame image is standardized using the maximum and minimum feature values to obtain multiple pixel feature values: ； in, Represents pixel feature values. Represents the smallest feature value. y represents the maximum feature value, and y represents the current feature value.
7. The video coding method for target detection tasks according to claim 6, characterized in that, The process of reconstructing all the pixel feature values to obtain a feature video sequence, performing recovery processing on the feature video sequence to obtain the encoded video of the target video, and processing the encoded video using a machine vision task network to output the task result of the target video specifically includes: The pixel feature values are encoded and decoded using a video encoder to output a reconstructed feature video sequence; Using the large feature value and the small feature value, the certificate pixel values in the feature video sequence are converted into floating-point feature values; Perform inverse transformation on all the floating-point feature values to reconstruct all single-frame feature maps in the feature video sequence into corresponding multi-channel feature maps; Using the feature mean and feature standard deviation, the parameters in all the multi-channel feature maps are fine-tuned to obtain multiple fine-tuned feature maps that meet preset conditions; The fine-tuned feature maps are restored to obtain the encoded video of the target video. The encoded video is then input into a machine vision task network for recognition and analysis, and the task result of the target video is output.
8. A video coding system for target detection tasks, characterized in that, The video coding system for object detection tasks is used to implement the video coding method for object detection tasks as described in any one of claims 1-7, wherein the video coding system for object detection tasks includes: The image domain processing module is used to filter out the background of the target video and obtain multiple standardized image features for each frame of video data; The feature dimensionality reduction module is used to perform dimensionality reduction processing on all the standardized image features and then perform interval filtering. The filtered features are then fused to obtain a single-resolution feature, and all channels of the single-resolution feature are filtered to obtain dimensionality reduction features for different channels. The feature transformation module is used to arrange all the dimensionality reduction features into a single frame image, calculate the feature value range within the single frame image, and standardize the feature value range to obtain multiple pixel feature values. The feature recovery module is used to reconstruct all the pixel feature values to obtain a feature video sequence, and to perform recovery processing on the feature video sequence to obtain the encoded video of the target video.
9. A terminal, characterized in that, The terminal includes: a memory, a processor, and a video encoding program for target detection tasks stored in the memory and executable on the processor. When the video encoding program for target detection tasks is executed by the processor, it implements the steps of the video encoding method for target detection tasks as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a video coding program for an object detection task, which, when executed by a processor, implements the steps of the video coding method for an object detection task as described in any one of claims 1-7.