Video region-of-interest extraction and improved compression method based on compressed domain

Abstract

The invention discloses a video interest region extraction and improved compression method based on a compressed domain, which comprises the following steps: firstly, acquiring an original video image and performing coding conversion, performing compressed domain feature extraction on the image after coding conversion, and determining an interest region in the image based on the extracted compressed domain feature; and finally, carrying out video image adaptive coding and compression according to the extracted region of interest, and carrying out adaptive decoding and display recovery according to the region of interest when video image decoding is carried out. According to the scheme of the invention, the region of interest extraction and adaptive coding are directly carried out on the compressed domain after coding, so that video compression processing can be carried out on the region of interest, the defects of a conventional region of interest extraction method based on a pixel domain are overcome, the calculation complexity is reduced, the region of interest extraction efficiency and accuracy are improved, and the video compression efficiency is improved. Compared with the traditional method, the scheme provided by the invention has the advantages that the processing speed is obviously at a better level, the video compression speed and quality can be obviously improved, the video code rate is reduced, the video quality evaluation index performance is better, the video compression processing time delay is lower, and the real-time video compression processing can be met.

Description

Technical Field

[0001] The invention belongs to the field of video compression, and in particular relates to a video interest region extraction and improved compression method based on compression domain. Background Art

[0002] In video images, people often pay the most attention to the important parts of the image, such as people, moving objects, etc. These areas have a great impact on the visual perception quality of the entire scene. Therefore, if only the area of ​​interest is finely encoded during compression encoding, and other non-important areas are deeply quantized, it can improve the video compression efficiency and reduce the computational complexity without affecting people's visual perception quality. Based on this idea, video image compression technology for the area of ​​interest has become a hot topic of concern and research.

[0003] One of the keys to the video image compression technology for regions of interest is how to accurately and efficiently identify the regions of interest. Most of the existing models for extracting regions of interest are based on the pixel domain of video images. However, for video data that has been encoded and compressed, if a model based on the pixel domain is used for region of interest extraction, the video encoding data must first be decompressed into the pixel domain to extract image features and regions of interest, and then compressed back to the original bit stream for transmission. This method will cause a lot of waste of computing resources and time.

[0004] To address the above problems, a method for extracting region of interest in the video compression domain is proposed. The encoded compressed data can be directly used to extract the region of interest according to the statistical characteristics of the compression domain features, and the extracted region of interest can be filtered, denoised and other operations can be performed to further improve the accuracy of region of interest extraction, thereby reducing the process of data decompression and recompression, and improving the efficiency and real-time performance of video compression while ensuring the accuracy of region of interest extraction. Summary of the invention

[0005] In view of the above problems, the object of the present invention is to provide a video interest region extraction and improved compression method based on compression domain.

[0006] The specific technical solution for achieving the purpose of the present invention is:

[0007] A video interest region extraction and improved compression method based on compression domain comprises the following steps:

[0008] Step 1: Collect the original video image and perform encoding conversion;

[0009] Step 2: extracting compressed domain features from the encoded image;

[0010] Step 3: Determine the region of interest in the image based on the extracted compressed domain features;

[0011] Step 4: Adaptively encode and compress the video image according to the extracted region of interest;

[0012] Step 5: When decoding the video image, adaptive decoding and display restoration are performed according to the region of interest.

[0013] Compared with the prior art, the present invention has the following beneficial effects:

[0014] The solution of the present invention is to directly extract the region of interest and perform adaptive encoding on the compressed domain after encoding in the process of video compression processing based on the region of interest, so as to meet the requirements of video compression processing for the region of interest, solve the defects of the previous region of interest extraction method based on the pixel domain, reduce the complexity of calculation, and improve the efficiency and accuracy of region of interest extraction. By comparing with the classic region of interest extraction model (including four models of LTST, AGS, FRGNE, and PDB), and comparing the average processing time per frame of each model video, the processing speed of the solution of the present invention is obviously at a better level, which can significantly improve the video compression speed and quality, realize the reduction of video bit rate, and the video quality evaluation index has good performance, and the video compression processing delay is low, which can meet the requirements of real-time video compression processing.

[0015] The present invention is further described below in conjunction with specific implementation modes. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 The present invention is a flow chart of an improved compression method for extracting video region of interest based on compression domain. DETAILED DESCRIPTION

[0017] Example

[0018] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an kind" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0020] Unless otherwise specifically stated, the relative arrangement, numerical expressions and numerical values ​​of the parts and steps set forth in these embodiments do not limit the scope of the present application. At the same time, it should be understood that, for ease of description, the sizes of the various parts shown in the accompanying drawings are not drawn according to the actual proportional relationship. The technology, method and equipment known to those of ordinary skill in the relevant field may not be discussed in detail, but in appropriate cases, the technology, method and equipment should be considered as a part of the authorization specification. In all examples shown and discussed here, any specific value should be interpreted as being merely exemplary, rather than as a limitation. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so that once a certain item is defined in an accompanying drawing, it does not need to be further discussed in subsequent drawings.

[0021] Combination Figure 1 , a video interest region extraction and improved compression method based on compression domain, comprising the following steps:

[0022] Step 1: Collect the original video image and perform encoding conversion:

[0023] Firstly, the original video image data is obtained from the camera device;

[0024] The collected original video images are encoded and converted based on the adopted video coding standard, converting them from the original pixel format to the compressed format. The encoding process includes image segmentation, transformation, quantization, entropy coding and other steps to reduce the amount of data.

[0025] Step 2: Analyze the feature information in the compressed domain data and extract the compressed domain features of the encoded image:

[0026] Step 2-1, extracting compressed domain data from the video image after coding conversion, including DCT coefficients and quantization coefficients;

[0027] Step 2-2: Based on the extracted compressed domain data, determine the compressed domain features of the video image. In this embodiment, the residual transform coefficient norm RTCN is introduced. The area with large prediction residual in the image is likely to be the area of ​​interest. The DCT coefficient of the prediction residual strictly corresponds to the prediction residual, and there is a certain correlation between the statistical singularity of the spectrum and the regional significance in the video frame. Therefore, the video interest area can be characterized by the DCT coefficient of the prediction residual. For the compressed video bit stream, the l of the prediction residual pThe norm (p ≥ 0) is easy to calculate, which can reduce the complexity of the algorithm, improve the calculation rate, and better capture the important features and changes in the video frame. For an image region, the prediction residual transform coefficient norm (RTCN) is used as the different norms of the prediction residual DCT coefficients of the region after quantization:

[0028]

[0029] Among them, (i,j) represents the i-th row and j-th column of the image area, represents the discrete cosine transform coefficient of the i-th row and j-th column in the prediction residual area, M and N represent the range of the prediction residual area, p represents the number of frames of the current image, and R represents the total number of video frames;

[0030] The compressed domain features of the video image need to be normalized after extraction:

[0031]

[0032] Among them, RTCN min With RTCN max Respectively represent the minimum and maximum values ​​of the RTCN features of the current image.

[0033] Step 3: Determine the region of interest in the image based on the extracted compressed domain features:

[0034] Step 3-1, based on the extracted compressed domain features, by analyzing the differences in the feature data, the region of interest in the image is detected, that is, when the residual transform coefficient norm RTCN is greater than the set threshold, the region is considered to be the region of interest. In this embodiment, when the residual transform coefficient norm RTCN is set to ≥ 0.85, the region is considered to be the region of interest;

[0035] Step 3-2: Post-process the ROI determined in step 3-1, including spatial filtering (noise removal), temporal filtering (smoothing), etc. Specifically, the intra-frame and inter-frame correlation of the video sequence allows us to improve the accuracy of ROI extraction through spatial filtering and temporal filtering. The spatial filter can analyze and enhance the specific features of the pixel values ​​and their spatial distribution in the image, while the temporal filter can capture and enhance the dynamic features and changes between video frames.

[0036] By combining spatial filtering and temporal filtering, we can more accurately detect salient areas in the video, thereby improving the accuracy of interest region extraction. This method can make full use of the information in the video, reduce the complexity of the algorithm, and improve the efficiency and quality of video processing. These processes can improve the accuracy and effectiveness of interest region extraction, specifically:

[0037] Step 3-2-1: Use Gaussian filter h(s,t) to process the image of the region of interest to achieve spatial smoothing:

[0038]

[0039] Wherein, h(s,t) represents an m×n Gaussian filter, a=(m-1) / 2, b=(n-1) / 2;

[0040] Step 3-2-2, perform motion mean filtering on the region of interest after spatial filtering to obtain an image of the region of interest after spatial and temporal filtering:

[0041]

[0042] Wherein, t represents the current frame, N represents the Nth frame before the current frame t, and m∈{1,2,3,....,N-1}.

[0043] Step 4: Adaptively encode and compress the video image according to the extracted interest area:

[0044] Step 4-1, dividing the image into the region of interest and the region of non-interest according to the position and size of the region of interest;

[0045] Step 4-2: Adopt adaptive coding strategies for different regions, that is, select different coding methods and compression ratios according to the importance of the image region. Use high image quality and low compression ratio coding methods for the interest region to retain more details; use low image quality and high compression ratio coding methods for non-interest regions to reduce the amount of data stored and transmitted.

[0046] Step 5: When decoding the video image, adaptive decoding and display recovery are performed according to the region of interest:

[0047] Decoding: Decoding the compressed video image to convert it from the compressed format back to the original pixel domain representation. The decoding process includes inverse transformation (such as inverse DCT), inverse quantization, entropy decoding and other steps;

[0048] Restoration display: Display or save the decoded image. At this time, the information of the area of ​​interest will be restored with higher quality, and the non-interest area will be restored according to the predetermined quality.

[0049] This embodiment is compared with the classic ROI extraction model (including four models: LTST, AGS, FRGNE, and PDB) to compare the average processing time per frame of each model video. The results are shown in Table 1. It can be seen that the processing speed of this method is at a relatively good level.

[0050] Table 1 Comparison of average processing time per frame for each ROI extraction model

[0051] Model LTST AGS FGR PDB OURS Duration (seconds) 1.40 0.1 0.19 0.05 0.02

[0052] This method is applied to the video compression process by collecting camera images and performing video on demand tests. The results are shown in Table 2. It can be seen that this method can reduce the video bit rate, the video quality evaluation index performs well, and the video compression processing delay is low, which can meet the requirements of real-time video compression processing. The test data of this solution is composed of the following table:

[0053] Table 2 Comparison of test video information parameters

[0054]

[0055] The solution of the present invention is aimed at the video compression processing process based on the region of interest. The region of interest is directly extracted from the encoded compressed domain and adaptively encoded to meet the video compression processing for the region of interest. It solves the defects of the previous region of interest extraction method based on the pixel domain, reduces the complexity of calculation, and improves the efficiency and accuracy of region of interest extraction.

[0056] In addition, the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0057] Step 1: Collect the original video image and perform encoding conversion;

[0058] Step 2: extracting compressed domain features from the encoded image;

[0059] Step 3: Determine the region of interest in the image based on the extracted compressed domain features;

[0060] Step 4: Adaptively encode and compress the video image according to the extracted region of interest;

[0061] Step 5: When decoding the video image, adaptive decoding and display restoration are performed according to the region of interest.

[0062] The present invention also provides a computer storable medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0063] Step 1: Collect the original video image and perform encoding conversion;

[0064] Step 2: extracting compressed domain features from the encoded image;

[0065] Step 3: Determine the region of interest in the image based on the extracted compressed domain features;

[0066] Step 4: Adaptively encode and compress the video image according to the extracted region of interest;

[0067] Step 5: When decoding the video image, adaptive decoding and display restoration are performed according to the region of interest.

[0068] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A video interest region extraction and improved compression method based on compression domain, characterized in that: The following steps are involved: Step 1: Collect the original video image and perform encoding conversion; Step 2: extracting compressed domain features from the encoded image; Step 3: Determine the region of interest in the image based on the extracted compressed domain features; Step 4: Adaptively encode and compress the video image according to the extracted region of interest; Step 5: When decoding the video image, adaptive decoding and display restoration are performed according to the region of interest.

2. The improved compression method for extracting video region of interest based on compression domain according to claim 1 is characterized in that: The step 1 of collecting the original video image and performing encoding conversion is specifically as follows: Firstly, the original video image data is obtained from the camera device; The collected original video images are encoded and converted based on the adopted video coding standard, converting them from the original pixel format to the compressed format. The encoding process includes image segmentation, transformation, quantization, and entropy coding.

3. The improved compression method for extracting video region of interest based on compression domain according to claim 1 is characterized in that: The step 2 of extracting compressed domain features from the encoded image is specifically as follows: Step 2-1, extracting compressed domain data from the video image after coding conversion, including DCT coefficients and quantization coefficients; Step 2-2: Based on the extracted compressed domain data, determine the compressed domain features of the video image, that is, the residual transform coefficient norm: Among them, (i,j) represents the i-th row and j-th column of the image area, It represents the discrete cosine transform coefficient of the i-th row and j-th column in the prediction residual area, M and N represent the range of the prediction residual area, p represents the number of frames of the current image, and R is the total number of video frames.

4. The improved compression method for extracting video interest regions based on compression domain according to claim 2 is characterized in that: The compressed domain features of the video image in step 2-2 need to be normalized after extraction: Among them, RTCN min With RTCN max They represent the minimum and maximum values ​​of the RTCN features of the current image respectively.

5. The improved compression method for extracting video interest regions based on compression domain according to claim 3 or 4, characterized in that: Determining the region of interest in the image in step 3 is specifically as follows: Step 3-1, based on the extracted compression domain features, by analyzing the differences in the feature data, the region of interest in the image is detected, that is, when the residual transform coefficient norm RTCN is greater than the set threshold, the region is considered to be the region of interest; Step 3-2: Post-process the region of interest determined in step 3-1, including spatial filtering and temporal filtering.

6. The improved compression method for extracting video interest regions based on compression domain according to claim 5 is characterized in that: The post-processing of the region of interest in step 3-2 is specifically as follows: Step 3-2-1: Use Gaussian filter h(s,t) to process the image of the region of interest to achieve spatial smoothing: Wherein, h(s,t) represents an m×n Gaussian filter, a=(m-1) / 2, b=(n-1) / 2; Step 3-2-2, perform motion mean filtering on the region of interest after spatial filtering to obtain an image of the region of interest after spatial and temporal filtering: Wherein, t represents the current frame, N represents the Nth frame before the current frame t, and m∈{1,2,3,....,N-1}.

7. The improved compression method for extracting video region of interest based on compression domain according to claim 1 is characterized in that: The step 4 of adaptively encoding and compressing the video image according to the extracted region of interest is specifically as follows: Step 4-1, dividing the image into the region of interest and the region of non-interest according to the position and size of the region of interest; Step 4-2: Adopt adaptive coding strategies for different regions, that is, select different coding methods and compression ratios according to the importance of the image region. Use high image quality and low compression ratio coding methods for the interest region to retain more details; use low image quality and high compression ratio coding methods for non-interest regions to reduce the amount of data stored and transmitted.

8. The improved compression method for extracting video interest regions based on compression domain according to claim 1 is characterized in that: The adaptive decoding and display restoration according to the region of interest in step 5 is specifically as follows: Decoding: Decoding the compressed video image, converting it from the compressed format back to the original pixel domain representation. The decoding process includes inverse transformation, inverse quantization, and entropy decoding. Restoration display: Display or save the decoded image. At this time, the information of the area of ​​interest will be restored with higher quality, and the non-interest area will be restored according to the predetermined quality.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer storable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images