Video compression optimization method based on human vision region of interest
By performing feature extraction and attention evaluation on video images, the H265 encoding process is optimized, and the problem of H265 encoding is solved, which is fast and efficient video encoding, and is suitable for real-time video applications.
Patent Information
- Application Number
- CN202411013879.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-26
- Publication Date
- 2024-11-15
AI Technical Summary
The H265 encoding algorithm takes a long time in video processing, resulting in the inability to realize real-time encoding, especially in video conferencing and live video broadcast scenarios.
通过对视频图像进行分割、提取纹理特征、位置特征和运动特征,计算关注度系数,判断视觉感兴趣区域,并在H265压缩过程中,对视觉感兴趣区域采用预先定义的四叉树划分算法, The encoding process is terminated early for non-visual areas of interest.
It significantly reduces the encoding time of non-visual area of interest blocks, shortens the entire H265 compression process, improves the video encoding speed, and ensures that the encoding quality is not significantly affected.
Smart Images

Figure CN118972597A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of video processing, and in particular relates to a video compression optimization method based on human visual interest areas. Background Art
[0002] The H265 encoding algorithm is widely used in the field of video processing. In the H265 encoding process, the encoding is layered and the cost value of the encoding unit is recursively calculated. If better encoding is achieved, more time is required for algorithm calculation. Therefore, under normal circumstances, H265 has some disadvantages in video compression processing: encoding takes a long time and real-time encoding cannot be performed. Since H265 consumes too much time in encoding, the encoding capability of H265 cannot be used in real-time video conferencing, video live broadcast and other scenarios. Summary of the invention
[0003] In view of the above problems, the purpose of the present invention is to provide a video compression optimization method based on human visual area of interest, which utilizes the determination of human visual area of interest and applies it to the video compression process of H265, thereby improving the processing speed of H265 during video compression.
[0004] The specific technical solution for achieving the purpose of the present invention is:
[0005] A video compression optimization method based on human visual interest area includes the following steps:
[0006] Step 1: Segment the image in the video into multiple sub-blocks;
[0007] Step 2: Extract texture features of the image;
[0008] Step 3: Extract the position features of the image;
[0009] Step 4: Determine the motion characteristics and static characteristics of the image;
[0010] Step 5: Determine the attention coefficient of the image based on the result of step 4;
[0011] Step 6: Based on the attention coefficient of the image, determine whether it is an area of human visual interest;
[0012] Step 7: Use the H265 compression algorithm to compress the video image. For the visual interest area, the pre-defined H265 quadtree partitioning algorithm is used to compress the image. For the non-visual interest area block, the encoding process is terminated in advance during the H265 compression algorithm process, and finally the video image compression is completed.
[0013] Compared with the prior art, the present invention has the following beneficial effects:
[0014] The present invention corrects the quadtree division method of the H265 encoder in terms of position features, texture features and motion features of the human visual system; by cutting off the coding of non-interested areas and retaining the block details of the visually interesting area, the fast encoding of the video is achieved, while ensuring that the user cannot feel obvious distortion;
[0015] By modifying the quadtree partitioning process of H265, the improved encoder keeps the depth of the coding tree unit unchanged when detecting that the block to be encoded is a visual region of interest block. For non-visual region of interest blocks, we set the depth of the coding tree unit to 0, that is, the quadtree partitioning of the current block is terminated in advance. This not only reduces the encoding time of non-visual region of interest blocks, but also greatly shortens the entire H265 compression process.
[0016] The present invention is further described below in conjunction with specific implementation modes. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 It is a flow chart of the video compression optimization method based on human visual interest area of the present invention.
[0018] Figure 2 Schematic diagram of extracting image texture features in an embodiment of the present invention.
[0019] Figure 3 The figure is a schematic diagram comparing the effects of video compression based on four different algorithms in an embodiment of the present invention. DETAILED DESCRIPTION
[0020] Example
[0021] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0022] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an kind" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.
[0023] Unless otherwise specifically stated, the relative arrangement, numerical expressions and numerical values of the parts and steps set forth in these embodiments do not limit the scope of the present application. At the same time, it should be understood that, for ease of description, the sizes of the various parts shown in the accompanying drawings are not drawn according to the actual proportional relationship. The technology, method and equipment known to those of ordinary skill in the relevant field may not be discussed in detail, but in appropriate cases, the technology, method and equipment should be considered as a part of the authorization specification. In all examples shown and discussed here, any specific value should be interpreted as being merely exemplary, rather than as a limitation. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so that once a certain item is defined in an accompanying drawing, it does not need to be further discussed in subsequent drawings.
[0024] There are indeed some significant differences in the processing of static and dynamic scenes by the human visual system, and these differences reflect the characteristics of the visual region of interest (ROI). In static scenes, people tend to focus more on high-contrast, salient objects or areas, such as objects with clear edges or colors or shapes that are different from the surrounding background. These characteristics enable the human eye to quickly identify and focus on important information, and this mechanism is crucial for extracting target objects from complex backgrounds. Through eye tracking technology, we can record the eye movement trajectories of subjects when viewing static and dynamic scenes. Studies have shown that in static scenes, eyes tend to stay on high-contrast, salient objects or areas, while in dynamic scenes, eyes tend to track moving objects or parts that change in motion.
[0025] We have studied the human visual system in terms of position, texture and motion features, and established a model algorithm to correct the quadtree partitioning method of the H265 encoder. The algorithm achieves fast encoding of the video by cutting off the encoding of non-interesting areas and retaining the block details of the visual interest area, while ensuring that users cannot feel obvious distortion.
[0026] Combination Figure 1 , a video compression optimization method based on human visual interest area, characterized in that it includes the following steps:
[0027] Step 1: Segment the image in the video into multiple sub-blocks:
[0028] The image is segmented into multiple sub-blocks, and the coordinates of each sub-block are determined separately, and the coordinates of the upper left corner of the image are set to (0,0), that is, the origin. The upper edge of the image represents the x-axis, and the left side represents the y-axis; loop through each sub-block according to the coordinates, and add 1 to the position point in the corresponding pixel array.
[0029] Step 2: Extract texture features of the image:
[0030] In order to comprehensively consider the sensitivity of the human visual system to position information and texture information, this patent adopts a new calculation method to determine the texture features in the static attention coefficient of the image, that is, using entropy as the attention coefficient and combining the coordinates of the block as the weight coefficient to comprehensively consider the position and texture features;
[0031] Traverse the frequency of occurrence of pixels in each sub-block and determine the entropy of each image block as its texture feature:
[0032]
[0033] Among them, P i It represents the probability of gray level i appearing in the word block, and n represents the number of gray levels set.
[0034] Step 3: The human visual system's perception of spatial position is not linear, but rather shows a characteristic of being more sensitive to some positions. Generally speaking, the weight of positions farther from the visual center will decay faster, which is consistent with the center preference and visual sensitivity distribution of the human eye. Based on this, the position features of the image are extracted:
[0035]
[0036] Among them, w(i,j) represents the position feature value of the sub-block with coordinates (i,j), x i Represents the horizontal coordinate of the i-th point, y i Represents the ordinate of point i, and (midx, midy) represents the coordinates of the central area of the image.
[0037] Step 4: Determine the motion and static features of the image:
[0038] Step 4-1: First, according to the video playback order, determine the motion characteristics of the current block based on the frame difference method:
[0039] d(i,j)=|f k (i,j)-f k-1 (i,j)|
[0040]
[0041] Among them, f k (i, j) represents the sum of the pixels of the sub-block image with coordinates (i, j) in the kth frame, w d (i, j) is the dynamic eigenvalue, F(i, j) is the sum of the residual matrix, M and N are the number of pixels in the horizontal and vertical directions of the block;
[0042] Step 4-2: Determine the static characteristics of the current block:
[0043] w s (i,j)=H×w(i,j)
[0044] Among them, w(i,j) represents the position feature value of the sub-block with coordinates (i,j), and H represents the texture feature value of the sub-block.
[0045] Step 5: Determine the image's attention coefficient based on the result of step 4:
[0046] w z =αw s (z)+(1-α)w d (z)
[0047] Among them, w s (z) represents the static eigenvalue of the z-th sub-block after normalization, w d (z) represents the normalized dynamic feature value of the zth block, and α represents the weight factor between static features and dynamic features. By adjusting the value of α, the importance of static and dynamic features in image processing can be adjusted according to needs.
[0048] Step 6: Based on the attention coefficient of the image, determine whether it is an area of human visual interest:
[0049] According to the attention coefficient threshold w thr Determine whether each sub-block is a region of human visual interest;
[0050] The attention coefficient threshold w thr for:
[0051]
[0052] Where n represents the number of sub-blocks in the image.
[0053] When the attention coefficient of a block is greater than (w thr ), it is marked as a visual region of interest block, otherwise it is a non-visual region of interest block.
[0054] Step 7, use the H265 compression algorithm to compress the video image, wherein, for the visual interest area, the pre-defined H265 quadtree partitioning algorithm is used to compress the image, so as to ensure that the high-quality detail expression can still be obtained for the area of interest to the user; for the non-visual interest area block, its encoding process is terminated in advance during the H265 compression algorithm process, and finally the video image compression is completed, so as to improve the overall compression efficiency while maintaining the subjective feeling.
[0055] In summary, we have achieved a speed-up improvement of the H265 compression algorithm by setting the attention coefficient threshold and adopting different compression measures, while ensuring that the user's subjective experience is not affected. level ) is calculated as follows:
[0056]
[0057] By modifying the quadtree partitioning process of H265, the improved encoder keeps the depth of the coding tree unit unchanged when detecting that the block to be encoded is a visual region of interest block. For non-visual region of interest blocks, we set the depth of the coding tree unit to 0, that is, the quadtree partitioning of the current block is terminated in advance. This not only reduces the encoding time of non-visual region of interest blocks, but also greatly shortens the entire H265 compression process.
[0058] Combination Figure 3 For the algorithm, we use comparative tests and set up four groups of algorithms for comparison: A is the HM16.0 standard algorithm; B algorithm uses the moving part in the video as the region of interest to speed up the HEVC encoder; C algorithm is the RDO model; D is to quickly encode the motion information of visual interest through human visual habit analysis.
[0059] Through comparative experiments, this patent (D algorithm) uses a variety of visual interest area factors to calculate the attention coefficient to process the moving and static parts of the video. Therefore, it can save about 30% of the encoding time on average. In addition, the algorithm of this patent is more robust and can cope with video encoding requirements in various situations. In general, through the above optimization and improvement, this patent successfully improves the compression speed of H265.
[0060] The present invention also provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:
[0061] Step 1: Segment the image in the video into multiple sub-blocks;
[0062] Step 2: Extract texture features of the image;
[0063] Step 3: Extract the position features of the image;
[0064] Step 4: Determine the motion characteristics and static characteristics of the image;
[0065] Step 5: Determine the attention coefficient of the image based on the result of step 4;
[0066] Step 6: Based on the attention coefficient of the image, determine whether it is an area of human visual interest;
[0067] Step 7: Use the H265 compression algorithm to compress the video image. For the visual interest area, the pre-defined H265 quadtree partitioning algorithm is used to compress the image. For the non-visual interest area block, the encoding process is terminated in advance during the H265 compression algorithm process, and finally the video image compression is completed.
[0068] The present invention also provides a computer storable medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:
[0069] Step 1: Segment the image in the video into multiple sub-blocks;
[0070] Step 2: Extract texture features of the image;
[0071] Step 3: Extract the position features of the image;
[0072] Step 4: Determine the motion characteristics and static characteristics of the image;
[0073] Step 5: Determine the attention coefficient of the image based on the result of step 4;
[0074] Step 6: Based on the attention coefficient of the image, determine whether it is an area of human visual interest;
[0075] Step 7: Use the H265 compression algorithm to compress the video image. For the visual interest area, the pre-defined H265 quadtree partitioning algorithm is used to compress the image. For the non-visual interest area block, the encoding process is terminated in advance during the H265 compression algorithm process, and finally the video image compression is completed.
[0076] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.
Claims
1. A video compression optimization method based on human visual interest area, characterized in that: The following steps are involved: Step 1: Segment the image in the video into multiple sub-blocks; Step 2: Extract texture features of the image; Step 3: Extract the position features of the image; Step 4: Determine the motion characteristics and static characteristics of the image; Step 5: Determine the attention coefficient of the image based on the result of step 4; Step 6: Based on the attention coefficient of the image, determine whether it is an area of human visual interest; Step 7: Use the H265 compression algorithm to compress the video image. For the visual interest area, the pre-defined H265 quadtree partitioning algorithm is used to compress the image. For the non-visual interest area block, the encoding process is terminated in advance during the H265 compression algorithm process, and finally the video image compression is completed.
2. The video compression optimization method based on human visual interest area according to claim 1 is characterized in that: The step 1 of segmenting the image in the video is specifically as follows: The image is segmented into multiple sub-blocks, and the coordinates of each sub-block are determined respectively.
3. The video compression optimization method based on human visual interest area according to claim 2 is characterized in that: The texture features of the extracted image in step 2 are specifically: Traverse the frequency of occurrence of pixels in each sub-block and determine the entropy of each image block as its texture feature: Among them, P i It represents the probability of gray level i appearing in the word block, and n represents the number of gray levels set.
4. The video compression optimization method based on human visual interest area according to claim 2 is characterized in that: The position features of the extracted images in step 3 are specifically: Among them, w(i,j) represents the position feature value of the sub-block with coordinates (i,j), x i Represents the horizontal coordinate of the i-th point, y i Represents the ordinate of point i, and (midx, midy) represents the coordinates of the central area of the image.
5. The video compression optimization method based on human visual interest area according to claim 2 is characterized in that: The step 4 of extracting the motion features of the image is specifically as follows: Step 4-1: According to the video playback order, determine the motion characteristics of the current block based on the frame difference method: d(i,j)=|f k (i,j)-f k-1 (i,j)| Among them, f k (i, j) represents the sum of the pixels of the sub-block image with coordinates (i, j) in the kth frame, w d (i, j) is the dynamic eigenvalue, F(i, j) is the sum of the residual matrix, M and N are the number of pixels in the horizontal and vertical directions of the block; Step 4-2: Determine the static characteristics of the current block: w s (i,j)=H×w(i,j) Among them, w(i,j) represents the position feature value of the sub-block with coordinates (i,j), and H represents the texture feature value of the sub-block.
6. The video compression optimization method based on human visual interest area according to claim 5 is characterized in that: The attention coefficient w of the determined image in step 5 is specifically: In z =αw s (from)+(1-α)in d (With) Among them, w s (z) represents the static eigenvalue of the z-th sub-block after normalization, w d (z) represents the normalized dynamic feature value of the z-th sub-block, and α represents the weight factor between static features and dynamic features.
7. The video compression optimization method based on human visual interest area according to claim 6 is characterized in that: In step 6, according to the attention coefficient threshold w thr Determine whether each sub-block is a region of human visual interest; The attention coefficient threshold w thr for: Where n represents the number of sub-blocks in the image.
8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.
9. A computer storable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.
Citation Information
Cited By
Image compression method and device, equipment, medium and product
CN121284254A
