A vvc code rate control optimization method for 360-degree video compression encoding

Abstract

The application provides a VVC code rate control optimization algorithm for 360-degree video compression and coding. First, the significant feature information of a video frame is extracted in the spatial and temporal domains. Then, the video frame is divided into a demand area and a non-demand area according to the position information and the significant feature of each CTU. Finally, an improved code rate control algorithm is proposed for the two areas, which can explicitly change the bit allocation strategy. Experimental results show that the code rate control optimization method disclosed by the application improves the △BR error , SSIM, BD rate and rate-distortion curve indicators compared with VVC, and the fluctuation degree of bit allocation of each frame is smoother. The video sequence obtained based on the application meets the needs of the human eye better, and the subjective quality can be greatly optimized.

Description

Technical Field

[0001] This invention relates to the problem of bitrate control optimization technology in the field of video coding, and in particular to a bitrate control optimization method for Versatile Video Coding (VVC) for 360° video compression coding. Background Technology

[0002] In recent years, streaming media, live streaming, and short videos have gradually entered people's daily lives, demonstrating the importance of video in today's information society. On the other hand, as people's demands for video quality and variety continue to increase, ultra-high-definition video, 360° panoramic video, and high-dynamic range (HDR) video have gradually come into view, greatly enriching people's daily lives.

[0003] In July 2020, the Joint Video Exploration Team (JVET) announced the formal standardization of the Versatile Video Coding (H.266 / VVC) standard. Compared to H.265 / HEVC, H.266 / VVC reduces data requirements by approximately 50% while maintaining the same resolution. This is due to the innovative technologies introduced in almost every module of the H.266 / VVC encoding standard, which are more suitable for compressing and encoding special videos such as 360° panoramic video and HDR video. In future research, there is still considerable potential for optimization in each module and technology of the H.266 / VVC standard, which warrants further in-depth discussion and research.

[0004] Rate control, as one of the core modules of the H.266 / VVC video coding standard, not only achieves the most accurate target bit allocation possible, but also plays a significant role in minimizing compressed video distortion, maintaining the consistency of compressed video quality, and reducing system latency. VVC optimizes the rate control model in HEVC by expanding the size of the Group of Pictures (GOP) for different configuration files, introducing the concepts of skip and non-skip regions, proposing an adaptive lambda bit prediction algorithm, and optimizing the parameter update process. Furthermore, H.266 / VVC also proposes a rate control algorithm dependent on the quality factor and a rate control algorithm based on the loop filtering module, further improving coding performance.

[0005] Furthermore, with the emergence of various head-mounted displays such as HTC's Vive, Samsung's VR (Virtual Reality), and Oculus Rift, an increasing number of 360° panoramic videos are being released online. Since a panoramic video is a bounded sphere encompassing the entire environment, a resolution of 4K or higher is required to provide a satisfactory visual experience. However, the high data volume resulting from high resolution has become one of the obstacles to the widespread use of 360° panoramic videos. Therefore, effective compression of 360° panoramic videos has become a key task. Previously, the Joint Video Exploration Group studied the potential need to incorporate panoramic video coding technology into next-generation video coding standards and proposed a basic approach to panoramic video compression coding. Although existing research teams have conducted some research on 360° video coding and obtained some excellent results, these achievements still cannot meet the future needs of 360° video coding. In particular, especially in the bitrate control module, the improvement in coding performance is ultimately limited without explicitly changing the bit allocation algorithm.

[0006] Furthermore, equirectangular projection (ERP) is widely used in 360° video encoding projection because it is computationally simpler and has stronger visual appeal compared to other projection formats. However, in ERP projection, the number of sampling points at the North and South Poles is the same as the number of sampling points at the equator. This not only leads to wasted bit allocation in areas near the poles, but also results in severe distortion in the projected 2D video in these areas. From another perspective, the area corresponding to eye level is near the equator; while in a spherical model, the North and South Poles correspond to directly above and below the observer, respectively. Based on human bodily habits, the human eye is more likely to observe the eye level area, i.e., the area near the center of the video frame. Only in very rare cases is it necessary to raise or lower the head by 90° for extended periods to observe the North and South Poles. Because the probability of these extreme cases is extremely low, observation shows that under most normal circumstances, the human eye has a relatively high demand for areas near the equator, while its demand for areas near the North and South Poles is relatively low. Therefore, in most conventional 360° panoramic videos, the area near the equator is of greater importance to the human eye than the areas near the North and South Poles, according to human visual habits.

[0007] To address the aforementioned issues, our research group conducted exploratory research and improved the bitrate control model in VVC. First, we extracted salient feature information in both the spatial and temporal domains from the video frames. Then, based on the location information and salient features of each CTU, the video frames were divided into demand and non-demand regions. Finally, for these two regions, we proposed a bitrate control algorithm for 360° video, which can explicitly change the bit allocation strategy. By making these improvements to the bitrate control model in VVC, we can improve the subjective quality and overall coherence of the video sequence while satisfying accurate target bit allocation, thereby achieving the goal of bitrate control optimization. This is precisely the objective of this invention. Summary of the Invention

[0008] This invention patent proposes a bitrate control optimization method for 360° video compression coding, specifically targeting the bitrate control model of VVC. The method includes the following steps:

[0009] (1) Extract spatial saliency features from the current encoded frame, as follows:

[0010] First, the global saliency feature information in the spatial domain is obtained using the video frame image signature method. The so-called image signature is the value of the video frame after DCT transformation and sign function, and its calculation process is shown in equation (1):

[0011] IS(Ys)=sign[DCT(Ys)] (1)

[0012] Where Ys represents the original video frame in the video sequence, IS(Ys) represents the image signature corresponding to Ys, DCT(Ys) represents the DCT transform performed on the video frame, and sign[·] represents the sign function, which reassigns the input value to 1 or -1 depending on whether it is positive or negative, as follows:

[0013]

[0014] The obtained image signature is subjected to inverse DCT transformation and then Hadamard product operation, as shown in Equation (3), so as to obtain global salient feature information.

[0015] S SG =IDCT(IS(Ys))⊙IDCT(IS(Ys)) (3)

[0016] Among them, S SG This represents the spatial global saliency feature of the current frame, where IDCT(·) represents the inverse DCT transform, and the symbol ⊙ represents the Hadamard product operation. At this point, the spatial global saliency feature information of the current frame has been extracted.

[0017] (2) Extract local spatial saliency features from the current frame, as follows:

[0018] The extraction process of local saliency feature information also uses DCT transformation. The original video frame is divided into several non-overlapping 4×4 region blocks. Then, DCT transformation is performed on each region block to obtain 15 AC coefficients and 1 DC coefficient. The low-frequency AC coefficient can represent the texture feature information of the current 4×4 region block, while the DC coefficient can represent the brightness feature of the current 4×4 region block. In addition, the regional saliency feature of the current 4×4 region block is highly correlated with the saliency feature of the surrounding eight blocks. These eight related region blocks R are located in the upper left R1, upper R2, upper right R3, left R4, right R5, lower left R6, lower R7 and lower right R8 directions of the central block C. Therefore, when extracting the saliency feature information of the current central block C, the DC coefficient and AC coefficient information of the surrounding eight related blocks are also required. In summary, the DC feature extraction and AC feature extraction of the current 4×4 region block are shown in equations (4) and (5):

[0019]

[0020]

[0021] Among them, S SL (Y,DC,C) and S SL (Y, AC, C) represent the saliency features of the current central region block, and N represents the saliency features of the current central region block. L This represents the sum of the number of central blocks and associated blocks. Y DC00,c and Y DC00,i Y represents the DC coefficient of the central block and the associated block, respectively. ACmn,c and Y ACmn,i Let S represent the communication coefficients of the central block and the associated blocks, respectively. Therefore, the local saliency feature S of the current 4×4 region block... SL It can be obtained by calculation using equation (6):

[0022] S SL =γ Y,DC ×S SL (Y,DC)+γ Y,AC ×S SL (Y,AC) (6)

[0023] Among them, S SL (Y,DC) and S SL (Y, AC) represent the saliency information of the current block's DC and AC values, respectively, γ Y,DC and γ Y,ACThe weighting coefficients for the proportions of DC and AC information in the current block are 0.4 and 0.6, respectively. At this point, all global and local saliency features of the current frame image have been extracted. Therefore, the spatial saliency feature information S of the current frame... S The calculation process is as follows:

[0024] S S =α×S SG +β×S SL (7)

[0025] Among them, S SG and S SL These are the global and local salient feature information of the current frame, respectively, and α and β are the corresponding weight coefficients, both of which are 0.5 in this invention;

[0026] (3) Extract temporal saliency features from the current frame and combine the spatiotemporal saliency features to obtain the saliency features of the current frame, and obtain the saliency features of each CTU, as follows:

[0027] If only the salient feature information in the spatial domain (pixel domain) is extracted, the improved algorithm will not be applicable to video sequences with violent motion or scene switching, resulting in the proposed algorithm lacking universality. Therefore, the extraction of temporal salient feature information is particularly important in the establishment of the proposed algorithm. The square of the pixel difference between the corresponding positions of the current frame and the reference frame is taken as the temporal salient feature information of the current frame, as shown in Equation (8).

[0028] S T (i,j)=[Y S (i,j)-Y Sref (i,j)] 2 (8)

[0029] Among them, Y S (i,j) and Y Sref (i,j) represent the pixel values ​​at position (i,j) in the current frame and the reference frame, respectively. T (i,j) represents the temporal saliency feature information at position (i,j).

[0030] Finally, the spatial and temporal saliency feature information obtained above is converted into 4×4 channel information, and the spatial saliency feature value SS of the j-th CTU in the current frame is calculated according to equations (9) and (10). CTU (j) and the time-domain significance eigenvalue ST CTU (j):

[0031]

[0032]

[0033] Where, N s,j S represents the number of 4×4 feature blocks contained in the j-th CTU of the current frame. S (i|j) represents the spatial saliency feature information of the i-th 4×4 feature block in the j-th CTU, S T (i|j) represents the spatial saliency feature information of the i-th 4×4 feature block in the j-th CTU. Then the saliency feature SST of the current CTU is... CTU (j) consists of spatial saliency features and temporal saliency features, and is as follows:

[0034] SST CTU (j)=ε1SS CTU (j)+ε2ST CTU (j) (11)

[0035] The average saliency feature of the current frame is SST. avgCTU :

[0036]

[0037] If the saliency feature of the current CTU is greater than the average saliency feature of the current frame, then the current CTU is a salient region; otherwise, it is a non-salient region.

[0038] (4) Since the region near the equator is relatively more important than the regions at the poles, the current frame is divided into demand regions and non-demand regions based on the above salience characteristics and CTU location information:

[0039] Assume the center pixel Y of the current j-th CTU j The two-dimensional coordinates are (w, h), and the center point Y of the current video frame is... c The two-dimensional coordinates are (W / 2, H / 2), where W and H are the width and height of the video sequence, respectively. Then, the vertical distance D of each CTU relative to the center point is... V for:

[0040]

[0041] Assumption The threshold value for the vertical distance is:

[0042]

[0043] Where μ V The corresponding weighting coefficient is set to 0.8. If the current CTU's D... V If the value is greater than the corresponding threshold, the CTU is located in the region near the North and South Poles; otherwise, the current CTU is located in the region near the equator of the video frame.

[0044] Therefore, the proposed method for determining the region division is as follows: if the current CTU is a significant region and is located near the equator of the video frame, then the current CTU is determined to be a demand region; otherwise, it is a non-demand region.

[0045] (5) Perform CTU-level bit allocation for both demand and non-demand areas. In this specification, the allocation strategy for the demand area is taken as an example, as follows:

[0046] First, determine if the current CTU is a demand region. If the current CTU is a demand region, then based on the spatial-temporal saliency characteristic (SST) of the CTU... CTU (j) Obtain the remaining number of bits in the required region (BitLeft). de-area

[0047]

[0048] Among them, T curFrame N represents the target number of bits in the current frame. CTUde-area Indicates the number of demand regions (CTUs) in the current frame, Bit coded This indicates the number of bits that have been encoded and consumed.

[0049] Secondly, target bits are allocated based on the spatial saliency characteristics of each CTU within the demand region. The proposed algorithm allocates target bits based on the total spatial saliency characteristic value SS of the demand CTU. allCTUde-area And the current CTU's spatial characteristic value SS cuiCTU Calculate the spatial saliency feature weight (SSW) of the current CTU. curCTU The process is shown in equations (16) to (17):

[0050]

[0051]

[0052] Finally, based on the current CTU's spatial saliency feature weight SSW curCTU And the remaining bits in the demand area BitLeft de-area To calculate the target number of bits T of the current CTU curCTU

[0053] T curCTU =SSW curCTU BitLeft de-area (18)

[0054] Analysis shows that SSW curCTUThis represents the spatial saliency feature weight corresponding to the current CTU. The larger the value, the higher the saliency of the current CTU, the higher the importance that the human eye attaches to it, and the more bits are allocated to it. Conversely, the smaller the value, the lower the saliency of the current CTU, the lower the importance that the human eye attaches to it, and the fewer bits are allocated to it. Therefore, the above process can well reflect the relationship between human vision and saliency features.

[0055] (6) Encode the current CTU. If the current frame encoding is finished, directly encode the next frame. Otherwise, enter (3) again until the entire video sequence encoding is finished.

[0056] The advantages and beneficial technical effects of this invention compared with the prior art are as follows:

[0057] (1) The present invention proposes a VVC bitrate control optimization method for 360° video compression coding. Compared with the default bitrate control algorithm of VVC, the coding performance of the bitrate control method proposed in this invention is higher than that of the bitrate control model corresponding to the standard VVC algorithm.

[0058] (2) The present invention proposes a VVC bit rate control optimization method for 360° video compression coding, which utilizes the characteristics of human eye observation habits, combines saliency features and CTU position information to guide the CTU-level target bit allocation, thereby controlling the bit error within a small range and obtaining high subjective quality. Attached Figure Description

[0059] Figure 1 The flowchart shows the overall process of optimizing VVC bitrate control for 360° video compression coding.

[0060] Figure 2 The rate-distortion curves of the sequences DrivingInCity and KiteFlite based on the method of this invention are compared with those of VVC.

[0061] Figure 3 A comparison chart of bit fluctuation per frame consumption for the sequences DrivingInCity and KiteFlite based on the method of this invention and VVC.

[0062] Figure 4 The image shows a subjective comparison of frames 47 to 50 of the AerialCity sequence based on the method of this invention and VVC (VVC on the left and the optimized method of this invention on the right). Detailed Implementation

[0063] The present invention will be further described in detail below with reference to the embodiments. It should be noted that the following embodiments are only used to further illustrate the present invention and should not be construed as limiting the scope of protection of the present invention. Those skilled in the art can make some non-essential improvements and adjustments to the present invention based on the above-described invention, and these improvements and adjustments should still fall within the scope of protection of the present invention.

[0064] (1) The method proposed in this invention is implemented on the VVC standard test code VTM10.0 and 360Lib platform. The configuration file is encoder_lowdelay_P_main.cfg, and in the bitrate control parameter configuration, RateControl is set to 1, indicating that bitrate control is enabled;

[0065] (2) The test sequences for verifying the coding performance of the method proposed in this invention were two types of official standard test sequences with resolutions of 3840×1920 and 8192×4096.

[0066] (3) In the VVC standard algorithm program, set RateControl to 0 in the configuration file encoder_lowdelay_P_main.cfg to disable bitrate control, and then input all the standard video sequences to be tested to obtain the target bitrate of each sequence under the condition of disabling bitrate control.

[0067] (4) In the program of the algorithm proposed in this invention, the parameters in (1) are set in the configuration file encoder_lowdelay_P_main.cfg, and the TargetBitrate is set to the target bitrate corresponding to each sequence in (3). Then, all the standard video sequences to be tested are input to obtain the results of the algorithm proposed in this invention. The actual bitrate, the spherically weighted uniform peak signal-to-noise ratio (WS-PSNR), and the number of bits consumed in each frame are recorded and statistically analyzed.

[0068] (5) In the VVC default algorithm program, the parameters in (1) are set in the configuration file encoder_lowdelay_P_main.cfg, and the TargetBitrate is set to the target bitrate corresponding to each sequence in (3). Then, all the standard video sequences to be tested are input to obtain the results of the VVC default algorithm. The actual bitrate, the spherically weighted uniform peak signal-to-noise ratio (WS-PSNR), and the number of bits consumed in each frame are recorded and statistically analyzed.

[0069] (6) Process the results of the above three categories respectively, where the unit of bit rate is Kbps; calculate the bit error ΔBR of the present invention and VVC respectively using equation (19). error The smaller the index, the better; use equation (20) to calculate the improvement in structural complexity of the present invention relative to VVC, and the larger the index, the better;

[0070]

[0071] ΔSSIM=SSIM pro -SSIM VVC (20)

[0072] (7) Table 1 shows a performance comparison between VVC and the present invention. As can be seen from Table 1, the average ΔBR of the proposed algorithm... error The proposed algorithm achieves only about 30% of the bitrate control efficiency of the default VTM-360Lib algorithm, with an average SSIM improvement of 0.00845, demonstrating the accuracy and rationality of its bit allocation strategy. Furthermore, under the same WS-PSNR conditions, the proposed algorithm saves an average of 31.650% of the bitrate compared to the default VTM+360Lib algorithm. This indicates that compared to the standard algorithm, the proposed algorithm achieves better BD-Rate performance. w It has certain advantages in terms of evaluation indicators;

[0073] (8) Figure 2 This indicates that regardless of whether it is a high bitrate or a low bitrate range, the rate-distortion curve of the proposed algorithm is higher than that of the VVC standard algorithm. That is, under the same coding bitrate, the WS-PSNR of the video reconstructed by the proposed algorithm after compression is higher than that of the H.266 / VVC+360Lib standard algorithm, and its coding performance is better than that of the standard algorithm across the entire bitrate range.

[0074] (9) Figure 3 This indicates that under the standard VVC algorithm, the number of bits consumed between consecutive frames can fluctuate greatly, which may lead to extreme situations where the quality of a certain frame suddenly deteriorates or the quality of a certain frame suddenly improves, thus affecting the quality of the overall video sequence. However, under the proposed algorithm, the actual number of bits consumed between consecutive frames changes slowly, so the proposed algorithm can better meet the needs of human vision.

[0075] (10) In Figure 4In (a) and (b), the left side shows the subjective quality of the compressed and reconstructed video based on the VVC standard algorithm, while the right side shows the subjective quality of the compressed and reconstructed video based on the proposed algorithm. It is clearly visible that the proposed algorithm outperforms the VVC standard algorithm in areas such as the mast's edge, lake ripples, building tiles, and roads. Therefore, compared to the VVC standard algorithm, the proposed algorithm improves the subjective quality of the reconstructed video to a certain extent.

[0076] Table 1 Performance comparison of the present invention and VVC

[0077]

Claims

1. A VVC bitrate control optimization method for 360° video compression coding, characterized in that: (1) The global saliency feature information in the spatial domain is obtained by using the video frame image signature method. The so-called image signature is the value of the video frame after DCT transformation and sign function. The calculation process is shown in Equation (1): IS(Ys)=sign[DCT(Ys)] (1) Where Ys represents the original video frame in the video sequence, IS(Ys) represents the image signature corresponding to Ys, DCT(Ys) represents the DCT transform performed on the video frame, and sign[·] represents the sign function, which reassigns the input value to 1 or -1 depending on whether it is positive or negative, as follows: The obtained image signature is subjected to inverse DCT transformation, followed by Hadamard product operation, as shown in Equation (3), to obtain global salient feature information: S SG =IDCT(IS(Ys))⊙IDCT(IS(Ys)) (3) Among them, S SG This represents the spatial global saliency feature of the current frame. IDCT(·) represents the inverse DCT transform, and the symbol ⊙ represents the Hadamard product operation. At this point, the spatial global saliency feature information of the current frame has been extracted. (2) Extract local spatial saliency features of the current frame. The extraction process of local saliency feature information is also carried out by DCT transformation. The original video frame is divided into several non-overlapping 4×4 region blocks. Then, DCT transformation is performed on each region block to obtain 15 AC coefficients and 1 DC coefficient. The low-frequency AC coefficient can represent the texture feature information of the current 4×4 region block, while the DC coefficient can represent the brightness feature of the current 4×4 region block. In addition, the regional saliency features of the current 4×4 region block are highly correlated with the saliency features of the surrounding eight blocks. These eight related region blocks R are located in the left upper R1, upper R2, right upper R3, left R4, right R5, left lower R6, lower R7 and right lower R8 directions of the center block C. Therefore, when extracting the saliency feature information of the current center block C, the DC coefficient and AC coefficient information of the surrounding eight related blocks are also required. In summary, the DC feature extraction and AC feature extraction of the current 4×4 region block are shown in equations (4) and (5): Among them, S SL (Y,DC,C) and S SL (Y, AC, C) represent the saliency features of the current central region block, and N represents the saliency features of the current central region block. L Y represents the sum of the number of central blocks and associated blocks. DC00,c and Y DC00,i Y represents the DC coefficient of the central block and the associated block, respectively. ACmn,c and Y ACmn,i Let S represent the communication coefficients of the central block and the associated block, respectively. Therefore, the local saliency feature S of the current 4×4 region block... SL It can be obtained by calculation using equation (6): S SL =c Y,DC ×S SL (Y,DC)+γ Y,AC ×S SL (Y,AC) (6) Among them, S SL (Y,DC) and S SL (Y, AC) represent the saliency information of the current block's DC and AC values, respectively, γ Y,DC and γ Y,AC The weighting coefficients for the proportions of DC and AC information in the current block are 0.4 and 0.6, respectively. At this point, all global and local saliency features of the current frame image have been extracted. Therefore, the spatial saliency feature information S of the current frame... S The calculation process is as follows: S S =α×S SG +β×S SL (7) Among them, S SG and S SL These are the global and local salient feature information of the current frame, respectively, and α and β are the corresponding weight coefficients, both of which are 0.5 in this invention; (3) Extract temporal saliency features from the current frame and combine the spatiotemporal saliency features to obtain the saliency features of the current frame, and obtain the saliency features of each CTU, as follows: The square of the pixel difference between the corresponding positions in the current frame and the reference frame is used as the temporal saliency feature information of the current frame, as shown in Equation (8): S T (i,j)=[Y S (i,j)-Y Sref (i,j)] 2 (8) Among them, Y S (i,j) and Y Sref (i,j) represent the pixel values ​​at position (i,j) in the current frame and the reference frame, respectively. T (i,j) represents the temporal saliency feature information at position (i,j). Finally, the spatial and temporal saliency feature information obtained above is converted into 4×4 channel information, and the spatial saliency feature value SS of the j-th CTU in the current frame is calculated according to equations (9) and (10). CTU (j) and the time-domain significance eigenvalue ST CTU (j): Where, N s,j S represents the number of 4×4 feature blocks contained in the j-th CTU of the current frame. S (i|j) represents the spatial saliency feature information of the i-th 4×4 feature block in the j-th CTU, S T (i|j) represents the spatial saliency feature information of the i-th 4×4 feature block in the j-th CTU, then the saliency feature SST of the current CTU is... CTU (j) consists of spatial saliency features and temporal saliency features, and is as follows: SST CTU (j)=ε1SS CTU (j)+ε2ST CTU (j) (11) The average saliency feature of the current frame is SST. avgCTU : If the saliency feature of the current CTU is greater than the average saliency feature of the current frame, then the current CTU is a salient region; otherwise, it is a non-salient region. (4) Since the region near the equator is relatively more important than the regions at the poles, the current frame is divided into demand regions and non-demand regions based on the above salience characteristics and CTU location information: Assume the center pixel Y of the current j-th CTU j The two-dimensional coordinates are (w, h), and the center point Y of the current video frame is... c The two-dimensional coordinates are (W / 2, H / 2), where W and H are the width and height of the video sequence, respectively. Then, the vertical distance D of each CTU relative to the center point is... V for: Assumption The threshold value for the vertical distance is: Where μ V The corresponding weighting coefficient is set to 0.

8. If the current CTU's D V If the value is greater than the corresponding threshold, then the CTU is located in the region near the North and South Poles; otherwise, the current CTU is located in the region near the equator of the video frame. Thus, the proposed method for region division is as follows: if the current CTU is a significant region and is located in the region near the equator of the video frame, then the current CTU is determined to be a demand region; otherwise, it is a non-demand region. (5) Perform CTU-level bit allocation for both demand and non-demand areas. Taking the allocation strategy for the demand area as an example, the details are as follows: First, determine if the current CTU is a demand region. If the current CTU is a demand region, then based on the spatial-temporal saliency characteristic (SST) of the CTU... CTU (j) Obtain the remaining number of bits in the required region (BitLeft). de-area Among them, T curFrame N represents the target number of bits in the current frame. CTUde-area Indicates the number of demand regions (CTUs) in the current frame, Bit coded This represents the number of bits already encoded and consumed. Secondly, based on the spatial saliency characteristics of each CTU within the demand region, target bits are allocated. The proposed algorithm allocates target bits based on the total spatial saliency characteristic value SS of the demand CTU. allCTUde-area And the current CTU's spatial characteristic value SS cuiCTU Calculate the spatial saliency feature weight (SSW) of the current CTU. curCTU The process is shown in equations (16) to (17): Finally, based on the current CTU's spatial saliency feature weight SSW curCTU And the remaining bits in the demand area BitLeft de-area To calculate the target number of bits T of the current CTU curCTU T curCTU =SSW curCTU ·BitLeft de-area (18) Analysis shows that SSW curCTU This represents the spatial saliency feature weight corresponding to the current CTU. The larger the value, the higher the saliency of the current CTU, the higher the importance that the human eye attaches to it, and the more bits are allocated to it. Conversely, the smaller the value, the lower the saliency of the current CTU, the lower the importance that the human eye attaches to it, and the fewer bits are allocated to it. Therefore, the above process can well reflect the relationship between human vision and saliency features. (6) Encode the current CTU. If the current frame encoding is finished, directly encode the next frame. Otherwise, enter (3) again until the entire video sequence encoding is finished.

Images