H.266 / VVC video coding method based on space-time visual perception

Abstract

The invention discloses an H.266 / VVC video coding method based on space-time visual perception. The H.266 / VVC video coding method comprises the steps of extracting space-time features and saliency features, determining perception distortion and determining perception QP. According to the method, the space and time perception features of the CTU in each frame of image are adopted to determine the saliency feature matrix and the saliency value of the CU, and the technical problem that an existing saliency-based compression coding method depends on selection of a saliency region, and excessively pays attention to the saliency region but ignores perceptible distortion of a non-saliency region is solved. A perception distortion model is constructed by performing linear weighting on saliency features and spatial-temporal features, more subjective visual perception redundancy is removed, the video compression effect is improved, the video compression time is shortened, the video compression effect is further improved by using a saliency adaptive QP selection method, and the video compression efficiency is improved. The method has the advantages that the method better conforms to visual features of audiences in the video field with complex motion and texture, and the like.

Description

Technical Field

[0001] The present invention belongs to the technical field of video processing, and in particular relates to an H.266 / VVC video encoding method based on spatiotemporal visual perception. Background Art

[0002] In recent years, with the development of high-speed communication technology, high-performance communication technology, high-performance computing technology and advanced intelligent algorithms / systems, new types of videos such as ultra-high definition (UHD), virtual reality (VR) and user-generated video (UGC) have attracted more and more attention. This development trend has also brought unprecedented challenges to video compression technology. Compared with the previous high-efficiency video coding (H.265 / HEVC), the versatile video coding (H.266 / VVC) standard has achieved significant compression performance improvements. However, relying solely on traditional video coding methods cannot fully utilize the human visual perception model to maximize compression efficiency and visual quality. Therefore, how to ensure efficient video compression while improving the quality of human subjective perception has become an urgent problem to be solved.

[0003] Traditional video coding technology mainly compresses video data by reducing spatiotemporal redundancy, and fails to fully utilize the characteristics of the human visual system (HVS) to further improve coding performance. In recent years, research on HVS has found that people's attention is often focused on specific salient areas in the field of vision, while other scenes are ignored. Therefore, some studies have combined salient features with video coding methods to effectively eliminate subjective perception redundancy. However, these methods only consider salient feature factors and are prone to distortion when processing fast motion or complex scenes. Summary of the invention

[0004] The main technical problem to be solved by the present invention is to overcome the shortcomings of the above-mentioned prior art and provide an H.266 / VVC video encoding method based on spatiotemporal visual perception with high video compression quality and high compression efficiency.

[0005] The technical solution adopted to solve the above technical problems consists of the following steps:

[0006] (1) Extracting spatiotemporal features and saliency features

[0007] According to formula (1), the normalized spatial perceptual feature Γ of the coding tree unit CTU block in each frame image is determined n :

[0008]

[0009] in, Represents the normalized spatial perception information of the coding tree unit CTU block, n∈{1,2,...,N}, N represents the number of video frames, and its value is a finite positive integer, h represents the number of rows of the coding tree unit CTU block, w represents the number of columns of the coding tree unit CTU block, and h and w are finite positive integers.

[0010] According to formula (2), the normalized temporal perceptual feature T of the coding tree unit CTU block in each frame image is determined n :

[0011]

[0012] in, Represents the normalized temporal perception information of the coding tree unit CTU block.

[0013] According to equations (3) and (4), the number of columns w and rows h of the coding tree unit CTU block in each frame image is determined as follows:

[0014]

[0015]

[0016] Wherein, [·] represents a rounding function, W represents the width of the video resolution, H represents the height of the video resolution, and W and H are finite positive integers.

[0017] According to formula (5), the spatiotemporal perceptual characteristics of the coding tree unit CTU block in each frame image are determined

[0018]

[0019] in, represents the normalized temporal perception information of the coding tree unit CTU block in the i-th row and j-th column in the n-th frame, Represents the normalized spatial perceptual information of the coding tree unit CTU block in the i-th row and j-th column in the n-th frame, i∈{1,2,...,w}, j∈{1,2,...,h}, w and h are finite positive integers.

[0020] Determine the saliency value S of the n×n coding unit CU block in the image according to formula (6): CU :

[0021]

[0022]

[0023] Among them, S(i,j) represents the salient feature matrix of the n×n coding unit CU block, i represents the horizontal coordinate in the n×n block, i∈{1,2,...,n}, j represents the vertical coordinate in the n×n block, j∈{1,2,...,n}, and n is 2 0 ~2 6 .

[0024] Determine the average saliency value S of the current frame according to formula (7): avg :

[0025]

[0026]

[0027] Among them, F n (i, j) represents the salient feature matrix in the nth frame image, i represents the abscissa of the saliency map of each frame, i∈{1,2,...,W}, j represents the ordinate of the saliency map of each frame, j∈{1,2,...,H}.

[0028] (2) Determining Perceptual Distortion

[0029] Determine the perceptual distortion D of the current coding unit CU block according to formula (8): p :

[0030] D p =(α×Π+β×SW+γ)×D MSE (8)

[0031]

[0032] Among them, D MSE is the distortion size of the multifunctional video coding standard; α, β, γ are all weight parameters, α∈(0,1), β∈(0,1), γ∈[0.5,1]; Π represents the spatiotemporal perception of each coding tree unit CTU block; SW represents the significance of each coding unit CU block, S cu Represents the significance value of the current coding unit CU block, S avg Represents the average saliency value of the current frame.

[0033] The final perceptual rate-distortion cost J is determined according to formula (9):

[0034] J=D p +λR (9)

[0035] Among them, D p represents the perceptual distortion of the current coding unit CU block, λ represents the Lagrange multiplier, and R represents the encoding bit rate.

[0036] (3) Determine the perceived QP

[0037] Determine the perceptual QP of the current coding unit CU block according to formula (10):

[0038] QP=QP f +round(μ×min(ω s ,1)) (10)

[0039]

[0040] Among them, QP f represents the perceptual QP of the current frame, S max Indicates the maximum significance value of the coding unit CU block, S avg represents the average saliency value of the current frame, S cu Represents the significance value of each coding unit CU block; μ and k are coefficients, μ∈[-10,-6], k∈[0.5,1].

[0041] In the formulas (3) and (4) for extracting spatiotemporal features and saliency features in step (1) of the present invention, W represents the width of the video resolution, H represents the height of the video resolution, W takes a value of 416 to 2560, H takes a value of 240 to 1600, and W>H.

[0042] In formulas (3) and (4) for extracting spatiotemporal features and saliency features in step (1) of the present invention, W represents the width of the video resolution, and H represents the height of the video resolution. The optimal value of W is 1080, and the optimal value of H is 720.

[0043] In the equation (8) for determining the perceptual distortion in step (2) of the present invention, the optimal value of α is 0.4, the optimal value of β is 0.6, and the optimal value of γ is 0.8.

[0044] In the equation (10) for determining the perceptual QP in step (3) of the present invention, μ and k are coefficients, the optimal value of μ is -8, and the optimal value of k is 0.7.

[0045] In the formula (6) for extracting spatiotemporal features and salient features in step (1) of the present invention, the S(i,j) represents the salient feature matrix of the n×n coding unit CU block, i represents the horizontal coordinate in the n×n block, i∈{1,2,...,n}, j represents the vertical coordinate in the n×n block, j∈{1,2,...,n}, and the optimal value of n is 2 3 .

[0046] The present invention combines the video coding method with the human visual system, removes the subjective visual perception redundancy, and improves the video compression effect while improving the subjective visual perception quality of the human eye. Not only the significant features are considered, but also the spatial features of the image content and the temporal features of the video sequence are introduced, effectively allocating bit rate resources during the coding process, and improving the coding efficiency and video quality.

[0047] The video restoration quality of the proposed method is quantitatively evaluated in detail through the video rate distortion curve. Compared with the VTM16.0 version, multiple indicators based on EWPSNR, including BD-EWPSNR, BD-PSNR, and BD-Rate, are used to measure the performance of compressed video. Among them, BD-PSNR is used to evaluate video quality, while BD-EWPSNR specifically considers the impact of visual attention on quality. The video maintains a high quality after compression, which improves the compression efficiency. The method of the present invention is compared with the existing perceptual coding method. The experimental results show that in a variety of typical scenarios, especially in fast motion and complex content scenarios, the method of the present invention is superior to the existing perceptual coding method. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a flow chart of Example 1 of the present invention.

[0049] Figure 2 This is the original image of the 68th frame of the video sequence BasketballPass

[0050] Figure 3 Compare the results of Experiment 1.

[0051] Figure 4 Effect diagram of embodiment 1 of the present invention. DETAILED DESCRIPTION

[0052] The present invention is further described below in conjunction with the accompanying drawings and examples, but the present invention is not limited to the following embodiments.

[0053] Example 1

[0054] The H.266 / VVC video encoding method based on spatiotemporal visual perception of this embodiment comprises the following steps (see Figure 1 ):

[0055] (1) Extracting spatiotemporal features and saliency features

[0056] According to formula (1), the normalized spatial perceptual feature Γ of the coding tree unit CTU block in each frame image is determined n :

[0057]

[0058] in, represents the normalized spatial perception information of the coding tree unit CTU block, n∈{1,2,...,N}, N represents the number of video frames, and its value is a finite positive integer, h represents the number of rows of the coding tree unit CTU block, and w represents the number of columns of the coding tree unit CTU block. h and w are finite positive integers. In this embodiment, the value of h is 6 and the value of w is 9.

[0059] According to formula (2), the normalized temporal perceptual feature T of the coding tree unit CTU block in each frame image is determined n :

[0060]

[0061] in, Represents the normalized temporal perception information of the coding tree unit CTU block.

[0062] According to equations (3) and (4), the number of columns w and rows h of the coding tree unit CTU block in each frame image is determined as follows:

[0063]

[0064]

[0065] Wherein, [·] represents a rounding up function, W represents the width of the video resolution, H represents the height of the video resolution, W ranges from 416 to 2560, H ranges from 240 to 1600, and in this embodiment, W ranges from 1080, and H ranges from 720.

[0066] According to formula (5), the spatiotemporal perceptual characteristics of the coding tree unit CTU block in each frame image are determined

[0067]

[0068] in, represents the normalized temporal perception information of the coding tree unit CTU block in the i-th row and j-th column in the n-th frame, Represents the normalized spatial perception information of the coding tree unit CTU block in the i-th row and j-th column in the n-th frame, i∈{1,2,...,w}, j∈{1,2,...,h}, h and w are finite positive integers. In this embodiment, the value of h is 6, and the value of w is 9.

[0069] Determine the saliency value S of the n×n coding unit CU block in the image according to formula (6): CU :

[0070]

[0071]

[0072] Among them, S(i,j) represents the salient feature matrix of the n×n coding unit CU block, i represents the horizontal coordinate in the n×n block, i∈{1,2,...,n}, j represents the vertical coordinate in the n×n block, j∈{1,2,...,n}, and n is 2 0 ~2 6 In this embodiment, the value of n is 2 3 .

[0073] Determine the average saliency value S of the current frame according to formula (7): avg :

[0074]

[0075]

[0076] Among them, F n (i, j) represents the salient feature matrix in the nth frame image, i represents the abscissa of the saliency map of each frame, i∈{1,2,...,W}, j represents the ordinate of the saliency map of each frame, j∈{1,2,...,H}, W takes a value of 416 to 2560, H takes a value of 240 to 1600, and in this embodiment, W takes a value of 1080 and H takes a value of 720.

[0077] (2) Determining Perceptual Distortion

[0078] Determine the perceptual distortion D of the current coding unit CU block according to formula (8): p :

[0079] D p =(α×Π+β×SW+γ)×D MSE (8)

[0080]

[0081] Among them, D MSE is the distortion size of the multifunctional video coding standard; α, β, and γ are all weight parameters, α∈(0,1), β∈(0,1), γ∈[0.5,1]. In this embodiment, α is 0.4, β is 0.6, and γ is 0.8. Π represents the spatiotemporal perception of each coding tree unit CTU block; SW represents the significance of each coding unit CU block, S cu Represents the significance value of the current coding unit CU block, S avg Represents the average saliency value of the current frame.

[0082] The final perceptual rate-distortion cost J is determined according to formula (9):

[0083] J=D p +λR (9)

[0084] Among them, D p represents the perceptual distortion of the current coding unit CU block, λ represents the Lagrange multiplier, and R represents the coding bit rate;

[0085] (3) Determine the perceived QP

[0086] Determine the perceptual QP of the current coding unit CU block according to formula (10):

[0087] QP=QP f +round(μ×min(ω s ,1)) (10)

[0088]

[0089] Among them, QP f represents the perceptual QP of the current frame, S max Indicates the maximum significance value of the coding unit CU block, S avg represents the average saliency value of the current frame, S cu Represents the significance value of each coding unit CU block; μ and k are coefficients, μ∈[-10,-6], k∈[0.5,1]. In this embodiment, μ is -8 and k is 0.7.

[0090] The present invention combines the video coding method with the human visual system, removes the subjective visual perception redundancy, and improves the video compression effect while improving the subjective visual perception quality of the human eye. Not only the significant features are considered, but also the spatial features of the image content and the temporal features of the video sequence are introduced, effectively allocating bit rate resources during the coding process, and improving the coding efficiency and video quality.

[0091] Complete the H.266 / VVC video coding method based on spatiotemporal visual perception.

[0092] Example 2

[0093] The H.266 / VVC video encoding method based on spatiotemporal visual perception of this embodiment consists of the following steps:

[0094] (1) Extracting spatiotemporal features and saliency features

[0095] According to formula (1), the normalized spatial perceptual feature Γ of the coding tree unit CTU block in each frame image is determined n :

[0096] The expression of formula (1) is the same as that of Example 1.

[0097] In formula (1), represents the normalized spatial perception information of the coding tree unit CTU block, n∈{1,2,...,N}, N represents the number of frames of the video, and its value is a finite positive integer, h represents the number of rows of the coding tree unit CTU block, and w represents the number of columns of the coding tree unit CTU block. h and w are finite positive integers. In this embodiment, the value of h is 2, and the value of w is 4.

[0098] According to formula (2), the normalized temporal perceptual feature T of the coding tree unit CTU block in each frame image is determined n :

[0099] The expression of formula (2) is the same as that of Example 1.

[0100] According to equations (3) and (4), the number of columns w and rows h of the coding tree unit CTU block in each frame image is determined as follows:

[0101] The expressions of formula (3) and (4) are the same as those in Example 1.

[0102] In equations (3) and (4), [·] represents a rounding function, W represents the width of the video resolution, and H represents the height of the video resolution. W ranges from 416 to 2560, and H ranges from 240 to 1600. In this embodiment, W ranges from 416 to 240.

[0103] According to formula (5), the spatiotemporal perceptual characteristics of the coding tree unit CTU block in each frame image are determined

[0104] The expression of formula (5) is the same as that of Example 1.

[0105] In formula (1), represents the normalized temporal perception information of the coding tree unit CTU block in the i-th row and j-th column in the n-th frame, Represents the normalized spatial perception information of the coding tree unit CTU block in the i-th row and j-th column in the n-th frame, i∈{1,2,...,w}, j∈{1,2,...,h}, h and w are finite positive integers. In this embodiment, the value of h is 2, and the value of w is 4.

[0106] Determine the saliency value S of the n×n coding unit CU block in the image according to formula (6): CU :

[0107] The expression of formula (6) is the same as that of Example 1.

[0108] In formula (6), S(i,j) represents the saliency matrix of the n×n coding unit CU block, i represents the horizontal coordinate in the n×n block, i∈{1,2,...,n}, j represents the vertical coordinate in the n×n block, j∈{1,2,...,n}, and n is 2 0 ~2 6In this embodiment, the value of n is 2 0 .

[0109] Determine the average saliency value S of the current frame according to formula (7): avg :

[0110] The expression of formula (7) is the same as that of Example 1.

[0111] In formula (7), F n (i, j) represents the salient feature matrix in the nth frame image, i represents the abscissa of the saliency map of each frame, i∈{1,2,...,W}, j represents the ordinate of the saliency map of each frame, j∈{1,2,...,H}, W takes a value of 416 to 2560, H takes a value of 240 to 1600, and in this embodiment, W takes a value of 416, and H takes a value of 240.

[0112] (2) Determining Perceptual Distortion

[0113] Determine the perceptual distortion D of the current coding unit CU block according to formula (8): p :

[0114] The expression of formula (8) is the same as that of Example 1.

[0115] In formula (8), D MSE is the distortion size of the multifunctional video coding standard; α, β, and γ are all weight parameters, α∈(0,1), β∈(0,1), γ∈[0.5,1], in this embodiment, α is 0.1, β is 0.1, and γ is 0.5. Π represents the spatiotemporal perception of each coding tree unit CTU block; SW represents the significance of each coding unit CU block, S cu Represents the significance value of the current coding unit CU block, S avg Represents the average saliency value of the current frame.

[0116] The other steps of this step are the same as those in Example 1.

[0117] (3) Determine the perceived QP

[0118] The perceptual QP of the current coding unit CU block is determined according to formula (10).

[0119] The expression of formula (10) is the same as that of Example 1.

[0120] In formula (10), QP f represents the perceptual QP of the current frame, S max Indicates the maximum significance value of the coding unit CU block, S avg represents the average saliency value of the current frame, S cuRepresents the significance value of each coding unit CU block; μ and k are coefficients, μ∈[-10,-6], k∈[0.5,1]. In this embodiment, μ is -10 and k is 0.5.

[0121] Complete the H.266 / VVC video coding method based on spatiotemporal visual perception.

[0122] Example 3

[0123] The H.266 / VVC video encoding method based on spatiotemporal visual perception of this embodiment consists of the following steps:

[0124] (1) Extracting spatiotemporal features and saliency features

[0125] According to formula (1), the normalized spatial perceptual feature Γ of the coding tree unit CTU block in each frame image is determined n :

[0126] The expression of formula (1) is the same as that of Example 1.

[0127] In formula (1), represents the normalized spatial perception information of the coding tree unit CTU block, n∈{1,2,...,N}, N represents the number of video frames, and its value is a finite positive integer, h represents the number of rows of the coding tree unit CTU block, and w represents the number of columns of the coding tree unit CTU block. h and w are finite positive integers. In this embodiment, the value of h is 13, and the value of w is 20.

[0128] According to formula (2), the normalized temporal perceptual feature T of the coding tree unit CTU block in each frame image is determined n :

[0129] The expression of formula (2) is the same as that of Example 1.

[0130] According to equations (3) and (4), the number of columns w and rows h of the coding tree unit CTU block in each frame image is determined as follows:

[0131] The expressions of formula (3) and (4) are the same as those in Example 1.

[0132] In equations (3) and (4), [·] represents a rounding function, W represents the width of the video resolution, and H represents the height of the video resolution. The value of W is 416 to 2560, and the value of H is 240 to 1600. In this embodiment, the value of W is 2560, and the value of H is 1600.

[0133] According to formula (5), the spatiotemporal perceptual characteristics of the coding tree unit CTU block in each frame image are determined

[0134] The expression of formula (5) is the same as that of Example 1.

[0135] In formula (1), represents the normalized temporal perception information of the coding tree unit CTU block in the i-th row and j-th column in the n-th frame, Represents the normalized spatial perception information of the coding tree unit CTU block in the i-th row and j-th column in the n-th frame, i∈{1,2,...,w}, j∈{1,2,...,h}, h and w are finite positive integers. In this embodiment, the value of h is 13, and the value of w is 20.

[0136] Determine the saliency value S of the n×n coding unit CU block in the image according to formula (6): CU :

[0137] The expression of formula (6) is the same as that of Example 1.

[0138] In formula (6), S(i,j) represents the saliency matrix of the n×n coding unit CU block, i represents the horizontal coordinate in the n×n block, i∈{1,2,...,n}, j represents the vertical coordinate in the n×n block, j∈{1,2,...,n}, and n is 2 0 ~2 6 In this embodiment, the value of n is 2 6 .

[0139] Determine the average saliency value S of the current frame according to formula (7): avg :

[0140] The expression of formula (7) is the same as that of Example 1.

[0141] In formula (7), F n (i, j) represents the salient feature matrix in the nth frame image, i represents the abscissa of the saliency map of each frame, i∈{1,2,...,W}, j represents the ordinate of the saliency map of each frame, j∈{1,2,...,H}, W takes a value of 416 to 2560, H takes a value of 240 to 1600, and in this embodiment, W takes a value of 2560, and H takes a value of 1600.

[0142] (2) Determining Perceptual Distortion

[0143] Determine the perceptual distortion D of the current coding unit CU block according to formula (8): p :

[0144] The expression of formula (8) is the same as that of Example 1.

[0145] In formula (8), D MSEis the distortion size of the multifunctional video coding standard; α, β, and γ are all weight parameters, α∈(0,1), β∈(0,1), γ∈[0.5,1]. In this embodiment, α is 0.9, β is 0.9, and γ is 1.0. Π represents the spatiotemporal perception of each coding tree unit CTU block; SW represents the significance of each coding unit CU block, S cu Represents the significance value of the current coding unit CU block, S avg Represents the average saliency value of the current frame.

[0146] The other steps of this step are the same as those in Example 1.

[0147] (3) Determine the perceived QP

[0148] The perceptual QP of the current coding unit CU block is determined according to formula (10).

[0149] The expression of formula (10) is the same as that of Example 1.

[0150] In formula (10), QP f represents the perceptual QP of the current frame, S max Represents the maximum significance value of the coding unit CU block, S avg represents the average saliency value of the current frame, S cu Represents the significance value of each coding unit CU block; μ and k are coefficients, μ∈[-10,-6], k∈[0.5,1]. In this embodiment, μ is -6 and k is 1.

[0151] Complete the H.266 / VVC video coding method based on spatiotemporal visual perception.

[0152] In order to verify the effectiveness of the present invention, the inventors conducted a comparative simulation experiment using the H.266 / VVC video encoding method based on spatiotemporal visual perception of Example 1 of the present invention and the existing saliency-based video encoding method (hereinafter referred to as comparative experiment 1) to determine the bit rate-distortion equalization weighted peak signal-to-noise ratio BD-EWPSNR, bit rate gain BD-Rate, and bit rate-peak signal-to-noise ratio gain BD-PSNR of the two methods as comprehensive indicators for evaluating the quality of video compression performance.

[0153] The bit rate distortion equalization weighted peak signal-to-noise ratio BD-EWPSNR is determined by the following formula:

[0154]

[0155] Where ΔPSNR(R) is the difference in PSNR between the two encoders at bit rate R, w(R) is the weight corresponding to bit rate R, and c and d are the minimum and maximum bit rates, respectively.

[0156] The bit rate gain BD-Rate is determined as follows:

[0157]

[0158] Where a and b are the lower and upper limits of the PSNR range, respectively, and R1 and R2 are the bit rates of the two encoders at corresponding PSNR values.

[0159] The bit rate-peak signal-to-noise ratio gain BD-PSNR is determined by the following formula:

[0160]

[0161] Among them, the peak signal-to-noise ratio PSNR1 and the peak signal-to-noise ratio PSNR2 are the peak signal-to-noise ratio PSNR values ​​of the two encoders at the bit rate R respectively.

[0162] The experimental and calculation results are shown in Table 1. Figure 3 , Figure 4 shown.

[0163] Table 1 Experimental results of the method of the present invention and the comparative experimental method

[0164]

[0165] As shown in Table 1, compared with the method in comparative experiment 1, the method of the present invention has significantly improved effects, wherein BD-EWPSNR is increased by 0.09dB, BD-Rate is decreased by 0.67%, and BD-PSNR is decreased by 0.07dB. The experiment shows that the method of the present invention has better compression efficiency, especially in videos with complex motion and texture, which is more in line with the visual characteristics of the audience.

[0166] Figure 2 This is the original image from the 68th channel of the video sequence BasketballPass. Figure 3 This is the effect diagram of the comparative test 1. Figure 4 It is a diagram showing the effect of a computer simulation experiment using the method of Example 1 of the present invention.

[0167] exist Figure 3 In the non-salient area, the details of the fast-moving basketball are severely lost and the edge contour is not obvious.

[0168] exist Figure 4 In the embodiment 1 of the present invention, the reconstructed frame can well preserve the details of the fast-moving basketball in the non-salient area, the edge contour is sharper, the color is accurately restored, and the blur and artifacts are significantly reduced. At the same time, the quality of the athlete image restoration in the salient area is also better than the comparative test.

Claims

1. A H.266 / VVC video coding method based on spatiotemporal visual perception, Features It consists of the following steps: (1) Extracting spatiotemporal features and saliency features According to formula (1), the normalized spatial perceptual feature Γ of the coding tree unit CTU block in each frame image is determined n : in, Represents the normalized spatial perception information of the coding tree unit CTU block, n∈{1,2,...,N}, N represents the number of video frames, and its value is a finite positive integer, h represents the number of rows of the coding tree unit CTU block, w represents the number of columns of the coding tree unit CTU block, and h and w are finite positive integers; According to formula (2), the normalized temporal perceptual feature T of the coding tree unit CTU block in each frame image is determined n : in, Represents the normalized temporal perception information of the coding tree unit CTU block; According to equations (3) and (4), the number of columns w and rows h of the coding tree unit CTU block in each frame image is determined as follows: Where [·] represents a rounding function, W represents the width of the video resolution, H represents the height of the video resolution, and W and H are finite positive integers; According to formula (5), the spatiotemporal perceptual characteristics of the coding tree unit CTU block in each frame image are determined in, represents the normalized temporal perception information of the coding tree unit CTU block in the i-th row and j-th column in the n-th frame, represents the normalized spatial perceptual information of the coding tree unit CTU block in the i-th row and j-th column in the n-th frame, i∈{1,2,...,w}, j∈{1,2,...,h}, w and h are finite positive integers; Determine the saliency value S of the n×n coding unit CU block in the image according to formula (6): CU : Among them, S(i,j) represents the salient feature matrix of the n×n coding unit CU block, i represents the horizontal coordinate in the n×n block, i∈{1,2,...,n}, j represents the vertical coordinate in the n×n block, j∈{1,2,...,n}, and n is 2 0 ~2 6 ; Determine the average saliency value S of the current frame according to formula (7): avg : Among them, F n (i, j) represents the salient feature matrix in the nth frame image, i represents the abscissa of the saliency map of each frame, i∈{1,2,...,W}, j represents the ordinate of the saliency map of each frame, j∈{1,2,...,H}; (2) Determining Perceptual Distortion Determine the perceptual distortion D of the current coding unit CU block according to formula (8): p : D p =(α×Π+β×SW+γ)×D MSE (8) Among them, D MSE is the distortion size of the multifunctional video coding standard; α, β, γ are all weight parameters, α∈(0,1), β∈(0,1), γ∈[0.5,1]; Π represents the spatiotemporal perception of each coding tree unit CTU block; SW represents the significance of each coding unit CU block, S cu Represents the significance value of the current coding unit CU block, S avg represents the average saliency value of the current frame; The final perceptual rate-distortion cost J is determined according to formula (9): J=D p +λR (9) Among them, D p represents the perceptual distortion of the current coding unit CU block, λ represents the Lagrange multiplier, and R represents the coding bit rate; (3) Determine the perceived QP Determine the perceptual QP of the current coding unit CU block according to formula (10): QP=QP f +round(μ×min(ω s ,1)) (10) Among them, QP f represents the perceptual QP of the current frame, S max Represents the maximum significance value of the coding unit CU block, S avg represents the average saliency value of the current frame, S cu Represents the significance value of each coding unit CU block; μ and k are coefficients, μ∈[-10,-6], k∈[0.5,1].

2. The H.266 / VVC video encoding method based on spatiotemporal visual perception according to claim 1, Features: In formulas (3) and (4) for extracting spatiotemporal features and saliency features in step (1), W represents the width of the video resolution, H represents the height of the video resolution, the value of W is 416 to 2560, the value of H is 240 to 1600, and W>H.

3. The H.266 / VVC video encoding method based on spatiotemporal visual perception according to claim 1 or 2, Features: In formulas (3) and (4) for extracting spatiotemporal features and saliency features in step (1), W represents the width of the video resolution, H represents the height of the video resolution, and the value of W is 1080 and the value of H is 720.

4. The H.266 / VVC video encoding method based on spatiotemporal visual perception according to claim 1, Features: In the equation (8) for determining the perceptual distortion in step (2), the value of α is 0.4, the value of β is 0.6, and the value of γ is 0.

8.

5. The H.266 / VVC video encoding method based on spatiotemporal visual perception according to claim 1, Features: In the equation (10) for determining the perceptual QP in step (3), μ and k are coefficients, μ is -8, and k is 0.

7.

6. The H.266 / VVC video encoding method based on spatiotemporal visual perception according to claim 1, Features: In the formula (6) for extracting the spatiotemporal features and the salient features in step (1), the S(i,j) represents the salient feature matrix of the n×n coding unit CU block, i represents the horizontal coordinate in the n×n block, i∈{1,2,...,n}, j represents the vertical coordinate in the n×n block, j∈{1,2,...,n}, and n is 2 3 .

Images