SCV coding perceptual bit rate control method and device integrating spatial-frequency domain saliency features
By integrating the significance characteristics of the space frequency domain, the significance characteristics of the screen content video are obtained and the perceived rate distortion model is constructed, which solves the problem of poor encoding of screen content videos in the prior art, and improves the encoding quality and code rate control accuracy. , achieving better visual experience and compression performance.
Patent Information
- Application Number
- CN202410885047.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-03
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-07-03
AI Technical Summary
The existing video encoding standards are difficult to meet the compression needs of new video formats such as 4K, 8K, high dynamic range HDR and 360-degree panoramic videos. Especially in screen content videos, the compression effect of traditional encoding methods is poor, and the existing codes are Rate control algorithms are difficult to further compress time and spatial redundancy under the hybrid coding framework, resulting in increased computational complexity.
Using the SCV encoding perceived bitrate control method that integrates the significance characteristics of the space frequency domain, the significance characteristics of the space and frequency domain are obtained through convolution calculation and discrete cosine transformation, the significance factor is constructed and linear weighted, the target bit allocation is guided, and the target bit allocation is established. Perceived rate distortion model, optimized encoding parameters to improve encoding quality and perceived bitrate control accuracy.
It improves the encoding quality and rate distortion performance of screen content videos, enhances the perceived bit rate control accuracy, brings users a better visual experience, and reduces the computational complexity.
Smart Images

Figure CN118450127B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of video coding, and in particular to a SCV coding perceptual bit rate control method and device integrating spatial-frequency domain saliency features. Background Art
[0002] With the development of the times, the development of computer technology and multimedia technology has greatly enriched people's lives, and applications related to video technology have also made great progress. Human beings have entered an era full of visual information, and everyone can become a member of self-media, so humans prefer video applications. Video applications are now ubiquitous and have become an indispensable part of human society.
[0003] In recent years, with the rapid development of cloud computing and mobile Internet technologies, the application of screen content video (SCV) such as screen sharing, wireless display, and distance education has become more and more widespread. Screen content video is generated by computers and other devices. It is different from natural video captured by cameras. It usually has large uniform planes, repeated patterns and characters, limited color types but high saturation, high image contrast, and sharp edges. If the traditional natural video coding standard is still used to process screen content video, the compression effect is often poor. Therefore, the encoding and processing of screen content video is different from the encoding and processing of natural video.
[0004] Due to the rapid development of video technology, the application of new video formats such as 4K, 8K, high dynamic range HDR (High Dynamic Range) video, virtual reality video and 360-degree panoramic video has gradually appeared in the human field of vision. Therefore, the demand for compression performance in the field of video coding is also increasing, and the previous coding standards are gradually unable to meet the compression requirements of new videos. Therefore, the Joint Video Experts Group (JVET) of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG) jointly released the latest generation of video coding standard H.266, also known as VVC (Versatile Video Coding). Compared with the previous generation coding standard HEVC, VVC can achieve a compression performance gain of up to 50%. In addition, VVC has designed a variety of coding tools for screen content videos, including Intra Block Copy (IBC), Palette Mode (PLT), Adaptive Color Transfoorm (ACT) and Adaptive Motive Vector (AMV).
[0005] In order to efficiently utilize bandwidth resources and improve transmission efficiency, rate control is an indispensable part of the encoder. The main function of rate control is to establish a mathematical relationship model between the encoding rate and the quantization parameter, determine the encoding parameters based on the target rate, ensure that the encoding rate is close to the target rate, and meet the transmission bandwidth requirements while maintaining rate-distortion performance. At present, most rate control algorithms are considering eliminating spatial redundancy and temporal redundancy, but under the existing hybrid coding framework, temporal redundancy and spatial redundancy are difficult to further compress. Further improving the encoding performance may lead to an increase in computational complexity. Summary of the invention
[0006] The main purpose of the present invention is to overcome the above-mentioned defects in the prior art, and propose an SCV coding perceptual rate control method and device that integrates the spatial-frequency domain significance features, combines the screen content characteristics and the human eye visual perception characteristics, introduces perceptual factors in the rate control process, and allocates more bits to the areas that the human eye pays more attention to, thereby improving the screen content video coding quality, rate-distortion performance and perceptual rate control accuracy, and bringing users a better visual experience.
[0007] The present invention adopts the following technical solution:
[0008] On the one hand, a SCV coding perceptual bit rate control method integrating spatial-frequency domain saliency features includes:
[0009] S101, obtaining a screen content video frame, performing saliency modeling on the screen content video frame in the spatial domain through convolution calculation and similarity calculation, and obtaining a spatial domain saliency feature;
[0010] S102, using discrete cosine transform DCT, performing saliency modeling on the screen content video frame in the frequency domain to obtain frequency domain saliency features;
[0011] S103, respectively calculating the spatial domain significance feature and the frequency domain significance feature to obtain the spatial domain significance factor and the frequency domain significance factor, then linearly weighting the spatial domain significance factor and the frequency domain significance factor to obtain the significance factor of the current CTU, and then using the significance factor of the current CTU to guide the target bit allocation at the CTU level;
[0012] S104, constructing a saliency-inspired rate-distortion model based on the saliency factor and target bits of the current CTU;
[0013] S105, estimating coding parameters according to the constructed rate-distortion model, and updating the rate-distortion model according to actual coding parameters.
[0014] Preferably, the S101 specifically includes:
[0015] Perform Gaussian convolution on the current frame of the video and the current CTU as follows:
[0016]
[0017]
[0018] in, represents the transformed frame after Gaussian convolution transformation; Represents the pixel value of the image frame; Represents the convolution operation; represents the Gaussian convolution kernel function; Indicates the frame pixel number; Indicates the total number of pixels contained in a frame; Represents the Gaussian convolution kernel value corresponding to the i-th pixel of the frame; represents the pixel value of the i-th pixel of the frame; represents the transformed CTU after Gaussian convolution transformation; Indicates the pixel value of the image CTU; Indicates the CTU pixel number; Indicates the total number of pixels contained in CTU; Indicates the first The Gaussian convolution kernel value corresponding to the pixel; Indicates the first The pixel value of pixels;
[0019] The visual saliency inspired by motion blur is obtained by calculating the local similarity between the video frame and the video frame processed by motion blur. The motion convolution process of the current frame of the video and the current CTU is as follows:
[0020]
[0021]
[0022] in, represents the transformed frame after motion convolution transformation; Represents the transformed CTU after motion convolution transformation; represents the motion convolution kernel;
[0023] The similarity calculation function is used to calculate the Gaussian blur inspired salient image frame and the motion blur inspired salient image frame as follows:
[0024]
[0025]
[0026] in, Represents the similarity calculation function; Represents Gaussian blur inspired saliency image frame; Representing motion blur inspired saliency image frames;
[0027] The similarity calculation function is used to calculate the Gaussian blur inspired saliency image CTU and the motion blur inspired saliency image CTU as follows:
[0028]
[0029]
[0030] in, Represents the Gaussian blur inspired saliency image CTU; Represents motion blur inspired saliency image CTU;
[0031] Perform visual saliency inversion, calculated as follows:
[0032]
[0033]
[0034]
[0035]
[0036] in, represents the Gaussian blur inspired saliency image frame after inversion, represents an inverted motion blur inspired saliency image frame; represents the Gaussian blur inspired saliency image CTU after inversion, Represents the inverted motion blur inspired saliency image frame CTU;
[0037] Finally, the linear model is used to combine the two inverted saliency images to calculate the spatial saliency value of the current frame. And the spatial significance value of the current CTU , as follows:
[0038]
[0039]
[0040] in, and Is a non-negative number that adjusts the relative importance of Gaussian blur and motion blur.
[0041] Preferably, the S102 specifically includes:
[0042] The total energy of the image block is obtained by summing the values of the DCT coefficient matrix. The specific calculation is as follows:
[0043]
[0044]
[0045] in, Represents the frequency domain significance value of the current frame; Indicates the number of DCT coefficients of the current frame, Indicates the serial number is DCT coefficients of Indicates the frequency domain significance value of the current CTU; Indicates the number of DCT coefficients of the current CTU, Indicates the serial number is The DCT coefficients of .
[0046] Preferably, the S103 specifically includes:
[0047] The spatial domain significance features and frequency domain significance features are calculated separately to obtain the spatial domain significance factor and frequency domain significance factor. The specific calculation is as follows:
[0048]
[0049]
[0050] in, Represents the spatial significance factor of the current CTU; Indicates the frequency domain significance factor of the current CTU; Indicates the spatial significance value of the current CTU; Represents the spatial saliency value of the current frame; Indicates the frequency domain significance value of the current CTU; Represents the frequency domain significance value of the current frame;
[0051] Linearly weight the spatial domain significance factor and the frequency domain significance factor to obtain the fusion significance factor of the current CTU , calculated as follows:
[0052]
[0053] in, Represents the spatial significance factor of the current CTU; Indicates the frequency domain significance factor of the current CTU; , 1 [] ;
[0054] CTU-level target bits The allocation formula is as follows:
[0055]
[0056] in, Indicates the number of uncoded bits; represents the sum of the significance factors of all CTUs; Represents the sum of the significance factors of the encoded CTUs.
[0057] Preferably, the S104 specifically includes:
[0058] Based on the original rate-distortion model R-λ, the significance factor of the current CTU is combined , construct a rate-distortion model inspired by saliency, in which the CTU-level rate control parameter and quantization parameters The calculation is as follows:
[0059]
[0060]
[0061] Among them, λ is calculated by the original rate-distortion model, and the calculation is as follows:
[0062]
[0063] in, and are model parameters related to video characteristics; Indicates the average target number of bits per pixel of the current CTU, expressed as follows:
[0064]
[0065] in, Indicates the target bit of the current CTU, Indicates the number of pixels in the current CTU.
[0066] Preferably, the S105 specifically includes:
[0067] The perceptual rate-distortion model is used to estimate the optimal coding parameters for encoding;
[0068] After the actual encoding is completed, the perceptual rate-distortion model is updated with the actual encoding parameters.
[0069] On the other hand, an SCV coding perceptual bit rate control device integrating spatial-frequency domain saliency features comprises:
[0070] The spatial domain saliency feature acquisition module is used to acquire the screen content video, perform saliency modeling on the screen content video in the spatial domain through convolution calculation and similarity calculation, and obtain the spatial domain saliency feature;
[0071] The frequency domain saliency feature module is used to perform saliency modeling on the screen content video in the frequency domain using discrete cosine transform (DCT) to obtain frequency domain saliency features;
[0072] The significance factor and target bit acquisition module is used to calculate the spatial domain significance features and the frequency domain significance features respectively, and obtain the spatial domain significance factor and the frequency domain significance factor, and then linearly weight the spatial domain significance factor and the frequency domain significance factor to obtain the significance factor of the current CTU, and then use the significance factor of the current CTU to guide the target bit allocation of the CTU level;
[0073] A rate-distortion model building module is used to build a saliency-inspired rate-distortion model based on the saliency factor and target bits of the current CTU;
[0074] The rate-distortion model updating module is used to estimate coding parameters according to the constructed rate-distortion model and update the rate-distortion model according to the actual coding parameters.
[0075] Compared with the prior art, the present invention has the following beneficial effects:
[0076] (1) The present invention fully considers the content characteristics of screen content video and the characteristics of the human visual system, constructs a saliency model that can simulate the behavior of the human visual system based on the saliency characteristics of screen content video, and uses saliency factors to guide target bit allocation. Compared with the allocation method in the standard that does not consider screen content characteristics and visual redundancy, the target bit rate allocation scheme of the present invention is more accurate;
[0077] (2) The present invention fully considers the problem that the rate-distortion model in the standard does not take into account the characteristics of screen content video. By using the significance factor, a perceptual rate-distortion model of screen content video is constructed, which improves the quality and rate-distortion performance of the encoded video sequence and improves the perceptual bit rate control accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0078] Figure 1 It is a flow chart of the SCV coding perceptual bit rate control method integrating spatial-frequency domain saliency features of the present invention;
[0079] Figure 2 It is an overall flow chart of the SCV coding perceptual bit rate control method integrating spatial-frequency domain saliency features of the present invention;
[0080] Figure 3 It is a structural block diagram of the SCV coding perception bit rate control device integrating spatial-frequency domain saliency features of the present invention. DETAILED DESCRIPTION
[0081] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.
[0082] See also Figure 1 and Figure 2 As shown, the present invention provides an SCV coding perceptual bit rate control method integrating spatial-frequency domain saliency features, comprising the following steps.
[0083] S101, obtaining a screen content video frame, performing saliency modeling on the screen content video frame in the spatial domain through convolution calculation and similarity calculation, and obtaining a spatial domain saliency feature.
[0084] Specifically, the local similarity between the video frame and the video frame processed by Gaussian blur is calculated to represent the visual saliency inspired by Gaussian blur.
[0085] Perform Gaussian convolution on the current frame of the video and the current CTU as follows:
[0086]
[0087]
[0088] in, represents the transformed frame after Gaussian convolution transformation; Represents the pixel value of the image frame; Represents the convolution operation; represents the Gaussian convolution kernel function; Indicates the frame pixel number; Indicates the total number of pixels contained in a frame; Represents the Gaussian convolution kernel value corresponding to the i-th pixel of the frame; represents the pixel value of the i-th pixel of the frame; represents the transformed CTU after Gaussian convolution transformation; Indicates the pixel value of the image CTU; Indicates the CTU pixel number; Indicates the total number of pixels contained in CTU; Indicates the first The Gaussian convolution kernel value corresponding to the pixel; Indicates the first The pixel value of pixels.
[0089] Gaussian convolution kernel function It can be expressed as follows:
[0090]
[0091] in, represents standard deviation; Indicates the horizontal coordinate of the pixel; Indicates the vertical coordinate of the pixel.
[0092] The visual saliency inspired by motion blur is obtained by calculating the local similarity between a video frame and a video frame processed by motion blur.
[0093] The motion convolution process of the current frame of the video and the current CTU is as follows:
[0094]
[0095]
[0096] in, represents the transformed frame after motion convolution transformation; Represents the transformed CTU after motion convolution transformation; represents the motion convolution kernel;
[0097] It can be expressed by the following formula:
[0098]
[0099] in, is the amount of motion in pixels; Indicates the direction and level of movement The angle of the axes; Indicates the horizontal coordinate of the pixel; Indicates the vertical coordinate of the pixel; Represents matrix transpose.
[0100] Since the human visual system is more sensitive to structural changes in images, the gradient amplitude similarity deviation is used to measure the degree of structural degradation.
[0101] The similarity calculation function is used to calculate the Gaussian blur inspired salient image frame and the motion blur inspired salient image frame as follows:
[0102]
[0103]
[0104] in, Represents the similarity calculation function; Represents Gaussian blur inspired saliency image frame; Representing motion blur inspired saliency image frames.
[0105] The similarity calculation function is used to calculate the Gaussian blur inspired saliency image CTU and the motion blur inspired saliency image CTU as follows:
[0106]
[0107]
[0108] in, Represents the Gaussian blur inspired saliency image CTU; Represents motion blur inspired saliency image CTU.
[0109] The calculation method of is as follows.
[0110] First, the image gradient amplitude is extracted by the Scharr operator, which is expressed as follows:
[0111]
[0112] Among them, the horizontal component and the vertical component It can be expressed as follows:
[0113] p x = h x ⊗ r = 1 16 [ + 3 0 − 3 + 10 0 − 10 + 3 0 − 3 ] ⊗ r ;
[0114] p y = h y ⊗ r = 1 16 [ + 3 + 10 + 3 0 0 0 − 3 − 10 − 3 ] ⊗ r ;
[0115] in, Represents the horizontal component of the Scharr operator convolution kernel; Represents the vertical component of the Scharr operator convolution kernel; Represents the pixel values of an image frame.
[0116] Then, the similarity metric is used to reflect the difference in the gradient amplitude map between the original frame and the frame after convolution transformation, and the similarity calculation function is obtained. , calculated as follows:
[0117]
[0118] in, for or ; for , , or ; Represents the gradient amplitude of the original frame or original CTU. The calculation process uses the above image gradient amplitude Calculation formula; Represents the gradient amplitude map of the frame after convolution transformation or the gradient amplitude of CTU. The calculation process uses the above image gradient amplitude Calculation formula, in addition is a positive number to prevent the denominator from being zero.
[0119] Since the metric method based on saliency inspiration is used, it should be noted that in the calculation of gradient amplitude similarity, a larger value indicates that it is more similar to the surrounding elements, that is, the visual saliency is lower, so it is necessary to reverse the visual saliency and perform the following calculation:
[0120]
[0121]
[0122]
[0123]
[0124] in, represents the Gaussian blur inspired saliency image frame after inversion, represents an inverted motion blur inspired saliency image frame; represents the Gaussian blur inspired saliency image CTU after inversion, Represents the inverted motion blur inspired saliency image frame CTU;
[0125] Finally, the linear model is used to combine the two inverted saliency images to calculate the spatial saliency value of the current frame. And the spatial significance value of the current CTU , as follows:
[0126]
[0127]
[0128] in, and Is a non-negative number that adjusts the relative importance of Gaussian blur and motion blur.
[0129] S102, using discrete cosine transform DCT, performing saliency modeling on the screen content video frame in the frequency domain to obtain frequency domain saliency features.
[0130] After the screen content video frame is transformed by DCT, the energy of the image is concentrated on the low-frequency AC coefficient and DC coefficient. The DCT coefficients of a block consist of 63 AC coefficients and one DC coefficient; The DC coefficients and low-frequency AC coefficients of the DCT transform block represent most of the information of the current block, and different positions The DCT transform coefficients of the blocks are also different, so the DCT coefficients are suitable for representing the texture and brightness features of the image; by summing the values of the DCT coefficient matrix, the total energy of the image block is obtained to measure the complexity or content richness of the current block. The specific calculation is as follows:
[0131]
[0132]
[0133] in, Represents the frequency domain significance value of the current frame; Indicates the number of DCT coefficients of the current frame, Indicates the serial number is DCT coefficients of Indicates the frequency domain significance value of the current CTU; Indicates the number of DCT coefficients of the current CTU, Indicates the serial number is The DCT coefficients of .
[0134] S103, respectively operate the spatial domain significance feature and the frequency domain significance feature to obtain the spatial domain significance factor and the frequency domain significance factor, then linearly weight the spatial domain significance factor and the frequency domain significance factor to obtain the significance factor of the current CTU, and then use the significance factor of the current CTU to guide the target bit allocation at the CTU level.
[0135] Specifically, after obtaining the spatial domain significant features and the frequency domain significant features in S101 and S102 respectively, the spatial domain significant factor and the frequency domain significant factor are obtained, and the specific calculation is as follows:
[0136]
[0137]
[0138] in, Represents the spatial significance factor of the current CTU; Indicates the frequency domain significance factor of the current CTU; Indicates the spatial significance value of the current CTU; Represents the spatial saliency value of the current frame; Indicates the frequency domain significance value of the current CTU; Indicates the frequency domain significance value of the current frame.
[0139] Then, the spatial domain significance factor and the frequency domain significance factor are linearly weighted to obtain the significance factor of the current CTU: , calculated as follows:
[0140]
[0141] in, Represents the spatial significance factor of the current CTU, Indicates the frequency domain significance factor of the current CTU. In addition , 1 [] Therefore, the CTU-level target bit allocation formula is as follows:
[0142]
[0143] in, represents the number of uncoded bits, represents the sum of the significance factors of all CTUs, is the sum of the significance factors of the coded CTUs.
[0144] S104: construct a saliency-inspired rate-distortion model based on the saliency factor and target bits of the current CTU.
[0145] Specifically, based on the original rate-distortion model R-λ, the significance factor is combined to construct a rate-distortion model inspired by significance. The CTU-level bit rate control parameter in the rate-distortion model is and quantization parameters The calculation is as follows:
[0146]
[0147]
[0148] Among them, λ is calculated by the original rate-distortion model, and the calculation is as follows:
[0149]
[0150] In the formula, and are model parameters related to video characteristics, Indicates the average target number of bits per pixel of the current CTU, that is:
[0151]
[0152] in, Indicates the target bit of the current CTU, Indicates the number of pixels in the current CTU.
[0153] S105, estimating coding parameters according to the constructed rate-distortion model, and updating the rate-distortion model according to actual coding parameters.
[0154] The rate-distortion model is updated according to the actual coding parameters, including:
[0155] The perceptual rate-distortion model is used to estimate the optimal coding parameters for encoding;
[0156] After the actual encoding is completed, the perceptual rate-distortion model is updated with the actual encoding parameters.
[0157] See also Figure 3 As shown, the present invention also discloses an SCV coding perception bit rate control device integrating spatial-frequency domain saliency features, comprising:
[0158] The spatial domain saliency feature acquisition module 301 is used to acquire the screen content video, perform saliency modeling on the screen content video in the spatial domain through convolution calculation and similarity calculation, and obtain the spatial domain saliency feature;
[0159] The frequency domain saliency feature module 302 is used to perform saliency modeling on the screen content video in the frequency domain by using discrete cosine transform DCT to obtain frequency domain saliency features;
[0160] The significance factor and target bit acquisition module 303 is used to respectively calculate the spatial domain significance feature and the frequency domain significance feature to obtain the spatial domain significance factor and the frequency domain significance factor, and then linearly weight the spatial domain significance factor and the frequency domain significance factor to obtain the significance factor of the current CTU, and then use the significance factor of the current CTU to guide the target bit allocation of the CTU level;
[0161] A rate-distortion model building module 304, configured to build a saliency-inspired rate-distortion model based on a saliency factor and a target bit of a current CTU;
[0162] The rate-distortion model updating module 305 is used to estimate coding parameters according to the constructed rate-distortion model and update the rate-distortion model according to actual coding parameters.
[0163] The specific implementation of each module of the SCV coding perceptual rate control device integrating space-frequency domain significance features is the same as the SCV coding perceptual rate control method integrating space-frequency domain significance features, and this embodiment will not repeat the description.
[0164] The above is only a specific implementation of the present invention, but the design concept of the present invention is not limited thereto. Any non-substantial changes to the present invention using this concept shall be deemed as an infringement of the protection scope of the present invention.
Claims
1. A SCV coding perceptual bit rate control method integrating spatial-frequency domain saliency features, characterized in that: include: S101, obtaining a screen content video frame, performing saliency modeling on the screen content video frame in the spatial domain through convolution calculation and similarity calculation, and obtaining a spatial domain saliency feature; S102, using discrete cosine transform DCT, performing saliency modeling on the screen content video frame in the frequency domain to obtain frequency domain saliency features; S103, respectively calculating the spatial domain significance feature and the frequency domain significance feature to obtain the spatial domain significance factor and the frequency domain significance factor, then linearly weighting the spatial domain significance factor and the frequency domain significance factor to obtain the significance factor of the current CTU, and then using the significance factor of the current CTU to guide the target bit allocation at the CTU level; S104, constructing a saliency-inspired rate-distortion model based on the saliency factor and target bits of the current CTU; S105, estimating coding parameters according to the constructed rate-distortion model, and updating the rate-distortion model according to actual coding parameters; The S103 specifically includes: The spatial domain significance features and frequency domain significance features are calculated separately to obtain the spatial domain significance factor and frequency domain significance factor. The specific calculation is as follows: s curctu =S ctu / S pic ; d curctu =D ctu / D pic ; Among them, s curctu Indicates the spatial significance factor of the current CTU; d curctu Indicates the frequency domain significance factor of the current CTU; S ctu Indicates the spatial significance value of the current CTU; S pic represents the spatial saliency value of the current frame; D ctu Indicates the frequency domain significance value of the current CTU; D pic Represents the frequency domain significance value of the current frame; Linearly weight the spatial domain significance factor and the frequency domain significance factor to obtain the fusion significance factor w of the current CTU curctu , calculated as follows: w curctu =u×s curctu +(1-u)×d curctu ; Where u∈[0,1]; CTU level target bits curctu The allocation formula is as follows: Among them, Bits left Indicates the number of uncoded bits; w allctu represents the sum of the significance factors of all CTUs; w codedctu Represents the sum of the significance factors of the encoded CTUs.
2. The SCV coding perceptual bit rate control method integrating spatial-frequency domain saliency features according to claim 1 is characterized in that: The S101 specifically includes: Perform Gaussian convolution on the current frame of the video and the current CTU as follows: Among them, r g_pic represents the transformed frame after Gaussian convolution transformation; r pic Represents the pixel value of the image frame; represents the convolution operation; g represents the Gaussian convolution kernel function; i represents the frame pixel number; n pic Indicates the total number of pixels in a frame; g i Represents the Gaussian convolution kernel value corresponding to the i-th pixel of the frame; r i represents the pixel value of the i-th pixel of the frame; r g_ctu represents the transformed CTU after Gaussian convolution transformation; r ctu Indicates the pixel value of the image CTU; j indicates the CTU pixel number; n ctu Indicates the total number of pixels contained in CTU; g j represents the Gaussian convolution kernel value corresponding to the j-th pixel of CTU; r j represents the pixel value of the jth pixel of CTU; The visual saliency inspired by motion blur is obtained by calculating the local similarity between the video frame and the video frame processed by motion blur. The motion convolution process of the current frame of the video and the current CTU is as follows: Among them, r m_pic represents the transformed frame after motion convolution transformation; r m_ctu represents the transformed CTU after motion convolution transformation; m represents the motion convolution kernel; The similarity calculation function is used to calculate the Gaussian blur inspired salient image frame and the motion blur inspired salient image frame as follows: M g_pic (r pic )=G s (r pic ,r g_pic ); M m_pic (r pic )=G s (r pic ,r m_pic ); Among them, G s (·) represents the similarity calculation function; M g_pic (r pic ) represents the Gaussian blur inspired salient image frame; M m_pic (r pic ) represents a motion blur inspired saliency image frame; The similarity calculation function is used to calculate the Gaussian blur inspired saliency image CTU and the motion blur inspired saliency image CTU as follows: M g_ctu (r ctu )=G s (r ctu ,r g_ctu ); M m_ctu (r ctu )=G s (r ctu ,r m_ctu ); Among them, M g_ctu (r ctu ) represents the Gaussian blur inspired saliency image CTU; M m_ctu (r ctu ) represents the motion blur inspired saliency image CTU; Perform visual saliency inversion, calculated as follows: W g_pic (r pic )=1-M g_pic (r pic ); W m_pic (r pic )=1-M m_pic (r pic ); W g _ ctu (r ctu )=1-M g_ctu (r ctu ); W m_ctu (r ctu )=1-M m_ctu (r ctu ); Among them, W g_pic (r pic ) represents the Gaussian blur inspired saliency image frame after inversion, W m_pic (r pic ) represents the inverted motion blur inspired saliency image frame; W g_ctu (r ctu ) represents the Gaussian blur inspired saliency image CTU after inversion, W m_ctu (r ctu ) represents the inverted motion blur inspired saliency image frame CTU; Finally, the linear model is used to combine the two inverted saliency images to calculate the spatial saliency value S of the current frame. pic And the spatial significance value S of the current CTU ctu , as follows: Among them, q1 and q2 are non-negative numbers used to adjust the relative importance of Gaussian blur and motion blur.
3. The SCV coding perceptual bit rate control method integrating spatial-frequency domain saliency features according to claim 1 is characterized in that: The S102 specifically includes: The total energy of the image block is obtained by summing the values of the DCT coefficient matrix. The specific calculation is as follows: Among them, D pic Represents the frequency domain significance value of the current frame; N pic Indicates the number of DCT coefficients of the current frame, D k Indicates the DCT coefficient with serial number k; D ctu Indicates the frequency domain significance value of the current CTU; N ctu Indicates the number of DCT coefficients of the current CTU, D l Represents the DCT coefficient with sequence number l.
4. The SCV coding perceptual bit rate control method integrating spatial-frequency domain saliency features according to claim 1 is characterized in that: The S104 specifically includes: Based on the original rate-distortion model R-λ, the significance factor w of the current CTU is combined curctu , construct a rate-distortion model inspired by saliency, in which the CTU-level rate control parameter λ ctu and quantization parameter QP ctu The calculation is as follows: Among them, λ is calculated by the original rate-distortion model, and the calculation is as follows: λ=α×bpp β ; Among them, α and β are model parameters related to video characteristics; bpp represents the target number of bits per pixel of the current CTU, which is expressed as follows: Among them, Bits ctu (i) represents the target bits of the current CTU, N pixel Indicates the number of pixels in the current CTU.
5. The SCV coding perceptual bit rate control method integrating spatial-frequency domain saliency features according to claim 1, characterized in that: The S105 specifically includes: The perceptual rate-distortion model is used to estimate the optimal coding parameters for encoding; After the actual encoding is completed, the perceptual rate-distortion model is updated with the actual encoding parameters.
6. A SCV coding perceptual bit rate control device integrating spatial-frequency domain saliency features, characterized in that: include: The spatial domain saliency feature acquisition module is used to acquire the screen content video, perform saliency modeling on the screen content video in the spatial domain through convolution calculation and similarity calculation, and obtain the spatial domain saliency feature; The frequency domain saliency feature module is used to perform saliency modeling on the screen content video in the frequency domain using discrete cosine transform (DCT) to obtain frequency domain saliency features; The significance factor and target bit acquisition module is used to calculate the spatial domain significance features and the frequency domain significance features respectively, and obtain the spatial domain significance factor and the frequency domain significance factor, and then linearly weight the spatial domain significance factor and the frequency domain significance factor to obtain the significance factor of the current CTU, and then use the significance factor of the current CTU to guide the target bit allocation of the CTU level; A rate-distortion model building module is used to build a saliency-inspired rate-distortion model based on the saliency factor and target bits of the current CTU; A rate-distortion model updating module, used to estimate coding parameters according to the constructed rate-distortion model, and update the rate-distortion model according to actual coding parameters; The significance factor and target bit acquisition module is specifically used for: The spatial domain significance features and frequency domain significance features are calculated separately to obtain the spatial domain significance factor and frequency domain significance factor. The specific calculation is as follows: s curctu =S ctu / S pic ; d curctu =D ctu / D pic ; Among them, s curctu Indicates the spatial significance factor of the current CTU; d curctu Indicates the frequency domain significance factor of the current CTU; S ctu Indicates the spatial significance value of the current CTU; S pic represents the spatial saliency value of the current frame; D ctu Indicates the frequency domain significance value of the current CTU; D pic Represents the frequency domain significance value of the current frame; Linearly weight the spatial domain significance factor and the frequency domain significance factor to obtain the fusion significance factor w of the current CTU curctu , calculated as follows: w curctu =u×s curctu +(1-u)×d curctu ; Where u∈[0,1]; CTU level target bits curctu The allocation formula is as follows: Among them, Bits left Indicates the number of uncoded bits; w allctu represents the sum of the significance factors of all CTUs; w codedctu Represents the sum of the significance factors of the encoded CTUs.
