Video image bitrate allocation method, system and device, and storage medium

Abstract

Disclosed in embodiments of the present application are a video image bitrate allocation method, system and device, and a storage medium. In the technical solution provided by the embodiments of the present application, a saliency map of a target image is acquired, and real-time coding quality information of a target region in the target image is determined on the basis of the saliency map, wherein the target region is a region of interest (ROI) or a non-ROI; a coding quality redundancy amplitude of the target region is determined on the basis of the real-time coding quality information and a set quality discrimination threshold, and an offset bitrate of the target region is determined on the basis of the coding quality redundancy amplitude; and an initial bitrate of each coding block in the target region is acquired, and a coding bitrate of each coding block in the target region is configured on the basis of the initial bitrate and the offset bitrate. By using the described technical means, the bitrates of the ROI and the non-RON can be adaptively configured on the basis of the video content attribute of the video image, thereby reducing bandwidth costs while ensuring the coding quality, optimizing the video coding effect.

Description

Video image code rate allocation method, system, device and storage medium

[0001] The present application claims priority to the Chinese patent application No. 202410647881X filed on May 23, 2024 with the China Patent Office, the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] Embodiments of the present application relate to the technical field of video coding, in particular to a video image code rate allocation method, system, device and storage medium. BACKGROUND

[0003] At present, in the video coding scene, in order to improve the subjective quality of users, video image coding is usually performed based on video ROI (region of interest) technology. In the coding process, more code rate is allocated to the ROI region, so that the quality of the region of interest is obviously improved. The ROI coding implementation framework first segments the important face region by a face recognition algorithm to generate a saliency map; then the encoder sets different code rate weights for CTUs (coding tree units) in different regions according to the saliency map, which is manifested as generating a CTU-level QP_OFFSET (quantization parameter offset) table. In actual coding, the coding QP (quantization parameter) of each CTU is fine-tuned according to the QP_OFFSET table to achieve the effect of differentiated allocation of code rate.

[0004] However, when allocating code rates for different regions, a fixed offset value or historical information is usually used to allocate code rates for ROI regions and non-ROI regions. Due to the differences in video content attributes of different images, simple code rate allocation schemes are prone to cause large code rate allocation errors. For the case of complex image background, it may cause coding quality problems, and for the case of relatively simple image background, it is prone to cause code rate waste. SUMMARY

[0005] Embodiments of the present application provide a video image code rate allocation method, system, device and storage medium, which can adaptively allocate code rates for ROI regions and non-ROI regions according to video content attributes, and solve the error problem of code rate allocation between different regions in the ROI coding process.

[0006] In a first aspect, embodiments of the present application provide a video image code rate allocation method, comprising:

[0007] obtaining a saliency map of a target image, determining real-time coding quality information of a target region in the target image based on the saliency map, the target region being a region of interest or a region not of interest;

[0008] determine the coding quality redundancy range of the target region based on the real-time coding quality information and the set quality discrimination threshold, and determine the offset code rate of the target region based on the coding quality redundancy range;

[0009] obtain initial code rates of each coding block of the target region, and configure the coding code rates of each coding block of the target region based on the initial code rates and the offset code rate.

[0010] In a second aspect, an embodiment of the present application provides a video image code rate allocation system, comprising:

[0011] a quality determination module configured to obtain a saliency map of a target image, and determine real-time coding quality information of a target region in the target image based on the saliency map, the target region being a region of interest or a region of non-interest;

[0012] an offset determination module configured to determine a coding quality redundancy range of the target region based on the real-time coding quality information and the set quality discrimination threshold, and determine the offset code rate of the target region based on the coding quality redundancy range;

[0013] a code rate configuration module configured to obtain initial code rates of each coding block of the target region, and configure the coding code rates of each coding block of the target region based on the initial code rates and the offset code rate.

[0014] In a third aspect, an embodiment of the present application provides a video image code rate allocation device, comprising:

[0015] a memory and one or more processors;

[0016] the memory is configured to store one or more programs;

[0017] when the one or more programs are executed by the one or more processors, the one or more processors implement the video image code rate allocation method in the first aspect.

[0018] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores computer executable instructions, and the computer executable instructions are configured to perform the video image code rate allocation method in the first aspect when executed by a computer processor.

[0019] In a fifth aspect, an embodiment of the present application provides a computer program product, which contains instructions, and when the instructions are executed on a computer or a processor, the computer or the processor performs the video image code rate allocation method in the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0020] FIG. 1 is a flowchart of a video image code rate allocation method according to an embodiment of the present application;

[0021] FIG. 2 is an encoding flowchart of a video image according to an embodiment of the present application;

[0022] FIG. 3 is a flowchart of determining a quality discrimination threshold according to an embodiment of the present application;

[0023] FIG. 4 is a flowchart of determining an offset code rate according to an embodiment of the present application;

[0024] FIG. 5 is a flowchart of code rate configuration according to an embodiment of the present application;

[0025] FIG. 6 is a structural diagram of a video image code rate allocation system according to an embodiment of the present application;

[0026] FIG. 7 is a structural diagram of a video image code rate allocation device according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] To make the objectives, technical solutions and advantages of the present application clearer, the following will further describe the specific embodiments of the present application with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application. In addition, it should be noted that only the parts related to the present application are shown in the drawings for the convenience of description. Before discussing the example embodiments in more detail, it should be mentioned that some example embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The processes can be terminated when the operations are completed, but can also have additional steps not included in the drawings. The processes can correspond to methods, functions, procedures, subroutines, subprograms, etc.

[0028] The video image code rate allocation method provided by the present application aims to determine the offset code rate of a target region based on the encoding quality redundancy amplitude, to configure the code rate of each encoding block of the target region using the offset code rate, and to adaptively configure the code rate of the ROI region and the non-ROI region according to the video content properties of the video image.

[0029] With the progress of communication technology and the popularity of smart terminals, mobile Internet technology has penetrated into all aspects of society, and the video field has also developed vigorously, gradually replacing text and pictures and becoming the mainstream information carrier of the era. However, users' requirements for video quality and smoothness are gradually increasing, and the traffic required for video uploading and distribution is also showing an exponential growth trend, which brings obvious cost pressure to live, RTC and long / short video operators. In the face of the trade-off between quality and bandwidth cost, video ROI technology emerges as the times require, focusing on the quality of the area of interest of the human eye from the perspective of video saliency, and providing a new idea for subjective quality improvement or bandwidth saving.

[0030] Video ROI is a technology based on region of interest coding, which allocates more code rate to the ROI area according to the attention tendency of the human eye to the ROI area such as face, so that the quality of the area of interest is significantly improved. From the perspective of quality improvement, it can improve the user's viewing experience, and from the perspective of cost saving, it can provide ideas for manufacturers to save bandwidth by improving the quality of the area of interest first and then reducing the code rate. Therefore, coding based on the area of interest has been widely used in live, video conference and other video scenarios.

[0031] Region of interest coding technology is mainly applied in the code rate control link of video coding. Code rate control is a very important module in the video coding process, which controls the coding quality and compressed code stream size. Related code rate control technology generally adjusts the frame-level code rate according to the real-time code rate consumption and allocates the block-level code rate according to the texture complexity within the frame. However, code rate consumption belongs to time domain features, and texture complexity belongs to spatial features. Both of them are average indicators from the perspective of saliency and do not consider the degree of interest of the human eye. Therefore, when the code rate is limited or even sufficient, it may not be able to obtain satisfactory quality. Based on this situation, researchers introduced the concept of saliency into the coding field. In the actual coding process, more code rate is allocated to the ROI area of interest to the human eye to improve the subjective experience.

[0032] The code rate control method is generally divided into three levels of GOP level code rate allocation, frame level code rate allocation and block level code rate allocation, and the ROI-based code rate control method generally acts on the block level code rate allocation, reduces the coding loss of the ROI region by adjusting the code rate allocation weight of the ROI region, and achieves the effect of improving the subjective picture quality. A classic ROI coding implementation framework is to first segment the important face region through a face recognition algorithm to generate a saliency map; then the encoder sets different code rate weights for different regions of CTU according to the saliency map, which is manifested as generating a CTU-level QP_OFFSET table. In actual coding, the coding QP of each CTU is fine-tuned according to the QP_OFFSET table to achieve the effect of differentiated allocation of code rate. In specific implementation, the generation rule of the QP_OFFSET table can be flexibly set, such as setting different levels of QP_OFFSET for regions of different importance.

[0033] Under the above-mentioned classic framework of ROI coding, the code rate allocation method between the ROI region and the non-ROI region can be mainly divided into two categories: the first category is the QP direct constraint method, which realizes the preferential allocation of the code rate of the ROI region by setting fixed offset values for the ROI region and the non-ROI region, as shown in the following formula: QP ROI = QP ROI0 + QP1 QP fROI = QP fROI0 + QP2

[0034] Wherein, QP ROI is the adjusted quantization parameter of the ROI region, QP ROI0 is the initial quantization parameter of the ROI region, QP1 is the set quantization parameter offset of the ROI region; QP fROI is the adjusted quantization parameter of the non-ROI region, QP fROI0 is the initial quantization parameter of the non-ROI region, and QP2 is the set quantization parameter offset of the non-ROI region.

[0035] The second category is the bit rate constraint method, which measures the actual bit rate (average number of bits consumed by each pixel) generated by the ROI and non-ROI regions of the historical multiple frames to determine whether there is a problem of too small or too large bit allocation difference between the ROI and non-ROI regions, so as to guide the adjustment of the quantization granularity of the subsequent frames, and avoid the problems of obvious ROI region coding effect or poor picture quality of the non-ROI region.

[0036] However, the above two methods of allocating code rate between ROI region and non-ROI region lack consideration of video content attributes, and each pixel in the region is treated equally, which belongs to the average allocation and control in the content attribute dimension. According to the pixel equal control mode, when the background is complex, the high-frequency information of the image increases, and the information amount carried by each pixel increases, which is prone to the problem of insufficient code rate and poor encoding quality. When the background is simple, the low-frequency information of the image is dominant, and each pixel carries less information, which may cause problems such as code rate surplus and redundant encoding quality.

[0037] Therefore, the video image code rate allocation method provided by the embodiments of the present application is provided to solve the error problem of code rate allocation between different regions in the ROI encoding process.

[0038] Embodiments

[0039] FIG. 1 is a flowchart of a video image code rate allocation method provided by an embodiment of the present application. The video image code rate allocation method provided in the embodiment can be executed by a video image code rate allocation device. The video image code rate allocation device can be realized by software and / or hardware. The video image code rate allocation device can be composed of two or more physical entities, or can be composed of one physical entity. Generally, the video image code rate allocation device can be a computer, a mobile phone, a tablet computer, a video encoding server, or other video encoding devices.

[0040] The following describes the video encoding device as an example of the main body of the video image code rate allocation method. Referring to FIG. 1, the video image code rate allocation method specifically includes the following steps.

[0041] S110, obtaining a saliency map of a target image, determining real-time encoding quality information of a target region in the target image based on the saliency map, the target region being a region of interest or a region not of interest.

[0042] The embodiments of the present application aim to solve the problems of poor encoding quality and code rate waste caused by the incompatibility of ROI encoding code rate allocation and video content. The code rate allocation and encoding are performed by the quality-adaptive ROI encoding mode. Thus, in the complex and variable bandwidth environment and video content, the protection of the ROI region is achieved as much as possible, and the bandwidth is saved by reducing the code rate without damaging the subjective experience.

[0043] In reference to FIG. 2, the encoding loop represents a classic hybrid coding framework, including processes such as prediction, transform, quantization, entropy coding, filtering, etc. The video capture device can be a general term for coding devices such as smartphones, tablets, etc. When the user starts a live broadcast or video session, etc., the above-mentioned link will be automatically triggered. The video application first calls the terminal's capture hardware to capture video frames; then the video pre-processing module performs face recognition and other processes, and outputs the processed video frames and the Box information of ROI to the encoder; the quality adaptive ROI rate allocation module in the encoder collects and judges the historical coding quality information of ROI and non-ROI regions, and adaptively adjusts the inter-region rate allocation for the subsequent frame coding process to generate a QP OFFSET table; according to the generated QP OFFSET table, the QP of each CTU is adjusted, and the encoding loop such as prediction, transform, quantization, entropy coding, filtering, etc. is completed, and the encoded binary code stream is generated for transmission or local storage.

[0044] After the coding device completes the face / skin color detection and the saliency map generation, it enters the quality adaptive ROI rate allocation link. In this process, it needs to go through quality threshold setting, region quality discrimination, intra-frame adaptive rate offset, and rate adjustment processes in turn. Among them, the intra-frame adaptive rate offset module will affect the generation of the QP OFFSET table to ensure the quality of the ROI region; the rate adjustment module will affect the rate of the entire frame, control the size of the output code stream, and achieve bandwidth saving.

[0045] Exemplarily, after the device starts live broadcast, video conference, etc., the application App will first call the camera to capture video frames, and then use face recognition, skin color detection, etc. to detect face or skin color information. The face detection algorithm generally recognizes features such as eyes, nose, mouth, etc., and outputs the vertex coordinates of the Box frame covering the face; the skin color detection algorithm judges whether the block belongs to the skin color block according to whether the luminance and chroma pixel values meet the empirical skin color model, and finally outputs a skin color flag for each block.

[0046] The encoder of the coding device generally encodes in units of CTU or macroblock. The application idea of CTU and macroblock is consistent, and only the block size is different. The following will be described in terms of CTU-based block-level encoding. In the ROI-based encoding link, for the convenience of block-level retrieval during encoding, it is necessary to map the face / skin color detection results to a CTU-level marking matrix, which records which of the ROI region or the non-ROI region each CTU belongs to, which is equivalent to marking the saliency level of different regions in an image, so the matrix is also called a saliency map.

[0047] Further, by acquiring the saliency map, the real-time encoding quality of the corresponding target region is counted to complete the quality discrimination process of different regions. It should be noted that the target region of the present application can be a ROI region or a non-ROI region. By selecting the ROI region or the non-ROI region as the target region, the code rate of the region is further configured.

[0048] The real-time encoding quality information of the target region in the target image is determined based on the saliency map, and the real-time encoding quality information of the target region in the target image is determined based on the saliency map.

[0049] The average encoding quality of the corresponding target region in the last image group is acquired, and the real-time encoding quality information of the target region in the target image is calculated based on the frame number of the image group and the average encoding quality.

[0050] The average value of the encoding PSNR of the ROI / non-ROI region of the last GOP is calculated and counted, and is used as the real-time encoding quality reference. Taking the counting process of the non-ROI region as an example, the encoder generally uses the CTU row level PSNR (peak signal to noise ratio) calculation method, so the average value of the PSNR of all CTU rows of the non-ROI region can be directly used as the real-time encoding quality information of the non-ROI region. The real-time encoding quality information calculation formula is represented as:

[0051] Wherein, PSNR fROI represents the estimated real-time encoding quality information of the non-ROI region, psnri represents the PSNR value of one CTU row, and frame_num represents the frame number in one GOP.

[0052] Optionally, in the application, the real-time encoding quality information of the non-ROI region can also be determined based on the PSNR value of the non-ROI region of the last frame of video image and the set interframe correlation coefficient. The discrimination index of the real-time encoding quality information can not be limited to the PSNR value, but other commonly used quality discrimination indexes such as SSIM (structural similarity) and VMAF (video multi-evaluation method fusion) can also be used. The determination scheme of the real-time encoding quality information is not limited in the present application, and will not be described here.

[0053] S120, determining the encoding quality redundancy range of the target region based on the real-time encoding quality information and the set quality discrimination threshold, and determining the offset code rate of the target region based on the encoding quality redundancy range.

[0054] Further, after obtaining the saliency map, the target image will be sent to the encoder. The encoder first performs frame-level code rate allocation on the target image and sets the initial quantization parameter QP of all CTUs. Then, through quality adaptive ROI code rate allocation, the QP of the CTU level is adjusted according to the historical coding quality, and the QP OFFSET of the CTU level is obtained through the processes of quality threshold setting, region quality discrimination and intra-frame adaptive code rate offset in turn; for the scene with quality redundancy, the frame-level target code rate is further adjusted to save bandwidth.

[0055] Among them, the application determines the coding quality redundancy amplitude of the target region by real-time coding quality information and the set quality discrimination threshold value, so as to determine the code rate that can be offset by the target region by using the coding quality redundancy amplitude.

[0056] Referring to FIG. 3, before determining the coding quality redundancy amplitude of the target region based on real-time coding quality information and the set quality discrimination threshold value, it further includes:

[0057] 1201. Calculate the quality threshold offset value of the target region according to the size information and variance energy value of each coding block of the target region.

[0058] S1202, determine the quality discrimination threshold value based on the initial discrimination threshold value and the quality threshold offset value of the target region.

[0059] The application adopts PSNR as the discrimination index of coding quality. In actual coding, due to the complex and changeable video content, different quality discrimination threshold values should be designed for different scenes. According to the relationship between real-time coding PSNR and threshold value, the quality of region coding is further judged to be good or bad, which is used as the basis for subsequent code rate allocation. Taking the quality discrimination threshold setting process of the non-ROI region as an example, its calculation formula can be expressed as: Th fROI = Th fROI0 + Bias fROI_cplx

[0060] Among them, Th fROI represents the quality discrimination threshold value of the non-ROI region; Th fROI0 represents the initial discrimination threshold value of the non-ROI region, which is fitted and set according to the offline implementation results of different encoders, for example, 40dB; Bias fROI_cplx represents the quality threshold offset value calculated according to the video complexity, which is used to correct the quality discrimination threshold value according to the video content. Its calculation formula is:

[0061] Wherein, blk is the block size used by the target area when the video complexity is counted, which can be 64, 32, 16, 8 and the like; avg energy represents the variance energy value of the target area of the current frame averaged on each blk block. The more complex the video is, the higher the variance energy value is, the smaller the threshold offset is, and the lower the calculated quality discrimination threshold is. The more complex the video is, the more difficult the human eye is to identify the quality difference, which corresponds to the macro performance. Alternatively, the measurement method of the video complexity can also be calculated by using the texture gradient and the like, which is not limited in the present application. The quality discrimination threshold setting of the non-ROI area is taken as an example, and the quality discrimination threshold setting of the ROI area is also obtained in the same way.

[0062] Further, the real-time encoding information determined above is subtracted by the quality discrimination threshold, and the difference between the two is used as the encoding quality redundancy amplitude. Through the encoding quality redundancy amplitude, the offset code rate of the current non-ROI area can be adaptively calculated.

[0063] Wherein, the offset code rate of the target area is determined based on the encoding quality redundancy amplitude, comprising:

[0064] The set code rate step amplitude is obtained, and the offset code rate of the target area is calculated based on the code rate step amplitude and the encoding quality redundancy amplitude.

[0065] When the bandwidth saving of the fixed offset code rate ΔBr is extended to the fluctuation change scene, the encoding quality PSNR is introduced as the fluctuation scale. When the quality redundancy measured by the PSNR is larger, the weight proportion allowed to offset to the ROI area is higher. Through the fluctuation control of the target saving code rate ΔBr (i.e. the offset code rate), the adaptive code rate offset of the ROI area in the frame can be realized. Wherein, the calculation formula of the offset code rate is represented as: ΔBr=(PSNR fROI -Th fROI )*bias_step

[0066] Wherein, bias_step represents the code rate step amplitude allowed to be saved per 1 dB of quality redundancy, which is set according to the actual adjustment requirement.

[0067] S130, the initial code rate of each encoding block of the target area is obtained, and the encoding code rate of each encoding block of the target area is configured based on the initial code rate and the offset code rate.

[0068] Based on the determined offset code rate, the initial code rate of each coding block is obtained, and then the coding code rate of each coding block in the target region after code rate configuration is calculated in combination with the offset code rate. It can be understood that the offset code rate represents the code rate that can be adjusted by the target region under the condition of ensuring quality. Taking the non-ROI region as an example, the initial code rate of each coding block in the non-ROI region is subtracted by the divided offset code rate after the offset code rate is divided by the number of coding blocks in the non-ROI region, and the coding code rate of the non-ROI region is obtained. Similarly, the increased code rate value is obtained by dividing the offset code rate by the number of coding blocks in the ROI region, and the coding code rate of the ROI region is obtained by adding the divided code rate value to the initial code rate of the coding block in the ROI region.

[0069] Optionally, referring to FIG. 4, the initial code rate of each coding block in the target region is obtained, including:

[0070] S1301, obtaining an initial code rate offset ratio of the target region and area ratio information of the target region;

[0071] S1302, calculating a code rate weight offset ratio of the target region based on the initial code rate offset ratio, the area ratio information and the offset code rate, and calculating the initial code rate of each coding block in the target region based on the set basic code rate of each coding block in the target region and the code rate weight offset ratio.

[0072] When the PSNR fROI is higher than the set quality discrimination threshold Th fROI , it indicates that the coding quality of the non-ROI region is good, and there is quality redundancy, and the difference between the two represents the amplitude of the quality redundancy. The quality redundancy amplitude can be used to guide the design of the code rate ratio tilted to the ROI region, and can also be used to derive a reasonable code rate saving space. Assuming that the initial offset code rate ratio of the ROI region is 10%, when the expected bandwidth reduction amplitude is ΔBr (i.e. the offset code rate), the adaptive code rate offset ratio needs to ensure that the code rate of the ROI region will not be significantly reduced when the bandwidth decreases. Based on the invariance of the code rate of the ROI region, the equation (10%+roi_ratio+bias)*(1-ΔBr)=10%+roi_ratio

[0073] wherein roi_ratio is the area ratio of the ROI region such as a face, and 1-roi_ratio is the area ratio of the non-ROI region, and thus the target weight offset ratio is derived as follows:

[0074] Further, based on the weight offset ratio, the pre-configured basic rate of the non-ROI region is combined, and then the initial rate = (1-weight offset ratio) * basic rate. In the actual calculation process, the initial rate can also be directly replaced by the basic rate for calculation. The specific calculation method is not limited, and will not be described here.

[0075] Finally, after the intra-adaptive code rate offset, the classic code rate estimation model in the ROI coding framework can be used to estimate the CTU-level QP_OFFSET in each region. The code rate estimation model is represented as: MB bits = 937 * 2.13 -QP / 6

[0076] where MB bits represents the initial rate of the coding block. According to the offset rate of the ROI and the non-ROI region, the above formula is applied respectively, and the QP offset value QP_OFFSET of the coding block in different regions can be obtained. Further, in the actual encoding process, the QP_OFFSET value of different regions is added to the encoding QP value of each coding block in the encoder, which can complete the QP update of the coding block and smoothly enter the normal encoding process. The final implementation effect is that when there is quality redundancy in the non-ROI region, the encoding quality of the ROI region is improved, and the bandwidth cost is reduced to a certain extent. By transferring part of the code rate from the non-ROI region to the ROI region when detecting the quality redundancy in the non-ROI region, the video quality of the region of interest is improved, and the subjective experience is optimized.

[0077] Optionally, after determining the offset code rate of the target region based on the coding quality redundancy amplitude, the method further comprises:

[0078] updating the constant quality encoding information of the target image based on the coding quality redundancy amplitude or the offset code rate, to perform frame-level encoding of the target image based on the constant quality encoding information.

[0079] The application also realizes adaptive control in the code rate adjustment strategy, which saves the bandwidth cost while ensuring that the ROI region quality remains basically unchanged. Taking the crf (constant quality encoding) code rate control mode as an example, when detecting that there is quality redundancy in the non-ROI region and completing the code rate inclination to the ROI region, the target crf value of the encoder can be adjusted according to the quality redundancy amplitude or the target saved code rate ΔBr to reduce the code stream size.

[0080] wherein updating the constant quality encoding information of the target image based on the coding quality redundancy amplitude or the offset code rate comprises:

[0081] The quality encoding adjustment information is calculated based on the set step amplitude and the encoding quality redundancy amplitude or offset code rate, and the constant quality encoding information is calculated based on the initial quality encoding information of the target image and the quality encoding adjustment information.

[0082] Specifically, the sample is adjusted as follows: crf targ = crf init + (PSNR fROI - Th fROI ) * crf_step

[0083] Wherein, crf targ is the crf value corresponding to the constant quality encoding information, crf init is the crf value corresponding to the initial quality encoding information, and crf_step represents the crf step amplitude allowed to increase per 1 dB of quality redundancy. When the quality redundancy of the non-ROI region is greater, the target crf is higher, and the bandwidth cost is reduced more obviously.

[0084] So far, by monitoring the real-time encoding quality of the ROI region and the non-ROI region, the code rate tilt amplitude of the ROI region is adaptively controlled, and the target code rate of encoding is adjusted. Whether there is subjective quality improvement and code rate saving space can be judged according to the historical encoding quality of the ROI and the non-ROI region; when the subjective quality is poor, the ROI region quality is improved; when there is quality redundancy in the background, the code rate consumption is reduced. Avoid the problems of low encoding quality, partial region code rate redundancy caused by complex / simple video content, etc., while providing a solution for quality improvement and code rate saving, which can be diversified according to actual needs. In the case of ensuring the quality, the bandwidth cost is saved for video operation manufacturers; in the case of changing content, a better subjective experience is provided for users.

[0085] The above, by obtaining the saliency map of the target image, determining the real-time encoding quality information of the target region in the target image based on the saliency map, the target region is the region of interest or the region not of interest; determining the encoding quality redundancy amplitude of the target region based on the real-time encoding quality information and the set quality discrimination threshold, determining the offset code rate of the target region based on the encoding quality redundancy amplitude; obtaining the initial code rate of each encoding block of the target region, configuring the encoding code rate of each encoding block of the target region based on the initial code rate and the offset code rate. By using the above technical means, the offset code rate of the target region is determined based on the encoding quality redundancy amplitude, and the code rate of each encoding block of the target region is configured using the offset code rate. In this way, the code rate of the ROI region and the non-ROI region can be adaptively configured according to the video content properties of the video image, avoiding the situation that the code rate configuration error between the ROI region and the non-ROI region leads to low quality encoding or code rate waste, while ensuring the encoding quality and reducing the bandwidth cost, optimizing the video encoding effect.

[0086] On the basis of the above-mentioned embodiments, Fig. 6 is a structural schematic diagram of a video image code rate allocation system provided by the present application. Referring to Fig. 6, the video image code rate allocation system provided by the present embodiment specifically comprises a quality determination module 21, an offset determination module 22 and a code rate configuration module 23.

[0087] The quality determination module 21 is configured to acquire a saliency map of a target image, determine real-time encoding quality information of a target region in the target image based on the saliency map, the target region being a region of interest or a region of non-interest;

[0088] The offset determination module 22 is configured to determine an encoding quality redundancy amplitude of the target region based on the real-time encoding quality information and a set quality discrimination threshold, and determine an offset code rate of the target region based on the encoding quality redundancy amplitude.

[0089] The code rate configuration module 23 is configured to acquire initial code rates of each coding block of the target region, and configure encoding code rates of each coding block of the target region based on the initial code rates and the offset code rate.

[0090] Specifically, the determination of the real-time encoding quality information of the target region in the target image based on the saliency map comprises:

[0091] acquiring an encoding quality mean value of a corresponding target region in a latest image group, and calculating the real-time encoding quality information of the target region in the target image based on the number of frames of the image group and the encoding quality mean value.

[0092] Specifically, before the determination of the encoding quality redundancy amplitude of the target region based on the real-time encoding quality information and the set quality discrimination threshold, the method further comprises:

[0093] calculating a quality threshold offset value of the target region according to size information and variance energy values of each coding block of the target region;

[0094] determining the quality discrimination threshold based on an initial discrimination threshold of the target region and the quality threshold offset value.

[0095] Specifically, the determination of the offset code rate of the target region based on the encoding quality redundancy amplitude comprises:

[0096] acquiring a set code rate step amplitude, and calculating the offset code rate of the target region based on the code rate step amplitude and the encoding quality redundancy amplitude.

[0097] Specifically, the acquisition of the initial code rates of each coding block of the target region comprises:

[0098] acquiring an initial code rate offset proportion of the target region and area proportion information of the target region;

[0099] The initial code rate offset ratio, the area proportion information and the offset code rate are used to calculate a code rate weight offset ratio of the target region, and the initial code rate of each coding block of the target region and the code rate weight offset ratio are used to calculate an initial code rate of each coding block of the target region.

[0100] Specifically, after the offset code rate of the target region is determined based on the coding quality redundancy amplitude, the method further includes:

[0101] The constant quality coding information of the target image is updated based on the coding quality redundancy amplitude or the offset code rate, so that the frame-level coding of the target image is performed based on the constant quality coding information.

[0102] The constant quality coding information of the target image is updated based on the coding quality redundancy amplitude or the offset code rate, so that the frame-level coding of the target image is performed based on the constant quality coding information.

[0103] The quality coding adjustment information is calculated based on the set step amplitude and the coding quality redundancy amplitude or the offset code rate, and the constant quality coding information is calculated based on the initial quality coding information of the target image and the quality coding adjustment information.

[0104] The above method includes: obtaining a saliency map of a target image; determining real-time coding quality information of a target region in the target image based on the saliency map, the target region being a region of interest or a non-region of interest; determining a coding quality redundancy amplitude of the target region based on the real-time coding quality information and a set quality discrimination threshold; determining an offset code rate of the target region based on the coding quality redundancy amplitude; obtaining initial code rates of each coding block of the target region; and configuring coding code rates of each coding block of the target region based on the initial code rates and the offset code rate. By determining the offset code rate of the target region based on the coding quality redundancy amplitude, the coding code rates of each coding block of the target region are configured using the offset code rate, so that the code rates of the ROI region and the non-ROI region can be adaptively configured according to the video content properties of the video image, the situation that the coding quality is low or the code rate is wasted due to the coding rate configuration error between the ROI region and the non-ROI region is avoided, the coding quality is ensured, the bandwidth cost is reduced, and the video coding effect is optimized.

[0105] The video image code rate allocation system provided by the embodiments of the present application can be configured to perform the video image code rate allocation method provided by the above embodiments, and has the corresponding functions and advantages.

[0106] On the basis of the foregoing actual examples, the embodiments of the present application further provide a video image code rate allocation device. Referring to FIG. 7, the video image code rate allocation device includes a processor 31, a memory 32, a communication module 33, an input device 34, and an output device 35. The memory, as a computer readable storage medium, can be configured to store software programs, computer executable programs, and modules, such as program instructions / modules (for example, a quality determination module, an offset determination module, and a code rate configuration module in a video image code rate allocation system) corresponding to the video image code rate allocation method described in any of the embodiments of the present application. The communication module is configured to perform data transmission. The processor performs various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory, that is, implements the video image code rate allocation method described above. The input device can be configured to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device can include a display device such as a display screen. The video image code rate allocation device provided above can be configured to perform the video image code rate allocation method provided in the foregoing embodiments, and has corresponding functions and beneficial effects.

[0107] On the basis of the foregoing embodiments, the embodiments of the present application further provide a computer readable storage medium storing computer executable instructions. The computer executable instructions, when executed by a computer processor, are configured to perform a video image code rate allocation method. The storage medium can be any of various types of memory devices or storage devices. Of course, the computer executable instructions of the computer readable storage medium provided in the embodiments of the present application are not limited to the video image code rate allocation method described above, but can also perform related operations in the video image code rate allocation method provided in any of the embodiments of the present application.

[0108] On the basis of the foregoing embodiments, the embodiments of the present application further provide a computer program product. The technical solutions of the present application, essentially or the part that contributes to the prior art, or the whole or part of the technical solutions can be embodied in the form of a software product. The computer program product is stored in a storage medium and includes instructions for causing a computer device, a mobile terminal, or a processor therein to execute all or part of the steps of the video image code rate allocation method described in the embodiments of the present application.

Claims

1. A method of video image bit rate allocation, wherein, The method comprises the following steps: obtaining a saliency map of a target image, determining real-time encoding quality information of a target region in the target image based on the saliency map, the target region being a region of interest or a region of non-interest; determining an encoding quality redundancy amplitude of the target region based on the real-time encoding quality information and a set quality discrimination threshold, and determining an offset code rate of the target region based on the encoding quality redundancy amplitude; obtaining initial code rates of each encoding block of the target region, and configuring encoding code rates of each encoding block of the target region based on the initial code rates and the offset code rate.

2. The video image bit rate allocation method of claim 1, wherein, The method further comprises the following steps: obtaining a set code rate step amplitude, and calculating the offset code rate of the target region based on the code rate step amplitude and the encoding quality redundancy amplitude.

3. The video image bit rate allocation method of claim 1, wherein, The method further comprises the following steps: obtaining an initial code rate offset ratio of the target region and area proportion information of the target region; calculating a code rate weight offset ratio of the target region based on the initial code rate offset ratio, the area proportion information and the offset code rate, and calculating the initial code rates of each encoding block of the target region based on a set basic code rate of each encoding block of the target region and the code rate weight offset ratio.

4. The video image bit rate allocation method of claim 1, wherein, The method further comprises the following steps: obtaining an encoding quality mean value of the target region in a latest group of pictures, and calculating the real-time encoding quality information of the target region in the target image based on the number of frames of the group of pictures and the encoding quality mean value.

5. The method of video image bit rate allocation according to claim 1, wherein, The method further comprises the following steps before determining the encoding quality redundancy amplitude of the target region based on the real-time encoding quality information and the set quality discrimination threshold: calculating a quality threshold offset value of the target region according to size information and variance energy values of each encoding block of the target region; determining the quality discrimination threshold based on an initial discrimination threshold of the target region and the quality threshold offset value.

6. The method of video image bit rate allocation according to any of claims 1-5, wherein, The method further comprises the following steps after determining the offset code rate of the target region based on the encoding quality redundancy amplitude: updating constant quality encoding information of the target image based on the encoding quality redundancy amplitude or the offset code rate, so as to perform frame-level encoding of the target image based on the constant quality encoding information.

7. The video image bit rate allocation method of claim 6, wherein, The method further comprises the following steps of updating the constant quality encoding information of the target image based on the encoding quality redundancy amplitude or the offset code rate: calculating quality encoding adjustment information based on a set step amplitude and the encoding quality redundancy amplitude or the offset code rate, and calculating the constant quality encoding information based on initial quality encoding information of the target image and the quality encoding adjustment information.

8. A video image bit rate allocation system wherein, The method comprises the following steps: a quality determination module configured to obtain a saliency map of a target image, and determine real-time encoding quality information of a target region in the target image based on the saliency map, the target region being a region of interest or a region of non-interest; The offset determination module is configured to determine a coding quality redundancy range of the target region based on the real-time coding quality information and a set quality discrimination threshold, and determine an offset code rate of the target region based on the coding quality redundancy range. The code rate configuration module is configured to obtain initial code rates of each coding block of the target region, and configure coding code rates of each coding block of the target region based on the initial code rates and the offset code rate.

9. A video image bit rate allocation apparatus, wherein, The method comprises: a memory and one or more processors; the memory is configured to store one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the video image code rate allocation method as claimed in any one of claims 1-7.

10. A computer readable storage medium, wherein, The computer readable storage medium stores computer executable instructions, which when executed by a computer processor, are configured to perform the video image code rate allocation method as claimed in any one of claims 1-7.

11. A computer program product, wherein, The computer program product contains instructions, which when executed on a computer or processor, cause the computer or processor to perform the video image code rate allocation method as claimed in any one of claims 1-7.

Images