Image coding method and device, computer equipment, chip and coding and decoding system
By combining scene switching of image frames and dynamic adjustment of quantization parameters and bit rate control of bandwidth changes, the problem of quantization parameters determination in image encoding and the problem of number of coded bits under bandwidth changes is solved, and the smoothness of video transmission and image quality are achieved. promote.
Patent Information
- Application Number
- CN202310628699.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-30
- Publication Date
- 2024-12-03
AI Technical Summary
The prior art is difficult to effectively determine the quantization parameters during image encoding, resulting in poor image quality after reconstruction, and it is difficult to adjust the number of encoded bits in real time in the scenario of bandwidth changes, affecting the smoothness of video transmission.
Combining the scene switching and bandwidth changes of image frames, the quantization parameters are dynamically adjusted to ensure that the number of encoded bits of the image code conforms to the network bandwidth. The rate quantization model is used for refined code rate control, including frame-level and coded block-level code rate control, and Perform frame skipping operation in low bandwidth situations.
Optimize the number of image coded bits in the bandwidth changing scenario to ensure the smoothness of video transmission and image quality, avoid video lag caused by the number of coded bits exceeding the bandwidth limit, and improve the visual experience of the viewer.
Smart Images

Figure CN119071485A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of multimedia technology, and in particular to an image encoding method, apparatus, computer equipment, chip and encoding and decoding system. Background Art
[0002] At present, the image quality and bit rate of video coding are controlled based on bit rate control technology. For example, the larger the value of the quantization parameter and the lower the bit rate, the less effective information in the image frame contained in the bitstream, and the smaller the bandwidth occupied by the transmission bitstream, resulting in poor quality of the reconstructed image; conversely, the smaller the value of the quantization parameter, the better the quality of the reconstructed image, and the higher the bit rate, the more redundant information of the image frame contained in the bitstream and the more bits of the bitstream, and the larger the bandwidth occupied by the transmission bitstream. Therefore, how to determine the quantization parameter used in image coding to ensure the quality of the reconstructed image is an urgent problem to be solved. Summary of the invention
[0003] The present application provides an image encoding method, apparatus, computer equipment, chip and encoding and decoding system, which determine the quantization parameters used for image encoding and decoding in combination with scene switching and bandwidth changes of image frames to ensure the quality of the reconstructed image.
[0004] In a first aspect, an image encoding method is provided, the method comprising: determining a quantization parameter used for encoding a plurality of image frames in an acquired video according to factors affecting image quality. The factors affecting image quality include a scene type and a bandwidth variation parameter of the image frame. The image frame is encoded according to the quantization parameter corresponding to the image frame to obtain the number of encoding bits of the image frame.
[0005] The bandwidth change parameter can be used to indicate the size relationship between the current bandwidth and the previous bandwidth. The scene types of the image frame may include switching scenes and non-switching scenes. The switching scene may refer to a change in the scene of the current frame relative to the scene of the previous frame. In order to ensure the image quality of the current frame in the switching scene, the number of coding bits of the current frame may be higher than the number of coding bits of the previous frame. The non-switching scene may refer to a scene in the subsequent frame being similar to the scene in the current frame. In order to reduce the number of coding bits of the image frame, the number of coding bits in the subsequent frame may be lower than the number of coding bits in the current frame.
[0006] During the video encoding process, the number of encoding bits of the image frame is controlled according to the bandwidth change and the scene type. When the current bandwidth is less than the previous bandwidth or the bandwidth is low, the value of the quantization parameter can be increased to reduce the number of encoding bits of the current frame when switching scenes, and control the current frame to be over-encoded, even if the video transmission meets the lower network bandwidth, to avoid the current bandwidth being unable to meet the number of encoding bits of the current frame. In this way, the number of encoding bits of the image frame is controlled in the bandwidth change scenario, especially in combination with the scene type of the image frame, so that the video is transmitted in real time in accordance with the network bandwidth. The picture is smooth during video playback, which improves the viewer's visual experience.
[0007] In one possible implementation, quantization parameters used for encoding multiple image frames are determined based on factors that affect image quality, including: determining a target number of coding bits for each of the multiple image frames based on the factors that affect image quality; and determining a quantization parameter for the image frame based on the target number of coding bits.
[0008] The target number of coding bits indicates the expected number of bits after lossy coding. The target number of coding bits is allocated to each image frame based on factors that affect image quality. The quantization parameter used for encoding each image frame is determined based on the target number of coding bits for each image frame, so that the quantization parameter is more accurate, and the actual number of coding bits obtained based on the quantization parameter encoding is more accurate, so as to more accurately control overcoding in bandwidth variation scenarios.
[0009] In another possible implementation, the multiple image frames include a previous frame, a current frame, and a subsequent frame, and the scene of the current frame is different from the scene of the previous frame. The quantization parameters used for encoding the multiple image frames are determined according to factors affecting image quality, including: according to the factors, the quantization parameters of the current frame are attenuated to obtain the quantization parameters of the subsequent frame. As the current bandwidth increases and becomes stable, the value of the quantization parameter can be gradually reduced, and the number of encoding bits of the subsequent frame can be increased to ensure image quality.
[0010] In another possible implementation, the quantization parameter of the current frame is attenuated to obtain the quantization parameter of the subsequent frame, including: attenuating the quantization parameter of the current frame based on the frame number of the subsequent frame to obtain the quantization parameter of the subsequent frame.
[0011] For the switching scenario, the quantization parameter of the current frame is set larger to make the number of coded bits of the current frame smaller, close to the target number of coded bits under low bandwidth constraints, thereby controlling the current frame over-coding under low bandwidth conditions. As the current bandwidth increases and becomes stable again, the larger quantization parameter of the current frame is regularly attenuated based on the number of frames in the subsequent frames, so that the regularity of the value of the quantization parameter of the subsequent frames becomes smaller, and the number of coded bits in the subsequent frames becomes larger, thereby gradually restoring a higher image quality.
[0012] In another possible implementation, the image frame is encoded according to the quantization parameters corresponding to the image frame to obtain the number of coding bits of the image frame, including: predicting the number of coding bits of the coding block in the image frame according to the model parameters and quantization parameters of the rate quantization model, the rate quantization model is used to indicate the relationship between the quantization parameters and the number of coding bits, and the model parameters include at least one of a coding block complexity coefficient, an image compensation coefficient and a quantity adjustment coefficient.
[0013] Each image frame can contain multiple coding blocks, and the granularity of coding blocks is finer than that of image frames. When the current bandwidth is stable, the encoder performs coding block-level rate control on the basis of frame-level rate control, making the rate control more refined and the result of the number of coding bits more accurate.
[0014] In another possible implementation, the method further includes: selecting a rate quantization model type based on a scene type of the image frame. Based on the different scene types of the image frame, when the scene is switched, a switching frame rate quantization model is selected; when the scene is not switched, a non-switching frame rate quantization model is selected, so that the bit rate control has scene adaptability.
[0015] In another possible implementation, the number of coding bits of a coding block in an image frame is predicted based on model parameters and quantization parameters of a rate quantization model, including: determining the number of coding bits of a current coding block based on the model parameters and the quantization parameters; updating the model parameters based on a bandwidth variation parameter and the number of coding bits of the current coding block to obtain updated model parameters; and predicting the number of coding bits of a subsequent coding block based on the updated model parameters and the quantization parameters.
[0016] Since the images presented by each coding block in an image frame may be different, if the model parameters of the rate quantization model are fixed values, the number of coding bits predicted for each coding block based on the fixed model parameters is the same, resulting in inaccurate prediction results. During the encoding process, the model parameters of the rate quantization model are updated according to the encoded data to improve the accuracy of the result of the rate quantization model predicting the number of coding bits of the coding block.
[0017] In another possible implementation, quantization parameters used for encoding multiple image frames are determined based on factors that affect image quality, including: when the current bandwidth meets the bandwidth threshold, performing a frame skipping operation on image frames other than the current frame among the multiple image frames according to a bandwidth change parameter; and determining quantization parameters used for encoding remaining image frames among the multiple image frames after frame skipping according to factors that affect image quality.
[0018] In extremely low bandwidth conditions, in order to ensure video quality and prevent video discontinuity caused by jumping switching frames, frame skipping operations are performed on image frames other than switching frames among multiple image frames. By reducing the number of image frames, the number of encoding bits after encoding the image frames is reduced.
[0019] In another possible implementation, the method further includes: receiving a control operation, the control operation including at least one of setting a bandwidth threshold, detecting bandwidth mutation, detecting scene switching, and setting a rate control parameter. The encoder receives the control operation of the external controller, so that the rate control supports multiple scenarios, and can control over-editing in the case of low bandwidth, bandwidth mutation, scene switching, etc. The rate control parameters can be set according to actual needs, so that the rate control is more refined and the result of the number of encoded bits is more accurate.
[0020] In another possible implementation, the method further includes: displaying an over-editing rate, where the over-editing rate is used to indicate the extent to which the number of coding bits of the image frame exceeds the target number of coding bits. Displaying the over-editing rate can facilitate the user to clearly and obviously know the over-editing situation. According to the real-time over-editing situation of the image frame, the controller flexibly adjusts the control operation to accurately control the encoder to encode the image frame.
[0021] In a second aspect, an image encoding device is provided, the device comprising an acquisition module for acquiring multiple image frames in a video; a code control module for determining quantization parameters used for encoding the multiple image frames based on factors affecting image quality, the factors including scene types and bandwidth variation parameters of the image frames, the scene types including switching scenes and non-switching scenes; and an encoding module for encoding the image frames according to the quantization parameters corresponding to the image frames to obtain the number of encoding bits of the image frames.
[0022] In one possible implementation, when the code control module determines the quantization parameters used for encoding multiple image frames based on factors affecting image quality, it is specifically used to: determine the target number of coding bits for each of the multiple image frames based on the factors affecting image quality; and determine the quantization parameters of the image frames based on the target number of coding bits.
[0023] In another possible implementation, the multiple image frames include a previous frame, a current frame, and a subsequent frame, and the scene of the current frame is different from the scene of the previous frame. When the code control module determines the quantization parameters used for encoding the multiple image frames based on factors affecting the image quality, it is specifically used to: based on the factors, attenuate the quantization parameters of the current frame to obtain the quantization parameters of the subsequent frame.
[0024] In another possible implementation, the code control module, according to the factors, attenuates the quantization parameter of the current frame to obtain the quantization parameter of the subsequent frame, and is specifically used to: attenuate the quantization parameter of the current frame based on the frame number of the subsequent frame to obtain the quantization parameter of the subsequent frame.
[0025] In another possible implementation, when the encoding module encodes the image frame according to the quantization parameters corresponding to the image frame to obtain the number of coding bits of the image frame, it is specifically used to: predict the number of coding bits of the coding block in the image frame according to the model parameters and quantization parameters of the rate quantization model, the rate quantization model is used to indicate the relationship between the quantization parameters and the number of coding bits, and the model parameters include at least one of the coding block complexity coefficient, the image compensation coefficient and the quantity adjustment coefficient.
[0026] In another possible implementation manner, the apparatus further includes a selection module configured to select a type of rate quantization model based on a scene type of the image frame.
[0027] In another possible implementation, when the prediction module predicts the number of coding bits of a coding block within an image frame according to the model parameters and quantization parameters of the rate quantization model, it is specifically used to: determine the number of coding bits of the current coding block according to the model parameters and the quantization parameters; update the model parameters according to the bandwidth change parameters and the number of coding bits of the current coding block to obtain updated model parameters; and predict the number of coding bits of a subsequent coding block according to the updated model parameters and the quantization parameters.
[0028] In another possible implementation, when the code control module determines the quantization parameters used for encoding multiple image frames based on factors that affect image quality, it is specifically used to: when the current bandwidth meets the bandwidth threshold, perform a frame skipping operation on image frames other than the current frame among the multiple image frames according to the bandwidth change parameter; and determine the quantization parameters used for encoding the remaining image frames after frame skipping among the multiple image frames according to the factors that affect image quality.
[0029] In another possible implementation, the apparatus further includes a receiving module for receiving a control operation, where the control operation includes at least one of setting a bandwidth threshold, detecting a bandwidth mutation, detecting a scene switch, and setting a bit rate control parameter.
[0030] In another possible implementation, the method further includes a display module, configured to display an over-coding rate, where the over-coding rate is used to indicate the extent to which the number of coding bits of the image frame exceeds the target number of coding bits.
[0031] In a third aspect, a computer device is provided, which includes a processor and a memory, wherein the memory is used to store a set of computer instructions; when the processor executes the set of computer instructions as an execution device in the first aspect or any possible implementation of the first aspect, the operation steps of the image encoding method in the first aspect or any possible implementation of the first aspect are executed.
[0032] In a fourth aspect, a chip is provided, the chip including a processor and a power supply circuit, the power supply circuit supplies power to the processor, and the processor is used to execute the operation steps of the image encoding method in the first aspect or any possible implementation of the first aspect. In a possible design, the chip also includes a memory for storing program instructions and / or data.
[0033] In a fifth aspect, a coding and decoding system is provided, which includes an encoder and a controller, wherein the encoder is used to receive control operations sent by the controller to execute the operation steps of the image coding method in the first aspect or any possible implementation of the first aspect.
[0034] In a sixth aspect, a computer-readable storage medium is provided, comprising: computer software instructions; when the computer software instructions are executed in a computing device, the computing device executes the operating steps of the image encoding method in the first aspect or any possible implementation of the first aspect.
[0035] In a seventh aspect, a computer program product is provided. When the computer program product is run on a computer, the computing device executes the operating steps of the image encoding method in the first aspect or any possible implementation of the first aspect.
[0036] The technical effects brought about by any design method in the second to seventh aspects can refer to the technical effects brought about by the first aspect or different design methods in the first aspect, and will not be repeated here.
[0037] Based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 A schematic diagram of a scene switching process provided by this application;
[0039] Figure 2 A schematic diagram of the structure of a coding and decoding system provided by this application;
[0040] Figure 3 A flowchart of an image encoding method provided by the present application;
[0041] Figure 4 A schematic diagram of a process for determining a quantization parameter provided in this application;
[0042] Figure 5 A schematic diagram of a flow chart of coding block-level rate control and frame skipping provided in this application;
[0043] Figure 6 A schematic diagram of a coding block level rate control process provided by this application;
[0044] Figure 7 A schematic diagram of a frame skipping process provided by this application;
[0045] Figure 8 A schematic diagram of a display interface provided for this application;
[0046] Fig. 9 A schematic diagram of the structure of an image encoding device provided by the present application;
[0047] Fig.10 A schematic diagram of the structure of a computer device provided in this application. DETAILED DESCRIPTION
[0048] The technical solutions involved in this application may not only be applied to video coding standards (such as H.264, HEVC and other standards), but may also be applied to future video coding standards (such as H.266 standards). The terms used in the implementation method section of this application are only used to explain the specific embodiments of this application, and are not intended to limit this application. The following is a brief introduction to some concepts that may be involved in this application.
[0049] Video contains multiple continuous images. When the continuous images change at more than 24 frames per second, according to the principle of visual persistence, the human eye cannot distinguish a single static image, and the multiple images that appear smooth and continuous are called video.
[0050] Video coding refers to the processing of a sequence of pictures to form a video or video sequence. In the field of video coding, the terms "picture", "frame" or "image" can be used as synonyms. Video coding is performed on the source side and generally includes processing (e.g., compressing) the original video picture to reduce the amount of data required to represent the video picture under the condition of meeting a certain image quality, so as to store and / or transmit it more efficiently. The "coding" of video pictures involved in the embodiments should be understood as "coding" involving video sequences. Video coding may also be referred to as image coding (image coding) also known as image compression (image compression). Image decoding (image decoding) refers to the inverse process of image coding.
[0051] Video sequences include a series of images (pictures), which are further divided into slices (slices), and slices are further divided into coding blocks (blocks). Video coding is performed in units of coding blocks. In some new video coding standards, the concept of coding blocks is further expanded. For example, in the H.264 standard, there are macroblocks (MB), which can be further divided into multiple prediction blocks (partitions) that can be used for predictive coding. In the high-efficiency video coding (HEVC) standard, basic concepts such as coding units (CU), prediction units (PU) and transform units (TU) are used to functionally divide various block units and describe them using a new tree-based structure. For example, CU can be divided into smaller CUs according to the quadtree, and smaller CUs can be further divided to form a quadtree structure. CU is the basic unit for dividing and encoding the coded image. There are similar tree structures for PU and TU. PU can correspond to prediction blocks and is the basic unit of predictive coding. CU is further divided into multiple PUs according to the division mode. TU may correspond to a transform block, which is a basic unit for transforming a prediction residual. However, whether CU, PU or TU, they all essentially belong to the concept of coding block (or coding unit).
[0052] For example, in HEVC, a CTU is split into multiple CUs using a quadtree structure represented as a coding tree. A decision is made at the CU level whether to use inter-picture (temporal) or intra-picture (spatial) prediction to encode a picture area. Each CU can be further split into one, two, or four PUs according to the PU split type. The same prediction process is applied within a PU, and the relevant information is transmitted to the decoder on a PU basis. After obtaining the residual block by applying the prediction process based on the PU split type, the CU can be divided into transform units (TUs) according to other quadtree structures similar to the coding tree for the CU. In the latest developments in video compression technology, quadtree and binarytree (QTBT) are used to split frames to split coding blocks. In the QTBT block structure, the CU can be square or rectangular in shape.
[0053] Lossless video coding means that the original video picture can be reconstructed, that is, the reconstructed video picture has the same quality as the original video picture (assuming there is no transmission loss or other data loss during storage or transmission).
[0054] Lossy video coding refers to reducing the number of bits required to represent a video picture by performing further compression, such as quantization, while the decoder side cannot fully reconstruct the video picture, that is, the quality of the reconstructed video picture is lower or worse than the quality of the original video picture.
[0055] Bitstream (Data Rate) refers to the binary stream generated after encoding an image or video. Bitstream is also called code stream or code rate, which is the number of bits transmitted per unit time. It is an important part of picture quality control in image coding. For images of the same resolution, the larger the bitstream of the image, the smaller the compression ratio and the better the picture quality.
[0056] Bit rate control refers to the function of adjusting the bit rate during the encoding and decoding process, which is abbreviated as bit rate control below.
[0057] The bit rate control modes include constant bit rate (CBR) and variable bit rate (VBR).
[0058] Constant Bit Rate (CBR) ensures that the bit rate is stable within the bit rate statistics time.
[0059] Dynamic bit rate (Variable Bit Rate, VBR) allows the bit rate to fluctuate within the bit rate statistics time, thereby ensuring the stable quality of the encoded image.
[0060] Quantization refers to the process of mapping the continuous value of a signal into multiple discrete amplitudes.
[0061] The quantization parameter (QP) is used to quantize the residual value generated by the prediction operation or the coefficient generated by the transform operation during the encoding process. During the decoding process, the syntax elements are inversely quantized to obtain the residual value or coefficient. The quantization parameter is a parameter used in the quantization process. Generally, the larger the value of the quantization parameter, the more obvious the quantization degree, the worse the quality of the reconstructed image, and the lower the bit rate; conversely, the smaller the value of the quantization parameter, the better the quality of the reconstructed image, and the higher the bit rate.
[0062] A group of pictures (GOP) refers to a set of pictures in a video sequence. A GOP is a group of continuous pictures, starting with an I frame (intra-coded frame), including multiple P frames (predictive frames) or B frames (bi-directional frames), and ending before the next I frame.
[0063] The present application provides an image encoding method, that is, according to factors affecting image quality, determining the quantization parameters used for encoding multiple image frames in an acquired video. Among them, the factors affecting image quality include the scene type and bandwidth variation parameters of the image frame. The image frame is encoded according to the quantization parameter corresponding to the image frame to obtain the number of encoding bits of the image frame. The bandwidth variation parameter can be used to indicate the size relationship between the current bandwidth and the previous bandwidth. The scene type of the image frame can include a switching scene and a non-switching scene.
[0064] To facilitate understanding of the scene switching in a video sequence, for example, Figure 1 A schematic diagram of a scene switching process provided by this application. Figure 1 As shown, scene switching detection is performed on a driving scene of a vehicle in a video. The vehicle drives on a road and enters a parking lot. The parking lot scene in the current frame switches scene relative to the vehicle driving scene in the previous frame. For ease of description, the current frame in which the scene changes relative to the scene in the previous frame can be referred to as a switching frame. For example, the current frame presenting the parking lot scene can be referred to as a switching frame.
[0065] Compared with the GOP-level bit rate control method, the scene in the video is detected first. If the scene of the current frame changes relative to the scene of the previous frame, that is, when a scene switch occurs, the current frame is set to an I frame. Then, during the encoding process, the number of coding bits of the I frame is adjusted, and a higher number of coding bits is allocated to the I frame to improve the impact of scene switching on image quality. However, this method is not suitable for scenarios with low bandwidth or bandwidth fluctuations. In this scenario, a higher number of coding bits will exceed the limit of the lower network bandwidth, resulting in video freezes.
[0066] The solution provided by the present application controls the number of coding bits of the image frame according to the bandwidth change and the scene type during the video encoding process. When the current bandwidth is less than the previous bandwidth, or the bandwidth is low, the value of the quantization parameter can be increased to reduce the number of coding bits of the current frame when switching scenes, and control the current frame to be over-encoded, so that the current bandwidth can be controlled to meet the lower network bandwidth during video transmission, so as to avoid the current bandwidth being unable to meet the number of coding bits of the current frame. In this way, the number of coding bits of the image frame is controlled in the bandwidth change scenario, especially in combination with the scene type of the image frame, so that the video meets the network bandwidth during real-time transmission. The picture is smooth during video playback, which improves the viewer's visual experience.
[0067] The implementation methods of the present application are described below with reference to the accompanying drawings.
[0068] Figure 2The present invention provides a schematic diagram of the structure of a coding system. The coding system 200 includes a source device 210 and a destination device 220. The source device 210 is used to compress and encode a video or an image to obtain a bit stream, and transmit the bit stream to the destination device 210. The destination device 210 decodes the bit stream, reconstructs the video or image, and displays the reconstructed image. The image coding method provided in the embodiment of the present application is applied to the source device 210, and the method for decoding the bit stream by the destination device 220 is not limited.
[0069] Specifically, the source device 210 includes an image collector 211 , a preprocessor 212 , an encoder 213 , and a communication interface 214 .
[0070] The image collector 211 is used to collect multiple image frames in the video. The image collector 211 may include or may be any type of image capture device, for example, for capturing real-world images, and / or any type of image or comment (for screen content encoding, some text on the screen is also considered to be an image or part of an image to be encoded) generation device, for example, a computer graphics processor for generating computer animation images, or any type of device for acquiring and / or providing real-world images, computer animation images (e.g., screen content, virtual reality (VR) images), and / or any combination thereof (e.g., augmented reality (AR) images). The image collector 211 may be a camera for capturing images or a memory for storing images, and the image collector 211 may also include any type of (internal or external) interface for storing previously captured or generated images and / or acquiring or receiving images. When the image collector 211 is a camera, the image collector 211 may be, for example, a local integrated camera or an integrated camera integrated in a source device; when the image collector 211 is a memory, the image collector 211 may be, for example, a local integrated memory or an integrated memory integrated in a source device. When the image collector 211 includes an interface, the interface may be, for example, an external interface for receiving images from an external video source, the external video source may be, for example, an external image capture device such as a camera, an external memory, or an external image generating device, the external image generating device may be, for example, an external computer graphics processor, a computer, or a server. The interface may be any type of interface according to any proprietary or standardized interface protocol, such as a wired or wireless interface, an optical interface.
[0071] An image can be considered as a two-dimensional array or matrix of pixels (picture elements). The pixels in the array can also be called sampling points. The number of sampling points in the array or image in the horizontal and vertical directions (or axes) defines the size and / or resolution of the image. In order to represent color, three color components are usually used, that is, the image can be represented as or include three sampling arrays. For example, in the RBG format or color space, the image includes corresponding red, green and blue sampling arrays. However, in video coding, each pixel is usually represented in a brightness / chrominance format or color space, such as for an image in the YUV format, including a brightness component indicated by Y (sometimes also indicated by L) and two chrominance components indicated by U and V. The brightness (luma) component Y represents the brightness or grayscale level intensity (for example, the two are the same in a grayscale image), while the two chroma (chroma) components U and V represent the chroma or color information components. Accordingly, an image in the YUV format includes a brightness sampling array of brightness sampling values (Y) and two chroma sampling arrays of chrominance values (U and V). An image in RGB format can be converted or transformed into a YUV format, and vice versa, which process is also referred to as color conversion or conversion. If the image is black and white, the image can only include a brightness sampling array. In the present application, the multiple image frames transmitted to the encoder 213 by the image collector 211 can also be referred to as raw image data.
[0072] The preprocessor 212 is used to receive the original image collected by the image collector 211, and preprocess the original image to obtain a preprocessed image. For example, the preprocessing performed by the preprocessor 212 includes refurbishment, color format conversion (for example, from RGB format to YUV format), color adjustment or denoising, etc.
[0073] The encoder 213 is used to receive the preprocessed image generated by the preprocessor 212, and compress and encode the preprocessed image to obtain a bit stream. For example, the encoder 213 may include a code control unit 2131 and a coding unit 2132, and the code control unit 2131 is used to determine the quantization parameters used for encoding multiple image frames according to factors affecting image quality, so that the coding unit 2132 encodes the image frame according to the quantization parameters corresponding to the image frame to obtain the number of coded bits of the image frame. For specific methods, reference may be made to the description of steps 320 to 330 in the following embodiments.
[0074] The communication interface 214 is used to receive the bit stream generated by the encoder 213, and send the bit stream to the destination device 220 through the communication channel 230, so that the destination device 220 can reconstruct the original image according to the bit stream.
[0075] The destination device 220 includes a display 221 , a post-processor 222 , a decoder 223 , and a communication interface 224 .
[0076] The communication interface 224 is used to receive the bit stream sent by the communication interface 214 and transmit the bit stream to the decoder 223, so that the decoder 223 can reconstruct the original image according to the bit stream.
[0077] The communication interface 214 and the communication interface 224 can be used to send or receive relevant data of the original image through a direct communication link between the source device 210 and the destination device 220, such as a direct wired or wireless connection, or through any type of network, such as a wired network, a wireless network or any combination thereof, any type of private network and public network or any type of combination thereof.
[0078] Both the communication interface 214 and the communication interface 224 can be configured as follows: Figure 2 A unidirectional communication interface, or a bidirectional communication interface, indicated by an arrow pointing from the source device 210 to the corresponding communication channel 230 of the destination device 220, and can be used to send and receive messages, etc. to establish a connection, confirm and exchange any other information related to the communication link and / or data transmission such as encoded bit stream transmission, etc.
[0079] The decoder 223 is used to decode the bit stream and reconstruct the original image. For example, the decoder 223 performs entropy decoding, dequantization and predictive reconstruction on the bit stream to obtain a reconstructed image. The decoder 223 may include a code control unit 2231 and a decoding unit 2232, wherein the code control unit 2231 is used to determine the quantization parameter used for decoding each coding unit in the current frame, so that the decoding unit 2232 decodes, dequantizes, and predicts and reconstructs the bit stream according to the quantization parameter to obtain a reconstructed image.
[0080] The post-processor 222 is used to receive the reconstructed image generated by the decoder 223 and perform post-processing on the reconstructed image. For example, the post-processing performed by the post-processor 222 includes color format conversion (for example, from YUV format to RGB format), color matching, repair or resampling, or any other processing.
[0081] The display 221 is used to display the reconstructed image. The display 221 may be or may include any type of display device for presenting the reconstructed image, such as an integrated or external display or monitor. For example, the display may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other type of display.
[0082] Among them, the encoder 213 and the decoder 223 can be implemented as any of a variety of suitable circuits, for example, one or more microprocessors, digital signal processors (digital signal processor, DSP), application-specific integrated circuits (application-specific integrated circuit, ASIC), field-programmable gate arrays (field-programmable gate array, FPGA), discrete logic, hardware or any combination thereof. If the technology is partially implemented in software, the device may store the instructions of the software in a suitable non-temporary computer-readable storage medium, and one or more processors may be used to execute the instructions in hardware to perform the technology of the present disclosure. Any of the foregoing contents (including hardware, software, a combination of hardware and software, etc.) can be regarded as one or more processors.
[0083] The image collector 211 and the encoder 213 may be integrated into one physical device or may be arranged on different physical devices, without limitation. Figure 2 The source device 210 shown includes an image collector 211 and an encoder 213, which means that the image collector 211 and the encoder 213 are integrated in one physical device, and the source device 210 can also be called an acquisition device. The source device 210 is, for example, a mobile phone, a tablet computer, a computer, a laptop computer, a camera, a camera, a wearable device, a vehicle-mounted device, a terminal device, a virtual reality (VR) device, an augmented reality (AR) device, a mixed reality (MR) device or an extended reality (XR) device or other image acquisition device. If the source device 210 does not include the image collector 211, it means that the image collector 211 and the encoder 213 are two different physical devices, and the source device 210 can obtain the original image from other devices (such as: an image acquisition device or an image storage device).
[0084] In addition, the display 221 and the decoder 223 may be integrated into one physical device, or may be arranged on different physical devices, without limitation. Figure 2The destination device 220 shown includes a display 221 and a decoder 223, indicating that the display 221 and the decoder 223 are integrated in one physical device, and the destination device 220 can also be called a playback device, and the destination device 220 has the function of decoding and displaying the reconstructed image. The destination device 220 is, for example, a display, a television, a digital media player, a video game console, a car computer, or other device that displays images. If the destination device 220 does not include the display 221, it means that the display 221 and the decoder 223 are two different physical devices. After the destination device 220 decodes the bitstream and reconstructs the original image, it transmits the reconstructed image to other display devices (such as: television, digital media player), and the reconstructed image is displayed by other display devices.
[0085] also, Figure 2 It is shown that the source device 210 and the destination device 220 can be integrated into one physical device, or can be set on different physical devices, which is not limited.
[0086] In these embodiments, the source device 210 or its corresponding functions and the destination device 220 or its corresponding functions may be implemented using the same hardware and / or software or by separate hardware and / or software or any combination thereof. Figure 2 The existence and division of different units or functions in the source device 210 and / or the destination device 220 shown may vary according to actual devices and applications, which is obvious to those skilled in the art.
[0087] The structure of the above-mentioned codec system is only a schematic illustration. In some possible implementations, the codec system may also include other devices, for example, the codec system may also include a terminal device or a cloud device. After the source device 210 acquires the original image, it preprocesses the original image to obtain a preprocessed image; and transmits the preprocessed image to the terminal device or the cloud device, which implements the function of encoding the preprocessed image.
[0088] Next, combine Figures 3 to 8 , the image encoding method provided by this application is described in detail. Figure 3 A schematic diagram of a flow chart of an image encoding method provided in this application. Figure 2 The following is an example in which the encoder 213 determines the quantization parameters used for encoding multiple image frames according to factors affecting image quality. Figure 3 As shown, the method includes the following steps.
[0089] Step 310: Acquire multiple image frames in the video.
[0090] As described in the above embodiment, in a possible implementation, the encoder 213 carries the image collector 211 to collect multiple image frames. Figure 2 Related introduction in .
[0091] In another possible implementation, the encoder 213 does not carry the image collector 211 , and receives multiple image frames collected by the image collector 211 , thereby acquiring multiple image frames.
[0092] In some embodiments, the video includes a plurality of smoothly continuous image frames, in which scene switching may occur, and the scene after the switching is different from the scene before the switching. For example, the plurality of image frames may include a preceding frame, a current frame, and a succeeding frame. The current frame refers to a frame of image or an original image that is being coded and decoded at the current moment. The preceding frame refers to a frame of image or an original image that has been coded and decoded at a moment before the current moment. The preceding frame may be a frame at a moment before or at a plurality of moments before the current moment. The succeeding frame refers to a frame of image or an original image that is to be coded and decoded at a moment after the current moment. The succeeding frame may be a frame at a moment after or at a plurality of moments after the current moment.
[0093] When a scene switch occurs in a video, the previous frame may be the last frame in the scene before the switch occurs, and the current frame may be the starting frame in the scene after the switch occurs. The scene of the current frame is different from that of the previous frame. The next frame may be the subsequent frame of the starting frame in the scene after the switch occurs, and the scene of the next frame is similar to that of the current frame. For example, a vehicle drives on the road and enters a parking lot. The parking lot scene in the current frame is different from that of the vehicle driving in the previous frame. The vehicle reverses into the parking lot and parks. The parking scene in the next frame is similar to that of the parking lot in the current frame. There is no scene switch in the video, and the scenes of the previous frame, current frame, and next frame are similar.
[0094] Step 320: Determine quantization parameters used for encoding multiple image frames according to factors affecting image quality.
[0095] Image quality affects the clarity and smoothness of the video. The higher the image quality, the higher the clarity and smoothness of the video. The lower the image quality, the lower the clarity and smoothness of the video. Factors affecting image quality may include the scene type of the image frame and the bandwidth change parameter. The scene type may include a switching scene and a non-switching scene. The bandwidth change parameter indicates the size relationship between the current bandwidth and the previous bandwidth.
[0096] Among them, the scene type of the image frame can be used as an internal factor, and the internal factor focuses on the impact of image parameter changes on image quality. In addition to the scene type of the image frame, the internal factors can also include the resolution of the image frame, the size of the image frame, the color of the image frame, etc. The bandwidth change parameter can be used as an external factor, and the external factor focuses on the impact of network bandwidth changes on image quality.
[0097] When encoding an image frame, the encoding scene is judged based on internal and external factors to determine the quantization parameters used for encoding the image frame. For example, the bandwidth change and the scene type of the image frame are detected to determine the encoding scene, and the quantization parameters used for encoding the image frame are determined for different encoding scenes.
[0098] In the first embodiment, the coding scene includes a switching scene and a non-switching scene when the bandwidth is stable. The bandwidth change is determined based on the bandwidth change parameter. If the current bandwidth meets a higher threshold and is close to the previous bandwidth value, it means that the current bandwidth is stable. When the bandwidth is stable, for the non-switching scene, a smaller number of coding bits is allocated so that the non-switching frame occupies less bandwidth, thereby improving the transmission rate of the coding bits. Compared with the non-switching scene, a larger number of coding bits is allocated when the scene is switched to make full use of the network bandwidth and ensure image quality.
[0099] Optionally, based on factors affecting image quality, a quantization parameter used for encoding each of the multiple image frames may be determined, and each image frame may be encoded based on the quantization parameter to obtain the number of encoding bits for each image frame to perform frame-level bit rate control.
[0100] Therefore, when the current bandwidth is less than the previous bandwidth, or the bandwidth is low, or the current bandwidth is low when the bandwidth changes suddenly, the value of the quantization parameter can be increased to reduce the number of coding bits for the switching frame and control the over-coding of the switching frame, while ensuring the image quality of the switching frame when switching scenes. As the current bandwidth increases and becomes stable, the value of the quantization parameter can be gradually reduced to increase the number of coding bits for the subsequent frames to ensure image quality.
[0101] In the second embodiment, the coding scenario includes a bandwidth stable scenario. When the current bandwidth is stable, the granularity of the rate control can be made finer, and the number of coding bits of each coding block in multiple coding blocks in each image frame can be determined according to the quantization parameter corresponding to each image frame.
[0102] In a third embodiment, the encoding scenario includes a switching scenario with extremely low bandwidth. When the current bandwidth is extremely low, in order to avoid the video screen being discontinuous due to the jump switching frame, a frame skipping operation is performed on the image frames other than the switching frame in the multiple image frames. When the current bandwidth is restored to stability, the frame skipping operation can be terminated.
[0103] It should be noted that when the frame skipping operation is performed, the skipped image frames are no longer encoded, and the frame-level bit rate control in the first embodiment described above can be performed on the image frames that are not skipped. That is, the quantization parameters used for encoding the remaining image frames after the frame skipping among the multiple image frames are determined according to the factors affecting the image quality, and the remaining image frames are encoded according to the quantization parameters to obtain the number of encoding bits of the remaining image frames.
[0104] When the frame skipping operation is terminated, the frame-level rate control in the first embodiment and the coding block-level rate control in the second embodiment are continued to be performed on each remaining image frame that is no longer skipped.
[0105] The following describes in detail the methods of frame-level rate control, coding block-level rate control, and frame skipping operation.
[0106] The method for encoding multiple image frames based on frame-level bit rate control is described in the following steps 321 to 323.
[0107] In some embodiments, the encoder 213 may determine the target number of coding bits for multiple image frames based on factors affecting image quality, where the target number of coding bits represents the expected number of bits after lossy coding. The quantization parameter is then determined based on the target number of coding bits to make the quantization parameter more accurate, so that the actual number of coding bits obtained by encoding based on the quantization parameter is more accurate, so as to more accurately control super-coding in bandwidth variation scenarios.
[0108] Step 321: Determine a target number of coding bits for each of a plurality of image frames according to factors affecting image quality.
[0109] It can be seen from step 310 that the multiple image frames in the video may include a previous frame, a current frame, and a subsequent frame. When the scene is switched, since the scene of the current frame is different from that of the previous frame, the current frame can be used as a switching frame relative to the previous frame; when the scene is not switched, since the scene of the subsequent frame is similar to that of the current frame, the current frame can be used as a non-switching frame. It can be seen that the type of the current frame is different according to different scene types.
[0110] In a possible implementation, the encoder 213 may determine the target number of coding bits for the image frame according to the bandwidth variation parameter and the type of the current frame. The target number of coding bits may be the number of coding bits that the encoder 213 is expected to allocate to the current frame in the case of lossy video coding.
[0111] For example, the bit rate may refer to the number of data bits transmitted per unit time. The encoder 213 allocates bit rates to image frames per unit time, and introduces bandwidth variation parameters in the bit rate allocation process to mitigate the impact of real-time bandwidth variation on frame-level bit rate control. A higher bit rate is allocated to switching frames relative to non-switching frames to avoid over-encoding when encoding switching frames. The bit rate allocation may satisfy the following calculation process.
[0112] Calculate the bitrate WantedBitsWindow allocated for the current frame t , satisfying formula (1).
[0113]
[0114] Where t represents the current frame, BitRate represents the target bit rate, fps represents the frame rate, PeakRatio is a proportional constant greater than 1, which is used to allocate a higher bit rate to the switching frame, and WantedBitsWindow t-1 Indicates the bit rate allocated for the previous frame. t-1 Indicates the cumulative encoding bit rate of multiple previous frames transmitted within the previous seconds of the current frame.
[0115] δ represents a bandwidth change parameter, which can be used as a weight factor to control the influence of the previous frame and the current frame on the bit rate allocation when the bandwidth changes. The bandwidth change parameter δ is calculated to satisfy formula (2).
[0116]
[0117] Wherein, BandWidth′ represents the current bandwidth when transmitting the current frame, and BandWidth represents the previous bandwidth when transmitting the previous frame. Indicates that the current bandwidth is greater than or equal to the previous bandwidth. Indicates that the current bandwidth is smaller than the previous bandwidth.
[0118] Step 322: Determine the quantization parameter of the image frame according to the target number of coding bits.
[0119] The encoder 213 can determine the quantization parameter of the current frame according to the target number of coding bits of the current frame, thereby determining the quantization parameter of each image frame. If the current frame is a switching frame, the encoder 213 calculates the quantization parameter of the switching frame according to the target number of coding bits of the switching frame and the complexity of the switching frame. The complexity can be a variable that can characterize the complexity of the image, such as image texture, image edge, image information entropy, etc. If the current frame is a non-switching frame, the encoder 213 calculates the quantization parameter of the non-switching frame according to the target number of coding bits of the non-switching frame, the bandwidth variation parameter, and the weighted complexity of multiple previous frames.
[0120] Optionally, the encoder 213 can adjust the quantization parameter of the current frame at the coarse-grained level based on the rate quantization (RQ) model and the video buffering verifier (VBV) method to obtain the quantization parameter of the current frame at the fine-grained level. The finer the quantization granularity, the higher the accuracy of the bit rate control based on the RQ model.
[0121] In the actual video transmission process, the actual number of coded bits of the current frame may exceed the target number of coded bits. When the network bandwidth suddenly changes, the current bandwidth is low or the bandwidth is extremely low, this over-coding phenomenon will be more obvious, resulting in screen freezes during video playback. Especially when there is a scene switch in the video, in order to ensure the video quality, the current frame contains complete image information, and in order to reduce the storage capacity of the video, the subsequent frame contains difference information that is different from the complete image information. Therefore, compared with the subsequent frame, the current frame is more likely to be over-coded.
[0122] In some embodiments, when switching scenes, the quantization parameter of the previous frame may be the first quantization parameter, and the quantization parameter of the current frame may be the second quantization parameter. In order to control the over-encoding of the current frame so that the number of coded bits of the current frame meets the bandwidth during real-time video transmission, the quantization parameter of the current frame is set to be larger, that is, the value of the second quantization parameter is greater than the value of the first quantization parameter, and the value of the quantization parameter of the subsequent frame may be less than the value of the second quantization parameter.
[0123] As the current bandwidth increases, when the bandwidth is restored to stability, in order to fully utilize the network bandwidth resources, the larger quantization parameter of the current frame is attenuated so that the quantization parameter of the subsequent frame becomes smaller and the number of coded bits of the subsequent frame becomes larger, thereby restoring the image quality. The specific method is described in the following steps 323.
[0124] Step 323: Attenuate the quantization parameter of the current frame according to the factors to obtain the quantization parameter of the subsequent frame.
[0125] The encoder 213 can first determine the quantization parameter used for encoding the current frame according to the factors affecting the image quality by the method described in the above steps 321 to 322. When the bandwidth is restored to stability, in order to restore the image quality, the quantization parameter of the current frame is attenuated to obtain the quantization parameter of the subsequent frame.
[0126] In a possible implementation, the encoder 213 may decay the quantization parameter of the current frame frame by frame based on the number of frames in the subsequent frame to obtain the quantization parameter of the subsequent frame. The present application provides two optional decay formulas. When the RQ model is a linear model, the quantization parameter Qp1 of the subsequent frame is calculated to satisfy formula (3). When the RQ model is a nonlinear model, the quantization parameter Qp2 of the subsequent frame is calculated to satisfy formula (4).
[0127]
[0128]
[0129] In the above formulas (3) and (4), represents the maximum value of the preset quantization parameter of the current frame, N represents the number of frames in the subsequent frame, and θ1, θ2, and θ3 are all fixed coefficients that can be adjusted according to the requirements for image quality. As N gradually increases, Qp1 and Qp2 gradually decrease.
[0130] When the overshoot situation of the current frame is alleviated, the encoder 213 can terminate the attenuation process of the quantization parameter. The condition for the overshoot situation to be alleviated satisfies formula (5).
[0131] RealSecBits + Bits < BitRate * β and
[0132] Bits < MaxFrmSize * γ Formula (5)
[0133] where RealSecBits represents the number of encoded bits of the prior frames within the prior second of the current second where the current frame is located, Bits represents the number of encoded bits of the current frame, BitRate represents the target bit rate, MaxFrmSize represents the frame-level target bit rate, and β and γ represent proportionality coefficients.
[0134] When formula (5) holds, both the second-level overshoot situation and the frame-level overshoot situation of the current frame are alleviated, so it is determined that the overshoot situation of the current frame has been alleviated, and the encoder 213 can terminate the attenuation process of the quantization parameter.
[0135] The above method for determining the quantization parameter will be described below with reference to the accompanying drawings. Figure 4 is a schematic diagram of a process for determining the quantization parameter provided by this application. As Figure 4 shown, first, the encoder 213 determines whether the type of the current frame is a switching frame (step 410).
[0136] Next, according to the method described in step 322, if the current frame is a switching frame, the quantization parameter of the switching frame is calculated based on the target bit rate and the switching frame complexity of the switching frame; if the current frame is a non-switching frame, the quantization parameter of the non-switching frame is calculated based on the target bit rate, the bandwidth change parameter, and the weighted complexity of multiple prior frames (step 420).
[0137] Next, based on the RQ model and the video buffer verification method, the quantization parameter of the current frame at the coarse-grained level is adjusted to obtain the quantization parameter of the current frame at the fine-grained level (step 430).
[0138] Then, when the current frame is a switching frame, it is determined whether the number of coded bits of the switching frame exceeds the target number of coded bits (step 440). When the number of coded bits of the switching frame exceeds the target number of coded bits, the attenuation amount of the image frame is calculated based on the quantization parameter of the switching frame being able to attenuate the quantization parameter of the image frame (step 450). The quantization parameter of the image frame includes the quantization parameter of the non-switching frame, and the attenuation amount of the image frame includes the maximum attenuation amount of the non-switching frame.
[0139] Finally, it is determined whether the number of frames in the subsequent frame is less than or equal to the attenuation number of the image frame (step 460). When the number of frames in the subsequent frame is less than or equal to the attenuation number of the image frame, based on the method described in step 323, according to formula (3) or formula (4), the quantization parameter of the switching frame is attenuated to obtain the quantization parameter of the subsequent frame (step 470).
[0140] Optionally, when the number of frames of the subsequent frame is greater than the attenuation number of the image frame, the quantization parameter of the subsequent frame is determined. For example, when the number of frames of the subsequent frame is greater than the attenuation number of the image frame, the quantization parameter of the current frame is attenuated frame by frame according to the attenuation number of the image frame to obtain the quantization parameter of the subsequent frame.
[0141] Step 330: Encode the image frame according to the quantization parameter corresponding to the image frame to obtain the number of encoding bits of the image frame.
[0142] The encoder 213 may determine the quantization parameter corresponding to each image frame according to the target number of coding bits for each image frame. Optionally, since the target number of coding bits for each image frame may be different, one image frame may correspond to one quantization parameter. Optionally, since the target number of coding bits for some image frames may be the same, some image frames in multiple image frames may correspond to the same quantization parameter, that is, the number of quantization parameters is less than the number of image frames. The encoder 213 performs frame-level bit rate control on each image frame according to the quantization parameter corresponding to each image frame, and the number of coding bits for each image frame can be obtained.
[0143] Each image frame may contain multiple coding blocks. To make the bit rate control more granular, when the current bandwidth is stable, the number of coding bits of each coding block in the multiple coding blocks in each image frame may be determined according to the quantization parameter corresponding to each image frame, and the sum of the number of coding bits of each coding block may be taken as the number of coding bits of the image frame. Figure 5 A schematic diagram of a coding block-level bit rate control and frame skipping process provided by this application. Figure 5 As shown, the encoder 213 performs a coding block level rate control method on multiple coding blocks including the following steps 331 to 332.
[0144] Step 331: Select the type of rate quantization model according to the scene type of the image frame.
[0145] The scene type of the image frame may include a switching scene and a non-switching scene, the type of the current frame may include a switching frame and a non-switching frame, and the type of the RQ model may include an RQ model of a switching frame and an RQ model of a non-switching frame. In order to enable the bit rate control to be targeted at different types of scenes, the encoder 213 selects the RQ model of the switching frame based on the difference in scene types when the scene is switched and the type of the current frame is a switching frame; and selects the RQ model of the non-switching frame when the scene is not switched and the type of the current frame is a non-switching frame.
[0146] After the RQ model is selected, the encoder 213 performs a method for encoding block level rate control according to the quantization parameter of the current frame based on the RQ model, as described in the following step 332 .
[0147] Step 332: predict the number of coding bits of the coding block within the image frame according to the model parameters and quantization parameters of the rate quantization model.
[0148] The RQ model is used to indicate the relationship between the quantization parameter and the number of coded bits. For example, the RQ model is used to indicate the inverse proportional relationship between the quantization parameter and the number of coded bits. Based on the RQ model, the number of coded bits can be obtained according to the model parameters and the quantization parameters.
[0149] In some embodiments, when the current bandwidth is stable, based on the RQ model, the number of coded bits of the current coding block is determined according to the model parameters and the quantization parameters; based on the RQ model, the model parameters are updated according to the number of coded bits of the current coding block to obtain the updated model parameters; based on the RQ model, the number of coded bits of the subsequent coding block is predicted according to the updated model parameters and the quantization parameters. In this way, the number of coded bits of each coding block in the image frame is predicted.
[0150] Before the coding block level bit rate control, the encoder 213 needs to determine whether the model parameters of the selected RQ model need to be initialized. If the current coding block is the initial coding block in the image frame, the model parameters need to be initialized; if the current coding block is a non-initial coding block in the image frame, the model parameters do not need to be initialized and the model parameters are directly updated. The model parameters of the RQ model include at least one of the coding block complexity coefficient, the image compensation coefficient and the quantity adjustment coefficient. In order to facilitate the description of the coding block level bit rate control process, taking the linear RQ model as an example, the expression of the linear RQ model satisfies formula (6).
[0151]
[0152] Among them, PredBits represents the number of coding bits predicted in the subsequent coding block. Coeff, Offset and Count are all model parameters of the RQ model, Coeff represents the coding block complexity coefficient, Offset represents the image compensation coefficient, and Count represents the quantity adjustment coefficient. Clpx represents the coding block complexity. Qscale represents the quantization parameter, and the quantization parameter Qscale corresponds to the quantization step Qstep one by one. For example, in the H.264 standard, for brightness coding, the quantization step Qstep has a total of 52 values, and the quantization parameter is the serial number corresponding to the quantization step, and the value of the quantization parameter is between 0 and 51.
[0153] Based on formula (6), if the current coding block is the initial coding block in the image frame, the encoder 213 needs to initialize the model parameters of the RQ model, set the initial value of Coeff to 0.25, set the initial value of Offset to 0, and set the initial value of Count to 1. Thus, the number of coding bits of the current coding block in the image frame is determined according to the initialized model parameters and quantization parameters.
[0154] In order to make the RQ model predict the number of coded bits more accurate, the RQ model parameters need to be updated according to the coded data during the coding process.
[0155] In some embodiments, bandwidth changes may affect bit rate control. When the current coding block is a non-initial coding block in an image frame, the encoder 213 may update the model parameters according to the bandwidth change parameters and the number of coding bits of the current coding block to obtain updated model parameters.
[0156] According to the bandwidth change parameter δ in step 321, if δ exceeds the bandwidth change threshold, indicating bandwidth fluctuation, the model parameters are not updated temporarily; if the bandwidth is relatively stable, the model parameters are updated. It is determined whether to update the model parameters to satisfy formula (7).
[0157]
[0158] Among them, δ t Indicates the bandwidth change threshold. When it is 1, it means updating the model parameters. When it is 0, it means not updating the model parameters.
[0159] When updating the model parameters, the update methods of Count, Coeff and Offset satisfy formula (8), formula (9) and formula (10) respectively.
[0160] Count′=Count·Decay+1 Formula (8)
[0161] Coeff′=Coeff·Decay+NewCoeff formula (9)
[0162] Offset′=Offset·Decay+NewOffset Formula (10)
[0163] Among them, Decay represents the attenuation coefficient, which is usually set to 0.5, Count, Coeff and Offset represent the model parameters before updating, and Count′, Coeff′ and Offset′ represent the model parameters after updating.
[0164] NewCoeff represents the adjustment coefficient used when updating Coeff. NewOffset represents the adjustment coefficient used when updating Offset. In order to prevent model parameter mutations from causing exceptions when updating the RQ model, the value ranges of NewCoeff and NewOffset need to be restricted.
[0165] The value range of NewCoeff satisfies formula (11).
[0166]
[0167] Among them, MinCoeff represents the minimum value of the complexity coefficient of the coding block, which is usually set to 1 / 16. Bit represents the number of coding bits of the current coding block. When the bit rate control type is constant bit rate, represents the coding block complexity coefficient Coeff obtained by transforming formula (6). When the rate control type is other, such as variable bit rate, the value of Offset is 0. represents the coding block complexity coefficient Coeff obtained by transforming formula (6).
[0168] Constrain the upper and lower limits of the value range of NewCoeff to satisfy formula (12).
[0169]
[0170] in, Indicates that if the value of NewCoeff is Between, the value range of NewCoeff is If the value of NewCoeff is less than Then the value range of NewCoeff is If the value of NewCoeff is greater than Then the value range of NewCoeff is
[0171] By transforming formula (6), the image compensation coefficient Offset is obtained, and the value range of Newoffset is determined based on the value range of NewCoeffClipped in formula (12). The value range of Newoffset satisfies formula (13).
[0172] NewOffset=Bits*Qscale-NewCoffClipped*Cplx Formula (13)
[0173] Based on formula (6), the encoder 213 predicts the number of coding bits in the post-coding block in each image frame according to the updated model parameters Count′, Coeff′, Offset′ and the quantization parameter, until the number of coding bits of all coding blocks in the image frame is predicted.
[0174] The above coding block level rate control method is described below with reference to the accompanying drawings. Figure 6 A schematic diagram of a coding block level rate control process provided by this application. Figure 6 As shown, first, the encoder 213 determines whether the type of the current frame is a switching frame (step 610).
[0175] Next, based on the method in step 331, an RQ model is selected according to the type of the current frame. When the type of the current frame is a switching frame, the RQ model of the switching frame is selected (step 620); when the type of the current frame is a non-switching frame, the RQ model of the non-switching frame is selected (step 630).
[0176] Then, it is determined whether the model parameters of the RQ model need to be initialized (step 640). When the current coding block is an initial coding block, the model parameters need to be initialized, and the initial values of the model parameters are set according to formula (6) in the above step 332 (step 650); when the current coding block is a non-initial coding block, the model parameters do not need to be initialized, and considering the impact of bandwidth changes on bit rate control, the model parameters are updated according to formulas (7) to (13) in the above step 332 (step 660).
[0177] Finally, according to formula (6) in step 332, the number of coding bits of the subsequent coding block is predicted based on the updated model parameters and the quantization parameter, so as to adjust the size of the quantization parameter based on the size of the predicted number of coding bits (step 670). When the predicted number of coding bits is close to the target number of coding bits, the quantization parameter does not need to be adjusted; when the predicted number of coding bits is too large relative to the target number of coding bits, the quantization parameter needs to be increased to reduce the number of coding bits; when the predicted number of coding bits is too small, the quantization parameter needs to be reduced to increase the number of coding bits.
[0178] In the case of extremely low bandwidth, over-encoding will be more obvious. At this time, the adjustment of quantization parameters reaches its limit and the number of encoding bits cannot be further reduced to control over-encoding. In this case, by reducing the number of image frames, the number of encoding bits after encoding the image frames is reduced. In order to ensure that the video image continuity is less affected by extremely low bandwidth, such as Figure 5 As shown, the method in which the encoder 213 performs frame skipping operation on part of the image frames includes the following steps 340.
[0179] Step 340: Perform a frame skipping operation on image frames other than the current frame among the multiple image frames according to the bandwidth change parameter.
[0180] When a scene switch occurs in a video, in order to ensure the video quality and prevent the video picture from being discontinuous due to the jump of the switching frame, the encoder 213 does not perform a frame skipping operation on the switching frame, but performs a frame skipping operation on the image frames other than the switching frame. For example, if the scene of the non-switching frame after the switching frame is similar to the scene of the switching frame, the frame skipping operation is performed on the non-switching frame.
[0181] In some embodiments, when the current bandwidth meets the extremely low bandwidth threshold, for example, when the current bandwidth is lower than 20Mbps, if the remaining number of coding bits is not enough to encode image frames other than switching frames, the encoder 213 can determine the frame skipping level when performing frame skipping operations on image frames other than switching frames according to the bandwidth change parameter, and the frame skipping mode can be determined according to the frame skipping level, and the frame skipping mode corresponds to the frame skipping level one by one. The remaining number of coding bits can refer to the difference between the sum of the target coding bit numbers of multiple image frames and the number of coded bits of the previous frame. The smaller the current bandwidth value, the more frame skipping, and the larger the current bandwidth value, the fewer frame skipping.
[0182] The frame skipping level L satisfies formula (14).
[0183]
[0184] Among them, δ represents the bandwidth change parameter, BitRate represents the target bit rate, fps represents the frame rate, and WindowBitSum represents the accumulated encoding bit rate of multiple previous frames. α and β represent the proportional coefficients of the frame-level target bit rate, usually α is set to 0.1 and β is set to 0.2.
[0185] The frame skipping mode W satisfies formula (15).
[0186]
[0187] For example, let ABCDEFGHIJ be ten frames of images, and perform frame skipping on these ten frames of images. Skip one frame every other frame, and the skipped image is BDFHJ, then the displayed image is ACEGI. Skip two frames every other frame, which means skipping two consecutive frames of images, and the skipped image is BCEFHI, then the displayed image is ADGJ. Skip three frames every other frame, which means skipping three consecutive frames of images, and the skipped image is BCDFGHJ, then the displayed image is AEI.
[0188] In some embodiments, in the process of determining the quantization parameters used for encoding multiple image frames according to factors affecting image quality, when the current bandwidth meets the extremely low bandwidth threshold, a frame skipping operation is performed on image frames other than the switching frame. The quantization parameters used for encoding the multiple image frames include the quantization parameters of the switching frame and the quantization parameters of the remaining image frames after the frame skipping operation is performed.
[0189] If the current bandwidth increases and returns to a stable state, the number of remaining coding bits satisfies the coding of the subsequent frames, and the encoder 213 terminates the frame skipping operation to fully utilize the network bandwidth resources and continues to encode the remaining image frames. Each image frame in the coding of the remaining image frames can refer to the frame-level bit rate control method described in steps 321 to 323, and each coding block in the coding image frame can refer to the coding block-level bit rate control method described in steps 331 to 332.
[0190] The method for performing the frame skipping operation is described below with reference to the accompanying drawings. Figure 7 A schematic diagram of a frame skipping process provided by this application. Figure 7 As shown, first, the encoder 213 determines whether to terminate the frame skipping operation based on the remaining number of coding bits (step 710). Next, when performing the frame skipping operation, it is determined whether the type of the current frame is a switching frame (step 720). When the type of the current frame is a non-switching frame, according to formula (14) in the above step 340, the frame skipping level is determined according to the bandwidth change parameter (step 730). Finally, if the current bandwidth becomes larger and returns to stability, it is determined whether to continue the frame skipping operation based on the remaining number of coding bits (step 740). When the remaining number of coding bits is sufficient to encode the subsequent frame, the encoder 213 terminates the frame skipping operation.
[0191] In order to make the bit rate control support multiple scenarios, the bit rate control can be performed in the case of low bandwidth, sudden bandwidth change, scene switching, etc., that is, the quantization parameter of the image frame to be encoded is determined. The encoder 213 can receive the control operation of the controller. The specific implementation method is described by the following steps 350.
[0192] Step 350: Receive a control operation.
[0193] In some embodiments, the encoder 213 may receive a control operation sent by the controller before encoding the image frame. The control operation includes at least one of setting a bandwidth threshold, detecting bandwidth mutation, detecting scene switching, and setting a bit rate control parameter. Receiving the control operation enables the encoder 213 to perform bit rate control for a variety of scenes, and the encoding supports a variety of scenes such as low bandwidth, bandwidth mutation, and scene switching.
[0194] Among them, setting a lower bandwidth threshold can construct a low-bandwidth scenario. The relationship between the current bandwidth and the previous bandwidth can be used to detect whether the bandwidth has changed suddenly. The inter-frame motion information of multiple image frames can be used to detect whether a scene switch has occurred. For example, in the auxiliary stream scenario during a video conference, that is, the screen sharing scenario, the scrolling of the shared screen can be used to detect whether a page switch has occurred. In the frame-level bitrate control and encoding block-level bitrate control process, the bitrate control parameters can be customized according to actual needs to make the bitrate control more refined and the encoding bit number result more accurate.
[0195] Before encoding the image frame, the encoder 213 performs extremely low bandwidth detection, bandwidth mutation detection and scene switching detection, and selects frame skipping operation (such as the above step 340) or frame level bit rate control (such as the above steps 321 to 323) or coding block level bit rate control (such as the above steps 331 to 332) according to the detection results of the coding scene.
[0196] When the encoder 213 encodes the image frame, the number of encoded bits is transmitted to the display so that the display displays the over-encoding rate. The specific implementation method is described in the following steps 360.
[0197] Step 360: The display shows the overstaffing rate.
[0198] In some embodiments, the source device 210 may further include a display to display the over-coding rate. The over-coding rate may be used to indicate the extent to which the number of coding bits of an image frame exceeds the target number of coding bits. The over-coding rate may refer to the ratio of the difference between the number of coding bits of an image frame and the target number of coding bits to the target number of coding bits, thereby calculating the over-coding rate. The display displays the over-coding rate so that the user can clearly and obviously know the over-coding situation. According to the real-time over-coding situation of the image frame, the controller flexibly adjusts the control operation to accurately control the encoder 213 to perform frame-level rate control, coding block-level rate control and frame skipping operations on the image frame.
[0199] Figure 8 This is a schematic diagram of a display interface provided by this application. Figure 8As shown in (a) of FIG. 1 , the user can click on the control operation buttons such as “Set bandwidth threshold”, “Detect bandwidth mutation”, “Detect scene switching” and “Set rate control parameters” in the interface to enable the encoder 213 to perform frame-level rate control according to the method described in steps 321 to 323, perform coding block-level rate control according to the method described in steps 331 to 332, and perform frame skipping according to the method described in step 340. Figure 8 As shown in (b) in FIG. 1 , when the encoder 213 is performing rate control, the interface of the display 221 can display the over-editing rate in real time.
[0200] The solution provided by the present application controls the number of coding bits of the image frame according to the bandwidth change and the scene type during the video encoding process. When the current bandwidth is less than the previous bandwidth, or the bandwidth is low, the value of the quantization parameter can be increased to reduce the number of coding bits of the current frame when switching scenes, and control the current frame to be over-encoded, so that the current bandwidth can be controlled to meet the lower network bandwidth during video transmission, so as to avoid the current bandwidth being unable to meet the number of coding bits of the current frame. In this way, the number of coding bits of the image frame is controlled in the bandwidth change scenario, especially in combination with the scene type of the image frame, so that the video meets the network bandwidth during real-time transmission. The picture is smooth during video playback, which improves the viewer's visual experience.
[0201] It is understandable that in order to implement the functions in the above embodiments, the computing device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and method steps of each example described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.
[0202] Combined with the above Figures 2 to 8 , describes in detail the image encoding method provided according to this embodiment, and will now be combined with Fig. 9 , describes the image encoding device provided according to this embodiment.
[0203] Fig. 9 This is a schematic diagram of the structure of an image encoding device provided by the present application. These image encoding devices can be used to implement the functions of the encoder in the above method embodiments, and thus can also achieve the beneficial effects possessed by the above method embodiments.
[0204] like Fig. 9 As shown, the image encoding device 900 includes an acquisition module 910, a code control module 920, an encoding module 930, a frame skipping module 940, a receiving module 950, a display module 960 and a storage module 970.
[0205] The acquisition module 910 is used to acquire multiple image frames in the video. For example, the acquisition module 910 is used to execute Figure 3 Step 310.
[0206] The code control module 920 is used to determine the quantization parameters used for encoding multiple image frames according to factors affecting image quality, including the scene type and bandwidth variation parameters of the image frames. Figure 3 Step 320.
[0207] The encoding module 930 is used to encode the image frame according to the quantization parameter corresponding to the image frame to obtain the number of encoding bits of the image frame. Figure 3 Step 330.
[0208] The frame skipping module 940 is used to perform a frame skipping operation on image frames other than the current frame among multiple image frames according to the bandwidth change parameter when the current bandwidth meets the bandwidth threshold; and to determine the termination of frame skipping according to the number of remaining coding bits and the bandwidth change parameter. Figure 5 Step 340.
[0209] The receiving module 950 is used to receive a control operation, where the control operation includes at least one of setting a bandwidth threshold, detecting a bandwidth mutation, detecting a scene switch, and setting a bit rate control parameter.
[0210] The display module 960 is used to display the over-coding rate, which is used to indicate the extent to which the number of coding bits of the image frame exceeds the target number of coding bits.
[0211] The storage module 970 may correspond to the above method embodiment and be used to store information such as rate quantization model, quantization parameters, and number of coding bits.
[0212] It should be understood that the image encoding device 900 according to the embodiment of the present application may correspond to executing the method described in the embodiment of the present application, and the above and other operations and / or functions of each unit in the image encoding device 900 are respectively to implement Figure 3 and Figure 5 For the sake of brevity, the corresponding processes of each method in are not repeated here.
[0213] Fig.10 1 is a schematic diagram of the structure of a computer device 1000 provided in this application. Fig.10As shown, the computer device 1000 includes a processor 1010, a bus 1020, a memory 1030, a memory unit 1050 (also referred to as a main memory unit), and a communication interface 1040. The processor 1010, the memory 1030, the memory unit 1050, and the communication interface 1040 are connected via the bus 1020.
[0214] It should be understood that in this embodiment, the processor 1010 may be a CPU, and the processor 1010 may also be other general-purpose processors, DSPs, ASICs, FPGAs or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0215] The communication interface 1040 is used to implement communication between the computer device 1000 and an external device or component. In this embodiment, the communication interface 1040 is used to exchange data with other computer devices.
[0216] The bus 1020 may include a path for transmitting information between the above components (such as the processor 1010, the memory unit 1050, and the memory 1030). In addition to the data bus, the bus 1020 may also include a power bus, a control bus, and a status signal bus. However, for the sake of clarity, various buses are labeled as bus 1020 in the figure. The bus 1020 may be a Peripheral Component Interconnect Express (PCIe) bus, or an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX), etc.
[0217] As an example, the computer device 1000 may include multiple processors. The processor may be a multi-core (multi-CPU) processor. The processor here may refer to one or more devices, circuits, and / or computing units for processing data (e.g., computer program instructions). The processor 1010 may call multiple image frames stored in the memory 1030, and determine the quantization parameters used for encoding the multiple image frames according to factors affecting the image quality. The image frames are encoded according to the quantization parameters corresponding to each image frame to obtain the number of encoding bits of the image frame.
[0218] It is worth mentioning that Fig.10 In the example, a computer device 1000 including one processor 1010 and one memory 1030 is taken as an example. Here, the processor 1010 and the memory 1030 are respectively used to indicate a type of device or equipment. In a specific embodiment, the number of each type of device or equipment can be determined according to business requirements.
[0219] The memory unit 1050 may correspond to the storage medium for storing information such as quantization parameters and the number of coding bits in the above method embodiment. The memory unit 1050 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).
[0220] The memory 1030 is used for storing rate quantization models, etc., and may be a solid state drive or a mechanical hard disk.
[0221] It should be understood that the computer device 1000 according to this embodiment may correspond to the image encoding device 900 in this embodiment, and may correspond to executing the image encoding device 900 according to the embodiment. Figure 3 and Figure 5 The corresponding subjects in the image encoding device 900, and the above and other operations and / or functions of each module in the image encoding device 900 are respectively implemented Figure 3 and Figure 5 For the sake of brevity, the corresponding process in will not be repeated here.
[0222] The method steps in this embodiment can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a computer device. Of course, the processor and the storage medium can also exist as discrete components in a network device or a terminal device.
[0223] The present application also provides a chip system, which includes a processor for implementing the functions of the data processing unit in the above method. In one possible design, the chip system also includes a memory for storing program instructions and / or data. The chip system can be composed of a chip, or can include a chip and other discrete devices.
[0224] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computer, the process or function described in the embodiment of the present application is executed in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program or instruction may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, for example, a floppy disk, a hard disk, a tape; it may also be an optical medium, for example, a digital video disc (DVD); it may also be a semiconductor medium, for example, a solid state drive (SSD).
[0225] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.
Claims
1. An image coding method, characterized in that: include: Get multiple image frames in the video; Determining quantization parameters used for encoding the multiple image frames according to factors affecting image quality, wherein the factors include scene types and bandwidth variation parameters of the image frames, and the scene types include switching scenes and non-switching scenes; The image frame is encoded according to the quantization parameter corresponding to the image frame to obtain the number of encoding bits of the image frame.
2. The method according to claim 1, characterized in that Determining quantization parameters used for encoding the multiple image frames according to factors affecting image quality includes: Determine a target number of coding bits for each of the plurality of image frames according to the factors; A quantization parameter of the image frame is determined according to the target number of encoding bits.
3. The method according to claim 1 or 2, characterized in that: The multiple image frames include a previous frame, a current frame, and a subsequent frame, the scene of the current frame is different from the scene of the previous frame, and the quantization parameters used for encoding the multiple image frames are determined according to factors affecting image quality, including: According to the factors, the quantization parameter of the current frame is attenuated to obtain the quantization parameter of the subsequent frame.
4. The method according to claim 3, characterized in that Attenuating the quantization parameter of the current frame to obtain the quantization parameter of the subsequent frame includes: The quantization parameter of the current frame is attenuated based on the frame number of the subsequent frame to obtain the quantization parameter of the subsequent frame.
5. The method according to any one of claims 1 to 4, characterized in that Encoding the image frame according to the quantization parameter corresponding to the image frame to obtain the number of encoding bits of the image frame, including: The number of coding bits of the coding block in the image frame is predicted according to the model parameters of the rate quantization model and the quantization parameter, the rate quantization model is used to indicate the relationship between the quantization parameter and the number of coding bits, and the model parameters include at least one of a coding block complexity coefficient, an image compensation coefficient and a quantity adjustment coefficient.
6. The method according to claim 5, characterized in that The method further comprises: The type of the rate quantization model is selected based on the scene type of the image frame.
7. The method according to claim 5 or 6, characterized in that: Predicting the number of coding bits of the coding block within the image frame according to the model parameters of the rate quantization model and the quantization parameter, comprising: Determine the number of coding bits of the current coding block according to the model parameter and the quantization parameter; Updating the model parameters according to the bandwidth change parameter and the number of coded bits of the current coding block to obtain updated model parameters; The number of coded bits of the subsequent coded block is predicted according to the updated model parameters and the quantization parameter.
8. The method according to any one of claims 1 to 7, characterized in that The multiple image frames include a previous frame, a current frame, and a subsequent frame, the scene of the current frame is different from the scene of the previous frame, and the determining, according to factors affecting image quality, quantization parameters used for encoding the multiple image frames includes: When the current bandwidth meets the bandwidth threshold, performing a frame skipping operation on image frames other than the current frame among the multiple image frames according to the bandwidth change parameter; A quantization parameter for encoding remaining image frames after frame skipping among the multiple image frames is determined according to factors affecting image quality.
9. The method according to any one of claims 1 to 8, characterized in that The method further comprises: A control operation is received, where the control operation includes at least one of setting a bandwidth threshold, detecting a bandwidth mutation, detecting a scene switch, and setting a bit rate control parameter.
10. The method according to any one of claims 1 to 9, characterized in that The method further comprises: The over-coding rate is displayed, where the over-coding rate is used to indicate the extent to which the number of coding bits of the image frame exceeds the target number of coding bits.
11. An image encoding device, characterized in that: include: An acquisition module, used for acquiring multiple image frames in a video; A code control module, used to determine quantization parameters used for encoding the multiple image frames according to factors affecting image quality, wherein the factors include scene types and bandwidth variation parameters of the image frames, wherein the scene types include switching scenes and non-switching scenes; The encoding module is used to encode the image frame according to the quantization parameter corresponding to the image frame to obtain the number of encoding bits of the image frame.
12. A computer device, characterized in that: It comprises a memory and a processor, wherein the memory is used to store a set of computer instructions; when the processor executes the set of computer instructions, the operation steps of the method described in any one of claims 1 to 10 are performed.
13. A chip, characterized in that: The chip includes a processor and a power supply circuit, and the power supply circuit supplies power to the processor; when the processor executes the set of computer instructions, the operation steps of the method described in any one of claims 1 to 10 are performed.
14. A coding and decoding system, characterized in that: The encoding and decoding system includes an encoder and a controller, and the encoder is used to receive the control operation sent by the controller to execute the operation steps of any one of the methods described in claims 1-10.
Citation Information
Cited By
Video coding method and system based on screen sharing
CN120512536A
Adaptive video compression coding method, system and device, and mobile platform
CN121309834A
