Video coding rate control method and device, electronic equipment and storage medium

Abstract

The invention discloses a video coding rate control method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a current image frame of a to-be-processed video stream according to a preset time period; determining each first target area of the current image frame and the image complexity of each first target area according to a preset algorithm; inputting each first target area, the image complexity of each first target area, the obtained target quantization parameter set of the previous image frame, the code rate of the previous image frame and the perceived image quality evaluation value of the previous image frame into a parameter fitting model to obtain a quantization parameter set of a preset number of current image frames; and determining a target quantization parameter set of the current image frame based on the quantization parameter set of the preset number of current image frames and the perception quality evaluation model, and coding the current image frame and the image frame in the next time period according to the target quantization parameter set of the current image frame.

Description

Technical Field

[0001] The present application relates to the technical field of video image processing, and in particular to a method, device, electronic device and storage medium for controlling a video encoding bit rate. Background Art

[0002] Video coding technology plays an increasingly important role in video servers such as video surveillance, video conferencing, video live broadcast, and video TV. The H.264 standard is a highly compressed digital video codec standard that can provide superior compression performance for high-definition video. However, in order to meet the needs of different network environments and devices, the video stream needs to be compressed to varying degrees, which often results in a loss of video quality.

[0003] Bit rate control is an important part of video coding. Different business scenarios have different requirements and restrictions on bit rates. Bit rate control controls the bit rate by adjusting the quantization parameter (QP) of the encoding so that the bit rate of the encoded video meets the predetermined target bit rate requirement, while optimizing the video quality and encoding efficiency as much as possible. The selection of quantization parameter is a key factor affecting the video bit rate. The quantization parameter reflects the compression of spatial details. If the quantization parameter is smaller, most of the details of the video frame will be retained. If the bit rate of the video frame increases, the quantization parameter will be larger, some details will be lost, the quality of the video frame will decrease, and the bit rate will decrease.

[0004] Therefore, how to adjust the quantization parameters of video encoding to achieve more efficient video compression while ensuring video quality is one of the technical problems that need to be solved urgently. Summary of the invention

[0005] The embodiments of the present application provide a video encoding bit rate control method, device, electronic device and storage medium, which realize dynamic adjustment of the quantization parameters of video encoding, thereby improving the video encoding efficiency while ensuring the video quality.

[0006] In a first aspect, an embodiment of the present application provides a video encoding rate control method, which is applied to an edge computing server, and the method includes:

[0007] Obtaining a current image frame of the video stream to be processed according to a preset time period;

[0008] Determine each first target area of ​​the current image frame and the image complexity of each first target area according to a preset algorithm;

[0009] Acquire a target quantization parameter set of an image frame previous to the current image frame, a bit rate of the previous image frame, and a perceived image quality assessment value of the previous image frame;

[0010] Inputting the first target regions, the image complexity of the first target regions, the target quantization parameter set of the previous image frame, the bit rate of the previous image frame, and the perceived image quality assessment value of the previous image frame into a parameter fitting model to obtain a preset number of quantization parameter sets of the current image frame, wherein each quantization parameter set includes a group of quantization parameters of the first target regions of the current image frame;

[0011] A target quantization parameter set of the current image frame is determined based on a preset number of quantization parameter sets of the current image frame and the perceptual quality assessment model, and the current image frame and image frames in the next time period are encoded according to the target quantization parameter set of the current image frame.

[0012] In one embodiment, the target quantization parameter set of the previous image frame includes target quantization parameters of each second target area of ​​the previous image frame, and the perceptual image quality assessment value of the previous image frame is obtained according to the previous image frame, the decoded data after encoding the previous image frame based on the target quantization parameter set of the previous image frame, and the perceptual quality assessment model.

[0013] In one embodiment, after obtaining the current image frame of the video stream to be processed, the method further includes:

[0014] Performing motion estimation according to the current image frame and the previous image frame to obtain motion complexity of the current image frame;

[0015] Determining a value of a motion estimation guidance parameter according to the motion complexity and a preset motion estimation threshold; and

[0016] While inputting the first target regions, the image complexity of the first target regions, the target quantization parameter set of the previous image frame, the bit rate of the previous image frame, and the perceived image quality assessment value of the previous image frame into the parameter fitting model, the method further includes:

[0017] The values ​​of the motion estimation guidance parameters are input into the parameter fitting model.

[0018] In one implementation, determining the value of the motion estimation guidance parameter according to the motion complexity and a preset motion estimation threshold specifically includes:

[0019] If it is determined that the motion complexity is less than or equal to the preset motion estimation threshold, setting the value of the motion estimation guidance parameter to a first set value;

[0020] If it is determined that the motion complexity is greater than the preset motion estimation threshold, the value of the motion estimation guidance parameter is normalized to a preset value interval, wherein the minimum endpoint value of the preset value interval is greater than or equal to the first set value.

[0021] In one implementation, determining each first target region of the current image frame and the image complexity of each first target region according to a preset algorithm specifically includes:

[0022] Performing image segmentation on the current image frame according to a preset image segmentation model to obtain each first target area of ​​the current image frame;

[0023] Determining the texture complexity, color complexity and spatial frequency of each first target area respectively;

[0024] For each first target region, the texture complexity, color complexity and spatial frequency of the first target region are input into an image complexity prediction model to obtain the image complexity of the first target region.

[0025] In one implementation, for each first target area, inputting the texture complexity, color complexity, and spatial frequency of the first target area into an image complexity prediction model to obtain the image complexity of the first target area specifically includes:

[0026] For each first target region, inputting the texture complexity, color complexity and spatial frequency of the first target region into a convolutional layer in an image complexity prediction model to obtain feature information of the first target region;

[0027] The feature information of the first target area is input into a pooling layer in the image complexity prediction model to obtain a predicted image complexity of the first target area.

[0028] In one implementation, determining a target quantization parameter set for the current image frame based on a preset number of quantization parameter sets for the current image frame and the perceptual quality assessment model specifically includes:

[0029] For each quantization parameter set, encoding each first target area according to the quantization parameter of each first target area in the quantization parameter set, to obtain first encoded data corresponding to the current image frame;

[0030] Decoding the first encoded data to obtain first decoded data corresponding to the current image frame;

[0031] Determining a first perceptual image quality assessment value of the current image frame based on the current image frame before encoding, the first decoded data, and the perceptual quality assessment model;

[0032] Based on the bit rate corresponding to each quantization parameter set and each first perceptual image quality assessment value, a quantization parameter set that meets a preset condition is selected as a target quantization parameter set for the current image frame.

[0033] In one implementation, determining a first perceptual image quality assessment value of the current image frame based on the current image frame before encoding, the first decoded data, and the perceptual quality assessment model specifically includes:

[0034] Inputting YUV data of the current image frame before encoding and YUV data of the first decoded data into the perceptual quality assessment model for feature extraction to obtain a first feature map corresponding to the current image frame and a second feature map corresponding to the first decoded data;

[0035] determining a distance between the first feature map and the second feature map;

[0036] A distance between the first feature map and the second feature map is determined as a first perceptual image quality assessment value of the current image frame.

[0037] In one implementation, based on the bit rate corresponding to each quantization parameter set and each first perceptual image quality assessment value, selecting a quantization parameter set that meets a preset condition as a target quantization parameter set for the current image frame specifically includes:

[0038] Obtaining a relationship curve between bit rate and quality according to the bit rate corresponding to each quantization parameter set and each first perceptual image quality assessment value;

[0039] The quantization parameter set corresponding to the bit rate at the point with the highest slope in the bit rate-quality relationship curve is used as the target quantization parameter set for the current image frame.

[0040] In a second aspect, an embodiment of the present application provides a video encoding bit rate control device, which is applied to an edge computing server, and the device includes:

[0041] A first acquisition unit, used for acquiring a current image frame of a video stream to be processed according to a preset time period;

[0042] A first determining unit, configured to determine each first target region of the current image frame and an image complexity of each first target region according to a preset algorithm;

[0043] A second acquisition unit, configured to acquire a target quantization parameter set of an image frame previous to the current image frame, a bit rate of the previous image frame, and a perceived image quality assessment value of the previous image frame;

[0044] a parameter fitting unit, configured to input the first target regions, the image complexity of the first target regions, the target quantization parameter set of the previous image frame, the bit rate of the previous image frame, and the perceived image quality assessment value of the previous image frame into a parameter fitting model, so as to obtain a preset number of quantization parameter sets of the current image frame, wherein each quantization parameter set includes a group of quantization parameters of the first target regions of the current image frame;

[0045] A processing unit is used to determine a target quantization parameter set of the current image frame based on a preset number of quantization parameter sets of the current image frame and the perceptual quality assessment model, and encode the current image frame and image frames in the next time period according to the target quantization parameter set of the current image frame.

[0046] In one embodiment, the target quantization parameter set of the previous image frame includes target quantization parameters of each second target area of ​​the previous image frame, and the perceptual image quality assessment value of the previous image frame is obtained according to the previous image frame, the decoded data after encoding the previous image frame based on the target quantization parameter set of the previous image frame, and the perceptual quality assessment model.

[0047] In one embodiment, the device further comprises:

[0048] A motion estimation unit, configured to, after acquiring a current image frame of a video stream to be processed, perform motion estimation based on the current image frame and the previous image frame to obtain a motion complexity of the current image frame;

[0049] A second determining unit, configured to determine a value of a motion estimation guidance parameter according to the motion complexity and a preset motion estimation threshold; and

[0050] The parameter fitting unit is further used to input the value of the motion estimation guidance parameter into the parameter fitting model while inputting the first target areas, the image complexity of the first target areas, the target quantization parameter set of the previous image frame, the bit rate of the previous image frame, and the perceptual image quality assessment value of the previous image frame into the parameter fitting model.

[0051] In one embodiment, the second determination unit is specifically used to set the value of the motion estimation guidance parameter to a first set value if it is determined that the motion complexity is less than or equal to the preset motion estimation threshold; if it is determined that the motion complexity is greater than the preset motion estimation threshold, normalize the value of the motion estimation guidance parameter to a preset value range, wherein the minimum endpoint value of the preset value range is greater than or equal to the first set value.

[0052] In one embodiment, the first determination unit is specifically used to perform image segmentation on the current image frame according to a preset image segmentation model to obtain each first target area of ​​the current image frame; respectively determine the texture complexity, color complexity and spatial frequency of each first target area; for each first target area, input the texture complexity, color complexity and spatial frequency of the first target area into the image complexity prediction model to obtain the image complexity of the first target area.

[0053] In one embodiment, the first determination unit is specifically used to input the texture complexity, color complexity and spatial frequency of each first target area into the convolution layer in the image complexity prediction model to obtain the feature information of the first target area; and input the feature information of the first target area into the pooling layer in the image complexity prediction model to obtain the predicted image complexity of the first target area.

[0054] In one embodiment, the processing unit is specifically used to, for each quantization parameter set, encode each first target area according to the quantization parameters of each first target area in the quantization parameter set to obtain first encoded data corresponding to the current image frame; decode the first encoded data to obtain first decoded data corresponding to the current image frame; determine a first perceptual image quality assessment value of the current image frame based on the current image frame before encoding, the first decoded data and the perceptual quality assessment model; and select a quantization parameter set that meets preset conditions as the target quantization parameter set for the current image frame based on the bit rate corresponding to each quantization parameter set and each first perceptual image quality assessment value.

[0055] In one embodiment, the processing unit is specifically used to input the YUV data of the current image frame before encoding and the YUV data of the first decoded data into the perceptual quality assessment model for feature extraction to obtain a first feature map corresponding to the current image frame and a second feature map corresponding to the first decoded data; determine the distance between the first feature map and the second feature map; and determine the distance between the first feature map and the second feature map as a first perceptual image quality assessment value of the current image frame.

[0056] In one embodiment, the processing unit is specifically used to obtain a relationship curve between bit rate and quality based on the bit rate corresponding to each quantization parameter set and each first perceptual image quality assessment value; and use the quantization parameter set corresponding to the bit rate at the highest slope in the relationship curve between bit rate and quality as the target quantization parameter set for the current image frame.

[0057] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the video encoding bit rate control method described in the present application when executing the program.

[0058] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the video encoding bit rate control method described in the present application.

[0059] The beneficial effects of this application are as follows:

[0060] In the video encoding bit rate control method, device, electronic device and storage medium provided by the embodiments of the present application, the edge computing server obtains the current image frame of the video stream to be processed according to a preset time period, determines each first target area of ​​the current image frame and the image complexity of each first target area according to a preset algorithm, obtains the target quantization parameter set of the previous image frame of the current image frame, the bit rate of the previous image frame and the perceived image quality evaluation value of the previous image frame, wherein the target quantization parameter set of the previous image frame includes the target quantization parameters of each second target area of ​​the previous image frame, and the perceived image quality evaluation value of the previous image frame is obtained based on the previous image frame and the target quantization parameter set of the previous image frame. The decoded data after frame encoding and the perceptual quality assessment model are obtained, and each first target area, the image complexity of each first target area, the target quantization parameter set of the previous image frame, the bit rate of the previous image frame, and the perceptual image quality assessment value of the previous image frame are input into the parameter fitting model to obtain a preset number of quantization parameter sets of the current image frames, wherein each quantization parameter set includes a group of quantization parameters of each first target area of ​​the current image frame, and then, based on the preset number of quantization parameter sets of the current image frames and the perceptual quality assessment model, the target quantization parameter set of the current image frame is determined, and the current image frame and the image frames in the next time period are evaluated according to the target quantization parameter set of the current image frame. Encoding is performed. In an embodiment of the present application, by determining each first target area of ​​the current image frame and the image complexity of each target area, combining the target quantization parameters of each second target area selected as the encoding parameter of the previous image frame, the bit rate of the previous image frame, and the perceived image quality evaluation value of the previous image frame, a plurality of groups of quantization parameters of each first target area of ​​the current image frame are predicted based on a pre-established parameter fitting model, that is, a preset number of quantization parameter sets of the current image frames are predicted, and then a group of quantization parameter sets are selected from the preset number of quantization parameter sets of the current image frames according to the perceived quality evaluation model as the target quantization parameter set of the current image frame, that is, a group of current image frames are selected. The target quantization parameters of each first target area are set, and each first target area of ​​the current image frame and each target area of ​​each image frame in the next time period are encoded based on the target quantization parameters of each first target area. Therefore, in the process of encoding the video stream, the quantization parameters are dynamically adjusted according to the preset time period, and a set of corresponding optimal quantization parameters are obtained in each time period to encode the video stream in each time period. In addition, in the embodiment of the present application, the decoded data after encoding the image frame (that is, the decoded image frame) is evaluated for quality based on the perceptual quality evaluation model according to the image frame and the target quantization parameter set of the image frame. Compared with PSNR (Peak signal-to-noise ratio,Peak signal-to-noise ratio), SSIM (Structural Similarity Index), and other video quality assessment methods can more accurately assess the image quality change of the image frame after encoding and decoding the image frame using the target quantization parameter predicted by the parameter fitting model compared to the image frame before encoding, thereby ensuring the video quality. This application uses an edge computing server to implement the quantization parameter prediction and video stream encoding and decoding process to assist the computing power in the bit rate control system and improve the efficiency of video compression. ,

[0061] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0063] Figure 1 A schematic diagram of an application scenario of the video encoding bit rate control method provided in an embodiment of the present application;

[0064] Figure 2 A schematic diagram of an implementation flow of a video encoding bit rate control method provided in an embodiment of the present application;

[0065] Figure 3 A schematic diagram of an implementation flow of determining each first target area of ​​a current image frame and the image complexity of each first target area provided in an embodiment of the present application;

[0066] Figure 4 A schematic diagram of an implementation flow of determining a target quantization parameter set for a current image frame provided in an embodiment of the present application;

[0067] Figure 5 A schematic diagram of an implementation flow of determining a first perceived image quality assessment value of a current image frame provided in an embodiment of the present application;

[0068] Figure 6 A schematic diagram of an implementation flow of determining a target quantization parameter set for a current image frame provided in an embodiment of the present application;

[0069] Figure 7 A schematic diagram of the structure of a video encoding bit rate control device provided in an embodiment of the present application;

[0070] Figure 8A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0071] In order to solve the problem in the background technology, the embodiments of the present application provide a video encoding bit rate control method, device, electronic device and storage medium.

[0072] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application may be combined with each other if there is no conflict.

[0073] First reference Figure 1, which is a schematic diagram of an application scenario of the video encoding rate control method provided in an embodiment of the present application. The video encoding rate control method provided in an embodiment of the present application can be applied to an edge computing server 10. The edge computing server 10 at least includes a parameter fitting module 101 and a codec module 102. The chip used for parameter fitting and codec in the edge computing server 10 can be, but is not limited to, a chip with NPU (Neural-network Processing Units, embedded neural network processor) computing power, such as RK3588 chip, etc. NPU adopts a data-driven parallel computing architecture and is good at processing massive multimedia data such as videos and images. NPU can be used to accelerate the operation of neural networks and solve the problem of low efficiency of traditional processor chips in neural network operations. Of course, CPU chips or GPU chips whose computing power can meet the computing requirements can also be used, and the embodiments of the present application are not limited to this.In the video encoding bit rate control method provided by the embodiment of the present application, the parameter fitting module 101 obtains the current image frame of the video stream to be processed according to a preset time period, determines each first target area of ​​the current image frame and the image complexity of each first target area according to a preset algorithm, and the parameter fitting module 101 obtains the target quantization parameter set of the previous image frame of the current image frame and the perceptual image quality evaluation value of the previous image frame from the local, wherein the target quantization parameter set of the previous image frame includes the target quantization parameters of each second target area of ​​the previous image frame, and the perceptual image quality evaluation value of the previous image frame is obtained according to the previous image frame, the decoded data after encoding the previous image frame based on the target quantization parameter set of the previous image frame, and the perceptual quality evaluation model, and the parameter fitting module 101 obtains the bit rate of the previous image frame from the encoding and decoding module 102, and the bit rate of the previous image frame is calculated by the parameter fitting module 101 passing the determined target quantization parameter set of the previous image frame to the encoding and decoding module 102, and the encoding and decoding module 102 encodes the previous image frame based on the target quantization parameter set of the previous image frame based on the preset encoding standard, wherein the preset visual The video coding standard may be but is not limited to the H.264 standard, which is not limited in the embodiment of the present application. The parameter fitting module 101 inputs each first target area of ​​the current image frame, the image complexity of each first target area, the target quantization parameter set of the previous image frame, the bit rate of the previous image frame, and the perceived image quality assessment value of the previous image frame into the parameter fitting model to obtain a preset number of quantization parameter sets of the current image frames, wherein each quantization parameter set includes a set of quantization parameters of each first target area of ​​the current image frame. Then, the parameter fitting module 101 determines a set of quantization parameter sets as the target quantization parameter set of the current image frame based on the preset number of quantization parameter sets of the current image frames and the perceived quality assessment model. The parameter fitting module 101 transmits the target quantization parameter set of the current image frame to the encoding and decoding module 102. The encoding and decoding module 102 encodes the current image frame and the image frames in the next time period according to the target quantization parameter set of the current image frame, and performs subsequent decoding processing. The parameter fitting module 101 and the encoding and decoding module 102 respectively use separate threads for parallel processing, thereby greatly improving the video encoding efficiency.

[0074] It should be noted that the video encoding bit rate control method provided in the embodiment of the present application can also be applied to terminal devices, such as smart phones, tablet computers, laptops, desktop computers, etc. The embodiment of the present application is not limited to this. The embodiment of the present application is only described by applying it to an edge computing server as an example.

[0075] Based on the above application scenarios, the following will refer to the attached Figures 2 to 6The exemplary embodiments of the present application are described in more detail. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principle of the present application, and the implementation of the present application is not limited in any way. On the contrary, the implementation of the present application can be applied to any applicable scenario.

[0076] like Figure 2 As shown, it is a schematic diagram of the implementation process of the video encoding rate control method provided in an embodiment of the present application. The video encoding rate control method can be applied to the above-mentioned edge computing server 10 and may include the following steps:

[0077] S21. Acquire a current image frame of the video stream to be processed according to a preset time period.

[0078] In specific implementation, for the video stream to be processed, the edge computing server can obtain the current image frame of the video stream to be processed according to a preset time period, wherein the preset time period can be set according to demand, for example, it can be set to 1 second, and the embodiment of the present application does not limit this.

[0079] S22. Determine each first target region of the current image frame and the image complexity of each first target region according to a preset algorithm.

[0080] When implementing it specifically, you can follow the following steps: Figure 3 The process shown determines each first target area of ​​the current image frame and the image complexity of each first target area, including the following steps:

[0081] S31. Perform image segmentation on the current image frame according to a preset image segmentation model to obtain various first target areas of the current image frame.

[0082] During specific implementation, the SAM (Segment Anything Model) algorithm in computer vision can be used to perform image segmentation on the current image frame to obtain various regions of interest in the current image frame, which can be recorded as the first target area. Any other image region of interest extraction algorithm can also be used to obtain various regions of interest in the current image frame. The embodiments of the present application are not limited to this.

[0083] S32. Determine the texture complexity, color complexity, and spatial frequency of each first target area respectively.

[0084] In specific implementation, the texture complexity of each first target area can be calculated by using but not limited to Fourier transform. For each first target area, the first target area can be converted into a grayscale image, and the grayscale image can be subjected to a two-dimensional Fourier transform to calculate the amplitude of the frequency, which can be converted into a logarithmic scale to enhance the visualization effect. The high-frequency part represents the details and texture information of the grayscale image, and the texture complexity of the first target area can be obtained by statistically analyzing the energy or amplitude of the high-frequency components.

[0085] During implementation, the color complexity of each first target area can be calculated by using but not limited to a color histogram. For each first target area, a color histogram of the first target area is calculated, and the number of non-zero entries in the color histogram is counted. If there are more non-zero entries in the color histogram, it means that the first target area has more colors. The entropy of the color histogram is used as the color complexity of the first target area. The entropy of the color histogram can provide a measure for the color distribution in the first target area.

[0086] During implementation, waveform transformation can be used but is not limited to calculate the spatial frequency of each first target area, where the spatial frequency is also known as spatial frequency complexity. For each first target area, a time window can be selected and the frames in the window can be analyzed using waveform transformation to obtain a multi-scale time-frequency representation, that is, the spatial frequency of the first target area. High-scale changes can represent fast motion, while low-scale changes can represent slow motion.

[0087] The embodiment of the present application may also adopt any other method of calculating the texture complexity, color complexity and spatial frequency of an image to determine the texture complexity, color complexity and spatial frequency of the first target area, and the embodiment of the present application is not limited to this.

[0088] S33. For each first target region, input the texture complexity, color complexity and spatial frequency of the first target region into an image complexity prediction model to obtain the image complexity of the first target region.

[0089] During specific implementation, the image complexity prediction model can be obtained by training a neural network model. The neural network model can be selected according to the needs, and the embodiments of the present application do not limit this.

[0090] In one embodiment, the image complexity prediction model may include a convolution layer and a pooling layer, and for each first target area, the following operations are performed:

[0091] The texture complexity, color complexity and spatial frequency of the first target area are input into the convolution layer in the image complexity prediction model to obtain feature information of the first target area, and the feature information of the first target area is input into the pooling layer in the image complexity prediction model to obtain the predicted image complexity of the first target area.

[0092] Thus, the image complexity of each first target area is obtained.

[0093] S23, obtaining a target quantization parameter set of an image frame previous to the current image frame, a bit rate of the previous image frame, and a perceived image quality assessment value of the previous image frame.

[0094] Among them, the target quantization parameter set of the previous image frame includes the target quantization parameters of each second target area of ​​the previous image frame, and each second target area of ​​the previous image frame is each interest area of ​​the previous image frame. The bit rate of the previous image frame is the bit rate corresponding to the encoding of the previous image frame using the video coding standard (such as: H.264 standard) according to the target quantization parameter set of the previous image frame. The perceptual image quality assessment value of the previous image frame is obtained based on the previous image frame, the decoded data after encoding the previous image frame based on the target quantization parameter set of the previous image frame and the perceptual quality assessment model. The target quantization parameter set of the previous image frame is also the target quantization parameter set adopted by each image frame in the previous time period.

[0095] During specific implementation, a neural network model is used in advance to train parameter fitting models to predict quantization parameters according to a preset time period. The neural network model can be selected by itself, such as but not limited to the LSTM (Long Short Term Memory Network) model. This embodiment of the present application does not limit this.

[0096] Initially, that is, when the current image frame is the first image frame of the video stream to be processed, the parameter fitting model can be trained by taking the video stream of the same application scenario as the video stream to be processed as a training sample, wherein the same application scenario, such as the video stream to be processed and the video stream as the training sample, can be a video stream generated by the same monitoring device monitoring a fixed place. The parameters of the model are adjusted by iterative fitting to obtain the trained parameter fitting model. The target quantization parameter set and the corresponding bit rate of the image frame finally output by the parameter fitting model can be used as the initial target quantization parameter set and initial bit rate, and the maximum target bit rate and the minimum target bit rate are set to constrain the fitting of the quantization parameters, so that when each set of quantization parameters in the predicted preset number of quantization parameter sets is used to encode the image frame, the corresponding bit rate shall not exceed the maximum target bit rate, nor be lower than the minimum target bit rate, but should be between the maximum target bit rate and the minimum target bit rate. The image quality evaluation value of the image frame finally output by the parameter fitting model can be used as the initial perceptual image quality evaluation value.

[0097] The perceptual quality assessment model can be obtained by training a perceptual neural network such as a multi-layer convolutional network. During training, the YUV data of the sample image before encoding and the YUV data of the decoded data (i.e., the decoded sample image frame) corresponding to the encoded data obtained by encoding the sample image frame (i.e., the encoded sample image frame) can be input into the perceptual neural network for feature extraction to obtain a feature map corresponding to the sample image frame before encoding and a feature map of the decoded sample image frame, and the distance between the feature map corresponding to the sample image frame before encoding and the feature map of the decoded sample image frame is calculated, and the distance value is used as the perceptual image quality assessment value of the sample image frame before and after encoding, and the parameters of the perceptual neural network are continuously adjusted until convergence to obtain a trained perceptual neural network model, wherein the distance between the feature map corresponding to the sample image frame before encoding and the feature map of the decoded sample image frame can be the L2 distance (i.e., Euclidean distance) between the two, or the L1 distance (i.e., Manhattan distance) between the two, or the Chebyshev distance, etc., which is not limited in the embodiments of the present application.

[0098] S24, input each first target area, the image complexity of each first target area, the target quantization parameter set of the previous image frame, the bit rate of the previous image frame, and the perceived image quality assessment value of the previous image frame into a parameter fitting model to obtain a preset number of quantization parameter sets of the current image frames.

[0099] In a specific implementation, each first target area of ​​the current image frame, the image complexity of each first target area, the target quantization parameter set of the previous image frame, the bit rate of the previous image frame, the maximum target bit rate and the minimum target bit rate of the previous image frame, and the perceptual image quality assessment value of the previous image frame are input into a parameter fitting model to obtain a preset number of quantization parameter sets of the current image frame, the maximum target bit rate and the minimum target bit rate of the current image frame, wherein each quantization parameter set includes a group of quantization parameters of each first target area of ​​the current image frame, and the preset number can be set according to demand. The embodiment of the present application does not limit this. For example, when the preset number is 20, the number of first target areas of the current image frame is 60, then the parameter fitting model can output 20 quantization parameter sets, each of which includes a group of quantization parameters of these 60 first target areas.

[0100] In one embodiment, after obtaining the current image frame of the video stream to be processed, motion estimation can also be performed based on the current image frame and the previous image frame to obtain the motion complexity of the current image frame, and the value of the motion estimation guidance parameter is determined based on the motion complexity and a preset motion estimation threshold. Thus, while inputting the first target areas of the current image frame, the image complexity of the first target areas, the target quantization parameter set of the previous image frame, the bit rate of the previous image frame, the maximum target bit rate, the minimum target bit rate, and the perceived image quality assessment value of the previous image frame into a parameter fitting model, the value of the motion estimation guidance parameter input parameter fitting model is used together with the first target areas of the current image frame, the image complexity of the first target areas, the target quantization parameter set of the previous image frame, the bit rate of the previous image frame, and the perceived image quality assessment value of the previous image frame as inputs to the parameter fitting model to predict the quantization parameter set of the current image frame. The motion estimation guidance parameters can be used to adjust the output parameters of the parameter fitting model, namely: each group of quantization parameter sets, the maximum target bit rate and the minimum target bit rate. When the amount of motion of the current image frame relative to the previous image frame is small, it means that the image picture changes little, then the maximum target bit rate and the minimum target bit rate can be lowered, and the quantization parameter value of each first target area in the output quantization parameter set can be increased. The larger the quantization parameter, the lower the corresponding bit rate. This can further improve the accuracy of the output parameters of the parameter fitting model.

[0101] Specifically, the value of the motion estimation guidance parameter may be determined in the following manner:

[0102] If it is determined that the motion complexity is less than or equal to the preset motion estimation threshold, the value of the motion estimation guidance parameter is set to a first set value; if it is determined that the motion complexity is greater than the preset motion estimation threshold, the value of the motion estimation guidance parameter is normalized to a preset value range, wherein the minimum endpoint value of the preset value range is greater than or equal to the first set value.

[0103] Among them, the preset motion estimation threshold, the first set value and the preset value range can be set arbitrarily, and the embodiment of the present application does not limit this.

[0104] As an example, the first set value can be set to 1, the preset value interval can be set to (1, 10), and the motion estimation guidance parameter can be used as a flag. If the motion complexity of the current image frame is less than or equal to the preset motion estimation threshold, the value of the flag can be determined to be 1; if the motion complexity of the current image frame is greater than the preset motion estimation threshold, the value of the flag can be normalized to (1, 10).

[0105] S25. Determine a target quantization parameter set for the current image frame based on a preset number of quantization parameter sets for the current image frames and a perceptual quality assessment model, and encode the current image frame and image frames in the next time period according to the target quantization parameter set for the current image frame.

[0106] When implementing it specifically, you can follow the following steps: Figure 4 The process shown determines the target quantization parameter set of the current image frame, including the following steps:

[0107] S41 . For each quantization parameter set, encode each first target area according to the quantization parameter of each first target area in the quantization parameter set to obtain first encoded data corresponding to the current image frame.

[0108] In specific implementation, each quantization parameter set output by the parameter fitting model is used to encode the current image frame to obtain encoding data corresponding to each current image frame, which can be recorded as first encoding data, that is, the encoded image frame corresponding to each current image frame.

[0109] Specifically, for each quantization parameter set output by the parameter fitting model, each first target area of ​​the current image frame can be encoded using the H.264 standard according to the quantization parameters of each first target area in the quantization parameter set to obtain the first encoded data corresponding to the current image frame, and a corresponding bit rate is generated for encoding the current image frame according to each quantization parameter set.

[0110] Assuming that 20 quantization parameter sets are output and the current image frame includes 60 first target areas, each quantization parameter set includes 60 quantization parameters of the first target areas. The corresponding first target areas are encoded according to the 60 quantization parameters in each quantization parameter set to obtain the first encoded data corresponding to the current image frame. In this way, the first encoded data corresponding to 20 current image frames can be obtained.

[0111] S42: Decode the first encoded data to obtain first decoded data corresponding to the current image frame.

[0112] In a specific implementation, each first encoded data is decoded respectively to obtain the first decoded data corresponding to each current image frame.

[0113] S43: Determine a first perceptual image quality assessment value of the current image frame based on the current image frame before encoding, the first decoded data, and the perceptual quality assessment model.

[0114] In specific implementation, for each quantization parameter set, the following Figure 5 The process shown determines a first perceptual image quality assessment value of a current image frame, comprising the following steps:

[0115] S51. Input the YUV data of the current image frame before encoding and the YUV data of the first decoded data into a perceptual quality assessment model for feature extraction to obtain a first feature map corresponding to the current image frame and a second feature map corresponding to the first decoded data.

[0116] S52: Determine the distance between the first feature map and the second feature map.

[0117] During specific implementation, the L2 distance between the first feature map and the second feature map may be calculated, or the L1 distance or Chebyshev distance between the first feature map and the second feature map may be calculated, which is not limited in the embodiments of the present application.

[0118] S53: Determine the distance between the first feature map and the second feature map as a first perceptual image quality assessment value of the current image frame.

[0119] In this way, the first perceived image quality evaluation values ​​before and after encoding can be obtained by encoding the current image frame using various quantization parameter sets.

[0120] S44. Based on the bit rate corresponding to each quantization parameter set and each first perceptual image quality assessment value, select a quantization parameter set that meets a preset condition as a target quantization parameter set for the current image frame.

[0121] When implementing it specifically, you can follow the following steps: Figure 6 The process shown in the figure determines the target quantization parameter set of the current image frame, which may include the following steps:

[0122] S61. Obtain a relationship curve between bit rate and quality according to the bit rate corresponding to each quantization parameter set and each first perceptual image quality assessment value.

[0123] In specific implementation, the bit rate can be used as the horizontal axis and the perceived image quality evaluation value can be used as the vertical axis to obtain a relationship curve between the bit rate and the quality according to the bit rate corresponding to each quantization parameter set and each corresponding first perceived image quality evaluation value.

[0124] S62: Use the quantization parameter set corresponding to the bit rate at the point with the highest slope in the bit rate-quality relationship curve as the target quantization parameter set for the current image frame.

[0125] Then, the current image frame and the image frames in the next time period are encoded according to the selected target quantization parameter set of the current image frame, and subsequent decoding operations are performed.

[0126] In this way, a set of most balanced quantization parameters can be selected from the target quantization parameter set used for encoding and decoding image frames in each time period, which can take into account both video encoding quality and encoding efficiency.

[0127] In actual business scenarios, by adopting the video encoding bit rate control method provided in the embodiment of the present application, the desktop side can comprehensively reduce the bit rate by more than 30%, among which the PPT screen can be compressed to 25%, and the pure motion screen can be compressed to 35%, and the picture quality is better. For the live video stream, the bit rate can be reduced from 656K to 329K, a compression of 50%, thereby ensuring that the live broadcast has better real-time performance and reducing latency by reducing picture quality.

[0128] In the process of encoding a video stream, the present application realizes dynamic adjustment of quantization parameters according to a preset time period, obtains a set of corresponding optimal quantization parameters in each time period, and encodes the video stream in each time period. In addition, in an embodiment of the present application, the decoded data after encoding the image frame is evaluated for quality based on a perceptual quality assessment model according to the image frame and the target quantization parameter set of the image frame. Compared with video quality assessment methods such as PSNR and SSIM, it can more accurately evaluate the change in image quality of the image frame after encoding and decoding the image frame using the target quantization parameter predicted by the parameter fitting model compared to the image frame before encoding, thereby ensuring the video quality. The present application uses an edge computing server to implement quantization parameter prediction and video stream encoding and decoding process to assist the computing power in the bit rate control system and improve the efficiency of video compression.

[0129] Based on the same inventive concept, an embodiment of the present application also provides a video encoding bit rate control device. Since the principle of solving the problem by the above-mentioned video encoding bit rate control device is similar to that of the above-mentioned video encoding bit rate control method, the implementation of the above-mentioned device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0130] like Figure 7 As shown, it is a structural schematic diagram of a video encoding bit rate control device provided in an embodiment of the present application, which can be applied to an edge computing server and may include:

[0131] A first acquisition unit 71 is used to acquire a current image frame of a video stream to be processed according to a preset time period;

[0132] A first determining unit 72, configured to determine each first target region of the current image frame and the image complexity of each first target region according to a preset algorithm;

[0133] A second acquisition unit 73 is used to acquire a target quantization parameter set of an image frame previous to the current image frame, a bit rate of the previous image frame, and a perceived image quality assessment value of the previous image frame;

[0134] A parameter fitting unit 74 is used to input the first target regions, the image complexity of the first target regions, the target quantization parameter set of the previous image frame, the bit rate of the previous image frame, and the perceived image quality assessment value of the previous image frame into a parameter fitting model to obtain a preset number of quantization parameter sets of the current image frame, wherein each quantization parameter set includes a set of quantization parameters of the first target regions of the current image frame;

[0135] The processing unit 75 is used to determine a target quantization parameter set of the current image frame based on a preset number of quantization parameter sets of the current image frame and the perceptual quality assessment model, and encode the current image frame and image frames in the next time period according to the target quantization parameter set of the current image frame.

[0136] In one embodiment, the target quantization parameter set of the previous image frame includes target quantization parameters of each second target area of ​​the previous image frame, and the perceptual image quality assessment value of the previous image frame is obtained according to the previous image frame, the decoded data after encoding the previous image frame based on the target quantization parameter set of the previous image frame, and the perceptual quality assessment model.

[0137] In one embodiment, the device further comprises:

[0138] A motion estimation unit, configured to, after acquiring a current image frame of a video stream to be processed, perform motion estimation based on the current image frame and the previous image frame to obtain a motion complexity of the current image frame;

[0139] A second determining unit, configured to determine a value of a motion estimation guidance parameter according to the motion complexity and a preset motion estimation threshold; and

[0140] The parameter fitting unit 74 is further used to input the value of the motion estimation guidance parameter into the parameter fitting model while inputting the first target areas, the image complexity of the first target areas, the target quantization parameter set of the previous image frame, the bit rate of the previous image frame, and the perceptual image quality assessment value of the previous image frame into the parameter fitting model.

[0141] In one embodiment, the second determination unit is specifically used to set the value of the motion estimation guidance parameter to a first set value if it is determined that the motion complexity is less than or equal to the preset motion estimation threshold; if it is determined that the motion complexity is greater than the preset motion estimation threshold, normalize the value of the motion estimation guidance parameter to a preset value range, wherein the minimum endpoint value of the preset value range is greater than or equal to the first set value.

[0142] In one embodiment, the first determination unit 72 is specifically used to perform image segmentation on the current image frame according to a preset image segmentation model to obtain each first target area of ​​the current image frame; respectively determine the texture complexity, color complexity and spatial frequency of each first target area; for each first target area, input the texture complexity, color complexity and spatial frequency of the first target area into the image complexity prediction model to obtain the image complexity of the first target area.

[0143] In one embodiment, the first determination unit 72 is specifically used to input the texture complexity, color complexity and spatial frequency of each first target area into the convolution layer in the image complexity prediction model to obtain the feature information of the first target area; input the feature information of the first target area into the pooling layer in the image complexity prediction model to obtain the predicted image complexity of the first target area.

[0144] In one embodiment, the processing unit 75 is specifically used to, for each quantization parameter set, encode each first target area according to the quantization parameters of each first target area in the quantization parameter set to obtain first encoded data corresponding to the current image frame; decode the first encoded data to obtain first decoded data corresponding to the current image frame; determine a first perceptual image quality assessment value of the current image frame based on the current image frame before encoding, the first decoded data and the perceptual quality assessment model; and select a quantization parameter set that meets preset conditions as the target quantization parameter set for the current image frame based on the bit rate corresponding to each quantization parameter set and each first perceptual image quality assessment value.

[0145] In one embodiment, the processing unit 75 is specifically used to input the YUV data of the current image frame before encoding and the YUV data of the first decoded data into the perceptual quality assessment model for feature extraction to obtain a first feature map corresponding to the current image frame and a second feature map corresponding to the first decoded data; determine the distance between the first feature map and the second feature map; and determine the distance between the first feature map and the second feature map as a first perceptual image quality assessment value of the current image frame.

[0146] In one embodiment, the processing unit 75 is specifically used to obtain a relationship curve between bit rate and quality based on the bit rate corresponding to each quantization parameter set and each first perceptual image quality assessment value; and use the quantization parameter set corresponding to the bit rate at the highest slope in the relationship curve between bit rate and quality as the target quantization parameter set for the current image frame.

[0147] Based on the same technical concept, the embodiment of the present application also provides an electronic device 800, referring to Figure 8 As shown, the electronic device 800 is used to implement the video encoding rate control method described in the above method embodiment. The electronic device 800 of this embodiment may include: a memory 801, a processor 802, and a computer program stored in the memory and executable on the processor, such as a video encoding rate control program. When the processor executes the computer program, the steps in each of the above video encoding rate control method embodiments are implemented.

[0148] The specific connection medium between the memory 801 and the processor 802 is not limited in the embodiment of the present application. Figure 8 In the embodiment, the memory 801 and the processor 802 are connected via a bus 803. The bus 803 is Figure 8 The connection between other components is shown by bold lines, and is not intended to be limiting. The bus 803 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0149] The memory 801 may be a volatile memory, such as a random-access memory (RAM); the memory 801 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), or the memory 801 may be any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 801 may be a combination of the above memories.

[0150] The processor 802 is used to implement the video encoding rate control method provided in the embodiment of the present application.

[0151] An embodiment of the present application also provides a computer-readable storage medium that stores computer-executable instructions required to execute the above-mentioned processor, which includes a program required to execute the above-mentioned processor.

[0152] In some possible implementations, various aspects of the video encoding bit rate control method provided in the present application may also be implemented in the form of a program product, which includes a program code. When the program product is run on an electronic device, the program code is used to enable the electronic device to perform the steps of the video encoding bit rate control method according to various exemplary implementations of the present application described above in this specification.

[0153] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0154] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (apparatus), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0155] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0156] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0157] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0158] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A video encoding bit rate control method, characterized in that: Applied to an edge computing server, the method comprises: Obtaining a current image frame of the video stream to be processed according to a preset time period; Determine each first target area of ​​the current image frame and the image complexity of each first target area according to a preset algorithm; Acquire a target quantization parameter set of an image frame previous to the current image frame, a bit rate of the previous image frame, and a perceived image quality assessment value of the previous image frame; Inputting the first target regions, the image complexity of the first target regions, the target quantization parameter set of the previous image frame, the bit rate of the previous image frame, and the perceived image quality assessment value of the previous image frame into a parameter fitting model to obtain a preset number of quantization parameter sets of the current image frame, wherein each quantization parameter set includes a group of quantization parameters of the first target regions of the current image frame; A target quantization parameter set of the current image frame is determined based on a preset number of quantization parameter sets of the current image frame and the perceptual quality assessment model, and the current image frame and image frames in the next time period are encoded according to the target quantization parameter set of the current image frame.

2. The method according to claim 1, characterized in that The target quantization parameter set of the previous image frame includes target quantization parameters of each second target area of ​​the previous image frame, and the perceptual image quality assessment value of the previous image frame is obtained according to the previous image frame, the decoded data after encoding the previous image frame based on the target quantization parameter set of the previous image frame, and the perceptual quality assessment model.

3. The method according to claim 2, characterized in that After obtaining the current image frame of the video stream to be processed, it also includes: Performing motion estimation according to the current image frame and the previous image frame to obtain motion complexity of the current image frame; Determining a value of a motion estimation guidance parameter according to the motion complexity and a preset motion estimation threshold; and While inputting the first target regions, the image complexity of the first target regions, the target quantization parameter set of the previous image frame, the bit rate of the previous image frame, and the perceived image quality assessment value of the previous image frame into the parameter fitting model, the method further includes: The values ​​of the motion estimation guidance parameters are input into the parameter fitting model.

4. The method according to claim 3, characterized in that Determining the value of the motion estimation guidance parameter according to the motion complexity and a preset motion estimation threshold specifically includes: If it is determined that the motion complexity is less than or equal to the preset motion estimation threshold, setting the value of the motion estimation guidance parameter to a first set value; If it is determined that the motion complexity is greater than the preset motion estimation threshold, the value of the motion estimation guidance parameter is normalized to a preset value interval, wherein the minimum endpoint value of the preset value interval is greater than or equal to the first set value.

5. The method according to claim 1, characterized in that Determining each first target area of ​​the current image frame and the image complexity of each first target area according to a preset algorithm specifically includes: Performing image segmentation on the current image frame according to a preset image segmentation model to obtain each first target area of ​​the current image frame; Determining the texture complexity, color complexity and spatial frequency of each first target area respectively; For each first target region, the texture complexity, color complexity and spatial frequency of the first target region are input into an image complexity prediction model to obtain the image complexity of the first target region.

6. The method according to claim 5, characterized in that For each first target area, inputting the texture complexity, color complexity and spatial frequency of the first target area into the image complexity prediction model to obtain the image complexity of the first target area specifically includes: For each first target area, inputting the texture complexity, color complexity and spatial frequency of the first target area into a convolutional layer in an image complexity prediction model to obtain feature information of the first target area; The feature information of the first target area is input into a pooling layer in the image complexity prediction model to obtain a predicted image complexity of the first target area.

7. The method according to claim 2 or 3, characterized in that: Determining a target quantization parameter set of the current image frame based on a preset number of quantization parameter sets of the current image frame and the perceptual quality assessment model specifically includes: For each quantization parameter set, encoding each first target area according to the quantization parameter of each first target area in the quantization parameter set, to obtain first encoded data corresponding to the current image frame; Decoding the first encoded data to obtain first decoded data corresponding to the current image frame; Determining a first perceptual image quality assessment value of the current image frame based on the current image frame before encoding, the first decoded data, and the perceptual quality assessment model; Based on the bit rate corresponding to each quantization parameter set and each first perceptual image quality assessment value, a quantization parameter set that meets a preset condition is selected as a target quantization parameter set for the current image frame.

8. The method according to claim 7, characterized in that Determining a first perceptual image quality assessment value of the current image frame based on the current image frame before encoding, the first decoded data, and the perceptual quality assessment model specifically includes: Inputting YUV data of the current image frame before encoding and YUV data of the first decoded data into the perceptual quality assessment model for feature extraction to obtain a first feature map corresponding to the current image frame and a second feature map corresponding to the first decoded data; determining a distance between the first feature map and the second feature map; A distance between the first feature map and the second feature map is determined as a first perceptual image quality assessment value of the current image frame.

9. The method according to claim 7, characterized in that Based on the bit rate corresponding to each quantization parameter set and each first perceptual image quality assessment value, selecting a quantization parameter set that meets a preset condition as a target quantization parameter set for the current image frame specifically includes: Obtaining a relationship curve between bit rate and quality according to the bit rate corresponding to each quantization parameter set and each first perceptual image quality assessment value; The quantization parameter set corresponding to the bit rate at the point with the highest slope in the bit rate-quality relationship curve is used as the target quantization parameter set for the current image frame.

10. A video encoding bit rate control device, characterized in that: Applied to an edge computing server, the device comprises: A first acquisition unit, used for acquiring a current image frame of a video stream to be processed according to a preset time period; A first determining unit, configured to determine each first target region of the current image frame and an image complexity of each first target region according to a preset algorithm; A second acquisition unit, configured to acquire a target quantization parameter set of an image frame previous to the current image frame, a bit rate of the previous image frame, and a perceived image quality assessment value of the previous image frame; a parameter fitting unit, configured to input the first target regions, the image complexity of the first target regions, the target quantization parameter set of the previous image frame, the bit rate of the previous image frame, and the perceived image quality assessment value of the previous image frame into a parameter fitting model, so as to obtain a preset number of quantization parameter sets of the current image frame, wherein each quantization parameter set includes a group of quantization parameters of the first target regions of the current image frame; A processing unit is used to determine a target quantization parameter set of the current image frame based on a preset number of quantization parameter sets of the current image frame and the perceptual quality assessment model, and encode the current image frame and image frames in the next time period according to the target quantization parameter set of the current image frame.

11. The device according to claim 10, characterized in that The target quantization parameter set of the previous image frame includes target quantization parameters of each second target area of ​​the previous image frame, and the perceptual image quality assessment value of the previous image frame is obtained according to the previous image frame, the decoded data after encoding the previous image frame based on the target quantization parameter set of the previous image frame, and the perceptual quality assessment model.

12. The device according to claim 11, characterized in that Also includes: A motion estimation unit, configured to, after acquiring a current image frame of a video stream to be processed, perform motion estimation based on the current image frame and the previous image frame to obtain a motion complexity of the current image frame; A second determining unit, configured to determine a value of a motion estimation guidance parameter according to the motion complexity and a preset motion estimation threshold; as well as The parameter fitting unit is further used to input the value of the motion estimation guidance parameter into the parameter fitting model while inputting the first target areas, the image complexity of the first target areas, the target quantization parameter set of the previous image frame, the bit rate of the previous image frame, and the perceptual image quality assessment value of the previous image frame into the parameter fitting model.

13. The device according to claim 12, characterized in that The second determination unit is specifically used to set the value of the motion estimation guidance parameter to a first set value if it is determined that the motion complexity is less than or equal to the preset motion estimation threshold; if it is determined that the motion complexity is greater than the preset motion estimation threshold, normalize the value of the motion estimation guidance parameter to a preset value range, wherein the minimum endpoint value of the preset value range is greater than or equal to the first set value.

14. The device according to claim 10, characterized in that The first determination unit is specifically used to perform image segmentation on the current image frame according to a preset image segmentation model to obtain each first target area of ​​the current image frame; respectively determine the texture complexity, color complexity and spatial frequency of each first target area; for each first target area, input the texture complexity, color complexity and spatial frequency of the first target area into the image complexity prediction model to obtain the image complexity of the first target area.

15. The device according to claim 14, characterized in that The first determination unit is specifically used to input the texture complexity, color complexity and spatial frequency of each first target area into the convolution layer in the image complexity prediction model to obtain the feature information of the first target area; and input the feature information of the first target area into the pooling layer in the image complexity prediction model to obtain the predicted image complexity of the first target area.

16. The device according to claim 11 or 12, characterized in that The processing unit is specifically configured to encode each first target area according to the quantization parameters of each first target area in the quantization parameter set for each quantization parameter set, so as to obtain first encoded data corresponding to the current image frame; Decoding the first encoded data to obtain first decoded data corresponding to the current image frame; Based on the current image frame before encoding, the first decoded data and the perceptual quality assessment model, a first perceptual image quality assessment value of the current image frame is determined; based on the bit rate corresponding to each quantization parameter set and each first perceptual image quality assessment value, a quantization parameter set that meets preset conditions is selected as the target quantization parameter set for the current image frame.

17. The device according to claim 16, characterized in that The processing unit is specifically used to input the YUV data of the current image frame before encoding and the YUV data of the first decoded data into the perceptual quality assessment model for feature extraction, to obtain a first feature map corresponding to the current image frame and a second feature map corresponding to the first decoded data; to determine the distance between the first feature map and the second feature map; and to determine the distance between the first feature map and the second feature map as a first perceptual image quality assessment value of the current image frame.

18. The device according to claim 16, characterized in that The processing unit is specifically used to obtain a relationship curve between bit rate and quality according to the bit rate corresponding to each quantization parameter set and each first perceptual image quality assessment value; and use the quantization parameter set corresponding to the bit rate at the highest slope in the relationship curve between bit rate and quality as the target quantization parameter set for the current image frame.

19. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the video encoding bit rate control method according to any one of claims 1 to 9 is implemented.

20. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the video encoding bit rate control method according to any one of claims 1 to 9 are implemented.

Images