Image coding method, apparatus, device, and storage medium

Abstract

The application provides an image coding method, device and equipment and a storage medium. The image coding method comprises the following steps: acquiring at least one image group; determining a quantization parameter of a first frame of a first image group according to frame information of the first frame, and determining quantization parameters of remaining frames according to frame information and adjacent frame information of the remaining frames, wherein the remaining frames are frames other than the first frame of the first image group in the at least one image group; and coding corresponding frames according to the quantization parameter of the first frame of the first image group and the quantization parameters of the remaining frames.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image coding, for example to an image coding method, device, equipment and storage medium. BACKGROUND

[0002] With the popularity and development of mobile communication technology, real-time video transmission has gradually appeared on a system on chip (SoC) in an embedded device, and has been more and more widely used in many fields such as video monitoring, network live broadcast, online conference and the like. Generally, the transmission of a video picture is affected by two factors: network bandwidth and video data volume. In order to reduce the data volume of the video data, the original data needs to be encoded and compressed in combination with the network condition before transmission. Different compression degrees will produce different image display effects. If the data volume after compression is too large, the transmission will be lost, and if the data volume after compression is too small, the image display picture will be poor. In order to adapt to different network bandwidths, the compression rate of the video needs to be controlled in real time. Therefore, a rate control module is an indispensable part of the encoding end before transmission, and has been widely valued in the theory of image transmission. The smoothness of video display and the clarity of the video picture are important indicators for people to measure and purchase embedded devices.

[0003] The existing algorithm determines the initial quantization parameter (QP) value through rough interval division according to the control mode of the bit number (bpp) occupied by each pixel, which results in too low algorithm accuracy. Since the selection of the initial QP has a great influence on the encoding effect of the first few frames of the image sequence, an inappropriate initial QP value may cause excessive occupation of the buffer (far exceeding the target bit number pre-allocated), at which time the encoder will try to reduce the target bit number allocated to the subsequent several frames to prevent the buffer from overflowing, thereby causing the image quality of the subsequent several frames to rapidly decrease (because too few bits are available for encoding), and even causing frame skipping. The existing algorithm allocates the remaining bits of the current group of pictures (GOP) to the remaining P frames in an average manner, without considering the influence of instantaneous scene changes. SUMMARY

[0004] The present application provides an image coding method, device, equipment and storage medium, which can solve the problem of improper selection of a quantization parameter in the prior art.

[0005] In a first aspect, an embodiment of the present application provides an image coding method, comprising: acquiring at least one group of pictures;

[0006] determine the quantization parameter of the first frame of the first image group according to the frame information of the first frame of the first image group, and determine the quantization parameter of the remaining frames according to the frame information and the adjacent frame information of the remaining frames, wherein the remaining frames are frames of the at least one image group except the first frame of the first image group;

[0007] encode the corresponding frames according to the quantization parameter of the first frame of the first image group and the quantization parameter of the remaining frames.

[0008] In a second aspect, an embodiment of the present application provides an image encoding apparatus, comprising: a first obtaining module, configured to obtain at least one image group;

[0009] a first determining module, configured to determine the quantization parameter of the first frame of the first image group according to the frame information of the first frame of the first image group, and determine the quantization parameter of the remaining frames according to the frame information and the adjacent frame information of the remaining frames, wherein the remaining frames are frames of the at least one image group except the first frame of the first image group;

[0010] a first encoding module, configured to encode the corresponding frames according to the quantization parameter of the first frame of the first image group and the quantization parameter of the remaining frames.

[0011] In a third aspect, an embodiment of the present application provides a device, comprising: one or more processors; a storage apparatus configured to store one or more programs; and when the one or more programs are executed by the one or more processors, the one or more processors implement the information determining method in the first aspect.

[0012] In a fourth aspect, an embodiment of the present application provides a storage medium, which stores a computer program, and the computer program is executed by a processor to implement any of the methods in the embodiments of the present application.

[0013] More details of the above embodiments and other aspects of the present application and implementation manners thereof are provided in the following description of drawings, specific embodiments and claims. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 a flowchart of an information determining method provided by the present application;

[0015] Figure 1a a flowchart of an encoding method provided by the present application;

[0016] Figure 2 a structure diagram of an information determining apparatus provided by the present application;

[0017] Figure 3 a structure diagram of a device provided by the present application. DETAILED DESCRIPTION

[0018] In order to make the purposes, technical solutions and advantages of the present application clearer, the embodiments of the present application will be described in detail below with reference to the drawings. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other as long as there is no conflict.

[0019] The steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer executable instructions. Moreover, although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown.

[0020] In an exemplary embodiment, Figure 1 A flowchart of an image encoding method provided by the present application is shown. The method can be applied to the case of encoding an image. The method can be executed by an image encoding device provided by the present application, which can be realized by software and / or hardware and integrated on a device.

[0021] As Figure 1 shown, an image encoding method provided by the present application includes S110-S130.

[0022] S110, acquiring at least one image group.

[0023] The image group (GOP, Grou Of Pictures) is an acquired image group, and the image group is composed of multiple frames.

[0024] S120, determining a quantization parameter of a first frame of a first image group according to frame information of the first frame, and determining quantization parameters of the remaining frames according to frame information and adjacent frame information of the remaining frames.

[0025] The remaining frames are frames other than the first frame of the first image group in the at least one image group.

[0026] The frame information includes complexity, remaining bit number and frame type, or complexity, frame type, pixel depth and entropy information.

[0027] The adjacent frame information includes average complexity of encoded frames, bit number occupied by encoded frames, actual buffer occupation, available bandwidth, target buffer frame rate, remaining bit number and motion change coefficient of the two frames before and after.

[0028] Specifically, the quantization parameter of the first frame of the first image group is determined according to the frame information of the first frame, and the quantization parameters of the remaining frames are determined according to the frame information and adjacent frame information, including: the quantization parameter of the first frame of the first image group is determined according to the complexity, frame type, pixel depth and entropy information of the first frame; the quantization parameter of the first frame of the remaining image groups is determined according to the complexity, frame type, pixel depth, entropy information and motion change coefficient of the previous and next two frames; the quantization parameter of the first type P frame in the remaining frames of the image group is the same as that of the adjacent first frame, wherein the first type P frame is the second frame of the image group; the quantization parameter of the second type P frame in the remaining frames of the image group is determined according to the average complexity of the encoded frames, the number of bits occupied by the encoded frames, the actual buffer occupation, the available bandwidth, the target buffer frame rate, the remaining number of bits and the motion change coefficient of the previous and next two frames, wherein the second type P frame is not the second frame of the image group; the quantization parameter of the B frame in the remaining frames of the image group is determined according to the average value of the quantization parameters of the adjacent P frames.

[0029] In a specific example, at least one image group is obtained, if the basic unit is set to a frame, the first frame of the first image group is encoded, the quantization parameter of the first frame is calculated before the first frame of the first image group is encoded (the first frame of the first image group is an I frame), the quantization parameter of the first frame is determined according to the complexity, frame type, pixel depth and entropy information, the first frame is encoded according to the quantization parameter of the first frame, the second frame is a P frame, the quantization parameter of the second frame is the same as that of the first frame, the second frame is encoded according to the quantization parameter of the first frame, if the third frame is a P frame, the quantization parameter of the third frame is determined according to the average complexity of the encoded frames, the number of bits occupied by the encoded frames, the actual buffer occupation, the available bandwidth, the target buffer frame rate, the remaining number of bits and the motion change coefficient of the previous and next two frames, the third frame is encoded according to the quantization parameter of the third frame, if the third frame is a B frame, it is not encoded first, the fourth frame is encoded first, if the fourth frame is a P frame, the quantization parameter of the fourth frame is determined according to the average complexity of the encoded frames, the number of bits occupied by the encoded frames, the actual buffer occupation, the available bandwidth, the target buffer frame rate, the remaining number of bits and the motion change coefficient of the previous and next two frames, the fourth frame is encoded according to the quantization parameter of the fourth frame, and the quantization parameter of the third frame is determined according to the quantization parameter of the fourth frame and the quantization parameter of the second frame. If the fourth frame is a B frame, it is not encoded first, until it is a P frame, and then the P frame is encoded.

[0030] S130, encoding the corresponding frame according to the quantization parameter of the first frame of the first image group and the quantization parameter of the remaining frames.

[0031] It should be noted that after the quantization parameter of a frame is determined, the current frame can be directly encoded according to the obtained quantization parameter, or the quantization parameter of the current frame can be first dynamically adjusted according to the previous frame, and then the current frame is encoded according to the adjusted quantization parameter. The embodiments of the present application do not have comparison restrictions.

[0032] The present application provides an image encoding method, comprising: obtaining at least one image group; determining a quantization parameter of a first frame of a first image group according to frame information of the first frame, and determining quantization parameters of the remaining frames according to frame information and adjacent frame information of the remaining frames, wherein the remaining frames are frames other than the first frame of the first image group in the at least one image group; and encoding corresponding frames according to the quantization parameter of the first frame of the first image group and the quantization parameters of the remaining frames, which solves the technical problem that the buffer is excessively occupied (far more than the target bit number pre-allocated) due to improper selection of the quantization parameter in the prior art, at this time, the encoder will try to reduce the target bit number allocated to the subsequent several frames to ensure that the buffer does not overflow, thereby causing the image quality of the subsequent several frames to rapidly decrease (because the number of bits available for encoding is too small), and even causing the technical problem of frame skipping, which can reduce the overall fluctuation of the encoded image and thus improve the quality of the graphical display, and ultimately improve the user experience.

[0033] On the basis of the above-mentioned embodiments, variant embodiments of the above-mentioned embodiments are provided, and it should be noted that, in order to make the description brief, only the differences from the above-mentioned embodiments are described in the variant embodiments.

[0034] In one embodiment, the determination of the quantization parameter of the first frame of the first image group according to the frame information and the determination of the quantization parameters of the remaining frames according to the frame information and the adjacent frame information comprise:

[0035] The determination of the quantization parameter of the first frame of the first image group according to the complexity, the frame type, the pixel depth and the entropy information of the first frame;

[0036] The determination of the quantization parameter of the first frame of the remaining image group according to the complexity, the frame type, the pixel depth, the entropy information and the motion change coefficient of the previous and subsequent two frames;

[0037] In the remaining frames of the image group, the quantization parameter of the first type P frame is the same as the quantization parameter of the adjacent first frame, wherein the first type P frame is the second frame of the image group;

[0038] In the remaining frames of the image group, the second type P frame, the quantization parameter of the second type P frame is determined according to the average complexity of the encoded frames, the number of bits occupied by the encoded frames, the actual buffer occupancy, the available bandwidth, the target buffer frame rate, the remaining number of bits and the motion change coefficient of the previous and subsequent two frames, wherein the second type P frame is not the second frame of the image group.

[0039] The B frame in the rest of the image group is determined according to the average of the quantization parameters of the adjacent P frames.

[0040] Specifically, the quantization parameter of the first frame of the first image group is determined according to the complexity, frame type, pixel depth and entropy information of the first frame of the first image group, the quantization parameter of the first frame of the rest of the image groups is determined according to the complexity, frame type, pixel depth, entropy information and motion change coefficient of the two frames before and after the first frame of the rest of the image groups, the quantization parameter of the first type P frame in the rest of the image group is the same as that of the adjacent first frame, wherein the first type P frame is the second frame of the image group, the quantization parameter of the second type P frame in the rest of the image group is determined according to the average complexity of the coded frames, the number of bits occupied by the coded frames, the actual buffer occupation, the available bandwidth, the target buffer frame rate, the remaining number of bits and the motion change coefficient of the two frames before and after, wherein the second type P frame is not the second frame of the image group, and the quantization parameter of the B frame in the rest of the image group is determined according to the average of the quantization parameters of the adjacent P frames.

[0041] In one embodiment, the quantization parameter of the first frame of the first image group is determined according to the frame information of the first frame of the first image group, and the quantization parameter of the rest of the frames is determined according to the frame information and adjacent frame information of the rest of the frames, including:

[0042] Obtaining a frame to be coded;

[0043] If the frame to be coded is the first frame of the first image group, the quantization parameter of the first frame of the first image group is determined according to the complexity, frame type, pixel depth and entropy information;

[0044] If the frame to be coded is the first frame of the rest of the image groups, the quantization parameter of the first frame of the rest of the image groups is determined according to the complexity, frame type, pixel depth, entropy information and motion change coefficient of the two frames before and after;

[0045] If the frame to be coded is the first type P frame, the quantization parameter of the first type P frame is determined according to the quantization parameter of the adjacent first frame, wherein the first type P frame is the second frame of the image group;

[0046] If the frame to be coded is the second type P frame, the quantization parameter of the second type P frame is determined according to the average complexity of the coded frames, the number of bits occupied by the coded frames, the actual buffer occupation, the available bandwidth, the target buffer frame rate, the remaining number of bits and the motion change coefficient of the two frames before and after, wherein the second type P frame is not the second frame of the image group;

[0047] If the frame to be encoded is a B frame, no encoding is performed, and a P frame adjacent to the B frame and after the B frame is encoded first, and then the quantization parameter of the B frame is determined according to the average of the quantization parameters of the adjacent P frames.

[0048] Specifically, if the frame to be encoded is the first frame of a group of pictures, the quantization parameter of the first frame of the group of pictures is determined according to the complexity, frame type, pixel depth and entropy information; if the frame to be encoded is the first frame of a group of pictures other than the first group of pictures, the quantization parameter of the first frame of the group of pictures is determined according to the complexity, frame type, pixel depth, entropy information and motion change coefficient of the two adjacent frames; if the frame to be encoded is a P frame and is not the second frame of a group of pictures, the quantization parameter of the P frame is determined according to the average complexity of the encoded frames, the number of bits occupied by the encoded frames, the actual buffer occupancy, the available bandwidth, the target buffer frame rate, the remaining number of bits and the motion change coefficient of the two adjacent frames; if the frame to be encoded is a B frame, no encoding is performed, and a P frame adjacent to the B frame and after the B frame is encoded first, and then the quantization parameter of the B frame is determined according to the average of the quantization parameters of the adjacent P frames.

[0049] In one embodiment, if the frame to be encoded is a second type of P frame, the quantization parameter of the second type of P frame is determined according to the average complexity of the encoded frames, the number of bits occupied by the encoded frames, the actual buffer occupancy, the available bandwidth, the target buffer frame rate, the remaining number of bits and the motion change coefficient of the two adjacent frames, wherein the second type of P frame is not the second frame of a group of pictures, and includes:

[0050] The target buffer occupancy of the second type of P frame is determined according to the average complexity of the encoded frames and the number of bits occupied by the encoded frames;

[0051] The target number of bits of the second type of P frame is determined according to the target buffer occupancy, the actual buffer occupancy, the available bandwidth, the target buffer frame rate, the remaining number of bits and the motion change coefficient of the two adjacent frames;

[0052] The encoding mode of the second type of P frame is estimated according to the target number of bits of the second type of P frame;

[0053] The number of header information bits of the second type of P frame is determined according to the estimated encoding mode of the second type of P frame and the motion change coefficient of the two adjacent frames;

[0054] The actual encoding mode of the second type of P frame is determined according to the number of header information bits of the second type of P frame;

[0055] The actual average absolute difference value of the second type of P frame is calculated according to the actual encoding mode of the second type of P frame.

[0056] The quantization parameter of the second type P frame is determined according to the actual average absolute value difference of the second type P frame, the target bit number of the second type P frame, the head information bit number of the second type P frame and the bit number occupied by the motion vector.

[0057] Specifically, a target bit number is allocated to the frame to be encoded; the bit number allocated to each frame is weightedly allocated in combination with the remaining allocated bit number in the GOP, the image complexity of the frame to be encoded, motion change information and other factors. First, the target buffer occupation of the frame to be encoded is adjusted according to the actual buffer occupation, since the target buffer capacity of the already encoded frame is not completely consistent with the actual buffer capacity, the difference between the two needs to be allocated to each frame to be encoded. Since the quantization parameter of the first P frame has been obtained in the GOP layer code rate control, the quantization parameter of the first P frame is the same as that of the adjacent I frame, here only the buffer size of other P frames in each GOP needs to be calculated. The target buffer size is set according to the encoding bit number of the first instant decoding update frame and the first P frame and the average image complexity of other already encoded B frames and P frames. After the first P frame in the ith GOP is encoded, the target buffer size is reset as:

[0058] S i (2)=V i (2),

[0059] Wherein, V i (2) is the actual occupation of the buffer after the first P frame in the ith GOP is encoded, and the target buffer size S i (j+1) of the subsequent P frame is calculated as follows:

[0060]

[0061] Wherein, N p (i) is the total number of P frames in the ith GOP, f is a predefined frame rate, R i (j) is the available bit rate when the frame is encoded, and L is the distance between adjacent P frames.

[0062] The average complexity factor of the P frame is The average complexity factor of the B frame is calculated by the following two formulas:

[0063]

[0064]

[0065] W p,i (j)=b p,i (j)×QP p,i (j)

[0066]

[0067] b p,i (j) and b b,i (j) are the actual bit numbers generated after encoding one P frame and one B frame respectively, QP p,i (j) is the quantization parameter of P frame, QP b,i (j) is the quantization parameter of B frame.

[0068] Then, the target bit number of each P frame is calculated, which is mainly determined by the following factors: target buffer occupancy, available bandwidth (target bit rate), actual buffer occupancy, and motion information; the final target bit rate is adjusted in combination with the available bandwidth, target buffer occupancy, and motion information. The calculation of the target bit number of the jth frame in the ith GOP is as follows:

[0069]

[0070] where γ is a constant, and its typical value is 0.5 in the absence of B frames, and 0 otherwise. In addition, when calculating the target bit number, the remaining bit number in the GOP should also be considered:

[0071]

[0072] where N p,r and N b,r are the numbers of P frames and B frames in the current GOP that have not yet been encoded, B i (j) represents the total bit number allocated to the remaining images in the GOP after encoding the jth frame in the ith GOP

[0073] The target bit number is the weighted sum of and multiplied by a motion change coefficient:

[0074]

[0075] where β is a constant, and its typical value is 0.5 in the absence of B frames, and 0.9 otherwise; the typical value of α is 0.01, and h(motion) is the motion change coefficient of the previous and next two frames.

[0076] Finally, the MAD (Mean Absolute Difference) value of the current frame (the average absolute value difference of the current basic unit) and the bit number of the encoding header information are predicted, the MAD prediction is realized according to the improved algorithm, and the spatial information and the temporal information are combined; the QP optimization is performed according to the MAD value and the rate-distortion model, and the final quantization parameter value is determined, as follows:

[0077] MAD of the current P frame is obtained by improving the algorithm MAD i (j), and then the quantization step corresponding to the target bit number is obtained by using the quadratic R-D model, i.e.:

[0078]

[0079] wherein m h,i (j) is the bit number occupied by the header information and the motion vector, c1 and c2 are model coefficients, QP step,i (j) is the quantization step, and the corresponding quantization parameter QP i (j) is obtained by QP step,i (j).

[0080] In one embodiment, the quantization parameter of the B frame is determined according to the average value of the quantization parameters of the adjacent P frames, comprising:

[0081] If the quantization parameters of the adjacent P frames are different, the quantization parameter of the B frame is the sum of the quantization parameters of the adjacent P frames and a second value divided by 2;

[0082] If the quantization parameters of the adjacent P frames are the same, the quantization parameter of the B frame is the sum of the quantization parameter of any adjacent P frame and a second value.

[0083] It should be noted that the second frame of each group of images is necessarily a P frame, not a B frame, and the second frame is encoded according to the initial QP. The third frame is first determined whether it is a B frame or a P frame, if it is a P frame, the corresponding QP of the P frame is calculated, if it is a B frame, it is not encoded first, and the P frame after encoding is encoded first.

[0084] Specifically, because the B frame is not used as a reference frame, its quantization parameter is generally greater than that of the adjacent P frame or I frame, thereby saving the bit number, so that the P frame or I frame can obtain more encoding bits, thereby obtaining higher image reconstruction quality; at the same time, in order to ensure the subjective continuity of the video quality, the difference between the quantization parameters of the adjacent two frames should not be greater than 2. The quantization parameter of the B frame can be calculated by simple linear interpolation: if there is only one B frame between the adjacent P frames, the quantization parameter of the B frame can be calculated by the average value of the quantization parameters of the two P frames;

[0085]

[0086] wherein QP i (j) is the quantization parameter of the previous frame of the frame to be encoded, QP i (j+1) is the quantization parameter of the frame to be encoded, and QP i (j+2) is the quantization parameter of the next frame of the frame to be encoded.

[0087] Assume the distance between adjacent P frames is L, that is, the number of B frames between adjacent P frames is L, and the quantization parameters of the two P frames are defined as QP i (j) and QP i (j+L+1), wherein QP i (j+L+1) is the quantization parameter of the P frame with a distance of L from QP i (j).

[0088] If there is more than one B frame between adjacent P frames, the quantization parameter of the B frame is dynamically adjusted according to the distance of the B frame from the front and rear P frames and the QP of the front and rear frames.

[0089]

[0090] k = 1, …, L, and a is given by the following formula:

[0091]

[0092] wherein QP i (j+k) is the quantization parameter of the P frame with a distance of more than one B frame from QP i (j).

[0093] In one embodiment, the quantization parameter of the first frame of the first group of images is determined according to the frame information of the first frame, and the quantization parameters of the remaining frames are determined according to the frame information and adjacent frame information of the remaining frames, comprising:

[0094] Obtaining the quantization parameter of the to-be-encoded frame;

[0095] Adjusting the quantization parameter of the to-be-encoded frame according to the quantization parameter of the previous encoded frame, wherein the distance between the previous encoded frame and the to-be-encoded frame is a first value.

[0096] The first value can be set as needed.

[0097] Specifically, in order to ensure the visual smoothness of subsequent video images, the following formula is used for adjustment:

[0098] QP i (j) = min{QP i (j-L-1) + 2, max{QP i (j-L-1) - 2, QP i (j)}, wherein QP i (j) is the quantization parameter of the to-be-encoded frame, QP i (j-L-1) is the quantization parameter of the previous encoded frame with a distance of L from the to-be-encoded frame. L is the first value. The final quantization parameter must be within the range of [0, 51] required by the H264 standard.

[0099] In one embodiment, the corresponding frame is encoded according to the quantization parameter of the first frame of the first group of images and the quantization parameter of the remaining frames, comprising:

[0100] The initial number of skipped frames is set to zero;

[0101] If the amount of data after encoding exceeds the set value of code rate, the number of skipped frames is gradually increased until the actual buffer occupancy meets the set condition.

[0102] Specifically, the main tasks of the encoding stage include three aspects: updating the parameters of the prediction model, updating the parameters of the secondary R-D model, and determining the number of skipped frames, and the number of bits actually encoded is added to the buffer.

[0103] In order to ensure that the updated buffer occupancy does not overflow, the number of skipped frames N skip is initially set to 0 and gradually increased until the following condition is met:

[0104] V(j+N skip )<V max ×0.8,

[0105] where N skip is the number of skipped frames, V max is the maximum buffer actual occupancy, and V(j+N skip ) is the buffer actual occupancy after skipping one frame.

[0106] Because there is no encoding operation (no bit number is generated), the occupancy of the buffer will be updated as follows every time one frame is skipped:

[0107]

[0108] where f is a predefined frame rate, V i (j+1) is the buffer actual occupancy before skipping, and V i (j+1+1) is the buffer actual occupancy after skipping one frame.

[0109] In one embodiment, the difference between the quantization parameters of two adjacent frames is less than or equal to a second value.

[0110] The second value can be 2, and the embodiments of the present application do not limit this.

[0111] In one embodiment, it further comprises:

[0112] obtaining a frame;

[0113] The quantization parameter of the first macroblock of the frame is determined according to the average of the quantization parameters of the macroblocks of the previous frame, and the quantization parameters of the remaining macroblocks are determined according to the macroblock information and the adjacent macroblock information;

[0114] encoding the corresponding macroblock according to the quantization parameter of the first macroblock and the quantization parameter of the remaining macroblocks.

[0115] The macroblock information can include a target bit number of the macroblock, and the adjacent macroblock information can include a quantization parameter of a previous macroblock and a motion change coefficient of the previous and next two macroblocks.

[0116] In one embodiment, the first macroblock of the frame is determined according to an average value of the quantization parameters of the macroblocks of a previous frame, and the remaining macroblocks are determined according to the macroblock information and the adjacent macroblock information, and the method comprises:

[0117] obtaining a macroblock to be encoded;

[0118] if the macroblock to be encoded is the first macroblock of the current frame, determining an average value of the quantization parameters of the macroblocks of a previous frame as the quantization parameter of the macroblock to be encoded;

[0119] encoding the macroblock to be encoded according to the quantization parameter of the macroblock to be encoded;

[0120] if the macroblock to be encoded is not the first macroblock of the current frame, determining a target bit number of the macroblock to be encoded according to a bit number occupied by a previous macroblock;

[0121] if the target bit number of the macroblock to be encoded is less than zero, determining a sum of the quantization parameter of the previous macroblock and a third value as the quantization parameter of the macroblock to be encoded;

[0122] if the target bit number of the macroblock to be encoded is greater than or equal to zero, estimating an encoding mode of the macroblock to be encoded according to the target bit number of the macroblock to be encoded;

[0123] determining a header information bit number of the macroblock to be encoded according to the estimated encoding mode of the macroblock to be encoded and a motion change coefficient of the previous and next two macroblocks;

[0124] determining an actual encoding mode of the macroblock to be encoded according to the header information bit number of the macroblock to be encoded;

[0125] calculating an actual average absolute difference value of the macroblock to be encoded according to the actual encoding mode of the macroblock to be encoded;

[0126] determining the quantization parameter of the macroblock to be encoded according to the actual average absolute difference value of the macroblock to be encoded, the target bit number of the macroblock to be encoded, the header information bit number of the macroblock to be encoded, and a bit number occupied by a motion vector.

[0127] The motion change coefficient of the previous and next two macroblocks refers to a motion change coefficient between the macroblock to be encoded and a previous macroblock.

[0128] It should be noted that after the quantization parameter of a macroblock is determined, the current macroblock can be encoded directly according to the obtained quantization parameter, or the quantization parameter of the current macroblock can be adjusted dynamically according to the previous macroblock, and then the current macroblock is encoded according to the adjusted quantization parameter. The embodiment of the present application does not limit comparison.

[0129] In a specific example, the code rate control of the base unit layer can realize more accurate control of the output bit rate. If the base unit is set as a frame, the layer control is ignored, and the QP value obtained by using the frame layer code rate control is used to encode all base units. If the base unit is set as a macroblock, the layer code rate control calculates a QP value for each macroblock. For low-delay and small-buffer applications, in order to avoid buffer overflow or underflow, the base unit layer code rate control is indispensable. The layer code rate control procedure is as follows: Step 1, check whether the current macroblock is the first macroblock of the current frame. If yes, calculate the average QP value of all macroblocks of the previous frame, and go to step 6. Step 2, update the remaining target bit number available for the current macroblock according to the actual number of bits generated by encoding the previous macroblock (simple subtraction operation). Check whether the remaining target bit number obtained in step 2 is less than 0. If yes, go to step 5. Step 3, calculate the value of the MAD (Mean Absolute Difference) of the current macroblock and the number of header bits using the improved algorithm, and then calculate the texture bit number allocated to the current macroblock. Step 4, calculate the QP value of the current macroblock using the quadratic rate-distortion model, and go to step 6. Step 5, encode the previous macroblock exceeding the bit quota allocated thereto, add △QP to the QP value of the previous macroblock as the final QP value of the current macroblock, and △QP is generally equal to 1. Step 6, encode the current macroblock using the QP value calculated in step 1, 4 or 5, perform RDO, and then record the actual MAD value, the number of header bits and the total consumed bit number (including texture bits) of the current macroblock. Step 7, update the parameters of the R-D model and the MAD prediction model using the linear regression method. Step 8, check whether the current macroblock is the last macroblock in the current frame. If yes, go to step 9, otherwise go to step 1. Step 9, end the base unit layer code rate control (the current frame encoding is completed), and update the buffer. If the buffer capacity is greater than a set threshold value, the next frame is skipped and not encoded.

[0130] Specifically, the code rate algorithm adopted by H.264 includes Group Of Pictures (GOP) layer code rate control, frame layer code rate control and basic unit layer code rate control, the size of the basic unit can be less than or equal to the size of a frame; less than a frame usually has several macroblocks. If the basic unit is equal to the size of a frame (i.e. a frame is divided into only one basic unit), the code rate control adjusts the quantization parameter of the frame based on the target bit number of the current frame. If the basic unit is composed of partial macroblocks, the quantization parameter of the macroblock is further adjusted according to the image complexity of the macroblock under the premise of the target bit number of the frame. At the same time, the quantization parameters of the front and rear two basic units cannot have too large a difference, the purpose being to ensure the visual smoothness. The front and rear two basic units can be the same frame or the front and rear frames, since the frame with higher encoding complexity will have more bit number after encoding, and conversely the frame with lower encoding complexity will have less bit number after encoding. In order to make the bit allocation as much as possible to conform to the actual encoding output code rate, a linear prediction model is used to predict the value of the Mean absolute difference (MAD) of the current basic unit. The MAD reflects the loss degree of the picture before and after encoding. The prior art relies on the MAD information of the previous basic unit for calculation, without fully considering the spatial domain information between the basic units within a frame, which will cause great prediction error when the scene changes or high-speed motion.

[0131] Specifically, first, the original encoding data is obtained from the video acquisition sequence, and then the encoding type of the first frame is judged. The encoding type mainly has three types of I, P and B frames. The I frame image can be independently encoded, the B frame image needs to be encoded by relying on the frames encoded before and after it, and usually the first frame of the first GOP is encoded as an I frame, and the first frame in the subsequent GOP can be I or P. Different frames need to be set with quantization parameters, and different quantization parameters will produce different compression effects. The original algorithm mainly processes the quantization parameter of the first frame in the GOP based on the resolution of the original data, calculates the bit number occupied by each pixel according to the size of the image, and then roughly estimates a quantization parameter QP value, which causes that the first frame image is too large or too small to code, which will greatly affect the quality or code rate allocation of the subsequent images. Usually, the code rate fluctuates too much, and the image display is sometimes blurred and sometimes clear. The original algorithm only considers the initial bpp to adjust the value of the initial QP, as follows:

[0132]

[0133] bpp = R i (1) / f×N pixel ,N pixelL1, L2, L3 are the number of pixels in a frame. For QCIF format video L1 = 0.1, L2 = 0.3, L3 = 0.6, for CIF format video L1 = 0.2, L2 = 0.6, L3 = 1.2, for higher resolution video format L1 = 0.6, L2 = 1.4, L3 = 2.4.

[0134] The main purpose of MAD prediction is to select the partition type and prediction mode used for encoding, different partition type and prediction mode will produce different image distortion, usually for the information-rich picture using smaller partition, for the information is relatively simple scene using larger partition. The original algorithm through the current frame and the previous frame corresponding position of the image prediction, through the linear model to carry out data prediction, for the frame with rich texture details or irregular local motion, linear prediction model can achieve good results, however, this kind of using time domain information prediction MAD way has obvious limitation, especially for the frame containing scene switching or high speed motion is more so.

[0135] The allocation of target bit number, mainly according to the target code rate and the code rate occupied by the frame has been encoded, to dynamically determine the code rate to be occupied by the frame to be encoded, the original algorithm in the calculation of the target code rate of the frame to be encoded uses the average value, first calculate the remaining bit number of the entire GOP, then the remaining bit number is evenly distributed to the unencoded frame. In this way, the quality of the image complex frame encoding will be very low, the image simple frame compression degree is not enough. As follows:

[0136] After the encoding of the jth frame in the ith GOP, the total bit number B allocated to the remaining images in the GOP i (j) is calculated as follows:

[0137]

[0138] Where, V i (j) is the bit number occupied by the virtual buffer after the encoding of the jth frame, j = 1 for the first frame of the GOP, R i (j) is the bit rate available when encoding the frame (channel capacity), f is the predefined frame rate, b i (j-1) is the actual number of bits generated when encoding the j-1th frame.

[0139] After the prediction mode and the target bit number are determined, the setting of the QP of the subsequent frame is performed. Different from the first frame QP, the setting of the subsequent frame QP depends on a quadratic rate-distortion model (R-D) reflecting the relationship between the code rate and the image distortion degree. The distortion degree reflects the display quality of the image, and the quality of the image is determined by the quantization parameter QP. Since there is a loss of data in the image coding process, the quantization parameter QP needs to be further adjusted in combination with the distortion degree and the code rate for subsequent coding. The basic unit layer code rate control operation is similar to the process of the frame layer code rate control. The difference is that the control acts on different partition blocks within the frame. When the coding of all the basic units of a frame is completed, the rate-distortion model coefficients need to be updated, so as to constantly adapt to the changing scene.

[0140] The embodiment of the present application is different from the setting of the initial QP of the first GOP in the prior art. Not only the bpp is considered, but also the image entropy coding information, the complexity of the image, and the specific scene motion change information are combined. The scene mainly includes the switching of different scenes such as weak light source and strong light source, motion and stillness, etc. Through the test of the coding conditions of different scenes, the approximate model of the complexity, bpp, entropy information, and scene transformation is fitted. The rate-distortion optimization is considered for the update of the initial QP. The instantaneous code rate, the complexity of the image itself, the scene transformation, etc. are considered for the allocation of the target bit number. For the selection of the prediction mode, the prediction accuracy is improved in combination with the time domain and space domain information, and the code rate control mode is dynamically adjusted. The code rate control mode includes constant code rate and variable code rate. The constant code rate refers to the guarantee of the invariable code rate, and the image quality can be lost. The variable code rate refers to the guarantee of the image quality, and the code rate can fluctuate within a certain range. The GOP layer code rate control mainly calculates the total bit number allocated to each GOP frame which has not been coded and the initial quantization parameter of the IDR frame and the I frame. The GOP structure can be assumed as IPPP…P or IBPBP, which represents the intra prediction frame, the forward prediction frame, and the bidirectional prediction frame. The calculation of the GOP layer code rate control is as follows:

[0141] In another specific example, as Figure 1aIn the flowchart shown, the first frame input data is read, it is judged whether the first frame is an I frame or a P frame, if it is an I frame, the pixel depth bpp, entropy information and content complexity of the I frame are calculated according to the first frame input data, if it is the first frame of the first image group, the quantization parameter is determined according to the pixel depth, entropy information, complexity and frame type of the first frame; if it is the first frame of the second image group, the quantization parameter is determined according to the pixel depth, entropy information, complexity, frame type and image group motion change coefficient of the first frame. The image group motion change coefficient refers to the motion change coefficient of the last frame of the first image group and the first frame of the second image group. The initial QP is calculated. The obtained QP is used for encoding, the QP is fine-tuned according to the encoding result, and the encoding is performed again; wherein the entropy information and content complexity are calculated as follows:

[0142]

[0143]

[0144] Wherein, N is the number of bits for encoding a single pixel, p(x) is the probability of the luminance value x, x is in the range of [0, 255], H y is the height of the luminance component of a frame of image, W y is the width, Y(h, w) is the pixel luminance value at coordinate (h, w), and Avg(Y) is the average luminance of a frame; similarly, H cb and H cr are the height of the chroma component of a frame of image, W cb and W cr are the width of the chroma component of a frame of image, and other parameter functions are similar to the Y component. The relationship between the initial QP and bpp, entropy, complexity and motion is obtained by fitting method, which is as follows:

[0145] QP init = f(bpp) + g(entrop, complexity) + h(motion),

[0146]

[0147] Wherein, bpp indicates that the initial QP is related to bpp, g(entrop, complexity) indicates that the initial QP is related to the content complexity of the I frame, and the values of corresponding parameters are as follows:

[0148] ε = {A, B, C, D, M, N, U, S1, S2, S3, T1, T2, T3}

[0149] = {-12.04, 23.16, -25.68, 28.51, 490.62, -217.49, 47.32, 14.67, -190.11, 600.72, 0.67, 7.93, 16.85}

[0150] The motion information reflects the change between the two frames, and usually includes motion to still, still to motion, and other scene changes. At this time, appropriate adjustment of QP can reduce the fluctuation of the code rate, as follows:

[0151]

[0152] In the above, h(motion) corresponds to different values, as shown in Table 1 below:

[0153] Table 1

[0154] Value of h(motion) Meaning -2 Severe motion change from previous to following frame -1 Minor motion change from previous to following frame 0 No motion change from previous to following frame 1 Minor motion change from following to previous frame 2 Severe motion change from following to previous frame

[0155] It should be noted that the actual number of bits occupied by the encoded frames in the GOP is counted, and the total target number of bits allocated to the entire GOP is combined with the code rate to calculate the remaining number of bits to be allocated. According to the obtained QP, the first I frame is encoded, and the actual number of bits obtained by encoding is compared with the target number of bits. If the fluctuation amplitude is not more than 5% of the target code rate, the algorithm terminates. Otherwise, further fine-tuning of QP is performed. If the actual code rate is low, the QP setting is reduced. If the actual code rate is high, the QP setting is increased. The amplitude of QP reduction can be dynamically adjusted according to the motion change. The control algorithm is within 3 times. Encoding is performed according to the adjusted QP. After the first frame is encoded, the type of the second frame is determined. If it is a P frame, the QP of the second frame is the same as that of the first frame, and the second frame is encoded according to the QP of the first frame. After the second frame is encoded, the type of the third frame is determined. If it is a P frame, frame-level MAD prediction is performed according to the improved algorithm, target bit allocation is performed according to the improved algorithm, and header information bit number is predicted according to the improved algorithm. The frame-level QP is calculated using the secondary R-D model, and the frame-level secondary rate-distortion model coefficients are updated. After the QP is calculated, encoding is performed according to the QP. This process continues until the last frame is encoded. If the basic unit layer is set to one macroblock, the QP of the macroblock is calculated.

[0156] Specifically, the data before and after optimization are compared in combination with test scenarios, mainly considering two factors: image quality and code rate fluctuation, wherein the image quality has average signal-to-noise ratio (PSNR) statistics, the PC processor is I5 2GHz, the memory is 2GB, the input original data sequence is encoded frame by frame without frame skipping, the encoding sequence is IPPP format (without using B frame), the rate-distortion optimization is turned on, the frame rate is 30 frames / second, the channel bandwidth (day standard bit rate) is set to 64Kbits / s (QCIF), 600Kbits / s (CIF), the number of encoded frames is 100, and the basic unit is set to 11 macroblocks. The specific experimental results are shown in Table 2:

[0157] Table 2

[0158]

[0159] As can be seen from the experimental results in the above table, for the test sequences such as Akiyo and News, which have relatively mild l-frame complexity, the optimized algorithm uses similar or lower bit rate to achieve obvious improvement in image reconstruction quality (measured by PSNR Avg); for sequences such as Carphone and Mobile, which have intense motion or complex image content and color levels, the optimized algorithm achieves more accurate code rate control (measured by Kbits / s) while losing little image reconstruction quality; for other sequences, the optimized algorithm has improved in both image reconstruction quality and code rate control accuracy.

[0160] The embodiment of the application can also run an intermediate layer through a central processor, responsible for resource scheduling of different hardware modules; collect original video sources through a graphics collector; the graphics collector and perform operations such as format conversion, scaling, complexity statistics and motion statistics on the collected data; compress and encode the original data through a graphics encoder; transmit the encoded data to a receiving end through a network; or can transmit the encoded data to the receiving end through a USB line.

[0161] The application provides an image encoding device, Figure 2 A structural schematic diagram of an image encoding device provided by the application is shown in the figure. Figure 2 As shown in the figure, the image encoding device in the embodiment of the application can be integrated on a device. The device comprises: a first acquisition module 21, configured to acquire at least one image group; a first determination module 22, configured to determine a quantization parameter of a first frame of a first image group according to frame information of the first frame, and determine quantization parameters of remaining frames according to frame information and adjacent frame information of the remaining frames, wherein the remaining frames are frames other than the first frame of the first image group in the at least one image group; and a first encoding module 23, configured to encode corresponding frames according to the quantization parameter of the first frame of the first image group and the quantization parameters of the remaining frames.

[0162] The device provided by the embodiment is used for implementing the method as shown in Figure 1 The device provided by the embodiment implements principles and technical effects similar to the method of the embodiment as shown in Figure 1 The device provided by the embodiment implements principles and technical effects similar to the method of the embodiment as shown in

[0163] On the basis of the above-mentioned embodiment, a variant embodiment of the above-mentioned embodiment is proposed. It should be noted that, in order to make the description brief, only the differences from the above-mentioned embodiment are described in the variant embodiment.

[0164] In one embodiment, the frame information includes complexity, remaining bit number and frame type, or complexity, frame type, pixel depth and entropy information.

[0165] In one embodiment, the adjacent frame information includes average complexity of the coded frame, bit number occupied by the coded frame, actual buffer occupation, available bandwidth, target buffer frame rate, remaining bit number and motion change coefficient of the two adjacent frames.

[0166] In one embodiment, the first determining module 22 is specifically configured to:

[0167] determine the quantization parameter of the first frame of the first group of images according to complexity, frame type, pixel depth and entropy information of the first frame of the first group of images;

[0168] determine the quantization parameter of the first frame of the remaining group of images according to complexity, frame type, pixel depth, entropy information of the first frame of the remaining group of images and motion change coefficient of the two adjacent frames;

[0169] of the remaining frames of the group of images, the quantization parameter of the first type P frame is the same as that of the adjacent first frame, wherein the first type P frame is the second frame of the group of images;

[0170] of the remaining frames of the group of images, the quantization parameter of the second type P frame is determined according to average complexity of the coded frame, bit number occupied by the coded frame, actual buffer occupation, available bandwidth, target buffer frame rate, remaining bit number and motion change coefficient of the two adjacent frames, wherein the second type P frame is not the second frame of the group of images;

[0171] of the remaining frames of the group of images, the quantization parameter of the B frame is determined according to the average value of the quantization parameters of the adjacent P frames.

[0172] In one embodiment, the first determining module 22 is specifically configured to:

[0173] obtain a to-be-encoded frame;

[0174] If the frame to be encoded is the first frame of the first GOP, the quantization parameter of the first frame of the first GOP is determined according to the complexity, the frame type, the pixel depth and the entropy information;

[0175] If the frame to be encoded is the first frame of the remaining GOP, the quantization parameter of the first frame of the remaining GOP is determined according to the complexity, the frame type, the pixel depth, the entropy information and the motion change coefficient of the previous and next frames;

[0176] If the frame to be encoded is the first type P frame, the quantization parameter of the first type P frame is determined according to the quantization parameter of the adjacent first frame, wherein the first type P frame is the second frame of the GOP;

[0177] If the frame to be encoded is the second type P frame, the quantization parameter of the second type P frame is determined according to the average complexity of the encoded frames, the number of bits occupied by the encoded frames, the actual buffer occupancy, the available bandwidth, the target buffer occupancy, the remaining number of bits and the motion change coefficient of the previous and next frames, wherein the second type P frame is not the second frame of the GOP;

[0178] If the frame to be encoded is a B frame, the B frame is not encoded, and a P frame adjacent to the B frame and after the B frame is encoded first, and then the quantization parameter of the B frame is determined according to the average value of the quantization parameters of the adjacent P frames.

[0179] In one embodiment, the first determining module 22 is specifically configured to:

[0180] determine the target buffer occupancy of the second type P frame according to the average complexity of the encoded frames and the number of bits occupied by the encoded frames;

[0181] determine the target number of bits of the second type P frame according to the target buffer occupancy, the actual buffer occupancy, the available bandwidth, the target buffer occupancy, the remaining number of bits and the motion change coefficient of the previous and next frames;

[0182] estimate the encoding mode of the second type P frame according to the target number of bits of the second type P frame;

[0183] determine the number of bits of the header information of the second type P frame according to the estimated encoding mode of the second type P frame and the motion change coefficient of the previous and next frames;

[0184] determine the actual encoding mode of the second type P frame according to the number of bits of the header information of the second type P frame;

[0185] calculate the actual average absolute difference value of the second type P frame according to the actual encoding mode of the second type P frame;

[0186] The quantization parameter of the second type P frame is determined according to an actual average absolute value difference of the second type P frame, a target bit number of the second type P frame, a head information bit number of the second type P frame, and a bit number occupied by a motion vector.

[0187] In one embodiment, the first determining module 22 is specifically configured to:

[0188] If the quantization parameters of the adjacent P frames are different, the quantization parameter of the B frame is a sum of the quantization parameters of the adjacent P frames and a second value divided by 2.

[0189] If the quantization parameters of the adjacent P frames are the same, the quantization parameter of the B frame is a sum of the quantization parameter of any adjacent P frame and a second value.

[0190] In one embodiment, the first determining module 22 is specifically configured to:

[0191] The quantization parameter of the to-be-encoded frame is obtained.

[0192] The quantization parameter of the to-be-encoded frame is adjusted according to the quantization parameter of the previous encoded frame, wherein the previous encoded frame and the to-be-encoded frame are spaced by a first value.

[0193] In one embodiment, the first encoding module 23 is specifically configured to:

[0194] The initial number of frame skips is set to zero.

[0195] If the amount of data after encoding exceeds the code rate setting value, the number of frame skips is gradually increased until the actual buffer occupancy meets the setting condition.

[0196] In one embodiment, the difference between the quantization parameters of the adjacent two frames is less than or equal to a second value.

[0197] In one embodiment, the method further comprises:

[0198] The second obtaining module is configured to obtain a frame.

[0199] The second determining module is configured to determine the quantization parameter of a first macroblock of the frame according to an average value of the quantization parameters of the macroblocks of the previous frame, and determine the quantization parameters of the remaining macroblocks according to the macroblock information and the adjacent macroblock information.

[0200] The second encoding module is configured to encode the corresponding macroblocks according to the quantization parameter of the first macroblock and the quantization parameters of the remaining macroblocks.

[0201] In one embodiment, the second determining module is specifically configured to:

[0202] The to-be-encoded macroblock is obtained.

[0203] if the macroblock to be encoded is the first macroblock of the current frame, determining an average of quantization parameters of macroblocks of a previous frame as the quantization parameter of the macroblock to be encoded;

[0204] encoding the macroblock to be encoded according to the quantization parameter of the macroblock to be encoded;

[0205] if the macroblock to be encoded is not the first macroblock of the current frame, determining a target bit number of the macroblock to be encoded according to a bit number occupied by a previous macroblock;

[0206] if the target bit number of the macroblock to be encoded is less than zero, determining a sum of the quantization parameter of the previous macroblock and a third value as the quantization parameter of the macroblock to be encoded;

[0207] if the target bit number of the macroblock to be encoded is greater than or equal to zero, estimating an encoding mode of the macroblock to be encoded according to the target bit number of the macroblock to be encoded;

[0208] determining a header information bit number of the macroblock to be encoded according to the estimated encoding mode of the macroblock to be encoded and motion variation coefficients of two macroblocks before and after the macroblock to be encoded;

[0209] determining an actual encoding mode of the macroblock to be encoded according to the header information bit number of the macroblock to be encoded;

[0210] calculating an actual average absolute difference value of the macroblock to be encoded according to the actual encoding mode of the macroblock to be encoded;

[0211] determining the quantization parameter of the macroblock to be encoded according to the actual average absolute difference value of the macroblock to be encoded, the target bit number of the macroblock to be encoded, the header information bit number of the macroblock to be encoded and a bit number occupied by a motion vector.

[0212] The image encoding method provided in the application comprises the following steps: acquiring at least one image group; determining a quantization parameter of a first frame of a first image group according to frame information of the first frame, and determining quantization parameters of the rest frames according to frame information and adjacent frame information of the rest frames, wherein the rest frames are frames other than the first frame of the first image group in the at least one image group; and encoding corresponding frames according to the quantization parameter of the first frame of the first image group and the quantization parameters of the rest frames. The application solves the technical problem that, in the prior art, due to improper selection of quantization parameters, the buffer is excessively occupied (far more than the target bit number pre-allocated), at this time, the encoder will try to reduce the target bit number allocated to subsequent frames to ensure that the buffer does not overflow, thereby causing the image quality of the subsequent frames to rapidly decrease (because the number of bits available for encoding is too small), and even causing the technical problem of frame skipping, and the application can reduce the overall fluctuation of the encoded images and thus improve the quality of graphic display, and ultimately improve the user experience.

[0213] The application provides a device, Figure 3A structural diagram of an apparatus provided in the present application is shown in Figure 3 The apparatus provided in the present application includes one or more processors 31 and storage devices 32. The processor 31 in the apparatus can be one or more processors 31, Figure 3 The storage device 32 is configured to store one or more programs. The one or more programs are executed by the one or more processors 31, so that the one or more processors 31 implement the method as described in the embodiments of the present application. Figure 1

[0214] The apparatus further includes a communication device 33, an input device 34 and an output device 35.

[0215] The processor 31, the storage device 32, the communication device 33, the input device 34 and the output device 35 in the apparatus can be connected by a bus or other means, Figure 3 for example, by a bus.

[0216] The input device 34 can be configured to receive input digital or character information and generate key signal input related to user settings and function control of the apparatus. The output device 35 can include a display device such as a display screen.

[0217] The communication device 33 can include a receiver and a transmitter. The communication device 33 is configured to perform information receiving and transmitting communication according to the control of the processor 31. The information includes but is not limited to uplink authorization information.

[0218] The storage device 32 as a computer readable storage medium can be configured to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the image encoding method as described in the embodiments of the present application. Figure 1 The storage device 32 can include a program storage area and a data storage area. The program storage area can store an operating system and application programs required by at least one function; the data storage area can store data created according to the use of the apparatus, etc. In addition, the storage device 32 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state memory device. In some examples, the storage device 32 can further include a storage memory remotely arranged with respect to the processor 31, which can be connected to the apparatus through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0219] ​The embodiment of the present application further provides a storage medium which stores a computer program, and the computer program is executed by a processor to implement the image coding method in any of the embodiments of the present application. The image coding method applied to a device comprises: acquiring at least one image group; determining a quantization parameter of a first frame of a first image group according to frame information of the first frame, and determining quantization parameters of remaining frames according to frame information and adjacent frame information of the remaining frames, wherein the remaining frames are frames of the at least one image group except the first frame of the first image group; and encoding corresponding frames according to the quantization parameter of the first frame of the first image group and the quantization parameters of the remaining frames.

[0220] The computer storage medium of the embodiment of the present application can adopt any combination of one or more computer readable media. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium may, for example, be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. More specific examples (non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM), a flash memory, an optical fiber, a portable CD-ROM, an optical storage device, a magnetic storage device, or any suitable combination of the above. The computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device.

[0221] The computer readable signal medium can include a data signal propagated in a baseband or as a part of a carrier wave, in which a computer readable program code is borne. Such a propagated data signal can take multiple forms, including but not limited to, an electromagnetic signal, an optical signal, or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that is not a storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device.

[0222] The program code contained in the computer readable medium can be transmitted by any suitable medium, including but not limited to wireless, wire, cable, radio frequency (RF), or the like, or any suitable combination thereof.

[0223] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0224] The specific embodiments described hereinabove are illustrative of specific embodiments of the present application and are not meant to be limiting of the scope of the application.

[0225] Those skilled in the art will appreciate that the term device encompasses any suitable type of user device, such as a set-top box, a smartphone, a tablet, a navigation device, a car TV, a personal digital assistant, etc.

[0226] In general, the various embodiments of the application can be implemented in hardware or special purpose circuits, software, logic or any combination thereof. For example, some aspects can be implemented in hardware, while other aspects can be implemented in

[0227] Embodiments of the application can be implemented by computer program instructions executed by a data processor of a mobile device, for example in a processor entity, or by hardware, or by a combination of software and hardware. The computer program instructions can be in the form of assembly instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or in any combination of one or more programming languages, executed on one or more computers.

[0228] The block diagrams of any logical flow of the present application in the accompanying drawings can represent program steps or can represent interconnected logic circuits, modules, and functions, or can represent a combination of program steps and logic circuits, modules, and functions. The computer program can be stored on a memory. The memory can have any type suitable for the local technical environment and can be implemented using any suitable data storage technology, such as, but not limited to, a Read-Only Memory (ROM), a Random Access Memory (RAM), an optical storage device and system, a compact disc (CD) or a digital versatile disc (DVD), and the like. The computer readable medium can include a non-transitory storage medium. The data processor can be of any type suitable for the local technical environment, and can include, but is not limited to, a general purpose computer, a special purpose computer, a microprocessor, a Digital Signal Processing (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FGPA) and processors based on a multi-core processor architecture, as non-limiting examples.

[0229] A detailed description of exemplary embodiments of the present application has been provided above with reference to the accompanying drawings. However, various modifications and changes can be made to the above embodiments by those skilled in the art without departing from the scope of the present application, which is defined by the appended claims. Accordingly, the proper scope of the present application is determined by the appended claims.

Claims

1. An image coding method, characterized in that: include: obtaining at least one image group; determining a quantization parameter of a first frame of the first image group according to frame information of the first frame of the first image group, and determining quantization parameters of the remaining frames according to frame information of the remaining frames and information of adjacent frames, wherein the remaining frames are frames of the at least one image group other than the first frame of the first image group; encoding corresponding frames according to the quantization parameter of the first frame of the first group of pictures and the quantization parameters of the remaining frames; The determining of the quantization parameter of the first frame of the first image group according to the frame information of the first frame of the first image group, and determining the quantization parameters of the remaining frames according to the frame information of the remaining frames and the adjacent frame information, includes: Obtaining the quantization parameter of the frame to be encoded; The quantization parameter of the frame to be encoded is adjusted according to the quantization parameter of a previous encoded frame, wherein a distance between the previous encoded frame and the frame to be encoded is a first value.

2. The method according to claim 1, characterized in that The frame information includes: complexity, number of remaining bits and frame type; or complexity, frame type, pixel depth and entropy information.

3. The method according to claim 2, characterized in that The adjacent frame information includes: average complexity of the encoded frame, number of bits occupied by the encoded frame, actual buffer occupancy, available bandwidth, target buffer frame rate, number of remaining bits, and motion change coefficients of the previous and next frames.

4. The method according to claim 3, characterized in that Determining a quantization parameter of a first frame of the first image group according to frame information of the first frame of the first image group, and determining quantization parameters of the remaining frames according to frame information of the remaining frames and information of adjacent frames, including: determining a quantization parameter of a first frame of the first image group according to complexity, frame type, pixel depth, and entropy information of the first frame of the first image group; Determine the quantization parameters of the first frames of the remaining image groups according to the complexity, frame type, pixel depth, entropy information of the first frames of the remaining image groups and the motion change coefficients of the two preceding and following frames; Among the remaining frames of the image group, a quantization parameter of a first-type P frame is the same as a quantization parameter of an adjacent first frame, wherein the first-type P frame is the second frame of the image group; determining a quantization parameter for a second-category P frame among the remaining frames of the group of pictures according to an average complexity of coded frames, a number of bits occupied by coded frames, an actual buffer occupancy, available bandwidth, a target buffer frame rate, a number of remaining bits, and a motion variation coefficient of two preceding and following frames, wherein the second-category P frame is not the second frame of the group of pictures; The quantization parameters of the B frames in the remaining frames of the image group are determined according to the average value of the quantization parameters of the adjacent P frames.

5. The method according to claim 3, characterized in that Determining the quantization parameter of the first frame of the first image group according to the frame information of the first frame of the first image group, and determining the quantization parameters of the remaining frames according to the frame information of the remaining frames and the adjacent frame information includes: Get the frame to be encoded; If the frame to be encoded is the first frame of the first picture group, determining a quantization parameter of the first frame of the first picture group according to the complexity, frame type, pixel depth and entropy information; If the frame to be encoded is the first frame of the remaining image groups, determining the quantization parameters of the first frame of the remaining image groups according to the complexity, frame type, pixel depth, entropy information, and motion change coefficients of the two preceding and following frames; If the frame to be encoded is a first-type P frame, determining a quantization parameter of the first-type P frame according to a quantization parameter of an adjacent first frame, wherein the first-type P frame is the second frame of the picture group; If the frame to be encoded is a second-category P frame, determining a quantization parameter of the second-category P frame according to an average complexity of encoded frames, a number of bits occupied by encoded frames, an actual buffer occupancy, available bandwidth, a target buffer frame rate, a number of remaining bits, and motion variation coefficients of two preceding and following frames, wherein the second-category P frame is not the second frame of the image group; If the frame to be encoded is a B frame, encoding is not performed. A P frame adjacent to and following the B frame is encoded first. After the P frame is encoded, the quantization parameter of the B frame is determined based on the average value of the quantization parameters of adjacent P frames.

6. The method according to claim 5, characterized in that If the frame to be encoded is a second-category P frame, a quantization parameter of the second-category P frame is determined based on the average complexity of the encoded frames, the number of bits occupied by the encoded frames, the actual buffer occupancy, the available bandwidth, the target buffer frame rate, the number of remaining bits, and the motion change coefficients of the previous and next frames, wherein the second-category P frame is not the second frame of the image group, and includes: determining a target buffer occupancy of the second type of P frames based on an average complexity of the coded frames and a number of bits occupied by the coded frames; Determine a target number of bits for the second type of P frame based on the target buffer occupancy, the actual buffer occupancy, the available bandwidth, the target buffer frame rate, the remaining number of bits, and the motion variation coefficient of the previous and next frames; estimating an encoding method for the second type of P frames according to a target number of bits of the second type of P frames; Determining the number of header information bits of the second-category P frame according to the estimated encoding mode of the second-category P frame and the motion change coefficients of the two preceding and following frames; determining an actual encoding method of the second type of P frame according to the number of bits of header information of the second type of P frame; The actual mean absolute value difference of the second type of P frames is calculated according to the actual encoding method of the second type of P frames; The quantization parameter of the second type P frame is determined according to the actual mean absolute value difference of the second type P frame, the target number of bits of the second type P frame, the number of header information bits of the second type P frame and the number of bits occupied by the motion vector.

7. The method according to claim 5, characterized in that The quantization parameter of the B frame is determined based on the average value of the quantization parameters of adjacent P frames, including: If the quantization parameters of adjacent P frames are different, the quantization parameter of the B frame is the sum of the quantization parameters of the adjacent P frames and the second value divided by 2; If the quantization parameters of adjacent P frames are the same, the quantization parameter of the B frame is the sum of the quantization parameter of any adjacent P frame and the second value.

8. The method according to claim 1, characterized in that Encoding corresponding frames according to the quantization parameter of the first frame of the first group of pictures and the quantization parameters of the remaining frames, comprising: The initial number of skipped frames is set to zero; If the amount of data after encoding exceeds the set value of the bit rate, the number of frame skipping is gradually increased until the actual buffer occupancy meets the set condition.

9. The method according to claim 1, characterized in that The difference between the quantization parameters of two adjacent frames is less than or equal to the second value.

10. The method according to claim 1, characterized in that Also includes: Get frame; The quantization parameter of the first macroblock of the frame is determined according to the average value of the quantization parameters of the macroblocks of the previous frame, and the quantization parameters of the remaining macroblocks are determined according to the macroblock information and the information of the adjacent macroblocks; The corresponding macroblocks are encoded according to the quantization parameter of the first macroblock and the quantization parameters of the remaining macroblocks.

11. The method according to claim 10, characterized in that The first macroblock of the frame determines a quantization parameter of the first macroblock according to an average value of quantization parameters of macroblocks of the previous frame, and the remaining macroblocks determine quantization parameters of the remaining macroblocks according to macroblock information and adjacent macroblock information, including: Get the macroblock to be encoded; If the macroblock to be encoded is the first macroblock of the current frame, an average value of the quantization parameters of the macroblocks of the previous frame is determined as the quantization parameter of the macroblock to be encoded; Encoding the macroblock to be encoded according to the quantization parameter of the macroblock to be encoded; If the macroblock to be encoded is not the first macroblock of the current frame, determining the target number of bits for the macroblock to be encoded according to the number of bits occupied by the previous macroblock; If the target number of bits of the macroblock to be encoded is less than zero, determining the sum of the quantization parameter of the previous macroblock and the third value as the quantization parameter of the macroblock to be encoded; If the target number of bits of the macroblock to be encoded is greater than or equal to zero, estimating the encoding mode of the macroblock to be encoded according to the target number of bits of the macroblock to be encoded; Determining the number of header information bits of the macroblock to be encoded according to the estimated encoding mode of the macroblock to be encoded and the motion change coefficients of the two preceding and following macroblocks; Determining an actual encoding method for the macroblock to be encoded according to the number of bits of header information of the macroblock to be encoded; Calculate the actual mean absolute value difference of the macroblock to be encoded according to the actual encoding method of the macroblock to be encoded; The quantization parameter of the macroblock to be encoded is determined according to the actual mean absolute value difference of the macroblock to be encoded, the target number of bits of the macroblock to be encoded, the number of header information bits of the macroblock to be encoded and the number of bits occupied by the motion vector.

12. An image coding device, characterized in that include: A first acquisition module, configured to acquire at least one image group; a first determining module, configured to determine a quantization parameter of a first frame of the first image group based on frame information of the first frame of the first image group, and determine quantization parameters of the remaining frames based on frame information of the remaining frames and information of adjacent frames, wherein the remaining frames are frames of the at least one image group other than the first frame of the first image group; A first encoding module, configured to encode corresponding frames according to the quantization parameter of the first frame of the first image group and the quantization parameters of the remaining frames; The first determination module is specifically configured to: obtain a quantization parameter of a frame to be encoded; and adjust the quantization parameter of the frame to be encoded according to a quantization parameter of a previous encoded frame, wherein a distance between the previous encoded frame and the frame to be encoded is a first value.

13. A device, characterized in that include: one or more processors; a storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the information determination method according to any one of claims 1 to 11.

14. A storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the information determination method according to any one of claims 1 to 11 is implemented.

Images