Coding processing method, device, equipment and readable storage medium

Abstract

The present invention provides a coding processing method, apparatus, device, and readable storage medium, relating to the field of communication technology. The method includes: determining a first frame image, where the first frame image is the current image frame to be encoded in the video to be encoded; obtaining the complexity of the first frame image, where the complexity is used to characterize the difference between the first frame image and a second frame image, and is determined based on the grayscale histogram difference, where the second frame image is the previous frame image of the first frame image; and determining a target number of bits for the first frame image in a sliding window based on the complexity of the first frame image. The solution of the present invention solves the problem that existing coding processing consumes a large number of bits and is prone to bit loss.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a coding processing method, apparatus, device and readable storage medium. Background Art

[0002] In actual applications, the bandwidth of network transmission is often limited. Therefore, in video-related services, it is often necessary to control the bit rate of the video after compression to reasonably utilize the network bandwidth.

[0003] For screen content videos with large-scale and high-frequency scene changes, during the scene switching process, due to the lack of reference in encoding of the sudden frame content, encoding its intra-frame mode through traditional encoding schemes consumes a large number of bits and easily causes bit loss. Summary of the Invention

[0004] The purpose of the present invention is to provide a coding processing method, apparatus, device and readable storage medium to solve the problem that existing coding processing consumes a large number of bits and easily causes bit loss of control.

[0005] In order to solve the above problems, the present invention is achieved as follows:

[0006] In a first aspect, an embodiment of the present invention provides an encoding processing method, including:

[0007] Determine a first frame of image, where the first frame of image is a current image frame to be encoded in the video to be encoded;

[0008] Obtaining a complexity of the first frame of image, where the complexity is determined based on a grayscale histogram difference between the first frame of image and a second frame of image, where the second frame of image is a previous frame of image of the first frame of image;

[0009] A target number of bits of the first frame of image in a sliding window is determined according to the complexity of the first frame of image.

[0010] In a second aspect, an embodiment of the present invention provides a coding processing device, comprising

[0011] A first processing module is configured to determine a first frame of image, where the first frame of image is a current image frame to be encoded in the video to be encoded;

[0012] an acquisition module, configured to acquire the complexity of the first frame image, where the complexity is used to characterize the difference between the first frame image and a second frame image, where the second frame image is a frame image preceding the first frame image;

[0013] The second processing module is used to determine a target number of bits of the first frame image in the sliding window according to the complexity of the first frame image.

[0014] In a third aspect, an embodiment of the present invention provides an encoding device, comprising: a processor, a memory, and a program or instruction stored in the memory and executable on the processor; when the processor executes the program or instruction, the encoding processing method as described above is implemented.

[0015] In a fourth aspect, an embodiment of the present invention provides a readable storage medium having a program or instruction stored thereon, which implements the steps in the encoding processing method described above when the program or instruction is executed by a processor.

[0016] The beneficial effects of the above technical solution of the present invention are as follows:

[0017] In an embodiment of the present invention, the complexity of the first frame of image, which is the image frame currently to be encoded in the video to be encoded, is obtained, and then the target number of bits of the first frame of image in the sliding window is further determined based on the complexity. Since the complexity can characterize the difference between the first frame of image and the previous frame of image, it is more suitable for the characteristics of screen content video and can achieve refined management of bit allocation in the sliding window without consuming more resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flowchart of a coding method according to an embodiment of the present invention;

[0019] Figure 2 This is a second flowchart of the encoding processing method according to an embodiment of the present invention;

[0020] Figure 3 Schematic diagram of the application of the encoding processing method according to an embodiment of the present invention

[0021] Figure 4 is a structural diagram of a coding processing device according to an embodiment of the present invention;

[0022] Figure 5 This is a structural diagram of an encoding device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0024] It should be understood that references throughout this specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic associated with the embodiment is included in at least one embodiment of the present invention. Therefore, the appearances of "in one embodiment" or "in an embodiment" throughout this specification do not necessarily refer to the same embodiment. Furthermore, these particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0025] In various embodiments of the present invention, it should be understood that the size of the serial numbers of the following processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0026] Additionally, the terms "system" and "network" are often used interchangeably herein.

[0027] In the embodiments provided herein, it should be understood that "B corresponding to A" means that B is associated with A and B can be determined based on A. However, it should also be understood that determining B based on A does not mean determining B based solely on A; B can also be determined based on A and / or other information.

[0028] like Figure 1 As shown, an encoding processing method according to an embodiment of the present invention includes:

[0029] Step 101: determining a first frame of image, where the first frame of image is a current image frame to be encoded in a video to be encoded;

[0030] Step 102: Obtain the complexity of the first frame of image, where the complexity is used to characterize the difference between the first frame of image and a second frame of image, where the second frame of image is a frame of image preceding the first frame of image.

[0031] Step 103: Determine a target number of bits of the first frame of image in a sliding window according to the complexity of the first frame of image.

[0032] The method of the embodiment of the present invention, according to the above steps 101-103, will obtain the complexity of the first frame image by determining the image frame currently to be encoded in the video to be encoded, that is, the first frame image, and then further determine the target number of bits of the first frame image in the sliding window based on the complexity. Since the complexity can characterize the difference between the first frame image and the previous frame image of the first frame image, it is more suitable for the characteristics of screen content video, and can achieve refined management of bit allocation in the sliding window without consuming more resources.

[0033] Optionally, step 101 includes:

[0034] According to the sliding window length, the first frame image in the current sliding window is used as the first frame image; wherein,

[0035] The sliding window length is a minimum value between a preset length and the number of uncoded frames in the video to be coded.

[0036] Here, the sliding window length is the minimum value between the preset length and the number of uncoded frames in the video to be coded, that is, the sliding window length N SW It can be expressed as N SW =min(SW,Pic left ), SW indicates preset length, Pic left Indicates the number of uncoded frames in the video to be coded. SW can be set to 8. In this way, during the video coding process, the first frame image in the sliding window will be determined as the first frame image for subsequent processing.

[0037] Assume that the sliding window length is N SW =8, then the sliding window length is the length of 8 image frames. At this time, the sequence numbers of the image frames corresponding to the sliding window (arranged in chronological order) are 1 to 8, and the first frame image in the sliding window is image frame 1, which is the first frame image.

[0038] Of course, as the sliding window slides, the image frames in the video to be encoded can be sequentially processed as the first frame of image until the encoding of the video to be encoded is completed.

[0039] In this embodiment, the sliding window also sets a corresponding number of bits, which can also be understood as the number of bits in the sliding window. In the initialization phase, the number of bits corresponding to the sliding window is T SW It is through the formula T SW =(R tar ×N SW ) / F calculated, R tar is the target bit rate, and F is the frame rate of the video to be encoded. tar It is pre-set or defined. After the first frame image is encoded, T SW Through formula T SW =T SWleft +R left / F left To update, T SWleft R is the number of bits remaining in the sliding window after the first frame of the image is encoded. left is the number of bits remaining in the video to be encoded, F left is the number of frames remaining to be encoded in the video to be encoded. leftThe acquisition of depends on the target number of bits and the actual number of bits consumed. In this way, the number of bits in the sliding window is cyclically updated, and the first frame image is always located at the starting position of the sliding window.

[0040] In this embodiment, the complexity can be measured using inter-frame luminance difference, DCT absolute sum luminance histogram, grayscale histogram, etc. However, considering that the grayscale histogram as a complexity measurement indicator has a higher tolerance for small motion of the screen content video and is more convenient to calculate, step 102 optionally includes:

[0041] The complexity of the first frame image is determined according to a grayscale histogram difference between the first frame image and the second frame image.

[0042] In conjunction with the fact that a frame can be divided into multiple coding units (LCUs), optionally, determining the complexity of the first frame image according to a grayscale histogram difference between the first frame image and the second frame image includes:

[0043] Obtaining a first ratio of the number of pixels having a pixel value of k in the j-th coding unit of the first frame image to the total number of pixels in the to-be-encoded video, and a second ratio of the number of pixels having a pixel value of k in the j-th coding unit of the second frame image to the total number of pixels in the to-be-encoded video;

[0044] According to the value of k, the difference between the first proportion and the second proportion corresponding to different values ​​of k is accumulated to obtain the complexity of the j-th coding unit;

[0045] Calculating the sum of the complexities of all coding units of the first frame of image to obtain the complexity of the first frame of image;

[0046] Wherein, k∈[0, L], L is a preset threshold, and j is an integer greater than or equal to 1.

[0047] Here, L is a preset threshold, such as L=255. For the j-th coding unit of the first frame image, according to the above steps, its complexity C j The calculation can be expressed as H j (k) is the first proportion of the number of pixels with pixel value k in the j-th coding unit of the first frame image to the total number of pixels in the video to be encoded, H j-1 (k) is the second proportion of the number of pixels with pixel value k in the j-th coding unit of the second frame image to the total number of pixels in the video to be encoded. Therefore, for the first frame image, the complexity of all coding units can be accumulated to obtain the complexity C of the first frame image. i , C i The calculation can be expressed as n is the number of coding units in the first frame image, wherein the coding unit may be a maximum coding unit LCU.

[0048] In this way, the complexity of the coding unit of the first frame image is first calculated, and then the complexity of each coding unit is accumulated to obtain the complexity of the first frame image. Since the complexity of each coding unit can be calculated in parallel, the processing efficiency of the encoding process is improved.

[0049] For example, using the BitstreamAnalyzer tool for bitstream analysis in a Security Context Constraint (SCC) test sequence, the sequence indicator curve shows a clear peak in the grayscale histogram, a complexity metric, for sudden frames in the sequence. For continuously changing or static frames, the corresponding indicator value fluctuates more smoothly. This metric can more accurately reflect the scene transition characteristics of the sequence.

[0050] In addition, due to possible mutations in the image frame, in order to perform targeted processing, optionally, in this embodiment, after step 102, the following steps are further included:

[0051] The frame type of the first frame image is determined according to the complexity of the first frame image; wherein the frame type includes: a sudden change frame, a normal frame, and a still frame.

[0052] The determining the frame type of the first frame of image according to the complexity of the first frame of image includes:

[0053] By formula Calculate and obtain the calculation result;

[0054] If the result is J i =1, then the first frame image is a sudden change frame; if the calculation result is J i =0, then the first frame image is a normal frame; if the calculation result is J i =0.5, then the first frame image is a still frame; wherein,

[0055] J i is a quantization function for determining the frame type, i is the frame number of the first frame image in the video to be encoded, C cur is the complexity of the first frame image, C pre is the sum of the complexity of the encoded frame images in the video to be encoded, N pre is the number of encoded frame images in the video to be encoded.

[0056] Here, 3×C pre / N pre and 0.05×C pre / N preThese are the thresholds for determining whether the first frame is a sudden change frame or a still frame, respectively. In this embodiment, both thresholds use mean values ​​to better reflect the average complexity of the encoded frames. Of course, the coefficients 3 and 0.05 in the thresholds are preferred implementations; other values ​​within a certain range may also be used, which will not be elaborated here.

[0057] Alternatively, as Figure 2 As shown, step 103 includes:

[0058] Step 201: obtaining an expected number of bits of the first frame of image in a sliding window according to the complexity of the first frame of image;

[0059] Step 202: determining whether the saturation of the buffer area satisfies a preset condition based on the expected number of bits and the frame type of the first image frame;

[0060] Step 203: If the preset condition is met, use the expected number of bits as the target number of bits;

[0061] Step 204, when the preset condition is not met, the expected number of bits is adjusted according to the frame type of the first frame image. If the saturation corresponding to the adjusted expected number of bits meets the preset condition, the adjusted expected number of bits is used as the target number of bits.

[0062] Here, the preset condition is used to ensure that the buffer reserves sufficient coding space for the next frame when encoding each frame in the sliding window, and to ensure that the buffer space has margin for bit compensation for continuously changing frames and abrupt frames. Thus, according to steps 201-204, bits are first pre-allocated within the sliding window based on the complexity of the first frame to obtain an expected number of bits. Then, based on this pre-allocated number of bits and the frame type of the first frame, a determination is made as to whether the buffer saturation meets the preset condition. Finally, based on the determination result, if the preset condition is met, the expected number of bits is used as the target number of bits. If the preset condition is not met, the expected number of bits is adjusted based on the frame type of the first frame. If the saturation corresponding to the adjusted expected number of bits meets the preset condition, the adjusted expected number of bits is used as the target number of bits. In this way, the number of bits allocated for the first frame in the sliding window ensures that the buffer reserves sufficient coding space for the next frame when encoding each frame in the sliding window, and that the buffer space has margin for bit compensation for continuously changing frames and abrupt frames, thereby improving image quality and resource utilization.

[0063] Optionally, step 201 includes:

[0064] By formula Calculate the expected number of bits Ti Wherein, i is the frame number of the first frame image in the video to be encoded, C cur is the complexity of the first frame image, C SW is the sum of the complexity of the frame images in the sliding window, T SW is the number of bits corresponding to the sliding window.

[0065] Since screen content videos generally contain many sudden frames and still frames, allocating bits to the first frame image based on its complexity, i.e., frame-level pre-allocation, can more accurately reserve sufficient resources for encoding sudden frames in the sliding window, thereby improving the quality of the entire video sequence while preventing bit rate runaway.

[0066] Optionally, the preset condition includes a compensation threshold Com(i) and an upper threshold Th(i);

[0067] Step 202 includes:

[0068] By formula Calculate the saturation of the buffer B(i);

[0069] If Com(i)≤B(i)≤Th(i), the preset condition is met, otherwise it is not met;

[0070] Among them, R tar is the target bit rate, F is the frame rate of the video to be encoded, B S is the buffer capacity;

[0071] J i Quantization function for determining frame type;

[0072] T i+1 is the expected number of bits of the next frame of image following the first frame of image.

[0073] Here, the establishment of the compensation threshold Com(i) and the upper threshold Th(i) provides a reference for the bit allocation of the first frame image, making the encoding more adaptable to the image characteristics of the screen content video. Among them, when the encoder transmits the encoded stream of the first frame image to the buffer area, the buffer area should ensure that sufficient buffer space is reserved for the next frame, that is, Here, T i+1 The predicted value of the number of bits allocated to the next frame of the first frame in the sliding window (i.e. the expected number of bits) is used to ensure that the remaining space in the buffer after encoding the first frame can meet the net increase after encoding the next frame. That is, the upper limit of the buffer saturation Th(i) is expressed as Considering that mutation frames and normal frames with complex contents have higher requirements for bit resources, when the first frame is a mutation frame or a normal frame, if the buffer saturation is low at this time, bit compensation can be appropriately performed on the current frame, and the compensation threshold Com(i) is set based on this. If B(i) is less than Com(i), bit compensation needs to be performed on the first frame of image. Therefore, B(i) ≥ Com(i). Therefore, the preset condition that B(i) must satisfy is Com(i) ≤ B(i) ≤ Th(i).

[0074] By using the formula After calculating B(i), and comparing it with the compensation threshold Com(i) and the upper threshold Th(i), it can be determined that B(i) meets the preset condition when Com(i)≤B(i)≤Th(i); otherwise, it does not meet the preset condition.

[0075] For the case where B(i) meets the preset conditions, as in step 203, the expected number of bits of the first frame image in the sliding window is the target number of bits. The encoding process at the LCU level can then be entered, and the actual number of encoded bits of the first frame image is used to correct the actual bit capacity of the buffer area and the number of bits in the sliding window after encoding the first frame image, thereby improving the accuracy of the prediction of the next frame.

[0076] In the case that B(i) satisfies the preset condition, as in step 204 , the expected number of bits needs to be adjusted until the saturation corresponding to the adjusted expected number of bits satisfies the preset condition.

[0077] In this embodiment, different adjustment steps are set according to the frame type. Therefore, optionally, step 204 includes:

[0078] If B(i) is less than Com(i), then by the formula Determine the adjusted expected number of bits T i ';

[0079] If B(i) is greater than Th(i), then by the formula Determine the adjusted expected number of bits T i '.

[0080] The expected number of bits T after adjustment in the above manner i ', for T i After updating, the formula The corresponding saturation is re-acquired, and then a judgment is made based on a preset condition until the saturation corresponding to the adjusted expected number of bits meets the preset condition.

[0081] In this embodiment, optionally, the method further includes:

[0082] After determining the target number of bits of the first frame image in the sliding window, the formula The number of bits T(i, j) allocated to the j-th coding unit of the first frame image; wherein, C curLCU is the complexity of the j-th coding unit, C leftcur is the sum of the complexities of the coding units that have not been allocated bits in the first frame of image, T left is the number of unallocated bits in the target number of bits.

[0083] In this way, the complexity of the coding unit in the complexity calculation of the first frame image can be reused, and the number of bits of the coding unit of the first frame image can be allocated based on the target number of bits, such as the bit number T(i, j) of the j-th coding unit, to perform LCU-level bit allocation.

[0084] In summary, further, the bit allocation within the sliding window of the method of the embodiment of the present invention adopts a feedback mechanism, which is more adapted to the characteristics of screen content encoding, and the bit resources allocated to it are reasonably planned for different frame types. When the storage pressure of the buffer area is large, the bit allocation of the current frame is reduced, and the reduction range is determined by the type of frame to ensure that the mutation frame can obtain more bit resources to ensure the encoding quality; when the storage pressure of the buffer area is small, the current frame is appropriately bit compensated. Moreover, the adjustment step size is also determined by the frame type. For low-latency encoding scenarios, the overall computational complexity overhead can be reduced, and the encoding time can be saved to a great extent. In terms of cache management, the basic management unit is accurately allocated to each frame in the encoding, and the cache resources are fully utilized to ensure that there will be no overflow or underflow of the cache area.

[0085] Specifically, the application of the method of the embodiment of the present invention is as follows: Figure 3 As shown:

[0086] First, the number of bits corresponding to the sliding window is determined, and then frame-level pre-allocation is performed to obtain the expected number of bits. Afterwards, the frame-level bit allocation result within the sliding window (i.e., the expected number of bits) is used as a prediction parameter to predict the saturation of the buffer area after the encoder encodes this frame with the expected number of bits. In addition, frame complexity is used as an indicator for classification, and frame types are divided into three categories: mutation frames, static frames, and ordinary frames. Since different types of frames have different requirements for the number of bits and cache space, different threshold criteria (i.e., preset conditions) are established accordingly to make the working state of the buffer area highly adaptable to the frame category. When the saturation prediction result does not meet the preset conditions, the bit allocation result is adjusted according to the frame type until the preset conditions are met. Then, LCU-level bit allocation is performed.

[0087] like Figure 4 As shown, an encoding processing device according to an embodiment of the present invention includes:

[0088] A first processing module 410 is configured to determine a first frame of image, where the first frame of image is a current image frame to be encoded in the video to be encoded;

[0089] An acquisition module 420 is configured to acquire the complexity of the first frame of image, where the complexity is used to represent a difference between the first frame of image and a second frame of image, where the second frame of image is a frame of image preceding the first frame of image;

[0090] The second processing module 430 is configured to determine a target number of bits of the first frame of image in the sliding window according to the complexity of the first frame of image.

[0091] Optionally, the first processing module is further configured to:

[0092] According to the sliding window length, the first frame image in the current sliding window is used as the first frame image; wherein,

[0093] The sliding window length is a minimum value between a preset length and the number of uncoded frames in the video to be coded.

[0094] Optionally, the acquisition module is further configured to:

[0095] The complexity of the first frame image is determined according to a grayscale histogram difference between the first frame image and the second frame image.

[0096] Optionally, the acquisition module includes:

[0097] A first acquisition submodule is configured to obtain a first ratio of the number of pixels with a pixel value of k in the j-th coding unit of the first frame image to the total pixel points of the to-be-encoded video, and a second ratio of the number of pixels with a pixel value of k in the j-th coding unit of the second frame image to the total pixel points of the to-be-encoded video;

[0098] A first processing submodule is configured to accumulate the differences between the first proportion and the second proportion corresponding to different values ​​of k according to the value of k, to obtain the complexity of the j-th coding unit;

[0099] a second processing submodule, configured to calculate the sum of the complexities of all coding units of the first frame of image to obtain the complexity of the first frame of image;

[0100] Wherein, k∈[0, L], L is a preset threshold, and j is an integer greater than or equal to 1.

[0101] Optionally, the device further comprises:

[0102] The third processing module is configured to determine a frame type of the first frame of image according to the complexity of the first frame of image; wherein the frame types include: a sudden change frame, a normal frame, and a still frame.

[0103] Optionally, the third processing module is further configured to:

[0104] By formula Calculate and obtain the calculation result;

[0105] If the result is J i =1, then the first frame image is a sudden change frame; if the calculation result is J i =0, then the first frame image is a normal frame; if the calculation result is J i =0.5, then the first frame image is a still frame; wherein,

[0106] J i is a quantization function for determining the frame type, i is the frame number of the first frame image in the video to be encoded, C cur is the complexity of the first frame image, C pre is the sum of the complexity of the encoded frame images in the video to be encoded, N pre is the number of encoded frame images in the video to be encoded.

[0107] Optionally, the second processing module includes:

[0108] a third processing submodule, configured to obtain an expected number of bits of the first frame of image in a sliding window according to the complexity of the first frame of image;

[0109] a fourth processing submodule, configured to determine whether a saturation of a buffer area satisfies a preset condition based on the expected number of bits and the frame type of the first frame of image;

[0110] a fifth processing submodule, configured to use the expected number of bits as the target number of bits when the preset condition is met;

[0111] The sixth processing submodule is used to adjust the expected number of bits according to the frame type of the first frame image when the preset condition is not met, and if the saturation corresponding to the adjusted expected number of bits meets the preset condition, the adjusted expected number of bits is used as the target number of bits.

[0112] Optionally, the third processing submodule is further configured to:

[0113] By formula Calculate the expected number of bits T i Wherein, i is the frame number of the first frame image in the video to be encoded, C curis the complexity of the first frame image, C SW is the sum of the complexity of the frame images in the sliding window, T SW is the number of bits corresponding to the sliding window.

[0114] Optionally, the preset condition includes a compensation threshold Com(i) and an upper threshold Th(i);

[0115] The fourth processing submodule is further configured to:

[0116] By formula Calculate the saturation of the buffer B(i);

[0117] If Com(i)≤B(i)≤Th(i), the preset condition is met, otherwise it is not met;

[0118] Among them, R tar is the target bit rate, F is the frame rate of the video to be encoded, B S is the buffer capacity;

[0119] J i Quantization function for determining frame type;

[0120]

[0121] Optionally, the sixth processing submodule is further configured to:

[0122] If B(i) is less than Com(i), then by the formula Determine the adjusted expected number of bits T i ';

[0123] If B(i) is greater than Th(i), then by the formula Determine the adjusted expected number of bits T i '.

[0124] Optionally, the device further comprises:

[0125] The fourth processing module is configured to, after determining the target number of bits of the first frame image in the sliding window, calculate the target bit number of the first frame image by the formula The number of bits T(i, j) allocated to the j-th coding unit of the first frame image; wherein, C curLCU is the complexity of the j-th coding unit, C leftcur is the sum of the complexities of the coding units that have not been allocated bits in the first frame of image, T left is the number of unallocated bits in the target number of bits.

[0126] The device will obtain the complexity of the first frame of image currently to be encoded in the video to be encoded, that is, the first frame of image, and then further determine the target number of bits of the first frame of image in the sliding window based on the complexity. Since the complexity can characterize the difference between the first frame of image and the previous frame of image, it is more suitable for the characteristics of screen content video and can achieve refined management of bit allocation in the sliding window without consuming more resources.

[0127] The implementation principle and technical effects of the device in the embodiment of the present invention are similar to those in the above-mentioned method embodiment, and will not be described in detail in this embodiment.

[0128] like Figure 5 As shown, an encoding device according to an embodiment of the present invention includes a processor 501, a memory 502, and a program or instruction stored in the memory 502 and executable on the processor 501; when the processor 502 executes the program or instruction, the following is achieved:

[0129] Determine a first frame of image, where the first frame of image is a current image frame to be encoded in the video to be encoded;

[0130] Obtaining a complexity of the first frame of image, where the complexity is used to characterize a difference between the first frame of image and a second frame of image, where the second frame of image is a frame of image preceding the first frame of image;

[0131] A target number of bits of the first frame of image in a sliding window is determined according to the complexity of the first frame of image.

[0132] Optionally, the processor 501 is further configured to implement:

[0133] According to the sliding window length, the first frame image in the current sliding window is used as the first frame image; wherein,

[0134] The sliding window length is a minimum value between a preset length and the number of uncoded frames in the video to be coded.

[0135] Optionally, the processor 501 is further configured to implement:

[0136] The complexity of the first frame image is determined according to a grayscale histogram difference between the first frame image and the second frame image.

[0137] Optionally, the processor 501 is further configured to implement:

[0138] Obtaining a first ratio of the number of pixels having a pixel value of k in the j-th coding unit of the first frame image to the total number of pixels in the to-be-encoded video, and a second ratio of the number of pixels having a pixel value of k in the j-th coding unit of the second frame image to the total number of pixels in the to-be-encoded video;

[0139] According to the value of k, the difference between the first proportion and the second proportion corresponding to different values ​​of k is accumulated to obtain the complexity of the j-th coding unit;

[0140] Calculating the sum of the complexities of all coding units of the first frame of image to obtain the complexity of the first frame of image;

[0141] Wherein, k∈[0, L], L is a preset threshold, and j is an integer greater than or equal to 1.

[0142] Optionally, the processor 501 is further configured to implement:

[0143] The frame type of the first frame image is determined according to the complexity of the first frame image; wherein the frame type includes: a sudden change frame, a normal frame, and a still frame.

[0144] Optionally, the processor 501 is further configured to implement:

[0145] By formula Calculate and obtain the calculation result;

[0146] If the result is J i =1, then the first frame image is a sudden change frame; if the calculation result is J i =0, then the first frame image is a normal frame; if the calculation result is J i =0.5, then the first frame image is a still frame; wherein,

[0147] J i is a quantization function for determining the frame type, i is the frame number of the first frame image in the video to be encoded, C cur is the complexity of the first frame image, C pre is the sum of the complexity of the encoded frame images in the video to be encoded, N pre is the number of encoded frame images in the video to be encoded.

[0148] Optionally, the processor 501 is further configured to implement:

[0149] Obtaining an expected number of bits of the first frame of image in a sliding window according to the complexity of the first frame of image;

[0150] determining, based on the expected number of bits and the frame type of the first frame of image, whether a saturation of the buffer satisfies a preset condition;

[0151] When the preset condition is met, taking the expected number of bits as the target number of bits;

[0152] When the preset condition is not met, the expected number of bits is adjusted according to the frame type of the first frame image. If the saturation corresponding to the adjusted expected number of bits meets the preset condition, the adjusted expected number of bits is used as the target number of bits.

[0153] Optionally, the processor 501 is further configured to implement:

[0154] By formula Calculate the expected number of bits T i Wherein, i is the frame number of the first frame image in the video to be encoded, C cur is the complexity of the first frame image, C SW is the sum of the complexity of the frame images in the sliding window, T SW is the number of bits corresponding to the sliding window.

[0155] Optionally, the preset condition includes a compensation threshold Com(i) and an upper threshold Th(i);

[0156] The processor 501 is further configured to implement:

[0157] By formula Calculate the saturation of the buffer B(i);

[0158] If Com(i)≤B(i)≤Th(i), the preset condition is met, otherwise it is not met;

[0159] Among them, R tar is the target bit rate, F is the frame rate of the video to be encoded, B S is the buffer capacity;

[0160] J i Quantization function for determining frame type;

[0161] T i+1 is the expected number of bits of the next frame of image following the first frame of image.

[0162] Optionally, the processor 501 is further configured to implement:

[0163] If B(i) is less than Com(i), then by the formula Determine the adjusted expected number of bits T i ';

[0164] If B(i) is greater than Th(i), then by the formula Determine the adjusted expected number of bits T i '.

[0165] Optionally, the processor 501 is further configured to implement:

[0166] After determining the target number of bits of the first frame image in the sliding window, the formula The number of bits T(i, j) allocated to the j-th coding unit of the first frame image; wherein, C curLCU is the complexity of the j-th coding unit, C leftcur is the sum of the complexities of the coding units that have not been allocated bits in the first frame of image, T left is the number of unallocated bits in the target number of bits.

[0167] The device provided in the embodiment of the present invention can execute the above method embodiment, and its implementation principle and technical effects are similar, so this embodiment will not be repeated here.

[0168] A readable storage medium according to an embodiment of the present invention stores a program or instruction thereon. When the program or instruction is executed by a processor, the steps in the encoding processing method described above are implemented and the same technical effect can be achieved. To avoid repetition, they will not be described here.

[0169] The processor is the processor in the encoding device described in the above embodiment. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0170] It should be further noted that many functional components described in this specification are referred to as modules in order to more particularly emphasize the independence of their implementation methods.

[0171] In embodiments of the present invention, modules can be implemented in software so that they can be executed by various types of processors. For example, an identified executable code module can include one or more physical or logical blocks of computer instructions, for example, which can be constructed as objects, procedures, or functions. Nevertheless, the executable code of the identified module does not need to be physically located together, but can include different instructions stored in different locations, which, when logically combined together, constitute the module and achieve the specified purpose of the module.

[0172] In fact, executable code module can be a single instruction or many instructions, and can even be distributed on a plurality of different code segments, distributed in the middle of different programs, and distributed across a plurality of memory devices.Similarly, operating data can be identified in the module, and can be implemented and organized in the data structure of any appropriate type according to any appropriate form.Described operating data can be collected as a single data set, or can be distributed in different locations (including on different storage devices), and can only be present on a system or network as an electronic signal at least in part.

[0173] When a module can be implemented using software, given the current state of hardware technology, those skilled in the art can build corresponding hardware circuits to implement the corresponding functions of the module, regardless of cost. The hardware circuits may include conventional very large scale integration (VLSI) circuits or gate arrays, as well as existing semiconductors such as logic chips and transistors, or other discrete components. Modules may also be implemented using programmable hardware devices, such as field programmable gate arrays, programmable array logic, or programmable logic devices.

[0174] The above exemplary embodiments are described with reference to the accompanying drawings. Many different forms and embodiments are possible without departing from the spirit and teachings of the present invention. Therefore, the present invention should not be construed as limited to the exemplary embodiments set forth herein. Rather, these exemplary embodiments are provided so that this disclosure will be complete and perfect and will convey the scope of the invention to those skilled in the art. In the drawings, component sizes and relative sizes may be exaggerated for clarity. The terminology used herein is for purposes of describing specific exemplary embodiments only and is not intended to be limiting. As used herein, the singular forms "a," "an," and "the" are intended to include plural forms, unless the context clearly indicates otherwise. It will be further understood that the terms "comprising" and / or "including," when used in this specification, indicate the presence of stated features, integers, steps, operations, components, and / or elements, but do not preclude the presence or addition of one or more other features, integers, steps, operations, components, elements, and / or groups thereof. Unless otherwise indicated, when stated, a range of values ​​includes the upper and lower limits of that range and any subranges therebetween.

[0175] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A coding processing method, characterized in that: include: Determine a first frame of image, where the first frame of image is a current image frame to be encoded in the video to be encoded; Obtaining a complexity of the first frame of image, where the complexity is used to characterize a difference between the first frame of image and a second frame of image, where the second frame of image is a frame of image preceding the first frame of image; determining a target number of bits of the first frame of image in a sliding window according to the complexity of the first frame of image; The determining, according to the complexity of the first frame of image, a target number of bits of the first frame of image in the sliding window includes: Obtaining an expected number of bits of the first frame of image in a sliding window according to the complexity of the first frame of image; determining, based on the expected number of bits and the frame type of the first frame of image, whether a saturation of the buffer satisfies a preset condition; When the preset condition is met, taking the expected number of bits as the target number of bits; When the preset condition is not met, the expected number of bits is adjusted according to the frame type of the first frame image. If the saturation corresponding to the adjusted expected number of bits meets the preset condition, the adjusted expected number of bits is used as the target number of bits.

2. The method according to claim 1, characterized in that The determining of the first frame image includes: According to the sliding window length, the first frame image in the current sliding window is used as the first frame image; wherein, The sliding window length is a minimum value between a preset length and the number of uncoded frames in the video to be coded.

3. The method according to claim 1, characterized in that The complexity of obtaining the first frame of image includes: The complexity of the first frame image is determined according to a grayscale histogram difference between the first frame image and the second frame image.

4. The method according to claim 3, characterized in that The determining the complexity of the first frame image according to the grayscale histogram difference between the first frame image and the second frame image includes: Obtaining a first ratio of the number of pixels having a pixel value of k in the j-th coding unit of the first frame image to the total number of pixels in the to-be-encoded video, and a second ratio of the number of pixels having a pixel value of k in the j-th coding unit of the second frame image to the total number of pixels in the to-be-encoded video; According to the value of k, the difference between the first proportion and the second proportion corresponding to different values ​​of k is accumulated to obtain the complexity of the j-th coding unit; Calculating the sum of the complexities of all coding units of the first frame of image to obtain the complexity of the first frame of image; Wherein, k∈[0, L], L is a preset threshold, and j is an integer greater than or equal to 1.

5. The method according to claim 1, wherein After obtaining the complexity of the first frame of image, the method further includes: The frame type of the first frame image is determined according to the complexity of the first frame image; wherein the frame type includes: a sudden change frame, a normal frame, and a still frame.

6. The method according to claim 5, characterized in that The determining the frame type of the first frame of image according to the complexity of the first frame of image includes: By formula Calculate and obtain the calculation result; If the result is J i =1, then the first frame image is a sudden change frame; if the calculation result is J i =0, then the first frame image is a normal frame; if the calculation result is J i =0.5, then the first frame image is a still frame; wherein, J i is a quantization function for determining the frame type, i is the frame number of the first frame image in the video to be encoded, C cur is the complexity of the first frame image, C pre is the sum of the complexity of the encoded frame images in the video to be encoded, N pre is the number of encoded frame images in the video to be encoded.

7. The method according to claim 1, characterized in that Obtaining an expected number of bits of the first frame of image in a sliding window according to the complexity of the first frame of image includes: By formula Calculate the expected number of bits T i Wherein, i is the frame number of the first frame image in the video to be encoded, C cur is the complexity of the first frame image, C SW is the sum of the complexity of the frame images in the sliding window, T SW is the number of bits corresponding to the sliding window.

8. The method according to claim 7, characterized in that The preset conditions include a compensation threshold Com(i) and an upper threshold limit Th(i); The determining, based on the expected number of bits and the frame type of the first frame image, whether the saturation of the buffer satisfies a preset condition includes: By formula Calculate the saturation of the buffer B(i); If Com(i)≤B(i)≤Th(i), the preset condition is met, otherwise it is not met; Among them, R tar is the target bit rate, F is the frame rate of the video to be encoded, B S is the buffer capacity; J i Quantization function for determining frame type; T i+1 is the expected number of bits of the next frame of image following the first frame of image.

9. The method according to claim 8, characterized in that The adjusting the expected number of bits according to the frame type of the first frame of image includes: If B(i) is less than Com(i), then by the formula Determine the adjusted expected number of bits T i '; If B(i) is greater than Th(i), then by the formula Determine the adjusted expected number of bits T i '.

10. The method according to claim 4, characterized in that Also includes: After determining the target number of bits of the first frame image in the sliding window, the formula The number of bits T(i, j) allocated to the j-th coding unit of the first frame image; wherein, C curLCU is the complexity of the j-th coding unit, C leftcur is the sum of the complexities of the coding units that have not been allocated bits in the first frame of image, T left is the number of unallocated bits in the target number of bits.

11. A coding processing device, characterized in that: include A first processing module is configured to determine a first frame of image, where the first frame of image is a current image frame to be encoded in the video to be encoded; an acquisition module, configured to acquire the complexity of the first frame image, where the complexity is used to characterize the difference between the first frame image and a second frame image, where the second frame image is a frame image preceding the first frame image; A second processing module, configured to determine a target number of bits of the first frame of image in a sliding window according to the complexity of the first frame of image; Wherein, the second processing module includes: a third processing submodule, configured to obtain an expected number of bits of the first frame of image in a sliding window according to the complexity of the first frame of image; a fourth processing submodule, configured to determine whether a saturation of a buffer area satisfies a preset condition based on the expected number of bits and the frame type of the first frame of image; a fifth processing submodule, configured to use the expected number of bits as the target number of bits when the preset condition is met; The sixth processing submodule is used to adjust the expected number of bits according to the frame type of the first frame image when the preset condition is not met, and if the saturation corresponding to the adjusted expected number of bits meets the preset condition, the adjusted expected number of bits is used as the target number of bits.

12. The device according to claim 11, characterized in that The first processing module is further configured to: According to the sliding window length, the first frame image in the current sliding window is used as the first frame image; wherein, The sliding window length is a minimum value between a preset length and the number of uncoded frames in the video to be coded.

13. The device according to claim 11, characterized in that The acquisition module is further used for: The complexity of the first frame image is determined according to a grayscale histogram difference between the first frame image and the second frame image.

14. The device according to claim 13, characterized in that The acquisition module includes: A first acquisition submodule is configured to obtain a first ratio of the number of pixels with a pixel value of k in the j-th coding unit of the first frame image to the total pixel points of the to-be-encoded video, and a second ratio of the number of pixels with a pixel value of k in the j-th coding unit of the second frame image to the total pixel points of the to-be-encoded video; A first processing submodule is configured to accumulate the differences between the first proportion and the second proportion corresponding to different values ​​of k according to the value of k, to obtain the complexity of the j-th coding unit; a second processing submodule, configured to calculate the sum of the complexities of all coding units of the first frame of image to obtain the complexity of the first frame of image; Wherein, k∈[0, L], L is a preset threshold, and j is an integer greater than or equal to 1.

15. The device according to claim 11, characterized in that Also includes: The third processing module is configured to determine a frame type of the first frame of image according to the complexity of the first frame of image; wherein the frame types include: a sudden change frame, a normal frame, and a still frame.

16. The device according to claim 15, characterized in that The third processing module is further configured to: By formula Calculate and obtain the calculation result; If the result is J i =1, then the first frame image is a sudden change frame; if the calculation result is J i =0, then the first frame image is a normal frame; if the calculation result is J i =0.5, then the first frame image is a still frame; wherein, J i is a quantization function for determining the frame type, i is the frame number of the first frame image in the video to be encoded, C cur is the complexity of the first frame image, C pre is the sum of the complexity of the encoded frame images in the video to be encoded, N pre is the number of encoded frame images in the video to be encoded.

17. The device according to claim 11, characterized in that The third processing submodule is further configured to: By formula Calculate the expected number of bits T i Wherein, i is the frame number of the first frame image in the video to be encoded, C cur is the complexity of the first frame image, C SW is the sum of the complexity of the frame images in the sliding window, T SW is the number of bits corresponding to the sliding window.

18. The device according to claim 17, characterized in that The preset conditions include a compensation threshold Com(i) and an upper threshold limit Th(i); The fourth processing submodule is further configured to: By formula Calculate the saturation of the buffer B(i); If Com(i)≤B(i)≤Th(i), the preset condition is met, otherwise it is not met; Among them, R tar is the target bit rate, F is the frame rate of the video to be encoded, B S is the buffer capacity; J i Quantization function for determining frame type; T i+1 is the expected number of bits of the next frame of image following the first frame of image.

19. The device according to claim 18, characterized in that The sixth processing submodule is further configured to: If B(i) is less than Com(i), then by the formula Determine the adjusted expected number of bits T i '; If B(i) is greater than Th(i), then by the formula Determine the adjusted expected number of bits T i '.

20. The device according to claim 14, characterized in that Also includes: The fourth processing module is configured to, after determining the target number of bits of the first frame image in the sliding window, calculate the target bit number of the first frame image by the formula The number of bits T(i, j) allocated to the j-th coding unit of the first frame image; wherein, C curLCU is the complexity of the j-th coding unit, C leftcur is the sum of the complexities of the coding units that have not been allocated bits in the first frame of image, T left is the number of unallocated bits in the target number of bits.

21. An encoding device comprising: A processor, a memory, and a program or instruction stored in the memory and executable on the processor; wherein the processor implements the encoding processing method according to any one of claims 1 to 10 when executing the program or instruction.

22. A readable storage medium having a program or instruction stored thereon, characterized in that: When the program or instruction is executed by a processor, the steps in the encoding processing method according to any one of claims 1 to 10 are implemented.

Images