Image coding method and related equipment

Abstract

The invention provides an image coding method, and relates to the field of image processing, and the method comprises the steps: determining a real-time code rate at a current moment; determining a code stream expected value of a next frame according to the real-time code rate and a target code rate; and for the intra-frame refresh frame, based on the code stream expected value, determining the column number of the contained intra-frame refresh macro block columns, and performing intra-frame refresh. According to the scheme, the smoothness of the code rate peak value can be realized by providing a dynamic coding method in the image coding process.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular, to an image encoding method and related devices. Background Art

[0002] In the prior art, almost all encoding technologies will periodically have frames with relatively large byte counts. Generally, such frames are I-frames (Intra frames). Since an I-frame contains complete image information, the code stream of the I-frame is relatively large. Therefore, when an I-frame appears, the code stream will have a peak.

[0003] This peak may cause stuttering, and in severe cases, may even cause frame loss; in more serious cases, after frame loss, the decoding end may repeatedly request I-frames, which may further cause the system to almost completely freeze. Therefore, how to avoid or reduce such sudden peaks has become a problem to be solved. Summary of the Invention

[0004] Embodiments of this application provide an image encoding method and related devices. By providing a dynamic encoding method, it is possible to smooth the peak of the code rate. The technical solution is as follows:

[0005] According to the first aspect of the embodiments of this application, an image encoding method is provided. The method includes:

[0006] Determine the real-time code rate at the current moment;

[0007] Determine the expected value of the code stream for the next frame according to the real-time code rate and the target code rate;

[0008] For an intra refresh frame, based on the expected value of the code stream, determine the number of columns of macroblock columns containing intra refresh, and perform intra refresh.

[0009] Embodiments of this application provide an image encoding method. By improving the IR method, adopting a dynamic period, and combining the feedback of the actual code rate size by the encoder, the size of the refreshed area in the intra refresh frame is adjusted, so as to achieve the purpose of smoothing bits, and further reduce the stuttering phenomenon during the transmission peak, and improve the user experience.

[0010] In one embodiment, the determining the real-time code rate at the current moment includes:

[0011] After subtracting 1 from the number of image frames collected in the most recent 1 second, determine the real-time code rate at the current moment according to the total number of bits of the code stream corresponding to the remaining frames.

[0012] In one embodiment, the determining the expected value of the code stream for the next frame according to the real-time code rate and the target code rate includes:

[0013] Subtract the real-time bit rate from the target bit rate to obtain the expected bitstream value for the next frame.

[0014] In one embodiment, for an intra refresh frame, based on the expected bitstream value, determine the number of columns of macroblocks with intra refresh included, and perform intra refresh, including:

[0015] When the refresh period is reached, determine that the current frame is the first intra refresh frame, denoted as IRF_1;

[0016] Encode IRF_1 as a normal P frame and record the generated first bit array;

[0017] Encode IRF_1 in the intra prediction mode and record the generated second bit array;

[0018] Based on the expected bitstream value, the first bit array, and the second bit array, use a judgment formula to determine that the number of columns of macroblocks with intra refresh included in the IRF_1 frame is from the first column to the kth column, and perform intra refresh; k is the maximum number of columns that satisfies the judgment formula.

[0019] In one embodiment, the judgment formula is as follows:

[0020]

[0021] And,

[0022] Where m is the number of columns of macroblocks encoded in the intra prediction mode for IRF_1, n is the number of columns of macroblocks encoded as a normal P frame in IRF_1; the minimum values of m and n are both 1, and the maximum value of n is 120.

[0023] In one embodiment, the method further includes:

[0024] Determine that the next frame is the second intra refresh frame, denoted as IRF_2;

[0025] Adopt the same steps as those for determining the number of columns of macroblocks with intra refresh included in IRF_1 to determine the number of columns of macroblocks with intra refresh included in IRF_2, and perform intra refresh;

[0026] And so on for subsequent frames until all macroblock columns are refreshed.

[0027] In one embodiment, the method further includes:

[0028] For an intra refresh frame, based on the expected bitstream value, determine the number of rows of macroblocks with intra refresh included.

[0029] In the second aspect of the embodiments of the present application, an image encoding system is further provided, which is characterized by including: an encoder;

[0030] The encoder is used to determine the real-time bitrate at the current moment;

[0031] According to the real-time bitrate and the target bitrate, determine the expected value of the bitstream for the next frame;

[0032] For an intra refresh frame, based on the expected value of the bitstream, determine the number of columns of macroblocks with intra refresh included, and perform intra refresh.

[0033] In the third aspect of the embodiments of the present application, an electronic device is further provided, including a processor and a memory. At least one computer instruction is stored in the memory, and the instruction is loaded and executed by the processor to execute the steps performed in the method described in the first aspect and any one of the above.

[0034] In the fourth aspect of the embodiments of the present application, a computer-readable storage medium is further provided. At least one computer instruction is stored in the storage medium, and the instruction is loaded and executed by the processor to execute the steps performed in the method described in the first aspect and any one of the above.

[0035] In the manner provided by the embodiments of the present application, according to the magnitude relationship between the target bitrate and the real-time bitrate, the number of intra macroblock columns in the IRF frame can be dynamically adjusted. Furthermore, when the bitstream is much lower than the target bitrate, more columns can be refreshed to complete the refresh process as soon as possible. When the bitstream is close to or higher than the target bitrate, fewer columns can be refreshed or the refresh can be paused to minimize bandwidth fluctuations, so as not to affect transmission and the final smoothness. It can not only utilize the advantages of periodic intra refresh but also maximize the effect of smoothing the peak bitrate, improving the user experience in bandwidth-sensitive scenarios.

[0036] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0038] Figure 1 is a schematic diagram of the frame length distribution of I frames in a bitstream;

[0039] Figure 2 is Figure 1 the corresponding bitrate distribution schematic diagram;

[0040] Figure 3 is a comparison diagram between conventional encoding and encoding using the periodic refresh I frame technique;

[0041] Figure 4 It is a schematic diagram of a refresh process for an intra-frame refresh frame;

[0042] Figure 5 It is a schematic flowchart of an image coding method provided by an embodiment of the present application;

[0043] Figure 6 It is a schematic diagram of a macroblock column in a natural mode;

[0044] Figure 7 It is a schematic diagram of an intra-frame refresh frame with 3 columns of intra-frame macroblocks refreshed in one frame. Detailed implementation manners

[0045] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0046] First, some terms related to the embodiments of the present application will be explained, and will not be elaborated hereinafter.

[0047] 1. I-frame: An independent frame. All blocks inside do not depend on the previously encoded frames, but only on the current frame. It contains complete image information and can be decoded and displayed independently. The I-frame can play the role of shielding error propagation and can also play a synchronization role, so it is a very important type of frame. In the encoding and decoding system, I-frames will appear periodically, and the period can be modified according to the actual situation.

[0048] 2. Intra refresh (IR): Traditional video coding usually inserts I-frames in a fixed GOP (group of pictures) structure, while Intra refresh disperses the macroblocks of the I-frame into multiple frames to gradually refresh the entire picture.

[0049] It should be understood that partial refresh means that only a part of the area (such as some macroblocks) is refreshed in each frame instead of refreshing the entire picture at once: gradually completed means that after several frames, all macroblocks of the entire picture will be refreshed once, which is equivalent to completing a complete I-frame refresh.

[0050] 3. All IntraFrame (All-I), is a frame type and coding mode in video coding. It means that all frames in a video sequence are encoded as I-frames without using P-frames (predicted frames) or B-frames (bi-directionally predicted frames).

[0051] Secondly, the prior art and the problems encountered in the prior art will be briefly introduced with reference to the accompanying drawings.

[0052] The disadvantage of I-frames is that the bitstream is relatively large. When encoding I-frames, there is no reference to previously encoded frames, that is, the temporal similarity is not utilized, only the spatial similarity is used. Therefore, the size of I-frames is generally relatively large. As a result, when an I-frame appears, there will be a peak in the bitstream.

[0053] In bandwidth-sensitive scenarios, this peak may cause stuttering and, in severe cases, may lead to frame loss. In more serious cases, after frame loss, the decoding end may request the encoding of an I-frame again. This I-frame is also very large, resulting in transmission blockage. Therefore, the decoding end may further repeatedly request I-frames, resulting in the so-called "I-frame storm". At this time, the system is almost completely frozen.

[0054] To address this problem, existing coding systems provide an important technology, that is, to smooth the peak by periodically refreshing I-frames (i.e., Intrarefresh, IR). Although this method can reduce the bitstream of a single frame, it cannot reduce the bit rate (bits per second, bps) counted in seconds. When transmitting over a network, there will still be a peak when an I-frame appears.

[0055] Figure 1 It is a schematic diagram of the frame length distribution of I-frames in a bitstream. As Figure 1 shown, the arrow is used to indicate the image frame with frame number 150. The image frame with frame number 150 is an I-frame; the size of this I-frame exceeds 60 kBytes, which is significantly larger than other nearby frames, such as P-frames.

[0056] Figure 2 For Figure 1 the corresponding bit rate distribution schematic diagram. As Figure 2 shown, the dashed line is used to indicate the target average bit rate (Avgkbps). This dashed line coincides with the horizontal line indicating 2000 kbps, indicating that the target average bit rate in this bitstream is 2000 kbps; after the I-frame with frame number 150 indicated by the arrow appears, the bit rate increases for a period of time. Based on this figure, it can be shown that this I-frame causes a peak situation where the bit rate exceeds the target average bit rate of 2000 kbps, and the peak of this bit rate appears near the I-frame with frame number 150.

[0057] Regarding the problem of the peak bit rate in the above example, the existing IR technology that periodically refreshes I-frames can distribute the intra-frame refresh technology that should originally be applied to one frame to multiple consecutive P-frames respectively.

[0058] Figure 3 It is a comparison chart of using conventional coding and coding with the technology of periodically refreshing I-frames.

[0059] Exemplarily, as Figure 3 shown in (a) of [reference], for 9 image frames with frame numbers from 1 to 9, after using conventional coding, the bar height of the 3rd frame is much higher than that of other frames; it should be noted that the height of the bar chart is used to represent the number of bits of the corresponding frame after coding is completed. Since the entire frame is coded in the intra-frame refresh mode, it can be seen that the number of bits of the 3rd frame, which is an I-frame, is significantly larger than that of other frames.

[0060] As Figure 3 shown in (b) of [reference], when coding the I-frame using the IR technology, the coding part that should originally appear at the position of the 3rd frame, with the strategy of intra-frame coding based on the entire frame, is evenly distributed to 4 consecutive frames, that is, distributed to the 3rd, 4th, 5th, and 6th frames; there will be four non-overlapping regions in these four frames that are coded using the intra-frame prediction mode respectively.

[0061] It can be understood that the intra-frame within the entire frame can achieve error masking and random access. Thus, the intra-frame within the entire frame that was originally completed in the 3rd frame, after using the IR technology, is distributed to the 3rd, 4th, 5th, and 6th frames and jointly completed by these four frames. This will be able to eliminate the peak of the number of bits of the 3rd frame. In addition, these four frames can be called Intra refresh Frames (IRF).

[0062] Exemplarily, Figure 4 is a schematic diagram of a refresh process for the above-mentioned 3rd, 4th, 5th, and 6th frames.

[0063] As Figure 4 shown, the entire refresh process needs to be completed by 4 consecutive intra-frame refresh frames (such as the 3rd, 4th, 5th, and 6th frames). Each time, a quarter of the intra-frame area is fixed for refreshing, and a vertical strip-shaped intra-frame refresh area is used, and the refresh order is from left to right.

[0064] Of course, a horizontal strip-shaped refresh area can also be sampled, and the refresh order is from top to bottom. The principle is the same and will not be elaborated here. It also needs to be noted that the number of the above-mentioned intra-frame refresh frames is only an example and is not specifically limited.

[0065] Based on the above description, first, compared with not transmitting I-frames and only transmitting P-frames, after the IR technology appears, multiple intra refresh frames are equivalent to being able to form the effect of a complete I-frame. If a certain P-frame has a decoding exception, as long as this assembled I-frame can be decoded normally, then the frames behind these intra refresh frames can still be decoded normally, thereby blocking error transmission.

[0066] Second, compared with the case of directly transmitting a complete I-frame (such as the case shown in (a) of Figure 3 ), using the IR technology for encoding and transmission can eliminate the bit peak of the conventional encoded I-frame, and to a certain extent, it can achieve the purpose of realizing a smooth bit rate.

[0067] However, the existing intra refresh technology also has certain limitations. For example, although it seemingly distributes the number of bits of a conventional encoded I-frame to several consecutive frames, the actual bit rate is statistically based on seconds, and the frame rate in general application scenarios is above 25fps; then, taking Figure 3 as an example, even if the bit rate of the original I-frame of the 3rd frame is distributed to the 3rd, 4th, 5th, and 6th frames, the time of these 4 frames is much shorter than 1 second; that is to say, from the perspective of the transmission of the ratio bit rate statistically based on seconds, the significance of this distribution is not great, and the size of the number of bits within this 1 second actually does not change much.

[0068] In view of this, the embodiments of the present application provide an image encoding method. By improving the IR method, adopting a dynamic period, and combining the feedback of the encoder on the actual bit rate size, the size of the refreshed area in the intra refresh frame is adjusted, so as to achieve the purpose of realizing smooth bits, and further, the stuttering phenomenon caused during the transmission peak can be reduced, and the user experience can be improved.

[0069] Next, the solution idea and specific method provided by the embodiments of the present application will be introduced in detail with reference to the accompanying drawings.

[0070] The present application utilizes the basic idea of intra refresh, that is, the refresh process of the frame rate prediction macroblock is completed dispersedly within several frames. However, the specific strategy of intra refresh is completely different from the prior art.

[0071] In the prior art, the refresh process is fixed. For example, assuming the image resolution is 1920×1080 pixels, the macroblock size is 16×16 pixels, and there are 120 columns of macroblocks in a full frame of the image. In this case, taking the example of refreshing one column of macroblocks each time, to perform an in-frame refresh of the full-frame macroblocks from left to right, 120 consecutive in-frame refresh frames are required; if refreshing two columns of macroblocks each time, to perform an in-frame refresh of the full-frame macroblocks from left to right, 60 consecutive in-frame refresh frames are required; if refreshing three columns of macroblocks each time, to perform an in-frame refresh of the full-frame macroblocks from left to right, 40 consecutive in-frame refresh frames are required; and so on, which will not be elaborated here.

[0072] Although the prior art can basically provide the range of each refresh, once this value is determined, the number of consecutive in-frame refresh frames is also determined accordingly. However, if the number of determined in-frame refresh frames is too small, that is, the number of in-frame predicted macroblock columns in each in-frame refresh frame is too large, the effect of peak clipping will be very poor; if the number of in-frame refresh frames is too large, that is, the number of in-frame predicted macroblock columns in each in-frame refresh frame is too small, then it may not be able to make good use of the bandwidth, and the refresh process is too slow, seriously affecting the effects of error blocking and random access.

[0073] In view of this, the image coding method provided by the embodiments of the present application aims to smooth the bitrate peak brought by I frames, make the bitrate as flat as possible, facilitate the transmission of the bitstream in bandwidth-sensitive application scenarios, and further reduce the stuttering phenomenon during the transmission peak, improving the user experience.

[0074] Under this idea, the image coding method provided by the present application can be summarized as: if the in-frame refresh period is reached, start the in-frame refresh process; during the refresh process, instead of fixedly performing the refresh according to the mode of refreshing multiple macroblock columns or macroblock rows per frame and how many consecutive frames are required to complete the refresh, dynamically control the number of in-frame refreshed macroblock columns or macroblock rows allocated in each in-frame refresh frame in combination with the encoder's feedback on the real-time bitrate.

[0075] For example, for a certain in-frame refresh frame, if the encoder feedbacks that the bitrate at the current moment has reached the target bitrate and there is almost no margin in the bandwidth, then no in-frame refreshed macroblock columns or macroblock rows may be arranged for this in-frame refresh frame, and wait until there is free bandwidth later to continue the refresh. On the contrary, if the encoder feedbacks that the bitrate at the current moment has not reached the target bitrate and there is a large margin in the bandwidth, then more in-frame macroblock rows or macroblock columns can be arranged for this in-frame refresh frame, so as to complete the refresh process within the full frame as soon as possible under the condition that the bandwidth permits.

[0076] It should be understood that whether refreshing by columns (i.e., from left to right) or by rows (i.e., from top to bottom), the principle is the same, which will not be elaborated here.

[0077] Figure 5 This is a schematic flowchart of an image encoding method provided by an embodiment of the present application. As Figure 5 shown, this method can be applied to an encoder; this method may include S101 to S108, and the following provides a detailed introduction to S101 to S108.

[0078] S101. Determine the real-time bitrate at the current moment.

[0079] In the embodiment of the present application, determining the real-time bitrate at the current moment means recording the total number of bits in the bitstream in the most recent 1 second. Among them, a bitrate monitor in the form of a sliding window can be maintained in the encoder, and the total number of bits in the bitstream in the most recent 1 second is recorded using this bitrate monitor.

[0080] For example, assuming the frame rate is 30fps, that is, 30 frames are transmitted per second, then the total number of bits in the bitstream of the most recent 29 image frames (that is, the number of frames in 1 second minus 1) can be recorded using this bitrate monitor. For example, the total number of bits in the bitstream is recorded into CurrentBitrate, and the assignment of this CurrentBitrate is used to represent the real-time bitrate (bits per second).

[0081] S102. Determine the expected value of the bitstream for the next frame according to the real-time bitrate and the target bitrate.

[0082] It should be understood that the real-time bitrate is used to indicate the real-time bitrate at the current moment determined in step S101.

[0083] It should be understood that the target bitrate can be preset in combination with transmission requirements, and the embodiment of the present application does not limit this. For example, the target bitrate is recorded into TargetBitrate, and the assignment of this TargetBitrate is used to indicate the magnitude of the set target bitrate.

[0084] It should also be understood that in an ideal situation, the actual bitrate should always be equal to the magnitude of the target bitrate. That is, in an ideal situation, the value of CurrentBitrate should be equal to the value of TargetBitrate.

[0085] However, in an actual situation, the two are not equal. Therefore, the expected value of the bitstream for the next frame can be obtained by subtracting the number of bits in the bitstream in the most recent 1 second from the preset target bitrate; the determined expected value of the bitstream can be recorded into ExpectBit, and the assignment of this ExpectBit is the size of the bitstream expected to be encoded for the next frame.

[0086] It should be noted that when the next frame is an intra refresh frame, the determined bitstream expected value can be used to limit how many columns or rows of macroblocks with forced intra prediction coding can be encoded in this intra refresh frame. If the determined bitstream expected value is relatively large, more intra macroblock columns can be arranged. On the contrary, if the determined bitstream expected value is relatively small, fewer intra macroblock columns can be arranged.

[0087] S103. When the refresh period is reached, determine that the current frame is the first intra refresh frame and record it as IRF_1.

[0088] S104. Encode IRF_1 according to a conventional P frame and record the generated first bit array.

[0089] For example, assume that the resolution of IRF_1 is set to 1080P (1920x1080 pixels) and the macroblock size is set to 16x16 pixels. Taking a full frame with 120x68 rows of macroblocks as an example, when encoding according to a conventional P frame, for example, taking macroblock columns as the unit, encoding each macroblock column from left to right for 120 macroblock columns, and recording the size of the number of bits corresponding to each macroblock column into the array MBColNormal

[120] . It should be understood that a total of 120 numbers of bits corresponding to 120 macroblock columns can be recorded.

[0090] Exemplarily, Figure 6 shows a schematic diagram of a macroblock column in a natural mode.

[0091] As Figure 6 shown, in the natural mode, the full frame can be logically divided into 120 macroblock columns, and the macroblock types therein are not forcibly set. Here, it is represented by a white background color.

[0092] It should be noted that in this step, the bitstream generated by encoding according to a conventional P frame is not used for transmission and is only temporarily cached in the encoder. Therefore, this process can also be called pre-encoding.

[0093] S105. Encode IRF_1 according to the intra prediction mode and record the generated second bit array.

[0094] For example, when encoding according to the intra prediction mode, for example, taking macroblock columns as the unit, encoding each macroblock column from left to right for 120 macroblock columns, and recording the size of the number of bits corresponding to each macroblock column into the array MBColIntra

[120] .

[0095] It should be understood that if the existing fixed refresh technology is adopted and it is set to refresh one column of macroblocks each time, 120 refreshes are required, that is, 120 IRF frames are required to complete the intra-frame refresh within the entire frame range. Therefore, a total of 120 bits corresponding to 120 columns of macroblocks can be recorded. Of course, the encoding method provided in the embodiments of the present application dynamically arranges the refreshed columns within the IRF frame, rather than refreshing one column fixedly each time. Therefore, the data recorded in the array will be less than 120.

[0096] It should be noted that in this step, the bitstream generated by encoding according to the intra-frame prediction mode is not used for transmission and is only temporarily cached in the encoder. Therefore, this process is only a kind of pre-encoding.

[0097] In addition, S104 and S105 can be executed successively or simultaneously, and the embodiments of the present application do not limit this.

[0098] S106. Based on the bitstream expected value, the first bit array, and the second bit array determined in the above S103 to S105, use the following judgment formula to determine the number of columns of intra-frame refreshed macroblocks that can be included within the IRF_1 frame.

[0099] Specifically, the judgment formula is as follows:

[0100]

[0101] And,

[0102] Wherein, m is the number of columns of macroblocks that can be encoded according to the intra-frame prediction mode in IRF_1, and n is the number of columns of macroblocks that can be encoded according to the conventional P frame in IRF_1; the minimum values of m and n are both 1, and the maximum value of n is 120.

[0103] Assign values to m and n step by step and derive the k value that satisfies the above judgment condition. This k value indicates that in the IRF_1 frame, from the first column to the kth column, the macroblock columns can be set as intra-frame refreshed macroblock columns, and from the (k + 1)th to the 120th column, the macroblock types still need to be normally encoded using natural determination. After encoding based on this allocation method, the bitrate of the IRF_1 frame is less than or equal to the bitstream expected value calculated above; if the macroblock columns are set as intra-frame refreshed from the first column to the (k + 1)th column, and from the (k + 2)th to the 120th column, after sampling the macroblock types using natural determination for normal encoding, the bitrate of the IRF_1 frame will be greater than the bitstream expected value calculated above; thus, the encoding structure of the IRF_1 frame is determined, and the available bitstream expected value is utilized to the maximum extent.

[0104] S107. Use the same method as in step S106 to determine the number of columns of intra-frame refreshed macroblocks that can be included within the next IRF_2 frame.

[0105] It should be understood that since the first column to the k-th column in IRF_1 have been refreshed intra-frame, therefore, the intra-frame refreshed macroblock columns of IRF_2 will be determined starting from the (k + 1)-th column. Using the same judgment formula in step S106, it can be determined that the refresh range of IRF_2 is from the macroblocks in the (k + 1)-th column to the (k + p)-th column.

[0106] S108. And so on for subsequent frames until all macroblock columns are refreshed.

[0107] Exemplarily, Figure 7 Fig. shows a schematic diagram of an intra-frame refreshed frame in which three columns of intra-frame macroblocks have been refreshed in one frame.

[0108] As Figure 7 shown, for an IRF frame, three columns of macroblocks in this IRF frame are forced to be encoded according to intra-frame prediction coding. For the macroblocks in the first column to the third column in Figure 7 , they are shown in the figure with a grid background color. Assuming that the next frame of this IRF allows four columns of intra-frame macroblocks to be refreshed, then the next IRF frame will have the macroblocks in the fourth column to the seventh column forced as intra-frame refreshed macroblock columns. And so on until a certain IRF frame contains the macroblock column with intra-frame prediction coding in the 120th column. At this time, it is equivalent to combining these IRF frames to achieve intra-frame prediction coding throughout the entire frame range, that is, the intra-frame refresh process.

[0109] It should be understood that Figure 7 , the grid area in

[0110] represents the refreshed area, which will expand from left to right successively as the IRF frames continuously appear until it fills the entire range of 1920×1080.

[0111] It should be noted that if, when determining the macroblock columns of a certain IRF frame, it is found that even if it contains only one column of intra-frame macroblocks, it will exceed the target bitstream size, then this IRF frame will not contain intra-frame refreshed macroblocks, and the entire frame will adopt the natural macroblock determination method. The refresh process will continue only when the bandwidth allows.

[0112] It should be understood that the method for dynamically adjusting the number of intra-frame macroblock rows in the IRF frame is similar to the above method and will not be elaborated here.

[0113] Based on the image encoding method described in the above embodiments, an embodiment of the present application further provides an electronic device, including a processor and a memory. At least one computer instruction is stored in the memory, and the instruction is loaded and executed by the processor to execute the steps in the image encoding method described in any of the above embodiments.

[0114] Based on the image encoding method described in the above embodiments, an embodiment of the present application further provides a computer-readable storage medium. For example, a non-transitory computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc. Computer instructions are stored on the storage medium for executing the steps in the image encoding method described in any of the above embodiments, which will not be elaborated herein.

[0115] Based on the image encoding method described in the above embodiments, an embodiment of the present application further provides a computer-readable storage medium. For example, a non-transitory computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc. Computer instructions are stored on the storage medium for executing the steps in the image encoding method described in any of the above embodiments, which will not be elaborated herein.

[0116] Those skilled in the art will readily conceive of other embodiments of the present application after considering the specification and practicing the disclosure herein. The present application is intended to cover any variations, uses, or adaptations of the present application, which follow the general principles of the present application and include known common knowledge or conventional technical means in the technical field not disclosed in the present application. The specification and embodiments are only regarded as exemplary, and the true scope and spirit of the present application are pointed out by the following claims.

[0117] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present application is only limited by the appended claims.

Claims

1. An image encoding method, characterized in that, The method includes: Determine the real-time bitrate at the current moment; Determine the expected bitstream value for the next frame according to the real-time bitrate and the target bitrate; For an intra refresh frame, based on the expected bitstream value, determine the number of columns of macroblocks with intra refresh included, and perform intra refresh.

2. The method according to claim 1, characterized in that, The determination of the real-time bitrate at the current moment includes: Subtract 1 from the number of image frames collected in the most recent 1 second, and determine the real-time bitrate at the current moment according to the total number of bits of the bitstream corresponding to the remaining frames.

3. The method according to claim 2, characterized in that, Determining the expected bitstream value for the next frame according to the real-time bitrate and the target bitrate includes: Subtract the real-time bitrate from the target bitrate to obtain the expected bitstream value for the next frame.

4. The method according to claim 3, characterized in that, For an intra refresh frame, based on the expected bitstream value, determine the number of columns of macroblocks with intra refresh included, and perform intra refresh, including: When the refresh period is reached, determine that the current frame is the first intra refresh frame, recorded as IRF_1; Encode IRF_1 according to the conventional P frame, and record the generated first bit array; Encode IRF_1 according to the intra prediction mode, and record the generated second bit array; Based on the expected bitstream value, the first bit array, and the second bit array, use the judgment formula to determine that the number of columns of macroblocks with intra refresh included in the IRF_1 frame is from the first column to the k-th column, and perform intra refresh; k is the maximum number of columns that satisfies the judgment formula.

5. The method according to claim 4, wherein The judgment formula is as follows: And, Where m is the number of columns of macroblocks encoded according to the intra prediction mode in IRF_1, and n is the number of columns of macroblocks encoded according to the conventional P frame in IRF_1; the minimum values of m and n are both 1, and the maximum value of n is 120.

6. The method according to claim 4 or 5, characterized in that, The method further includes: Determine that the next frame is the second intra refresh frame, recorded as IRF_2; Adopt the same steps as those for determining the number of columns of macroblocks with intra refresh included in IRF_1 to determine the number of columns of macroblocks with intra refresh included in IRF_2, and perform intra refresh; And so on for subsequent frames until all macroblock columns are refreshed.

7. The method according to claim 1, characterized in that, The method further includes: For an intra refresh frame, based on the expected bitstream value, determine the number of rows of macroblocks with intra refresh included.

8. An image coding system, characterized in that, It includes: An encoder; The encoder is used to determine the real-time bitrate at the current moment; Determine the expected bitstream value for the next frame according to the real-time bitrate and the target bitrate; For an intra refresh frame, based on the expected bitstream value, determine the number of columns of macroblocks with intra refresh included, and perform intra refresh.

9. An electronic device, characterized in that, It includes a processor and a memory, and at least one computer instruction is stored in the memory, and the instruction is loaded and executed by the processor to execute the steps performed in any one of the methods of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, At least one computer instruction is stored in the storage medium, and the instruction is loaded and executed by the processor to implement the steps performed in any one of the methods of claims 1 to 7.

Images