A display method of an image frame, a terminal device and a storage medium
Patent Information
- Application Number
- CN202610411579.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-30
- Publication Date
- 2026-06-12
AI Technical Summary
In the H.264 video coding standard, the timing of decoded frame output depends on the standard maximum reordered frame number. When this number is missing, the system degenerates into DPB-based management, which causes decoded frames to remain in the buffer for a longer period of time and cannot be released in time in the event of frame loss or bitstream abnormalities, resulting in delays in image frame display.
By acquiring the target frame count sequence, calculating the current out-of-order value, and dynamically updating the initial predicted maximum out-of-order value when the maximum reordered frame count is missing, the timing of image frame display is determined to ensure that decoded image frames are output in a timely manner according to the display order.
In the absence of a standard maximum number of reordered frames, the timing of image frame display is determined based on the dynamically calculated out-of-order value, avoiding blind waiting of decoded frames in the buffer, solving the problem of decoded frame output delay, and ensuring that image frames are displayed in order and in a timely manner.
Smart Images
Figure CN122205151A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of video display technology, and in particular relates to a method for displaying image frames, a terminal device, and a storage medium. Background Technology
[0002] H.264 is one of the mainstream standards widely used in video encoding and transmission. Its decoding process involves the management of the Decoded Picture Buffer (DPB). Generally, the decoded images stored in the DPB need to be output to the Output Picture Buffer (OPB) in a specific output order for subsequent display or processing. However, in practical engineering applications, the decoded frames output by the decoder usually need to go through a post-processing module before entering the OPB. Post-processing ensures that the data output timing of the OPB and the DPB do not interfere with each other.
[0003] Currently, the timing of decoded frame output mainly depends on the set standard maximum reordering frame count, controlling the output timing by limiting the number of image reordering frames. If the standard maximum reordering frame count is missing in the bitstream, the decoder cannot know the maximum number of reordering frames allowed in the bitstream, and the system can only degenerate into a management strategy based on the DPB size. However, the DPB size is usually large, which leads to a longer residence time of decoded frames in the buffer and a delay in decoded frame output. Summary of the Invention
[0004] This application provides a method for displaying image frames, a terminal device, and a storage medium, which can solve the problem of delayed output of decoded frames.
[0005] In a first aspect, embodiments of this application provide a method for displaying an image frame, comprising: In response to obtaining the decoded frame of the current image frame, a target frame counting sequence is obtained, wherein the target frame counting sequence contains image sequence count values of decoded image frames, and each of the image sequence count values is arranged in ascending order, and the decoded image frames include the current image frame; In response to the fact that the current image frame is not an instantaneous decoded refresh frame, a current out-of-order value is calculated based on the maximum index value of the target frame counting sequence and the position index number of the current image frame in the target frame counting sequence, wherein the current out-of-order value represents the number of out-of-order frames that exist in the decoded frames preceding the current image frame; If the current out-of-order value meets the preset conditions and the standard maximum reordering frame count does not exist, the currently stored initial predicted maximum out-of-order value is updated to the current out-of-order value to obtain the target predicted maximum out-of-order value. The preset conditions include the current out-of-order value not being equal to the preset maximum buffer value and the current out-of-order value being greater than the initial predicted maximum out-of-order value. The standard maximum reordering frame count is the maximum number of reordering frames allowed by the bitstream set in the image decoder. If the total number of decoded image frames is greater than the target predicted maximum out-of-order value, the target image frame is displayed. The target image frame is the decoded image frame corresponding to the minimum value among all the image sequence count values.
[0006] In this application, an initial predicted maximum out-of-order value is preset. This initial predicted maximum out-of-order value is dynamically changing. Even when there is no standard maximum reordering frame count in the system, the display timing of an image frame can still be determined based on the initial predicted maximum out-of-order value. Specifically, after obtaining the decoded frame of the current image frame, a target frame count sequence storing the image sequence count values of the decoded image frames is obtained. Then, based on the maximum index value of the target frame count sequence and the position index number of the current image frame in the target frame count sequence, a current out-of-order value representing the degree of out-of-order in the current bitstream is obtained. Since the image sequence count value of each decoded image frame changes, the current out-of-order value calculated based on the image sequence count value of the current image frame is dynamically changing, ensuring that the calculated current out-of-order value matches the actual situation of the current image frame. When there is no standard maximum reordering frame count and the current out-of-order value is greater than the initial predicted maximum out-of-order value, the currently stored initial predicted maximum out-of-order value is updated to the current out-of-order value to obtain the target predicted maximum out-of-order value. In other words, the calculated current out-of-order value replaces the standard maximum reordering frame count, and the image frame is then displayed based on the current out-of-order value. This application uses the calculated current out-of-order value as the timing condition for displaying image frames, thus establishing a new trigger condition for displaying image frames. Even if there is no standard maximum number of reordered frames in the system, as long as the total number of decoded image frames exceeds the current out-of-order level (current out-of-order value) in the bitstream, it is considered that the frame that appears earlier in the display order can be safely output. This application focuses on the timing reordering requirements of the current bitstream, so that decoded image frames do not have to wait blindly in the buffer. As long as the display trigger condition is met, the earliest image frame can be displayed immediately, solving the problem of extended waiting time for decoded frames.
[0007] Secondly, embodiments of this application provide an image frame display device, comprising: The data acquisition module is used to acquire a target frame count sequence in response to obtaining the decoded frame of the current image frame, wherein the target frame count sequence contains image sequence count values of decoded image frames, and each of the image sequence count values is arranged in ascending order, and the decoded image frames include the current image frame; The out-of-order calculation module is used to calculate the current out-of-order value based on the maximum index value of the target frame counting sequence and the position index number of the current image frame in the target frame counting sequence in response to the current image frame not being an instantaneous decoded refresh frame. The current out-of-order value represents the number of out-of-order frames that exist in the decoded frames preceding the current image frame. The data update module is used to update the currently stored initial predicted maximum out-of-order value to the current out-of-order value if the current out-of-order value meets the preset conditions and the standard maximum reordering frame number does not exist, thereby obtaining the target predicted maximum out-of-order value. The preset conditions include the current out-of-order value not being equal to the preset maximum buffer value and the current out-of-order value being greater than the initial predicted maximum out-of-order value. The standard maximum reordering frame number is the maximum number of reordering frames allowed by the bitstream set in the image decoder. The image output module is used to display the target image frame if the total number of cached decoded image frames is greater than the target predicted maximum out-of-order value. The target image frame is the decoded image frame corresponding to the minimum value among all the image sequence count values.
[0008] Thirdly, embodiments of this application provide a terminal device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for displaying image frames as described in any of the first aspects above.
[0009] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method for displaying an image frame as described in any of the first aspects above.
[0010] Fifthly, embodiments of this application provide a computer program product that, when run on a terminal device, causes the terminal device to execute the image frame display method described in any of the first aspects above. Attached Figure Description
[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0012] Figure 1 This is a schematic diagram of the structure for outputting image frames according to an embodiment of this application; Figure 2 This is a flowchart illustrating an embodiment of the initialization and IDR frame output method provided in this application; Figure 3 This is a flowchart illustrating a method for determining the degree of disorder in an image frame according to an embodiment of this application; Figure 4 This is a schematic flowchart of an image frame output method provided in an embodiment of this application; Figure 5 This is a schematic flowchart of an image frame display method provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of an image frame display device provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation
[0013] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.
[0014] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.
[0015] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0016] Video streams typically need to be decoded by a decoder before display. H.264 is one of the mainstream standards widely used in video encoding and transmission. Its decoding process includes: using a decoder to decode the video stream, and placing the decoded frames into a Decoded Picture Buffer (DPB) for buffering. Generally, the decoded images stored in the DPB need to be output to the Output Picture Buffer (OPB) in a specific output order for subsequent display or processing. However, in practical engineering applications, the decoded frames output by the decoder usually need to go through a post-processing module before entering the OPB. Specifically, for example... Figure 1 The diagram shows the process of outputting image frames.
[0017] Currently, there are two main ways to output image frames: one is based on the DPB size, where the first decoded image frame is output and displayed after the DPB buffer is full; the other is based on the standard maximum reordering frame count (max_num_reoder_frames), which is the maximum number of reordering frames allowed in the bitstream set in the image decoder. In other words, because the decoding order and display order of the images are inconsistent, there may be out-of-order frames in the decoded frames. max_num_reoder_frames defines the tolerable level of out-of-order; once the number of decoded frames exceeds max_num_reoder_frames, an image frame can be output for display.
[0018] In existing systems, not all system semantics store `max_num_reoder_frames`. If the `max_num_reoder_frames` semantic does not exist, the system will automatically output images according to the DPB size. Since the DPB buffer is generally large, the system needs to buffer a large number of image frames before outputting, resulting in prolonged waiting time for already decoded image frames in the buffer and delayed image frame output. Furthermore, in the case of frame loss or bitstream anomalies, the images before the frame loss cannot be released in time, causing image frame display delays.
[0019] Based on the above problems, this application proposes a method for displaying image frames, and the method of this application will be described in detail below.
[0020] First, refer to Figure 2 The flowchart shown illustrates that the method of this application includes: S11. After decoding the current image frame to obtain the decoded frame of the current image frame, determine whether the current image frame is an Instantaneous Decoding Refresh (IDR) frame.
[0021] S12, if the current image frame is an IDR frame, obtain the frame count sequence (pocs[ ]) stored at the current time, initialize the frame count sequence stored at the current time, and obtain the initialized frame count sequence. In this application, the initialized frame count sequence is denoted as the candidate frame count sequence (pocs[ ]).
[0022] In this embodiment, initializing the frame count sequence stored at the current time can be done by setting the values of each storage bit of the frame count sequence to the initial value. For example, the initial value can be -1 or -2, because the image order count (poc) of an image frame cannot be -1 or -2. When the poc value is -1, it indicates an invalid poc.
[0023] The frame count sequence (pocs[ ]) stores the POC values of the decoded image frames in ascending order, and the last storage bit in the frame count sequence is placed first, that is, the POC values are placed from the last storage bit in the frame count sequence to the first storage bit; the total number of storage bits in the frame count sequence (i.e. the total number of storage bits) can be set in advance, for example, the total number of storage bits in the frame count sequence can be 16 or 15, etc.
[0024] For example, if the total number of storage bits in the frame count sequence is 16, and there is currently an image frame with a POC value of 3, then place 3 in the last storage bit of the frame count sequence.
[0025] In another implementation, if the current image frame is not an IDR frame, the following step S13 is executed directly.
[0026] S13, determine whether the system has a standard maximum number of reordered frames (max_num_reoder_frames).
[0027] Specifically, if the bitstream_restrictopn_flag is the first character (e.g., 1), then it is determined that max_num_reoder_frames exist in the system; if the bitstream_restrictopn_flag is the second character (e.g., 0), then it is determined that max_num_reoder_frames do not exist in the system.
[0028] S14. If it is determined that max_num_reoder_frames exists, obtain the original maximum out-of-order value (b_frames), determine the maximum value between max_num_reoder_frames and b_frames, update b_frames to the maximum value between max_num_reoder_frames and b_frames, and obtain the updated original maximum out-of-order value. In this application, the updated original maximum out-of-order value is recorded as the initial maximum out-of-order value (b_frames).
[0029] In this embodiment, b_frames represents the maximum number of out-of-order image frames that may occur during decoding, used to record the maximum possible number of out-of-order frames. This value changes dynamically during the selection of output frames. The initial value can be set to 0, and b_frames changes continuously as more and more image frames are decoded. The b_frames at the current time is the b_frames that were finally stored after the previous image frame was processed using the method of this application.
[0030] Since b_frames is calculated, whether the final image frame is output depends on the size of b_frames. In this application, it does not depend on max_num_reoder_frames. Normally, b_frames is less than or equal to max_num_reoder_frames. If b_frames is too small, it will cause the final displayed image frames to be disordered. In order to ensure that the image frame output order is correct, b_frames is taken as the larger value between b_frames and max_num_reoder_frames.
[0031] In another implementation, if it is determined that max_num_reoder_frames do not exist, the procedure of step S15 below can be performed.
[0032] S15, insert the image sequence count value of the current image frame into the candidate frame count sequence to obtain the target frame count sequence (pocs[ ]).
[0033] In this embodiment, if the current image frame is an Instant Decoding Refresh (IDR) frame, the frame count sequence (pocs[]) is initialized, so the candidate frame count sequence (pocs[]) is the frame count sequence initialized from the frame count sequence stored at the current time. When the current image frame is not an IDR frame, the stored frame count sequence is not initialized, so the candidate frame count sequence is the frame count sequence stored at the current time.
[0034] In this embodiment, when the image sequence count value (poc) of the current image frame is inserted into the candidate frame count sequence (pocs[ ]), the insertion is performed according to the size of the poc value. If there is a poc value in the candidate frame count sequence that is the same as the poc value of the current image frame, the poc value of the current image frame is inserted after the same poc value. After obtaining the target frame count sequence (pocs[ ]), the position index number of the poc value of the current image frame in the target frame count sequence is recorded. When indexing the poc value in the frame count sequence, the index can start from position index number 0. For example, the position index number of the poc value in the first storage position is 0. If there are 16 storage positions in the frame count sequence, the position index number of the poc value in the 16th storage position is 15.
[0035] For example, if the candidate frame counting sequence is [… -1, -1, 3, 4, 5], and the current image frame's POC value is 4, then the current image frame's POC value is inserted after the 4 in [… -1, -1, 3, 4, 5], resulting in [… -1, -1, 3, 4, 4, 5], where the second 4 is the current image frame's POC.
[0036] In addition, inserting the image sequence count value of the current image frame into the candidate frame count sequence can also facilitate the observation of the disorder of image frames. For example, if there are two stored POC values, 1 and 3, and the POC value of the current image frame is 2, then 2 should be inserted between 1 and 3. At this time, there is one disordered frame after 2, and the disorder level is 1.
[0037] It should be noted that in the subsequent steps, pocs[ ] are all target frame count sequences.
[0038] S16, determine whether the current image frame is an IDR frame.
[0039] S17. If the current image frame is an IDR frame, display all the decoded image frames in order of ascending value of the image sequence count in the target frame counting sequence.
[0040] Specifically, the decoded image frames are placed in a buffer image sequence (delayed_pics[]), and the image frames in the buffer image sequence are placed in the order of decoding. The size of delayed_pics[] can be set as needed.
[0041] All images in delayed_pics[] are placed into the output image sequence (out_pics[]) in ascending order of their PoC values to output all decoded image frames. The image frames in out_pics[] are the image frames to be output. Delayed_pics[] is cleared, last_outputed_poc is set to 0, and the current frame is placed into out_pics[] to ensure that the current image frame can also be output and displayed. The process ends. last_outputed_poc is the PoC value of the previous output image, and its initial value can be -1.
[0042] S18, if the current image frame is not an IDR frame, calculate the current disorder value based on the maximum index value of the target frame counting sequence and the position index number of the current image frame in the target frame counting sequence.
[0043] The current out-of-order value (out_of_order) represents the number of out-of-order frames present in the decoded frames preceding the current image frame.
[0044] Specifically, the current disordered value is obtained by subtracting the position index number of the current image frame in the target frame count sequence from the maximum index value of the target frame count sequence (pocs[ ]).
[0045] For example, if there are 16 storage bits in the target frame counting sequence, then the maximum index value is 15. The position index number of the current image frame's POC value in the target frame counting sequence is 4. Then 15-4=11, and the current out-of-order value is 11.
[0046] Please continue to refer to the following. Figure 3 The process after step S18 will be explained further.
[0047] S19, determine whether the current image frame is a bi-predictive coded frame (B-frame).
[0048] S20, if it is determined that the current image frame is a B-frame, determine whether the current image frame is a non-reference frame and whether max_num_reoder_frames does not exist in the system.
[0049] In this embodiment, the method for determining whether max_num_reoder_frames exists is described in step S13 above, and will not be repeated here.
[0050] A non-reference frame is an image frame that is not used as a reference by other frames.
[0051] S21. If the current image frame is a non-reference frame and there are no max_num_reoder_frames in the system, determine whether the current out-of-order value is 0.
[0052] S22, if the current out-of-order value is 0, update the current out-of-order value to the first preset value, and the updated current out-of-order value is the target out-of-order value.
[0053] In this embodiment, since out-of-order frames may exist in the bitstream when B-frames are present, and the number of out-of-order frames is generally greater than or equal to 2 when the B-frame is a non-reference frame, even if out_of_order is 0, frame loss may still occur, causing out_of_order to be 0. To make the value of out_of_order relatively accurate, when the current image frame is a B-frame, out_of_order is corrected to a non-zero integer. The first preset value can be set to 2; however, in actual use, the first preset value can also be set to other values, which is not restricted here. The first preset value is greater than or equal to 2.
[0054] S23, if the current out-of-order value is not 0, add the second preset value to the current out-of-order value to obtain the target out-of-order value.
[0055] In this embodiment, the current out-of-order value is not 0, so the out-of-order calculated in step S18 may be 1. When the B frame is a non-reference frame, the number of out-of-order frames is generally greater than or equal to 2. Therefore, to make out-of-order more accurate, a second preset value can be added to out-of-order. The second preset value can be 1, etc., and is not limited here. The second preset value is a positive integer greater than or equal to 1.
[0056] After steps S22 and S23 above, the following step S26 is performed.
[0057] S24. If it is determined that the current image frame is not a B frame, determine whether there is an image sequence count value in the (i-1)th storage bit of the target frame counting sequence (pocs[ ]) and whether the difference between the image sequence count value (pocs[i]) in the i-th storage bit and the image sequence count value (pocs[i-1]) in the (i-1)th storage bit is greater than the fourth preset value.
[0058] In this embodiment, the i-th storage bit is the last storage bit in the target frame counting sequence. The image sequence count values of the decoded image frames are stored sequentially from the last storage bit to the first storage bit in the target frame counting sequence. If there is an image sequence count value in the (i-1)-th storage bit, then there is also an image sequence count value in the i-th storage bit. The fourth preset value can be 2, etc., and is not limited here.
[0059] S25, if the (i-1)th storage bit of the target frame counting sequence contains an image sequence count value, and the difference between the image sequence count value in the i-th storage bit and the image sequence count value in the (i-1)th storage bit is greater than the fourth preset value, the maximum value between out_of_order and the third preset value is determined as the target out-of-order value (out_of_order).
[0060] Specifically, if the (i-1)th storage bit of the target frame counting sequence contains an image sequence count value, and the difference between the image sequence count value in the ith storage bit and the image sequence count value in the (i-1)th storage bit is greater than the fourth preset value, it indicates that there are out-of-order frames or that B frames in the playback order have not been decoded (i.e., frame dropping).
[0061] Compare the current disordered value with the third preset value, update the current disordered value to the maximum of the current disordered value and the third preset value, and the updated current disordered value is the target disordered value. The third preset value can be 1, etc., and is not restricted here.
[0062] In another approach, if the (i-1)th storage bit of the target frame counting sequence does not contain an image sequence count value, and / or the difference between the image sequence count value at the i-th storage bit and the image sequence count value at the (i-1)th storage bit is not greater than a fourth preset value, the following step S26 is executed.
[0063] In another approach, after step S20, if the current image frame is a reference frame and / or max_num_reoder_frames exist in the system, then step S25 is performed to determine the maximum value between the current disordered value and the third preset value as the target disordered value.
[0064] S26, determine whether the target out-of-order value is equal to the preset maximum buffer value.
[0065] In this embodiment, the maximum buffer value can be preset, such as 16 or 17, and there is no limitation here.
[0066] S27. If the target out-of-order value is equal to the maximum buffer value, display all image frames decoded before the current image frame in sequence.
[0067] Specifically, all images in delayed_pics[] are placed into the output image sequence (out_pics[]) in ascending order of their POC values to output all decoded image frames before the current image frame. Delayed_pics[] and POCs[] are then cleared. The POC value of the current image frame is inserted into the last storage bit of POCs[], and last_outputed_poc is set to -1.
[0068] If the target out-of-order value is equal to the preset maximum buffer value, it indicates that a serious frame loss has occurred and the buffered decoded image frames need to be output sequentially. Since there may be frame loss at this time, the current image frame may not be output immediately. Therefore, the image frames decoded before the current image frame are output at this time, and the current image frame is not output.
[0069] In another implementation, if the target out-of-order value is not equal to the maximum buffer value, the following step S28 is executed.
[0070] S28, store the current image frame in delayed_pics[ ]. delayed_pics[ ] is stored in the order in which decoding is completed.
[0071] Please continue reading below. Figure 4 The flowchart shown below will continue to explain the process after step S28.
[0072] S29. Find the image frame with the smallest POC value in delayed_pics[ ] that can be displayed at the current time. Select the image frame as the output frame (out). In this application, the output frame is the target image frame.
[0073] S30, determine if there is an output frame (out).
[0074] In this embodiment, since the image frame with the smallest POC value in delayed_pics[] may not have been fully decoded, or may need to be used as a reference frame for other frames, the image frame with the smallest POC value in delayed_pics[] may not be suitable for output at the current time. Therefore, it is necessary to determine whether there is an output frame.
[0075] S31, if an output frame exists, determine whether the poc value (out.poc) of the output frame is less than or equal to the image sequence count value (last_outputed_poc) of the previous image frame, where the previous image frame is the last image frame displayed before the current time.
[0076] In another implementation, the process terminates if no output frame exists.
[0077] S32, if the poc value of the output frame is less than or equal to last_outputed_poc, all image frames decoded before the current image frame will be displayed sequentially.
[0078] In this embodiment, since images are generally displayed in ascending order of their PoC values, the PoC value of a later-displayed image frame is greater than that of a previously displayed image frame. If the PoC value of an output frame is less than or equal to last_outputed_poc, it indicates that an error has occurred in the display of the current image frame.
[0079] Specifically, except for the output frame, all images in delayed_pics[] are placed into the output image sequence (out_pics[]) in ascending order of their POC values; the delayed_pics[] output frame is cleared; the POCs[] is cleared, and the POC value of the output frame (i.e., out.poc) is inserted into the last storage position of the POCs[], and last_outputed_poc is set to -1.
[0080] The process ends after step S32.
[0081] S33. If the poc value of the output frame is greater than last_outputed_poc, determine whether the target out-of-order value (out_of_order) is greater than the initial predicted maximum out-of-order value (b_frames) and whether max_num_reoder_frames does not exist in the system.
[0082] S34. If the target out-of-order value (out_of_order) is greater than the initial predicted maximum out-of-order value (b_frames) and there is no max_num_reoder_frames in the system, update the currently stored initial predicted maximum out-of-order value to the target out-of-order value to obtain the updated b_frames. The updated b_frames is recorded as the target predicted maximum out-of-order value (b_frames).
[0083] In another implementation, if the target out-of-order value is less than or equal to the initial predicted maximum out-of-order value (b_frames) and / or max_num_reoder_frames exists in the system, then the process of step S35 below is executed directly.
[0084] S35, determine whether the total number of cached decoded image frames (pics) is greater than the target prediction maximum out-of-order value (b_frames), and / or whether the number of cached image frames (fullness) in the DPB is greater than the DPB size.
[0085] S36. If pics is greater than the target prediction maximum out-of-order value (b_frames), and / or fullness is greater than the DPB size, display the output frame.
[0086] In this embodiment, if pics is greater than the target prediction maximum out-of-order value (b_frames) and / or fullness is greater than the DPB size, it means that the number of frames decoded in display order is sufficient to display the found output frames as output.
[0087] Specifically, delete the output frame (out) from delayed_pics[ ], put the output frame into out_pics[ ], pics--, and at the same time record the poc value of out to last_outputed_poc, return to step S29 above, and continue to select output frames.
[0088] In another implementation, the process ends if pics is less than or equal to the target prediction maximum out-of-order value (b_frames) and fullness is less than or equal to the DPB size.
[0089] In another possible implementation, such as Figure 5 As shown, the method of this application may also include: S101, in response to obtaining the decoded frame of the current image frame, a target frame counting sequence is obtained, wherein the target frame counting sequence contains image sequence count values of decoded image frames, and each of the image sequence count values is arranged in ascending order, and the decoded image frames include the current image frame.
[0090] S102, in response to the fact that the current image frame is not an instantaneous decoded refresh frame, a current out-of-order value is calculated based on the maximum index value of the target frame counting sequence and the position index number of the current image frame in the target frame counting sequence, wherein the current out-of-order value represents the number of out-of-order frames that exist in the decoded frames preceding the current image frame.
[0091] In this embodiment, the current disordered value is obtained by subtracting the position index number of the current image frame in the target frame counting sequence from the maximum index value of the target frame counting sequence.
[0092] S103, if the current out-of-order value meets the preset conditions and the standard maximum reordering frame count does not exist, update the currently stored initial predicted maximum out-of-order value to the current out-of-order value to obtain the target predicted maximum out-of-order value. The preset conditions include the current out-of-order value not being equal to the preset maximum buffer value and the current out-of-order value being greater than the initial predicted maximum out-of-order value. The standard maximum reordering frame count is the maximum number of reordering frames allowed in the bitstream set in the image decoder.
[0093] S104, if the total number of cached decoded image frames is greater than the target predicted maximum out-of-order value, display the target image frame, which is the decoded image frame corresponding to the minimum value among all the image sequence count values.
[0094] In one possible implementation, the process of step S101 may include: S1011, in response to obtaining the decoded frame of the current image frame, if the standard maximum reordering frame number does not exist, the image sequence count value of the current image frame is inserted into the candidate frame count sequence to obtain the target frame count sequence. Wherein, when the current image frame is an instant-decode refresh frame, the candidate frame count sequence is the frame count sequence initialized with the frame count sequence stored at the current time; when the current image frame is not an instant-decode refresh frame, the candidate frame count sequence is the frame count sequence stored at the current time.
[0095] S1012, if the standard maximum reordering frame number exists, obtain the original prediction maximum out-of-order value stored at the current time, wherein the original prediction maximum out-of-order value is the maximum number of image frames whose order is disordered during decoding.
[0096] S1013, based on the standard maximum reordering frame number, update the original predicted maximum out-of-order value to obtain the initial predicted maximum out-of-order value.
[0097] S1014, insert the image sequence count value of the current image frame into the current frame count sequence to obtain the target frame count sequence.
[0098] In one possible implementation, the process of step S1013 may include: If the original maximum out-of-order value is greater than or equal to the standard maximum reordering frame number, the original maximum out-of-order value remains unchanged, and the original maximum out-of-order value is the initial maximum out-of-order value.
[0099] If the original maximum out-of-order value is less than the standard maximum reordering frame count, the original maximum out-of-order value is updated to the standard maximum reordering frame count, and the updated original maximum out-of-order value is the initial maximum out-of-order value.
[0100] In one possible implementation, after step S101, the above method may further include: If the current image frame is an instant-decoded refresh frame, all the decoded image frames are displayed sequentially in ascending order of the image sequence count values in the target frame counting sequence.
[0101] In one possible implementation, after step S102, the above method may further include: In response to the current image frame being a bidirectional predictive coded frame, the current out-of-order value is updated to obtain a target out-of-order value. The target out-of-order value is greater than or equal to a preset minimum out-of-order value. The minimum out-of-order value is the minimum out-of-order value present in the decoded image frame when the bidirectional predictive coded frame is present, and the minimum out-of-order value is greater than 0.
[0102] Accordingly, the implementation process of step S103 above may include: if the target out-of-order value meets the preset conditions and the standard maximum reordering frame number does not exist, the currently stored initial predicted maximum out-of-order value is updated to the target out-of-order value to obtain the target predicted maximum out-of-order value, wherein the preset conditions include the target out-of-order value not being equal to the preset maximum buffer value and the target out-of-order value being greater than the initial predicted maximum out-of-order value.
[0103] In one possible implementation, step S102 may include: If the current image frame is a non-reference frame and there is no standard maximum reordering frame number, determine whether the current disorder value is 0.
[0104] If the current disorder value is 0, the current disorder value is updated to a first preset value, and the updated current disorder value is the target disorder value. The minimum disorder value includes the first preset value.
[0105] If the current disordered value is not 0, add the second preset value to the current disordered value to obtain the target disordered value.
[0106] If the current image frame is a reference frame and / or there exists a standard maximum number of reordered frames, determine the maximum value between the current disordered value and the third preset value, update the current disordered value to the maximum value, and the updated current disordered value is the target disordered value, wherein the minimum disordered value includes the third preset value.
[0107] In one possible implementation, after updating the current out-of-order value in response to the current image frame being a bidirectional predictive coded frame to obtain the target out-of-order value, the method further includes: If the target out-of-order value is equal to the preset maximum buffer value, all image frames decoded before the current image frame will be displayed sequentially.
[0108] In one possible implementation, step S103 may include: S1031, if the target out-of-order value is not equal to the preset maximum buffer value, and the target image frame exists in the decoded image frames, determine whether the image sequence count value of the target image frame is greater than the image sequence count value of the previous image frame, wherein the previous image frame is the last image frame displayed before the current time.
[0109] S1032, if the image sequence count value of the target image frame is greater than the image sequence count value of the previous image frame, determine whether the target disorder value is greater than the stored initial predicted maximum disorder value.
[0110] S1033, if the target out-of-order value is greater than the initial predicted maximum out-of-order value and there is no standard maximum reordering frame number, update the currently stored initial predicted maximum out-of-order value to the target out-of-order value to obtain the target predicted maximum out-of-order value.
[0111] In one possible implementation, after step S102, the above method may further include: If the current image frame is not the bidirectional predictive coded frame, determine whether there is an image sequence count value in the (i-1)th storage bit of the target frame counting sequence, and whether the difference between the image sequence count value in the ith storage bit and the image sequence count value in the (i-1)th storage bit is greater than a fourth preset value; wherein, the image sequence count value of the decoded image frame is stored sequentially from the last storage bit to the first storage bit of the target frame counting sequence, and the ith storage bit is the last storage bit in the target frame counting sequence; If the (i-1)th storage bit of the target frame counting sequence contains an image sequence count value, and the difference is greater than the fourth preset value, the maximum value between the current disordered value and the third preset value is determined as the target disordered value.
[0112] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0113] Corresponding to the image frame display method described in the above embodiments, Figure 6 This diagram illustrates a structural block diagram of an image frame display device provided in an embodiment of this application. For ease of explanation, only the parts related to the embodiments of this application are shown.
[0114] Reference Figure 6 The device 300 may include: a data acquisition module 310, a randomized calculation module 320, a data update module 330, and an image output module 340.
[0115] The data acquisition module 310 is used to acquire a target frame count sequence in response to obtaining the decoded frame of the current image frame. The target frame count sequence contains image sequence count values of decoded image frames, and each image sequence count value is arranged in ascending order. The decoded image frames include the current image frame. The out-of-order calculation module 320 is used to calculate the current out-of-order value based on the maximum index value of the target frame counting sequence and the position index number of the current image frame in the target frame counting sequence in response to the current image frame not being an instantaneous decoded refresh frame. The current out-of-order value represents the number of out-of-order frames that exist in the decoded frames preceding the current image frame. The data update module 330 is used to update the currently stored initial predicted maximum out-of-order value to the current out-of-order value if the current out-of-order value meets the preset conditions and the standard maximum reordering frame number does not exist, thereby obtaining the target predicted maximum out-of-order value. The preset conditions include the current out-of-order value not being equal to the preset maximum buffer value and the current out-of-order value being greater than the initial predicted maximum out-of-order value. The standard maximum reordering frame number is the maximum reordering frame number allowed by the bitstream set in the image decoder. The image output module 340 is used to display a target image frame if the total number of cached decoded image frames is greater than the target predicted maximum out-of-order value. The target image frame is the decoded image frame corresponding to the minimum value among all the image sequence count values.
[0116] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.
[0117] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0118] This application also provides a terminal device, see [link to relevant documentation] Figure 7 The terminal device 400 may include: at least one processor 410, a memory 420, and a computer program stored in the memory 420 and executable on the at least one processor 410. When the processor 410 executes the computer program, it implements the steps in any of the above method embodiments, for example... Figure 5 Steps S101 to S104 in the illustrated embodiment. Alternatively, when the processor 410 executes the computer program, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 6 The functions of the data acquisition module 310 to the image output module 340 are shown.
[0119] For example, a computer program may be divided into one or more modules / units, one or more of which are stored in memory 420 and executed by processor 410 to complete this application. The one or more modules / units may be a series of computer program segments capable of performing a specific function, which are used to describe the execution process of the computer program in terminal device 400.
[0120] Those skilled in the art will understand that Figure 7 This is merely an example of a terminal device and does not constitute a limitation on the terminal device. It may include more or fewer components than shown, or combine certain components, or different components, such as input / output devices, network access devices, buses, etc.
[0121] The processor 410 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.
[0122] The memory 420 can be an internal storage unit of the terminal device or an external storage device, such as a plug-in hard drive, a smart media card (SMC), a secure digital (SD) card, or a flash card. The memory 420 is used to store the computer program and other programs and data required by the terminal device. The memory 420 can also be used to temporarily store data that has been output or will be output.
[0123] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.
[0124] The image frame display method provided in this application embodiment can be applied to terminal devices such as computers, tablets, laptops, netbooks, and personal digital assistants (PDAs). This application embodiment does not impose any restrictions on the specific type of terminal device.
[0125] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0126] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0127] In the embodiments provided in this application, it should be understood that the disclosed terminal devices, apparatuses, and methods can be implemented in other ways. For example, the terminal device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, apparatuses, or units, and may be electrical, mechanical, or other forms.
[0128] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0129] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0130] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by one or more processors, it can implement the steps of the various method embodiments described above.
[0131] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by one or more processors, it can implement the steps of the various method embodiments described above.
[0132] Similarly, as a computer program product, when the computer program product is run on a terminal device, it enables the terminal device to implement the steps in the above-described method embodiments.
[0133] The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.
[0134] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.
Claims
1. A method for displaying an image frame, characterized in that, include: In response to obtaining the decoded frame of the current image frame, a target frame counting sequence is obtained, wherein the target frame counting sequence contains image sequence count values of decoded image frames, and each of the image sequence count values is arranged in ascending order, and the decoded image frames include the current image frame; In response to the fact that the current image frame is not an instantaneous decoded refresh frame, a current out-of-order value is calculated based on the maximum index value of the target frame counting sequence and the position index number of the current image frame in the target frame counting sequence, wherein the current out-of-order value represents the number of out-of-order frames that exist in the decoded frames preceding the current image frame; If the current out-of-order value meets the preset conditions and the standard maximum reordering frame count does not exist, the currently stored initial predicted maximum out-of-order value is updated to the current out-of-order value to obtain the target predicted maximum out-of-order value. The preset conditions include the current out-of-order value not being equal to the preset maximum buffer value and the current out-of-order value being greater than the initial predicted maximum out-of-order value. The standard maximum reordering frame count is the maximum number of reordering frames allowed by the bitstream set in the image decoder. If the total number of cached decoded image frames is greater than the target predicted maximum out-of-order value, the target image frame is displayed. The target image frame is the decoded image frame corresponding to the minimum value among all the image sequence count values.
2. The image frame display method as described in claim 1, characterized in that, The step of obtaining the target frame count sequence in response to obtaining the decoded frame of the current image frame includes: In response to obtaining the decoded frame of the current image frame, if the standard maximum reordering frame count does not exist, the image sequence count value of the current image frame is inserted into the candidate frame count sequence to obtain the target frame count sequence. Wherein, when the current image frame is an instant-decode refresh frame, the candidate frame count sequence is a frame count sequence initialized with the frame count sequence stored at the current time; when the current image frame is not an instant-decode refresh frame, the candidate frame count sequence is the frame count sequence stored at the current time. If the standard maximum reordering frame number exists, obtain the original prediction maximum out-of-order value stored at the current time, wherein the original prediction maximum out-of-order value is the maximum number of image frames that are out of order during decoding. Based on the standard maximum reordering frame number, update the original predicted maximum out-of-order value to obtain the initial predicted maximum out-of-order value; The image sequence count value of the current image frame is inserted into the current frame count sequence to obtain the target frame count sequence.
3. The image frame display method as described in claim 2, characterized in that, The step of updating the original predicted maximum out-of-order value based on the standard maximum reorder frame number to obtain the initial predicted maximum out-of-order value includes: If the original maximum out-of-order value is greater than or equal to the standard maximum reordering frame number, the original maximum out-of-order value remains unchanged, and the original maximum out-of-order value is the initial maximum out-of-order value. If the original maximum out-of-order value is less than the standard maximum reordering frame count, the original maximum out-of-order value is updated to the standard maximum reordering frame count, and the updated original maximum out-of-order value is the initial maximum out-of-order value.
4. The method for displaying an image frame as described in claim 1, characterized in that, After obtaining the target frame count sequence, the method further includes: If the current image frame is an instant-decoded refresh frame, all the decoded image frames are displayed sequentially in ascending order of the image sequence count values in the target frame counting sequence.
5. The method for displaying an image frame as described in any one of claims 1 to 4, characterized in that, After calculating the current disorder value based on the maximum index value of the target frame counting sequence and the position index number of the current image frame in the target frame counting sequence, the method further includes: In response to the current image frame being a bidirectional predictive coded frame, the current out-of-order value is updated to obtain a target out-of-order value. The target out-of-order value is greater than or equal to a preset minimum out-of-order value. The minimum out-of-order value is the minimum out-of-order value present in the decoded image frame when the bidirectional predictive coded frame is present, and the minimum out-of-order value is greater than 0. Accordingly, if the current out-of-order value meets the preset conditions and the standard maximum reordering frame count does not exist, the currently stored initial predicted maximum out-of-order value is updated to the current out-of-order value to obtain the target predicted maximum out-of-order value, including: If the target out-of-order value meets the preset conditions and the standard maximum reordering frame count does not exist, the currently stored initial predicted maximum out-of-order value is updated to the target out-of-order value to obtain the target predicted maximum out-of-order value. The preset conditions include the target out-of-order value not being equal to the preset maximum buffer value and the target out-of-order value being greater than the initial predicted maximum out-of-order value.
6. The method for displaying an image frame as described in claim 5, characterized in that, The step of updating the current disordered value to obtain the target disordered value includes: If the current image frame is a non-reference frame and there is no standard maximum reordering frame number, determine whether the current disorder value is 0; If the current disorder value is 0, the current disorder value is updated to a first preset value, and the updated current disorder value is the target disorder value. The minimum disorder value includes the first preset value. If the current disordered value is not 0, add the second preset value to the current disordered value to obtain the target disordered value; If the current image frame is a reference frame and / or there exists a standard maximum number of reordered frames, determine the maximum value between the current disordered value and the third preset value, update the current disordered value to the maximum value, and the updated current disordered value is the target disordered value, wherein the minimum disordered value includes the third preset value.
7. The method for displaying an image frame as described in claim 5, characterized in that, If the target out-of-order value meets a preset condition and the standard maximum reordering frame count does not exist, the currently stored initial predicted maximum out-of-order value is updated to the target out-of-order value to obtain the target predicted maximum out-of-order value, including: If the target out-of-order value is not equal to the preset maximum buffer value, and the target image frame exists in the decoded image frames, determine whether the image sequence count value of the target image frame is greater than the image sequence count value of the previous image frame, wherein the previous image frame is the last image frame displayed before the current time. If the image sequence count value of the target image frame is greater than the image sequence count value of the previous image frame, determine whether the target disorder value is greater than the stored initial predicted maximum disorder value; If the target out-of-order value is greater than the initial predicted maximum out-of-order value and there is no standard maximum reordering frame count, the currently stored initial predicted maximum out-of-order value is updated to the target out-of-order value to obtain the target predicted maximum out-of-order value.
8. The method for displaying an image frame as described in claim 1, characterized in that, After calculating the current disorder value based on the maximum index value of the target frame counting sequence and the position index number of the current image frame in the target frame counting sequence, the method further includes: If the current image frame is not a bidirectional predictive coded frame, determine whether there is an image sequence count value in the (i-1)th storage bit of the target frame counting sequence, and whether the difference between the image sequence count value in the ith storage bit and the image sequence count value in the (i-1)th storage bit is greater than a fourth preset value; wherein, the image sequence count value of the decoded image frame is stored sequentially from the last storage bit to the first storage bit of the target frame counting sequence, and the ith storage bit is the last storage bit in the target frame counting sequence; If the (i-1)th storage bit of the target frame counting sequence contains an image sequence count value, and the difference is greater than the fourth preset value, the maximum value between the current disordered value and the third preset value is determined as the target disordered value.
9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the image frame display method as described in any one of claims 1 to 8.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the image frame display method as described in any one of claims 1 to 8.