Image display methods, devices and storage media

Abstract

This application discloses an image display method, device, and storage medium. The image display method includes: decoding each video frame in a target bitstream according to the decoding order, and parsing the display order of each video frame; comparing and analyzing the display order between the current decoded video frame and the previous decoded video frames to obtain a comparison result; if the comparison result indicates that the current decoded video frame needs to be updated with an output delay, then calculating the output display delay corresponding to the current decoded video frame based on the display order of the current decoded video frame and the display order of the previous decoded video frames; and performing output display processing on the current decoded video frame based on the output display delay corresponding to the current decoded video frame. This application improves the flexibility and accuracy of output display delay. While ensuring that the number of delayed output frames is minimized, a bitstream with a normal output order can be obtained, resulting in normal image display without issues such as screen tearing or stuttering, thus improving the user experience.

Description

Technical Field

[0001] This application relates to the field of video and image encoding and decoding technology, and in particular to an image display method, device and storage medium. Background Technology

[0002] Currently, in order to improve the compression rate during video and image bitstream encoding and decoding, bi-directional interpolated prediction frames (B-frames) or multi-reference frame encoding are often used. However, the delay coding problem is unavoidable, which leads to output display order problems in the images acquired by the user. If the output display order is incorrect, the displayed image content will be stuttered or have a distorted screen.

[0003] Therefore, ensuring that users can obtain normal bitstreams in real time and that the decoding library outputs normal bitstreams with low latency is an urgent problem that needs to be solved. Summary of the Invention

[0004] To address the aforementioned problems, this application provides at least one image display method, device, and storage medium.

[0005] The first aspect of this application provides an image display method, which includes: decoding each video frame in a target bitstream according to the decoding order, and parsing the display order of each video frame; comparing and analyzing the display order between the current decoded video frame and the previous decoded video frames to obtain a comparison result; wherein the decoding order of the previous decoded video frames is before the decoding order of the current decoded video frame; if the comparison result indicates that the current decoded video frame needs to be updated with an output delay, then calculating the output display delay corresponding to the current decoded video frame based on the display order of the current decoded video frame and the display order of the previous decoded video frames; wherein the output display delay represents the time required from the completion of decoding of the current decoded video frame to the output display; and performing output display processing on the current decoded video frame based on the output display delay corresponding to the current decoded video frame.

[0006] In one embodiment, a comparative analysis is performed on the display order between the current decoded video frame and the previous decoded video frame to obtain a comparison result. This includes: calculating the difference between the display order of the current decoded video frame and the display order of the previous decoded video frame to obtain a display order difference; obtaining the difference threshold corresponding to the current decoded video frame; if the display order difference is greater than the difference threshold, the comparison result indicates that the current decoded video frame needs to undergo an output delay update; if the display order difference is not greater than the difference threshold, the comparison result indicates that the current decoded video frame does not need to undergo an output delay update.

[0007] In one embodiment, obtaining the difference threshold corresponding to the current decoded video frame includes: classifying the display order of the current decoded video frame and the previous decoded video frame into odd and even numbers to obtain an odd and even classification result; if the odd and even classification result indicates that the display order of the current decoded video frame and the previous decoded video frame are both even numbers, then a first threshold value is selected as the difference threshold corresponding to the current decoded video frame; if the odd and even classification result indicates that there is an odd number in the display order of the current decoded video frame and the previous decoded video frame, then a second threshold value is selected as the difference threshold corresponding to the current decoded video frame.

[0008] In one embodiment, the output display delay corresponding to the current decoded video frame is calculated based on the display order of the current decoded video frame and the display order of the previous decoded video frames. This includes: inputting the display order difference into a first preset delay calculation formula to obtain a first initial output delay corresponding to the current decoded video frame; obtaining the output display delay corresponding to the previous decoded video frame to obtain the previous output delay; and selecting the maximum value from the first initial output delay and the previous output delay as the output display delay corresponding to the current decoded video frame.

[0009] In one embodiment, a comparative analysis is performed on the display order between the current decoded video frame and the previous decoded video frame to obtain a comparison result. This includes: statistically analyzing the trend of the display order of the current decoded video frame relative to the display order of the previous decoded video frame to obtain a display order change trend; if the display order change trend meets a preset condition, then the change difference between the current decoded video frame and the previous decoded video frame is calculated; if the change difference is greater than a difference threshold, the comparison result indicates that the current decoded video frame needs to undergo an output delay update; if the change difference is not greater than a difference threshold, the comparison result indicates that the current decoded video frame does not need to undergo an output delay update.

[0010] In one embodiment, if the display order change trend indicates that the display order of the current decoded video frame increases relative to the display order of the previous decoded video frames, it is determined that the display order change trend meets a preset condition; a maximum order flag value and a minimum order flag value are pre-stored; the change difference between the current decoded video frame and multiple previous decoded video frames is calculated, including: updating the display order of the current decoded video frame to the maximum order flag value, and updating the previous maximum order flag value to the minimum order flag value; calculating the difference between the maximum order flag value and the minimum order flag value to obtain the change difference between the current decoded video frame and the previous decoded video frames.

[0011] In one embodiment, the output display delay corresponding to the current decoded video frame is calculated based on the display order of the current decoded video frame and the display order of the previous decoded video frames. This includes: inputting the change difference into a second preset delay calculation formula to obtain a second initial output delay corresponding to the current decoded video frame; obtaining the output display delay corresponding to the previous decoded video frame to obtain the previous output delay; and selecting the maximum value from the second initial output delay and the previous output delay as the output display delay corresponding to the current decoded video frame.

[0012] In one embodiment, if a delayed output syntax exists in the target bitstream, the delayed output syntax is parsed to obtain the default output delay; the frame type of the current decoded video frame is parsed; if the current decoded video frame belongs to the target type frame, the current decoded video frame is either processed for immediate output or processed for output display according to the default output delay; if the current decoded video frame does not belong to the target type frame, the display order between the current decoded video frame and the previous decoded video frames is compared and analyzed to obtain a comparison result; if the comparison result indicates that the current decoded video frame needs to be updated with an output delay, the output display delay corresponding to the current decoded video frame is calculated based on the display order of the current decoded video frame and the display order of the previous decoded video frames; and the current decoded video frame is processed for output display based on the output display delay corresponding to the current decoded video frame.

[0013] A second aspect of this application provides an image display apparatus, comprising: a frame parsing module, configured to decode each video frame in a target bitstream according to the decoding order, and parse the display order of each video frame; a sequence analysis module, configured to compare and analyze the display order between the current decoded video frame and the previous decoded video frame to obtain a comparison result; wherein the decoding order of the previous decoded video frame is before the decoding order of the current decoded video frame; a delay calculation module, configured to calculate the output display delay corresponding to the current decoded video frame based on the display order of the current decoded video frame and the display order of the previous decoded video frames if the comparison result indicates that the current decoded video frame needs to be updated with an output delay; wherein the output display delay represents the time required from the completion of decoding of the current decoded video frame to its output display; and a display processing module, configured to perform output display processing on the current decoded video frame based on the output display delay corresponding to the current decoded video frame.

[0014] A third aspect of this application provides an electronic device, including a memory and a processor, wherein the processor is configured to execute program instructions stored in the memory to implement the above-described image display method.

[0015] The fourth aspect of this application provides a computer-readable storage medium having program instructions stored thereon, which, when executed by a processor, implement the above-described image display method.

[0016] The above scheme decodes each video frame in the target bitstream according to the decoding order and analyzes the display order of each video frame. It compares and analyzes the display order between the current decoded video frame and the previous decoded video frames to obtain the comparison result. If the comparison result shows that the current decoded video frame needs to be updated with an output delay, the output display delay corresponding to the current decoded video frame is calculated based on the display order of the current decoded video frame and the display order of the previous decoded video frames. Based on the output display delay corresponding to the current decoded video frame, the current decoded video frame is processed for output display to improve the flexibility and accuracy of the output display delay. Under the premise of minimizing the number of delayed output frames, a bitstream with a normal output order can be obtained, the screen is displayed normally, and there are no problems such as screen tearing or stuttering, thus improving the user experience.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0019] Figure 1 This is a schematic diagram illustrating an implementation environment related to an exemplary embodiment of the image display method of this application;

[0020] Figure 2 This is a flowchart illustrating an exemplary embodiment of the image display method of this application;

[0021] Figure 3 This is a schematic diagram illustrating, as shown in an exemplary embodiment of this application, that the video frames of the target bitstream need to undergo output delay updates;

[0022] Figure 4 This is a schematic diagram illustrating the need for output delay updates of video frames in the target bitstream, as shown in another exemplary embodiment of this application.

[0023] Figure 5 This is a block diagram illustrating an image display device according to an exemplary embodiment of this application;

[0024] Figure 6 This is a schematic diagram of the structure of an electronic device shown in an exemplary embodiment of this application;

[0025] Figure 7 This is a schematic diagram illustrating the structure of a computer-readable storage medium, as shown in an exemplary embodiment of this application. Detailed Implementation

[0026] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0027] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0028] In this document, the term "and / or" is merely a description of the association information of related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0029] The image display method provided in the embodiments of this application will be described below.

[0030] Please refer to Figure 1 The diagram illustrates an implementation environment provided in one embodiment of this application. This implementation environment may include a source data terminal 110 and a video playback terminal 120, which are interconnected.

[0031] The source data terminal 110 can be a network camera, smartphone, tablet, laptop, desktop computer, smartwatch, server, etc., but is not limited to these. The number of source data terminals 110 can be one or more.

[0032] The video playback terminal 120 may include a processing module for decoding and a display module that can provide display functions.

[0033] It should be noted that the processing module and the display module can be integrated into a single device, such as the video playback terminal 120, which can be a smartphone, tablet, laptop, desktop computer, etc., that simultaneously possesses decoding and image display functions. Alternatively, the processing module and the display module can be integrated into different devices, such as the processing module being a server, and the display module being a liquid crystal display (LCD), organic light emitting diode (OLED) display, plasma display, projector, micro-LED display, liquid crystal on silicon (LCoS), digital light processor (DLP), or any other type of display that communicates with the server. This application does not limit the implementation method of the video playback terminal 120.

[0034] In one example, source data terminal 110 generates encoded bitstream data and sends it to video playback terminal 120. Video playback terminal 120 receives the bitstream data sent by source data terminal 110, decodes the bitstream data through a processing module, and then outputs the decoded video frames to a display module for playback.

[0035] It should be noted that, Figure 1 The application scenarios can be various video service scenarios, such as video conferencing scenarios, video call scenarios, online education scenarios, remote tutoring scenarios, low-latency live streaming scenarios, cloud gaming scenarios, wireless screen projection scenarios, and wireless extended screen scenarios, etc. This application embodiment does not limit these.

[0036] Please see Figure 2 , Figure 2 This is a flowchart illustrating an exemplary embodiment of the image display method of this application. This image display method can be applied to… Figure 1 The implementation environment shown is specifically executed by the video playback terminal in that implementation environment. It should be understood that this method can also be applied to other exemplary implementation environments and specifically executed by devices in other implementation environments. This embodiment does not limit the implementation environment to which the method is applicable.

[0037] like Figure 2 As shown, the image display method includes at least steps S210 to S240, which are described in detail below:

[0038] Step S210: Decode each video frame in the target bitstream according to the decoding order, and parse the display order of each video frame.

[0039] Encoding refers to the process of converting a file in an original video format into a file in another video format using compression technology.

[0040] For example, the encoding process mainly includes intra prediction, interprediction, transform, quantization, and entropy encoding.

[0041] The decoding process is equivalent to the reverse process of the encoding process.

[0042] The encoding and decoding standards include, but are not limited to, H.261, H.263, H.264, and H.265.

[0043] Decode each video frame in the target bitstream according to the decoding order, and parse the display order of each video frame.

[0044] Decoding order refers to the order in which the encoded images in the target bitstream are decoded, while display order refers to the order in which the decoded images are displayed.

[0045] Optionally, the playback order count (POC) of each video frame can be parsed. The POC is used to identify the playback order of the video frames.

[0046] The target bitstream can contain three types of encoded images: intra-coded pictures (I-frames), predictive-frames (P-frames), and bi-directional interpolated prediction frames (B-frames). I-frames are the reference frames in decoding and are images formed using intra-frame compression coding. P-frames are predicted based on preceding I-frames or P-frames. B-frames are bi-directionally predicted based on the nearest adjacent (either preceding or following) I-frames or P-frames. If the video sequence does not contain B-frames, the decoding order is the same as the display order. If the video sequence contains B-frames, some video images that appear later in time need to be encoded before the earlier ones. The pre-encoded video images will appear first in the encoded video sequence and will be decoded first, but because they appear later in time, they should be displayed later, resulting in a discrepancy between the decoding and display order.

[0047] Step S220: Compare and analyze the display order between the current decoded video frame and the previous decoded video frame to obtain the comparison result; wherein, the decoding order of the previous decoded video frame is earlier than the decoding order of the current decoded video frame.

[0048] The display order of the current decoded video frame is compared with the display order of the previous decoded video frame to obtain a comparison result. This comparison result reflects the order relationship between the display order of the current decoded video frame and the previous decoded video frame, so as to analyze whether the current decoded video frame needs to be updated with an output delay.

[0049] Specifically, the preceding decoded video frame is a decoded video frame whose decoding order is before the current decoded video frame's decoding order and adjacent to the current decoded video frame's decoding order.

[0050] The steps for determining the preceding decoded video frames may include: taking the decoding order of the current decoded video frames as the reference decoding order, determining the previous decoding order corresponding to the reference decoding order, and taking the decoded video frames corresponding to the previous decoding order as the preceding decoded video frames.

[0051] Then, the display order of the preceding decoded video frames is obtained, and the display order of the preceding decoded video frames is compared with the display order of the current decoded video frames to obtain the comparison result.

[0052] For example, the display order of the current decoded video frame can be compared with the display order of the previous decoded video frame to obtain a comparison result; alternatively, the display order of the current decoded video frame can be compared with the display order of the previous decoded video frame to obtain an order difference, and this application does not limit this.

[0053] Step S230: If the comparison result shows that the current decoded video frame needs to be updated with an output delay, then calculate the output display delay corresponding to the current decoded video frame based on the display order of the current decoded video frame and the display order of the previous decoded video frames; where the output display delay represents the time required from the completion of decoding of the current decoded video frame to the output display.

[0054] If the current decoded video frame needs an output delay update, the output delay is recalculated based on the display order of the current decoded video frames.

[0055] Specifically, based on the display order of the current decoded video frame and the previous decoded video frame, the output display delay corresponding to the current decoded video frame is calculated. This output display delay represents the time required from the completion of decoding the current decoded video frame to the output display.

[0056] It should be noted that if the comparison results show that the current decoded video frame does not need to be updated with an output delay, the output display processing can be performed based on the output display delay of the latest decoded video frame.

[0057] By using the display order of the current decoded video frames and the display order of the previous decoded video frames, the output display delay can be flexibly determined according to the actual decoding situation. This ensures that the bitstream with the normal output order can be obtained while minimizing the number of delayed output frames, resulting in normal screen display without issues such as screen tearing or stuttering.

[0058] Step S240: Based on the output display delay corresponding to the current decoded video frame, perform output display processing on the current decoded video frame.

[0059] The output display process is performed on each decoded video frame according to the output display delay of each decoded video frame, such as displaying the output in a preset video display window.

[0060] The output display delay corresponding to the decoded video frame is calculated by following the above steps, which improves the flexibility and accuracy of the output display delay. Thus, when displaying the decoded video frame, the output display delay corresponding to each decoded video frame is used to process the output display of these decoded video frames.

[0061] The following sections will provide further detailed descriptions of some embodiments of this application.

[0062] After obtaining the target bitstream, the video playback device parses the target bitstream, such as parsing parameters like the frame type and POC of the video frames in the target bitstream.

[0063] In some implementations, the video playback strategy selected by the user is obtained. The video playback strategy includes a decoding order playback strategy and a display order playback strategy. If the user selects the decoding order playback strategy, the output is displayed according to the decoding order of each video frame. If the user selects the display order playback strategy, it is necessary to determine that the actual output display order of each video frame is the same as the output display order set in the POC.

[0064] The target bitstream can be configured with a delayed output syntax, and a default output delay can be obtained by parsing the delayed output syntax.

[0065] For example, parsing the delayed output syntax in the target bitstream yields a default output delay of D. D can be directly set as the maximum number of delayed frames, with a value range of [0, Mmax], where Mmax is the maximum number of decoded image buffers. Then, it's possible to output D frames with a delay for all video frames, or only I frames can be output instantly.

[0066] However, while the above-mentioned delayed output method can ensure that users can obtain the bitstream in the normal output display order, it has the problem of high latency in video frame output, resulting in a poor user experience.

[0067] Therefore, this application flexibly calculates the corresponding output display delay for each video frame in order to minimize the delay of video frame output. Please refer to the following embodiments for details.

[0068] First, determine if there is an output delay in the current decoded video frame.

[0069] In some implementations, determining whether there is an output delay in the current decoded video frame includes: calculating the difference between the display order of the current decoded video frame and the display order of the previous decoded video frames to obtain a display order difference; obtaining a difference threshold corresponding to the current decoded video frame; if the display order difference is greater than the difference threshold, the comparison result indicates that the current decoded video frame needs to be updated with an output delay; if the display order difference is not greater than the difference threshold, the comparison result indicates that the current decoded video frame does not need to be updated with an output delay.

[0070] The display order difference between the current decoded video frame and the previous decoded video frame is calculated to obtain the display order difference value, which is then used to determine the relationship between the display order difference value and the difference threshold.

[0071] The difference threshold can be a pre-set value or a value flexibly determined according to the actual situation.

[0072] For example, obtaining the difference threshold corresponding to the current decoded video frame includes: classifying the display order of the current decoded video frame and the previous decoded video frame into odd and even numbers to obtain an odd and even classification result; if the odd and even classification result shows that the display order of the current decoded video frame and the previous decoded video frame are both even numbers, then a first threshold value is selected as the difference threshold corresponding to the current decoded video frame; if the odd and even classification result shows that there is an odd number in the display order of the current decoded video frame and the previous decoded video frame, then a second threshold value is selected as the difference threshold corresponding to the current decoded video frame.

[0073] For example, if the difference threshold is set to m1, and the display order of both the current decoded video frame and the previous decoded video frame is even, then m1 = 2; if the display order of either the current decoded video frame or the previous decoded video frame is odd, then m1 = 1. It is understood that different encoding / decoding methods may require different methods for determining the difference threshold, and the threshold can be flexibly determined based on the actual application. This application does not impose any limitations on this.

[0074] Furthermore, if the display order of the current decoded video frame is set to POCcuur and the display order of the preceding decoded video frame is set to POCprev, then it is determined whether the relationship between the display order difference and the difference threshold satisfies the following formula 1:

[0075] POCcurr - POCprev > m1 (Formula 1)

[0076] If the relationship between the display order difference and the difference threshold satisfies Formula 1, the comparison result indicates that the current decoded video frame needs to undergo output delay update; otherwise, the comparison result indicates that the current decoded video frame does not need to undergo output delay update.

[0077] For example, using H.264 as an example, please refer to [link / reference]. Figure 3 , Figure 3 This is a schematic diagram illustrating the need for output delay updates of video frames in the target bitstream, as shown in an exemplary embodiment of this application. Figure 3 As shown, the I0 frame is parsed first according to the decoding order; then the P2 frame is parsed, and the display order of the P2 frame is POCcurr = 4. Since the preceding frame of the P2 frame is the I0 frame, the display order of the preceding decoded video frames is POCprev = 0. Based on the above embodiment, the difference threshold m1 = 2 is determined. Then, the display order difference is calculated to be 4. The relationship between this display order difference and the difference threshold satisfies Formula 1, indicating that the P2 frame needs an output delay update.

[0078] In some implementations, determining whether there is output delay in the current decoded video frame includes: statistically analyzing the trend of the display order of the current decoded video frame relative to the display order of the previous decoded video frame to obtain the display order change trend; if the display order change trend meets a preset condition, then calculating the change difference between the current decoded video frame and the previous decoded video frame; if the change difference is greater than the difference threshold, the comparison result indicates that the current decoded video frame needs to be updated with output delay; if the change difference is not greater than the difference threshold, the comparison result indicates that the current decoded video frame does not need to be updated with output delay.

[0079] The display order change trend includes an increase in the display order of the current decoded video frame relative to the display order of the previous decoded video frame, and a decrease in the display order of the current decoded video frame relative to the display order of the previous decoded video frame.

[0080] For example, if the display order change trend indicates that the display order of the current decoded video frame increases relative to the display order of the previous decoded video frames, it is determined that the display order change trend meets the preset conditions; the maximum order flag value and the minimum order flag value are pre-stored; the change difference between the current decoded video frame and multiple previous decoded video frames is calculated, including: updating the display order of the current decoded video frame to the maximum order flag value, and updating the previous maximum order flag value to the minimum order flag value; calculating the difference between the maximum order flag value and the minimum order flag value to obtain the change difference between the current decoded video frame and the previous decoded video frames.

[0081] The initial values ​​of the maximum sequence flag POCmax and the minimum sequence flag POCmin are set to 0. During the decoding process, the changing trend of the POC value of each decoded video frame is statistically analyzed. If the POC suddenly increases, POCmax is updated and the previous POCmax value is used as the new POCmin value. If the POC suddenly decreases, POCmax and POCmin are not updated.

[0082] Calculate the difference between POCmax and POCmin to obtain the change difference between the current decoded video frame and the previous decoded video frame. Then, determine the relationship between the change difference and the difference threshold.

[0083] For example, if the difference threshold is set to m2, then m2 = 2 if the display order of the current decoded video frame and the previous decoded video frame are both even numbers; if the display order of the current decoded video frame and the previous decoded video frame is odd, then m2 = 1.

[0084] To determine whether the relationship between the change difference and the difference threshold satisfies the following formula 2:

[0085] POCmax - POCmin > m2 (Formula 2)

[0086] If the relationship between the change difference and the difference threshold satisfies Formula 2, the comparison result shows that the current decoded video frame needs to be updated with an output delay; otherwise, the comparison result shows that the current decoded video frame does not need to be updated with an output delay.

[0087] For example, or as Figure 3 As shown, the I0 frame is parsed first according to the decoding order, at which point POCmax = POCmin = 0; then the P2 frame is parsed, and the display order of the P2 frame is POCcurr = 4. Since the previous frame of the P2 frame is the I0 frame, the display order of the preceding decoded video frames is POCprev = 0. The POC suddenly increases, so POCmax = 4 and POCmin = 0 are updated. Based on the above embodiment, the difference threshold m2 = 2 is determined. Then, the change difference is calculated to be 4. The relationship between this change difference and the difference threshold satisfies Formula 2, indicating that the P2 frame needs an output delay update.

[0088] If the judgment result obtained from the above embodiments indicates that the current decoded video frame needs to be updated with an output delay, then the output display delay corresponding to the current decoded video frame is calculated based on the display order of the current decoded video frame and the display order of the previous decoded video frames.

[0089] In some implementations, the output display delay corresponding to the current decoded video frame is calculated based on the display order of the current decoded video frame and the display order of the previous decoded video frames. This includes: inputting the display order difference into a first preset delay calculation formula to obtain a first initial output delay corresponding to the current decoded video frame; obtaining the output display delay corresponding to the previous decoded video frame to obtain the previous output delay; and selecting the maximum value from the first initial output delay and the previous output delay as the output display delay corresponding to the current decoded video frame.

[0090] Based on the display order difference obtained from the above embodiments, the output display delay corresponding to the current decoded video frame is calculated.

[0091] Specifically, the display order difference is input into the first preset delay calculation formula to obtain the first initial output delay corresponding to the current decoded video frame. The first preset delay calculation formula is expressed as the following formula 3:

[0092] Mcurr = (POCcurr– POCprev) / n1 + k (Formula 3)

[0093] Where Mcurr is the first initial output delay, POCcurr is the display order of the current decoded video frames, POCprev is the display order of the previous decoded video frames, the calculation coefficient n1 is determined in the same way as m1 in Formula 1, and k is an adjustable coefficient that can be set according to the decoder's decoding parameters or user requirements.

[0094] It is understood that the calculation coefficient n1 can be determined differently for different encoding and decoding methods. The calculation coefficient can be flexibly determined according to the actual application. This application does not limit this.

[0095] After obtaining the first initial output delay, it can be directly used as the output display delay corresponding to the current decoded video frame. Alternatively, the output display delay corresponding to the previous decoded video frame can be obtained to get the previous output delay. The maximum value between the first initial output delay and the previous output delay can be selected as the output display delay corresponding to the current decoded video frame. If there is a maximum number of delay frames, the minimum value between the first initial output delay and the maximum number of delay frames can be selected to get the intermediate output delay. Then, the maximum value between the intermediate output delay and the previous output delay can be selected as the output display delay corresponding to the current decoded video frame.

[0096] For example, let the maximum delay number of frames be D, and the preceding output delay be Mprev. The intermediate output delay is determined by the following formula 4:

[0097] Mcurr = min(Mcurr, D) (Formula 4)

[0098] Where D>0.

[0099] Then, the output display delay M corresponding to the current decoded video frame is determined using the following formula 5:

[0100] M = max(Mprev, Mcurr) (Formula 5)

[0101] Optionally, the feasibility of the output display delay M can also be verified. Specifically, the maximum number of decoded image buffers Mmax is obtained, and it is checked whether the output display delay M exceeds Mmax. If it does not exceed Mmax, the output display delay M is feasible, and the current decoded video frame is delayed by M frames; otherwise, the output display delay M is not feasible, and the output display delay M can be reset to 0.

[0102] Furthermore, continue processing subsequent decoded video frames in the manner described above. If subsequent decoded video frames do not require output delay updates, then Mcurr is not updated, meaning the output display delay of subsequent decoded video frames is the same as the output display delay of the current decoded video frame. If subsequent decoded video frames require output delay updates, then POCcurr, POCprev, and Mcurr are updated, and then the new output display delay can be further determined according to Formulas 4 and 5.

[0103] For example, or as Figure 3 As shown, the I0 frame is parsed first according to the decoding order; then the P2 frame is parsed, resulting in a display order POCcurr = 4 for the P2 frame. Since the preceding frame of the P2 frame is the I0 frame, the display order of the preceding decoded video frames is POCprev = 0. Therefore, the display order difference is 4. Based on the above embodiment, the difference threshold calculation coefficient n1 = 2. If k = 0, then it is calculated according to formula 3:

[0104] Mcurr=(POCcurr–POCprev) / n1+k=2

[0105] Then you can set the output display delay M to 2.

[0106] Continuing to parse frame B1, we obtain POCcurr=2 and POCprev=4 for frame B1. It is determined that frame B1 does not need to perform output delay update, that is, M is not updated.

[0107] Continuing to parse frame P4, we obtain POCcurr=8 and POCprev=2 for frame B1. It is determined that frame P4 needs to be updated with an output delay, so the output display delay M is updated to 3. Continue parsing and continuously update M.

[0108] In some implementations, the output display delay corresponding to the current decoded video frame is calculated based on the display order of the current decoded video frame and the display order of the previous decoded video frames. This includes: inputting the change difference into a second preset delay calculation formula to obtain the second initial output delay corresponding to the current decoded video frame; obtaining the output display delay corresponding to the previous decoded video frame to obtain the previous output delay; and selecting the maximum value from the second initial output delay and the previous output delay as the output display delay corresponding to the current decoded video frame.

[0109] Based on the variation difference obtained in the above embodiments, the output display delay corresponding to the current decoded video frame is calculated.

[0110] Specifically, the difference in value is input into the second preset delay calculation formula to obtain the second initial output delay corresponding to the current decoded video frame. The second preset delay calculation formula is expressed as the following formula 3:

[0111] Mcurr = (POCmax–POCmin) / n2 + k (Formula 6)

[0112] Where Mcurr is the second initial output delay, POCmax is the maximum sequence flag value corresponding to the current decoded video frame, POCmin is the minimum sequence flag value corresponding to the previous decoded video frame, the calculation coefficient n2 is determined in the same way as m2 in Formula 2, and k is an adjustable coefficient that can be set according to the decoder's decoding parameters or user requirements.

[0113] After obtaining the second initial output delay, it can be directly used as the output display delay corresponding to the current decoded video frame. Alternatively, the output display delay corresponding to the previous decoded video frame can be obtained to get the previous output delay. The maximum value between the second initial output delay and the previous output delay can be selected as the output display delay corresponding to the current decoded video frame. If there is a maximum delay frame number, the minimum value between the second initial output delay and the maximum delay frame number can be selected first to obtain the intermediate output delay. Then, the maximum value between the intermediate output delay and the previous output delay can be selected as the output display delay corresponding to the current decoded video frame.

[0114] For example, the output display delay corresponding to the current decoded video frame can be determined by referring to Formulas 4 and 5 above, which will not be elaborated here.

[0115] Optionally, the feasibility of the output display delay M can also be verified. Specifically, the maximum number of decoded image buffers Mmax is obtained, and it is checked whether the output display delay M exceeds Mmax. If it does not exceed Mmax, the output display delay M is feasible, and the current decoded video frame is delayed by M frames; otherwise, the output display delay M is not feasible, and the output display delay M can be reset to 0.

[0116] Furthermore, continue processing subsequent decoded video frames in the manner described above. If subsequent decoded video frames do not require output delay updates, then Mcurr is not updated, meaning the output display delay of subsequent decoded video frames is the same as the output display delay of the current decoded video frame. If subsequent decoded video frames require output delay updates, then POCcurr, POCprev, and Mcurr are updated, and then the new output display delay can be further determined according to Formulas 4 and 5.

[0117] For example, or as Figure 3 As shown, the I0 frame is parsed first according to the decoding order, at which point POCmax = POCmin = 0; then the P2 frame is parsed, and the display order of the P2 frame is obtained as POCcurr = 4. Since the previous frame of the P2 frame is the I0 frame, the display order of the preceding decoded video frames is obtained as POCprev = 0. POC suddenly increases, at which point POCmax = 4 and POCmin = 0 are updated. Based on the above embodiment, the difference threshold calculation coefficient n1 = 2 is determined. If k = 0, then it is calculated according to formula 6:

[0118] Mcurr=(POCmax–POCmin) / n2+k=2

[0119] Then you can set the output display delay M to 2.

[0120] Continuing to analyze frame B1, we find that POCcurr = 2 and POCprev = 4 for frame B1. Since POC decreases, it is determined that frame B1 does not need to be updated with output delay. Therefore, the values ​​of POCmax and POCmin are not updated, i.e., M is not updated.

[0121] Continuing to parse frame P4, we obtain POCcurr = 8 and POCprev = 2 for frame B1. The sudden increase in POC indicates that frame P4 needs an output delay update. POCmax is updated to 8 and POCmin is updated to 4. The output display delay M is updated to 2. We continue parsing and updating M continuously.

[0122] In some implementations, the initial value of the output display delay M can be preset. For example, a pre-parsing method can be used to pre-parse a bitstream of a specified number of frames N or a specified bitstream length S in the target bitstream. Based on the delay output syntax obtained from the pre-parsing of the target bitstream, the comparison results of the display order between the POC and the actual video frames, a more accurate initial value of the output display delay M can be determined to reduce stuttering caused by delay during the decoding process.

[0123] For example, 200 frames in the target bitstream are pre-parsed. Based on the comparison of the display order among the 200 frames, the output display delay of each frame is determined, resulting in multiple candidate output display delays. The maximum value is selected from the multiple candidate output display delays and used as the initial value of the output display delay M.

[0124] Then, the output display delay M is updated in a timely manner according to the above embodiment to avoid phenomena such as screen distortion.

[0125] In some implementations, if a delayed output syntax exists in the target bitstream, the delayed output syntax is parsed to obtain a default output delay D; the frame type to which the current decoded video frame belongs is parsed; if the current decoded video frame belongs to the target type frame, the current decoded video frame is processed for immediate output or for output display according to the default output delay; if the current decoded video frame does not belong to the target type frame, the output display delay corresponding to the current decoded video frame can be updated according to the above embodiments to process the current decoded video frame for output display.

[0126] Among them, target type frames can be I-frames, and non-target type frames can be B-frames or P-frames.

[0127] For example, using H.265 as an example, please refer to [link / reference]. Figure 4 , Figure 4 This is a schematic diagram illustrating the need for output delay updates of video frames in a target bitstream, as shown in another exemplary embodiment of this application. Figure 4 As shown:

[0128] 1. Parse the delayed output syntax in the target bitstream to obtain the default output delay D = 2;

[0129] 2. Analyze each video frame;

[0130] a) Parse the I0 frame, at which point POCmin = POCmax = 0; where the I0 frame is a target type frame, then the I0 frame is processed for immediate output or the I0 frame is processed for output display according to the default output delay;

[0131] b) Next, parse frame P4, POCcurr = 4, frame I0 is the preceding decoded video frame corresponding to frame P4, POCprev = 0, update POCmax = 4:

[0132] Based on Formula 2, it is determined that an output delay update is required.

[0133] According to formula 6, if k = 0, n² = 1:

[0134] Mcurr=(POCmax–POCmin) / n2=4

[0135] At this point, the output display delay M = 4;

[0136] c) Next, frames B2, B1, and B3 are parsed. The POC values ​​all decrease. At this time, POCmax and POCmin are not updated.

[0137] e) Next, parse frame P8. POCcurr = 8, the POC value increases, POCmax is updated to 8, and POCmin is updated to the previous POCmax value of 4.

[0138] Mcurr=(POCmax–POCmin) / n2=4

[0139] At this point, the output display delay M = 4;

[0140] f) Continue parsing other frames and calculate and update the output display delay M according to the actual situation until all frames are decoded.

[0141] The image display method provided in this application decodes each video frame in the target bitstream according to the decoding order and parses the display order of each video frame; it compares and analyzes the display order between the current decoded video frame and the previous decoded video frames to obtain the comparison result; if the comparison result shows that the current decoded video frame needs to be updated with an output delay, it calculates the output display delay corresponding to the current decoded video frame based on the display order of the current decoded video frame and the display order of the previous decoded video frames; based on the output display delay corresponding to the current decoded video frame, it performs output display processing on the current decoded video frame to improve the flexibility and accuracy of the output display delay. Under the premise of minimizing the number of delayed output frames, it can obtain a bitstream with a normal output order, and the screen display is normal without problems such as screen tearing or stuttering, thus improving the user experience.

[0142] Figure 5 This is a block diagram illustrating an image display device according to an exemplary embodiment of this application. Figure 5 As shown, the exemplary image display device 500 includes: a frame parsing module 510, a sequence analysis module 520, a delay calculation module 530, and a display processing module 540. Specifically:

[0143] The frame parsing module 510 is used to decode each video frame in the target bitstream according to the decoding order, and to parse the display order of each video frame;

[0144] The sequence analysis module 520 is used to compare and analyze the display order between the current decoded video frame and the previous decoded video frame to obtain the comparison result; wherein, the decoding order of the previous decoded video frame is before the decoding order of the current decoded video frame.

[0145] The delay calculation module 530 is used to calculate the output display delay corresponding to the current decoded video frame based on the display order of the current decoded video frame and the display order of the previous decoded video frames if the comparison result shows that the current decoded video frame needs to be updated with an output delay; wherein, the output display delay represents the time required from the completion of decoding of the current decoded video frame to the output display.

[0146] The display processing module 540 is used to perform output display processing on the current decoded video frame based on the output display delay corresponding to the current decoded video frame.

[0147] It should be noted that the image display device and the image display method provided in the above embodiments belong to the same concept. The specific ways in which each module and unit performs operations have been described in detail in the method embodiments, and will not be repeated here. In practical applications, the image display device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation.

[0148] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of an embodiment of the electronic device of this application. The electronic device 600 includes a memory 601 and a processor 602. The processor 602 is used to execute program instructions stored in the memory 601 to implement the steps in any of the above-described image display method embodiments. In a specific implementation scenario, the electronic device 600 may include, but is not limited to, a microcomputer or a server. In addition, the electronic device 600 may also include mobile devices such as laptops and tablets, which are not limited here.

[0149] Specifically, processor 602 controls itself and memory 601 to implement the steps in any of the above-described image display method embodiments. Processor 602 may also be referred to as a Central Processing Unit (CPU). Processor 602 may be an integrated circuit chip with signal processing capabilities. Processor 602 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor. Furthermore, processor 602 may be implemented using integrated circuit chips.

[0150] Please see Figure 7 , Figure 7 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present application. The computer-readable storage medium 700 stores program instructions 710 that can be executed by a processor. The program instructions 710 are used to implement the steps in any of the above-described image display method embodiments.

[0151] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0152] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0153] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0154] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. An image display method, characterized in that, include: Decode each video frame in the target bitstream according to the decoding order, and parse the display order of each video frame; The display order of the current decoded video frame is compared and analyzed with that of the previous decoded video frame to obtain the comparison result; wherein the decoding order of the previous decoded video frame is before the decoding order of the current decoded video frame. If the comparison result indicates that the current decoded video frame needs to be updated with an output delay, then the output display delay corresponding to the current decoded video frame is calculated based on the display order of the current decoded video frame and the display order of the preceding decoded video frames; wherein, the output display delay represents the time required from the completion of decoding of the current decoded video frame to its output display; Based on the output display delay corresponding to the current decoded video frame, output display processing is performed on the current decoded video frame.

2. The method according to claim 1, characterized in that, The comparison and analysis of the display order between the current decoded video frame and the previously decoded video frame to obtain the comparison results include: The display order difference is calculated by comparing the display order of the current decoded video frame with the display order of the previous decoded video frame. Obtain the difference threshold corresponding to the current decoded video frame; If the difference in display order is greater than the difference threshold, the comparison result indicates that the current decoded video frame needs to be updated with an output delay. If the difference in display order is not greater than the difference threshold, the comparison result indicates that the current decoded video frame does not need to undergo output delay update.

3. The method according to claim 2, characterized in that, The step of obtaining the difference threshold corresponding to the current decoded video frame includes: The display order of the current decoded video frame and the previous decoded video frame is classified as odd or even to obtain the odd or even classification result. If the odd-even classification result indicates that the display order of the current decoded video frame and the previous decoded video frame is both even, then the first threshold value is selected as the difference threshold corresponding to the current decoded video frame. If the odd-even classification result indicates that there is an odd number in the display order of the current decoded video frame and the previous decoded video frame, then the second threshold value is selected as the difference threshold corresponding to the current decoded video frame.

4. The method according to claim 2, characterized in that, The step of calculating the output display delay corresponding to the current decoded video frame based on the display order of the current decoded video frame and the display order of the preceding decoded video frames includes: The display order difference is input into the first preset delay calculation formula to obtain the first initial output delay corresponding to the current decoded video frame; and the output display delay corresponding to the previous decoded video frame is obtained to obtain the previous output delay; The maximum value between the first initial output delay and the preceding output delay is selected as the output display delay corresponding to the current decoded video frame.

5. The method according to claim 1, characterized in that, The comparison and analysis of the display order between the current decoded video frame and the previously decoded video frame to obtain the comparison results include: The display order change trend is obtained by statistically analyzing the change trend of the display order of the current decoded video frame relative to the display order of the previous decoded video frame; If the display order change trend meets the preset conditions, then calculate the change difference between the current decoded video frame and the previous decoded video frame; If the difference in change is greater than the difference threshold, the comparison result indicates that the current decoded video frame needs to be updated with an output delay. If the difference in change is not greater than the difference threshold, the comparison result indicates that the current decoded video frame does not need to undergo output delay update.

6. The method according to claim 5, characterized in that, If the display order change trend indicates that the display order of the current decoded video frame increases relative to the display order of the previous decoded video frame, then it is determined that the display order change trend meets the preset condition; The maximum and minimum sequence flag values ​​are pre-stored; the calculation of the difference between the current decoded video frame and multiple previous decoded video frames includes: Update the display order of the current decoded video frames to the maximum order flag value, and update the previous maximum order flag value to the minimum order flag value; Calculate the difference between the maximum sequence flag value and the minimum sequence flag value to obtain the change difference between the current decoded video frame and the previous decoded video frame.

7. The method according to claim 5, characterized in that, The step of calculating the output display delay corresponding to the current decoded video frame based on the display order of the current decoded video frame and the display order of the preceding decoded video frames includes: The change difference is input into the second preset delay calculation formula to obtain the second initial output delay corresponding to the current decoded video frame; and the output display delay corresponding to the previous decoded video frame is obtained to obtain the previous output delay; The maximum value between the second initial output delay and the preceding output delay is selected as the output display delay corresponding to the current decoded video frame.

8. The method according to claim 1, characterized in that, Also includes: If the target bitstream contains a delayed output syntax, then the delayed output syntax is parsed to obtain the default output delay; Analyze the frame type to which the currently decoded video frame belongs; If the current decoded video frame belongs to the target type frame, then the current decoded video frame is processed for immediate output, or the current decoded video frame is processed for output display according to the default output delay; If the current decoded video frame does not belong to the target type frame, the display order between the current decoded video frame and the previous decoded video frame is compared and analyzed to obtain the comparison result; if the comparison result shows that the current decoded video frame needs to be updated with output delay, the output display delay corresponding to the current decoded video frame is calculated based on the display order of the current decoded video frame and the display order of the previous decoded video frame. Based on the output display delay corresponding to the current decoded video frame, output display processing is performed on the current decoded video frame.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the processor being configured to execute program instructions stored in the memory to implement the steps of the method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program instructions that can be executed by a processor to implement the steps of the method as described in any one of claims 1-8.

Images