Decoding time detection method, decoding method and related equipment

Abstract

The present disclosure provides a decoding time detection method, a decoding method and related equipment, and relates to the field of computer and communication technology. The decoding time detection method includes: determining the number of frames stored in the decoder s, s is an integer greater than or equal to 0; according to the number of frames stored s, obtaining the first load decoding time of the decoder decoding the kth video frame to be decoded in the video to be decoded, k is a positive integer greater than 1; according to the kth video frame to be decoded and the k-1th video frame to be decoded, obtaining the second load decoding time of the decoder decoding the kth video frame to be decoded in the video to be decoded; according to the first load decoding time and the second load decoding time, determining the target decoding time of the decoder decoding the kth video frame to be decoded in the video to be decoded. The scheme provided in the embodiment of the present disclosure can improve the detection accuracy of the single-frame decoding time of the decoder, and can be applied to cloud game scenarios based on cloud servers.

Description

Technical Field

[0001] The present disclosure relates to the field of computer and communication technology, and in particular to a decoding time detection method, a decoding time detection device, a decoding method, a decoding device, an electronic device, and a computer-readable storage medium. Background Art

[0002] In some application scenarios, it is necessary to accurately estimate the single-frame decoding time of the hardware chip corresponding to the decoder for decoding the video bitstream in the terminal device. However, the single-frame decoding time estimation method in the related art does not take into account the decoding frame hoarding of the hardware chip and the network jitter, resulting in the estimated single-frame decoding time being inaccurate and unable to meet the needs of these application scenarios.

[0003] Therefore, a new decoding time detection method, decoding time detection device, decoding method, decoding device, electronic device and computer-readable storage medium are needed.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure. Summary of the invention

[0005] The embodiments of the present disclosure provide a decoding time detection method, a decoding time detection device, a decoding method, a decoding device, an electronic device and a computer-readable storage medium, which can improve the detection accuracy of a single-frame decoding time of a decoder.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] The present disclosure provides a decoding time detection method, the method comprising: determining the number of frames stored in a decoder s, where s is an integer greater than or equal to 0; obtaining a first load decoding time for the decoder to decode a kth video frame to be decoded in a video to be decoded according to the number of frames stored, where k is a positive integer greater than 1; obtaining a second load decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded according to the kth video frame to be decoded and the k-1th video frame to be decoded in the video to be decoded; and determining a target decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded according to the first load decoding time and the second load decoding time.

[0008] The present disclosure provides a decoding method, which includes: determining the number of frames stored in a decoder s, where s is an integer greater than or equal to 0; obtaining a first load decoding time for the decoder to decode a kth video frame to be decoded in a video to be decoded according to the number of frames stored, where k is a positive integer greater than 1; obtaining a second load decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded according to the kth video frame to be decoded and the k-1th video frame to be decoded in the video to be decoded; determining a target decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded according to the first load decoding time and the second load decoding time; and decoding the video to be decoded according to the target decoding time of the kth video frame to be decoded.

[0009] An embodiment of the present disclosure provides a decoding time detection device, which includes: a decoder hoarding frame number determination unit, used to determine the decoder hoarding frame number s, s is an integer greater than or equal to 0; a first load decoding time acquisition unit, used to obtain a first load decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded according to the hoarding frame number s, k is a positive integer greater than 1; a second load decoding time acquisition unit, used to obtain a second load decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded according to the kth video frame to be decoded and the k-1th video frame to be decoded in the video to be decoded; a target decoding time acquisition unit, used to determine a target decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded according to the first load decoding time and the second load decoding time.

[0010] In some exemplary embodiments of the present disclosure, the decoder frame hoarding quantity determination unit includes: a test code stream obtaining unit, used to obtain a test code stream having the same video size as the video to be decoded; a test video frame input unit, used to input K test video frames in the test code stream to the decoder, K is a positive integer greater than or equal to 1, and the last first preset number of test video frames in the K test video frames are not instant decoding refresh frames; a test decoding frame counting unit, used to count T test decoding frames output by the decoder when decoding the test code stream, T is a positive integer greater than or equal to 1; and a hoarding frame quantity obtaining unit, used to obtain the hoarding frame quantity s of the decoder according to the K test video frames and the T test decoding frames.

[0011] In some exemplary embodiments of the present disclosure, the decoder frame number determination unit includes: a preset to-be-decoded video frame obtaining unit, used to obtain the first second preset number of to-be-decoded video frames in the to-be-decoded video; a to-be-decoded video frame input unit, used to obtain K of the second preset number of to-be-decoded video frames input to the decoder at the i-th moment. i Frame video frame to be decoded, Ki is a positive integer greater than or equal to 1, i is a positive integer greater than or equal to 1 and less than or equal to the second preset number; a decoded video frame output unit, used to record the T output of the decoder decoding the second preset number of to-be-decoded video frames at the i-th moment. i Frame decoded video frame, T i is a positive integer greater than or equal to 1; a predetermined time frame number acquisition unit is used to obtain the number of frames according to the K i The frame to be decoded video frame and the T i The frame decoder decodes the video frame to obtain the number of frames stored by the decoder at the i-th moment; and the frame number determination unit is used to determine the number of frames stored by the decoder s according to the number of frames stored by the decoder at the i-th moment.

[0012] In some exemplary embodiments of the present disclosure, the preset to-be-decoded video frame obtaining unit includes: a video frame type obtaining unit, used to obtain the frame type of the to-be-decoded video frame in the to-be-decoded video; an instant decoding refresh frame elimination unit, used to eliminate the to-be-decoded video frames whose frame type is the instant decoding refresh frame and the third preset number of to-be-decoded frames before and after it from the to-be-decoded video; a second number of to-be-decoded video frame selection unit, used to select the first second preset number of to-be-decoded video frames from the remaining to-be-decoded video.

[0013] In some exemplary embodiments of the present disclosure, the to-be-decoded video frame input unit comprises: a slow start unit for converting K at a predetermined speed at a jth moment. j The frame to be decoded video frame is input to the decoder so that there is no to-be-decoded video frame in the buffer of the decoder; wherein j is a positive integer greater than or equal to 1 and less than or equal to i.

[0014] In some exemplary embodiments of the present disclosure, the first load decoding time obtaining unit includes: a first timestamp obtaining unit, used to obtain a first timestamp of the k+s-th video frame to be decoded in the video to be decoded input into the decoder according to the number of frames s; a second timestamp obtaining unit, used to obtain a second timestamp of the k-th video frame to be decoded in the video to be decoded when the decoder decodes the k-th video frame to be decoded in the video to be decoded; a first load decoding time calculation unit, used to obtain the first load decoding time according to the first timestamp and the second timestamp.

[0015] In some exemplary embodiments of the present disclosure, the second load decoding time obtaining unit includes: a third timestamp obtaining unit, used to obtain a third timestamp of the kth video frame to be decoded in the video to be decoded when the decoder decodes it; a fourth timestamp obtaining unit, used to obtain a fourth timestamp of the k-1th video frame to be decoded in the video to be decoded when the decoder decodes it; and a second load decoding time calculation unit, used to obtain the second load decoding time based on the third timestamp and the fourth timestamp.

[0016] In some exemplary embodiments of the present disclosure, the target decoding time obtaining unit includes: a comparison unit, used to compare the first load decoding time and the second load decoding time; a target decoding time determination unit, used to determine the minimum value of the first load decoding time and the second load decoding time as the target decoding time for the decoder to decode the kth video frame to be decoded.

[0017] In some exemplary embodiments of the present disclosure, the device also includes: a cloud gaming performance determination unit, which is used to determine whether to send the cloud gaming video to a terminal device including the decoder based on the target decoding time of the kth video frame to be decoded in the video to be decoded by the decoder if the video to be decoded is a cloud gaming video.

[0018] The embodiment of the present disclosure provides a decoding device, which includes: a decoder frame number determination unit, which is used to determine the decoder frame number s, where s is an integer greater than or equal to 0; a first load decoding time acquisition unit, which is used to obtain a first load decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded according to the frame number s, where k is a positive integer greater than 1; a second load decoding time acquisition unit, which is used to obtain a second load decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded according to the kth video frame to be decoded and the k-1th video frame to be decoded in the video to be decoded; a target decoding time acquisition unit, which is used to determine a target decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded according to the first load decoding time and the second load decoding time; and a to-be-decoded video decoding unit, which is used to decode the video to be decoded according to the target decoding time of the kth video frame to be decoded.

[0019] An embodiment of the present disclosure provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the decoding time detection method and the decoding method described in the above embodiments are implemented.

[0020] An embodiment of the present disclosure provides an electronic device, comprising: at least one processor; a storage device, configured to store at least one program, and when the at least one program is executed by the at least one processor, the at least one processor implements the decoding time detection method and decoding method as described in the above embodiments.

[0021] The embodiment of the present disclosure provides a computer program product or a computer program, which includes a computer instruction, and the computer instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device performs the decoding time detection method and the decoding method provided in the above various optional embodiments.

[0022] In the technical solutions provided by some embodiments of the present disclosure, on the one hand, by determining the number of frames hoarded by the decoder s, the first load decoding time of the decoder decoding the kth video frame to be decoded in the video to be decoded can be obtained according to the determined number of frames hoarded by the decoder s, that is, the present disclosure embodiment optimizes the detection method of the single frame decoding time of the decoder with the decoding hoarding situation, which can make the single frame decoding time estimation of the decoder more accurate; on the other hand, further according to the kth video frame to be decoded and the k-1th video frame to be decoded in the video to be decoded, the second load decoding time of the decoder decoding the kth video frame to be decoded in the video to be decoded is obtained, so as to adapt to the scene such as uneven frame delivery of the decoder caused by factors such as network jitter. At the same time, the present disclosure embodiment also finally determines the target decoding time of the decoder decoding the kth video frame to be decoded in the video to be decoded according to the first load decoding time and the second load decoding time determined in the above manner, optimizes the decoder with the decoding hoarding situation, and effectively avoids the deviation of the single frame decoding time estimation caused by the uneven frame delivery caused by abnormal situations such as network jitter, so as to accurately detect the single frame decoding time of the decoder. The solution provided by the embodiments of the present disclosure can be applied to, for example, cloud gaming or other video application scenarios where the estimation accuracy of the time required to decode a single frame is relatively high.

[0023] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0025] Figure 1 The flowchart of a decoding time detection method according to an embodiment of the present disclosure is schematically shown.

[0026] Figure 2The schematic diagram schematically shows a decoder storing 3 frames according to an embodiment of the present disclosure.

[0027] Figure 3 Schematically shows Figure 1 Step S110 in the embodiment is a flowchart in an exemplary embodiment.

[0028] Figure 4 The figure schematically shows a schematic diagram of calculating the number of hoarded frames according to an embodiment of the present disclosure.

[0029] Figure 5 Schematically shows Figure 1 Step S110 in the embodiment is a flowchart in an exemplary embodiment.

[0030] Figure 6 Schematically shows Figure 1 Step S120 in the embodiment is a flowchart in an exemplary embodiment.

[0031] Figure 7 Schematically shows Figure 1 Step S130 in the embodiment is a flowchart in an exemplary embodiment.

[0032] Figure 8 The flowchart of a decoding time detection method according to an embodiment of the present disclosure is schematically shown.

[0033] Fig. 9 A schematic diagram schematically shows the application of the decoding time detection method provided by an embodiment of the present disclosure to a cloud gaming scenario.

[0034] Fig.10 The flowchart of a decoding method according to an embodiment of the present disclosure is schematically shown.

[0035] Fig.11 The block diagram of a decoding time detection device according to an embodiment of the present disclosure is schematically shown.

[0036] Fig.12 A block diagram of a decoding device according to an embodiment of the present disclosure is schematically shown.

[0037] Fig.13 A schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0038] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more comprehensive and complete and will fully convey the concept of the example embodiments to those skilled in the art.

[0039] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the present disclosure.

[0040] The block diagrams shown in the accompanying drawings are only functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or in at least one hardware module or integrated circuit, or in different networks and / or processor devices and / or microcontroller devices.

[0041] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0042] The method for calculating the single-frame decoding time consumption of the hardware decoder in the related art is to record the timestamp of the video frame input to the decoder and the timestamp of the video frame output after the decoder decodes the input video frame, and calculate it according to the difference between the two timestamps, that is, according to the following formula:

[0043]

[0044] In the above formula (1), Indicates the timestamp when the code stream of the kth frame of the video is sent to the decoder. Indicates the timestamp of the video frame k when it is decoded and output from the decoder (called outgoing frame), ΔT k Indicates the decoding time of the kth frame of the video, where k is a positive integer greater than 1.

[0045] However, the time-consuming calculation scheme for decoding a single frame in the above-mentioned related art does not take into account the possibility that the hardware chip of the decoder may have a decoding frame hoarding situation. Decoding frame hoarding means that the decoder will output a decoded image only after a certain number of video frame code streams are sent to the decoder.

[0046] For this type of hardware chip, the single-frame decoding time calculation method in the related technology is not accurate, which is specifically manifested in the following two aspects (here taking 3 frames as an example):

[0047] First, the calculated decoding time of the kth frame includes the decoding time of the subsequent k+1th frame and k+2th frame.

[0048] Second, if the input time of the k+3th frame fluctuates due to network jitter or other reasons, the output time of the kth frame will also fluctuate. Since these jitters may not be estimated, the decoding time of the kth frame may be inaccurately calculated.

[0049] In many video application scenarios, such as cloud gaming, it is necessary to accurately calculate the single-frame decoding time of the decoder's hardware chip to determine whether the corresponding terminal device's hardware chip can enjoy a high-quality cloud gaming service experience. However, the technical solutions in the related art cannot solve the problem of the accuracy of the single-frame decoding time of the decoder with decoding frame hoarding, and cannot meet the business needs of such scenarios.

[0050] The embodiments of the present disclosure provide a decoding time detection method to solve at least one technical problem existing in the above-mentioned related technologies. The decoding time detection method and decoding method provided by the embodiments of the present disclosure can be performed by any electronic device, such as a terminal device or a server. Among them, the terminal device may include but is not limited to any one or more of a smart phone, a tablet computer, a desktop computer, a laptop computer, a wearable smart device, a smart TV, etc. Among them, the server can be an independent server, or a server cluster or distributed system composed of multiple servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, and big data and artificial intelligence platforms.

[0051] Figure 1 The flowchart of a decoding time detection method according to an embodiment of the present disclosure is schematically shown. Figure 1 The embodiments are described by taking the execution of a terminal device as an example, but the present disclosure is not limited thereto. Figure 1 As shown, the method provided by the embodiment of the present disclosure may include the following steps.

[0052] In step S110 , the number s of frames stored in the decoder is determined, where s is an integer greater than or equal to 0.

[0053] In the disclosed embodiment, the decoder is a hardware chip disposed in a terminal device for decoding a video to be decoded, and the decoder may have a decoding frame storage capability.

[0054] In the disclosed embodiment, decoding frame hoarding refers to the phenomenon that the hardware decoder starts to output corresponding video frames to be decoded only after a certain number of video frames to be decoded in the video to be decoded are input.

[0055] For the same decoder, the number of stored frames may vary depending on the video size of the video to be decoded. For example, for some decoder hardware chips, the number of stored frames s may be 3 for a video stream of a video to be decoded of 1080p (p is the abbreviation of Progressive scan, which means line-by-line scanning. Usually, the screen resolution of 1080p is 1920*1080 pixels). For a hardware chip with a number of stored frames of 3, the decoded video frame of the first frame of the video to be decoded will be output from the decoder only after the bitstream of the fourth frame of the video to be decoded is input.

[0056] For example, Figure 2 As shown, taking the decoder storing 3 frames as an example, after inputting the input frame data of the M+3th video frame to be decoded, the M+2th video frame to be decoded, and the M+1th video frame to be decoded into the decoder, the decoder outputs the decoded frame data of the Mth decoded video frame after the Mth video frame to be decoded is decoded. At this time, the M+4th video frame to be decoded has not yet been input into the decoder, and M is a positive integer greater than or equal to 1.

[0057] In the embodiment of the present disclosure, the number of stored frames of the decoder can be obtained by using a test bitstream test, or the number of stored frames of the decoder can be obtained directly by using the video to be decoded. For details, see the following Figure 3 and Figure 5 Embodiment of the invention.

[0058] In step S120, according to the number of stored frames s, a first load decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded is obtained, where k is a positive integer greater than 1.

[0059] In the disclosed embodiment, under the influence of external factors such as network jitter, the intervals of sending frames to the decoder are often uneven. In certain cases, many video frames to be decoded may be sent at one time or no video frames to be decoded may be sent for a period of time.

[0060] When no video frames to be decoded are input into the decoder for a period of time, or the speed of inputting the video frames to be decoded is slow, the hardware chip of the decoder can decode the input video frames to be decoded in time, and there are no video frames to be decoded in the buffer of the decoder. The state of no video frames to be decoded in the buffer is called the decoder being in a low load state (or a first load state).

[0061] In the disclosed embodiment, the first load decoding time refers to estimating the time taken by a decoder in a low load state to decode a single frame of the kth video frame to be decoded in the video to be decoded according to the number of frames stored in the decoder.

[0062] The specific method for calculating the first load decoding time can be referred to as follows Figure 6 Embodiment of the invention.

[0063] In step S130, a second load decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded is obtained according to the kth video frame to be decoded and the k-1th video frame to be decoded in the video to be decoded.

[0064] When many video frames to be decoded are sent at one time, the hardware chip of the decoder may not be able to complete the decoding of all the input video frames to be decoded in a short period of time, resulting in the accumulation of input video frames to be decoded in the decoder's buffer, and the decoding of the video frames to be decoded in the buffer has not yet begun. This situation is called decoding frame accumulation, and the state in which decoding frames accumulate in the decoder's buffer is called the decoder being in a high load state (or a second load state).

[0065] In the embodiment of the present disclosure, the second load decoding time refers to the estimated time consumed by a decoder in a high load state to decode a single frame of the kth video frame to be decoded in the video to be decoded.

[0066] The specific method for calculating the second load decoding time can be referred to as follows Figure 7 Embodiment of the invention.

[0067] In step S140, a target decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded is determined according to the first load decoding time and the second load decoding time.

[0068] In the embodiment of the present disclosure, the first load decoding time of the decoder in a low load state and the second load decoding time of the decoder in a high load state obtained by the above calculation can be comprehensively considered to finally determine the target decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded.

[0069] The decoding time detection method provided by the embodiment of the present disclosure, on the one hand, by determining the number of frames hoarded by the decoder s, the first load decoding time of the decoder decoding the kth video frame to be decoded in the video to be decoded can be obtained according to the determined number of frames hoarded by the decoder s, that is, the embodiment of the present disclosure optimizes the detection method of the single frame decoding time of the decoder with the decoding hoarding situation, which can make the single frame decoding time estimation of the decoder more accurate; on the other hand, further according to the kth video frame to be decoded and the k-1th video frame to be decoded in the video to be decoded, the second load decoding time of the decoder decoding the kth video frame to be decoded in the video to be decoded is obtained, so as to adapt to the scene such as uneven frame delivery of the decoder caused by factors such as network jitter. At the same time, the embodiment of the present disclosure also finally determines the target decoding time of the decoder decoding the kth video frame to be decoded in the video to be decoded according to the first load decoding time and the second load decoding time determined in the above manner, optimizes the decoder with the decoding hoarding situation, and effectively avoids the deviation of the single frame decoding time estimation caused by the uneven frame delivery caused by abnormal situations such as network jitter, so as to accurately detect the single frame decoding time of the decoder. The solution provided by the embodiments of the present disclosure can be applied to, for example, cloud gaming or other video application scenarios where the estimation accuracy of the time required to decode a single frame is relatively high.

[0070] Figure 3 Schematically shows Figure 1 Step S110 in the embodiment is a flow chart of an exemplary embodiment. Figure 3 As shown above Figure 1 Step S110 in the embodiment may further include the following steps.

[0071] In step S111, a test bitstream having the same video size as the video to be decoded is obtained.

[0072] For example, if the video size of the video to be decoded is 1080p, the video size of the corresponding test bitstream is also 1080p, but the present disclosure is not limited thereto.

[0073] In step S112, K test video frames in the test bitstream are input to the decoder, where K is a positive integer greater than or equal to 1, and the last first preset number of test video frames in the K test video frames are not instant decoding refresh frames.

[0074] In the disclosed embodiment, the instantaneous decoding refresh (IDR) frame is a special I frame, that is, all reference frames after this frame will only refer to this frame. Among them, the I frame (intra-frame coded frame) is an independent frame with all information, which can be decoded independently without referring to other images. Both I frames and IDR frames use intra-frame prediction. In encoding and decoding, the first I frame is called an IDR frame to facilitate the control of the encoding and decoding process. The role of the IDR frame is to refresh immediately so that errors will not propagate. Starting from the IDR frame, a new video sequence is recalculated to start encoding or decoding.

[0075] In step S113, a number of T test decoding frames output by the decoder after decoding the test bitstream is counted, where T is a positive integer greater than or equal to 1.

[0076] In step S114, the number of frames stored in the decoder s is obtained according to the K-frame test video frames and the T-frame test decoding frames.

[0077] For example, Figure 4 As shown, assuming that K test video frames are input into the decoder and T test decoded frames are decoded and output, the number of frames s corresponding to the video size of the video to be decoded by the decoder can be calculated according to the following formula:

[0078] s=KT (2)

[0079] That is, when calculating the number of stored frames of the hardware chip of a decoder of a specific video size, you can first send a certain number (K ​​frames) of test video frames to the decoder, and count the number of decoded frames (T frames). Based on the difference between the two, you can get the number of stored frames of the hardware chip of the decoder.

[0080] The above method of calculating the number of hoarded frames requires that the frame type of the test video frame sent to the decoder at the end is not an IDR frame, because all reference frames after the IDR frame will only refer to this frame. Due to the semantics of the IDR frame, most hardware chips with decoding hoarded frames will clean up the original internal hoarded frames accordingly after receiving the IDR frame, which will cause the calculated number of hoarded frames to be inaccurate. For hardware chips with a hoarded frame number of s, the hoarded frame test results of the IDR frame and the (s-1) frames before and after the IDR frame are inaccurate and should be excluded.

[0081] In the disclosed embodiment, each K test video frame in the test code stream is input to the encoder in chronological order, and the T test decoded frames output by the corresponding decoder are recorded. In order to test the accuracy of the result, the last test result can be selected as the final number of frames stored in the decoder, that is, the difference between the last K test video frames input to the decoder and the T test decoded frames output by the corresponding decoder is used as the number of frames stored in the decoder s. Considering that the number of frames stored in the hardware chip is 3 in most cases, it is sufficient to ensure that the last 3 frames (the first preset number) of the test code stream are not IDR frames during the test. When encoding the test code stream, the frame type of each encoded test video frame can be controlled.

[0082] The decoding time detection method provided by the embodiment of the present disclosure can ensure that the test bit stream used to test the number of hoarded frames of the decoder does not include IDR frames, or the last 3 frames are not IDR frames, by controlling the frame type of each encoded test video frame, thereby ensuring the accuracy of the calculated number of hoarded frames of the decoder.

[0083] Figure 5 Schematically shows Figure 1 Step S110 in the embodiment is a flow chart of an exemplary embodiment. Figure 5 As shown above Figure 1 Step S110 in the embodiment may further include the following steps.

[0084] In step S115, the first second preset number of to-be-decoded video frames in the to-be-decoded video are obtained.

[0085] In an exemplary embodiment, obtaining the first second preset number of video frames to be decoded in the video to be decoded may include: obtaining the frame type of the video frames to be decoded in the video to be decoded; eliminating the video frames to be decoded whose frame type is the instant decoding refresh frame and the third preset number of video frames before and after it from the video to be decoded; and selecting the first second preset number of video frames to be decoded from the remaining video to be decoded.

[0086] In step S116, the K frames of the second preset number of to-be-decoded video frames input to the decoder at the i-th moment are obtained. i Frame video frame to be decoded, K i is a positive integer greater than or equal to 1, and i is a positive integer greater than or equal to 1 and less than or equal to the second preset number.

[0087] In an exemplary embodiment, obtaining K of the second preset number of to-be-decoded video frames input to the decoder at the i-th time i The frame to be decoded may include: at the jth moment, K jThe frame to be decoded video frame is input to the decoder so that there is no to-be-decoded video frame in the buffer of the decoder; wherein j is a positive integer greater than or equal to 1 and less than or equal to i.

[0088] use Figure 5 When the real-time detection method of the embodiment is used, it is necessary to implement slow start, that is, the first few frames of video frames to be decoded cannot be sent to the decoder too fast, so that the hardware chip of the decoder has sufficient time to complete the decoding of each frame, and there will be no phenomenon that the decoder has not completed decoding and the code stream data of the video frames to be decoded are accumulated, so that the calculation of the number of stored frames is more accurate. In the disclosed embodiment, the predetermined speed can be determined according to the specific decoder and the video frames to be decoded, as long as it can satisfy the fact that there are no video frames to be decoded in the buffer of the decoder, that is, no storage of frames occurs. For example, a critical speed can be determined according to a test, and this critical speed makes it possible for the decoder to not have the upper limit of the speed that can be reached by storing frames, and the predetermined speed can be selected to be less than or equal to the critical speed.

[0089] In step S117, the T value output by the decoder after decoding the second preset number of to-be-decoded video frames at the i-th moment is recorded. i Frame decoded video frame, T i is a positive integer greater than or equal to 1.

[0090] In step S118, according to the K i The frame to be decoded video frame and the T i The frame decodes the video frame and obtains the number of frames stored by the decoder at the i-th moment.

[0091] In step S119, the number of frames stored by the decoder s is determined according to the number of frames stored by the decoder at the i-th moment.

[0092] In actual business, if there is no Figure 3 The test bitstream in the embodiment can still obtain the number of stored frames of the decoder by real-time decoding of the video to be decoded. For example, the number of input frames (inputting the video frames to be decoded to the decoder) and output frames (decoding the video frames to be decoded and outputting the decoded video frames after the decoder decodes the video frames to be decoded) at each moment can be recorded in real time during the decoding process of the video to be decoded, and the number of stored frames at the corresponding moment can be obtained according to the difference between the two at the corresponding moment, and the minimum value of the number of stored frames at all moments is taken as the number of stored frames s of the decoder, and it can be updated in real time according to the decoding process:

[0093] s=min{K i -T i} (3)

[0094] Generally, the test can be completed using the first 10 frames (the second preset number) of the video stream of the video to be decoded. Similarly, in order to ensure the accuracy of the above calculation results, the first 10 frames of the video to be decoded should exclude the IDR frame. Considering that the number of frames is 3 in most cases, the calculation of formula (3) excludes the data of the IDR frame and the 2 frames before and after it (the third preset number) in the video frame to be decoded, that is, the IDR frame itself and the 2 frames before and after the IDR frame need to be excluded.

[0095] The decoding time detection method provided in the embodiment of the present disclosure can directly obtain the number of frames stored by the decoder by recording the real-time decoding process of the video frame to be decoded, without the need for additional test bit streams.

[0096] Figure 6 Schematically shows Figure 1 Step S120 in the embodiment is a flow chart in an exemplary embodiment. Figure 6 As shown above Figure 1 Step S120 in the embodiment may further include the following steps.

[0097] In step S121, according to the number of stored frames s, a first timestamp of when the k+sth video frame to be decoded in the video to be decoded is input into the decoder is obtained.

[0098] In step S122, a second timestamp of when the decoder decodes the kth video frame to be decoded in the video to be decoded is obtained.

[0099] In step S123, the first load decoding time is obtained according to the first timestamp and the second timestamp.

[0100] In the disclosed embodiment, the first timestamp of sending a frame to a decoder (inputting a video frame to be decoded to the decoder) and the second timestamp of outputting the frame can be recorded respectively, and the first load decoding time can be calculated according to the difference between the two:

[0101]

[0102] In the above formula (4), ΔT k represents the target decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded, represents a second timestamp of the kth video frame to be decoded in the video to be decoded, It represents the first timestamp of the k+sth video frame to be decoded in the video to be decoded input into the decoder, and s is the number of frames stored in the hardware chip of the decoder obtained by calculation in the above embodiment, that is, the first load decoding time (single-frame hardware decoding time or single-frame decoding time) of the kth video frame to be decoded in the video to be decoded is calculated according to the difference between the above first timestamp and the second timestamp.

[0103] The decoding time detection method provided by the embodiment of the present disclosure avoids the influence of the decoding frame hoarding situation of the hardware chip of the decoder on the calculation result when detecting the single-frame hardware decoding time of the decoder, and can improve the estimation accuracy of the single-frame hardware decoding time, because for a hardware chip with s number of hoarded frames, it is necessary to input the k+sth video frame to be decoded before the kth video frame to be decoded is decoded. For a hardware chip that does not hoard frames, s=0, and its calculation result is not affected.

[0104] Figure 7 Schematically shows Figure 1 Step S130 in the embodiment is a flow chart in an exemplary embodiment. Figure 7 As shown above Figure 1 Step S130 in the embodiment may further include the following steps.

[0105] In step S131, a third timestamp of when the decoder decodes the kth video frame to be decoded in the video to be decoded is obtained.

[0106] In step S132, a fourth timestamp of when the decoder decodes the k-1th video frame to be decoded in the video to be decoded is obtained.

[0107] In step S133, the second payload decoding time is obtained according to the third timestamp and the fourth timestamp.

[0108] In the embodiment of the present disclosure, the frame output speed of the hardware chip of the decoder can also be calculated in the following manner, where the second load decoding time is used as a measure. The second load decoding time is calculated by recording the third timestamp of the last frame output of the decoder (for example, the decoded frame of the k-1th video frame to be decoded) and the fourth timestamp of the current frame output (for example, the decoded frame of the kth video frame to be decoded), and the difference between the two is used to calculate the second load decoding time:

[0109]

[0110] In the above formula, represents the fourth timestamp of the k-1th frame of the video to be decoded when it is decoded from the decoder, represents the third timestamp of the kth frame of the video to be decoded from the decoder, ΔP kThe second load decoding time representing the time when the decoder is in a high load state (or a full load state) to decode the kth video frame to be decoded is calculated according to the difference between the third timestamp and the fourth timestamp.

[0111] The above formula (5) is suitable for the scenario where the decoder is running at full load, where the decoder is in full load state means that there are still a certain number of video frames to be decoded in the decoder buffer, and the decoder will not be idle at this time.

[0112] In an exemplary embodiment, determining the target decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded based on the first load decoding time and the second load decoding time may include: comparing the first load decoding time and the second load decoding time; determining the minimum value of the first load decoding time and the second load decoding time as the target decoding time for the decoder to decode the kth video frame to be decoded.

[0113] In the embodiment of the present disclosure, the first load decoding time and the second load decoding time can be combined to accurately calculate the target decoding time ΔR of the hardware chip of the decoder with the decoding frame hoarding condition. k The formula is:

[0114] ΔR k =min{ΔT k ,ΔP k} (6)

[0115] The decoding time detection method provided by the embodiment of the present disclosure is that the intervals of sending frames to the decoder are often uneven under the influence of external reasons such as network jitter. In certain cases, many frames may be sent at a time (the decoder is in a high-load state) or no frames may be sent for a period of time (the decoder is in a low-load state). When the decoder is in a high-load state, since there is an accumulation of video frames to be decoded inside the decoder, there are many frames to be decoded, so the decoder will keep sending frames. At this time, it is more accurate to use the above formula (5) for calculation. When the decoder is in a low-load state, it is more accurate to use the improved solution of the above formula (4).

[0116] Figure 8 The flowchart of a decoding time detection method according to an embodiment of the present disclosure is schematically shown. Figure 8 The method provided in the embodiment may be executed by any electronic device, such as a terminal device including a decoder, but the present disclosure is not limited thereto.

[0117] like Figure 8 As shown, the method provided by the embodiment of the present disclosure may include the following steps.

[0118] In step S801, the number of frames s stored by the decoder corresponding to the video size of the video to be decoded is determined.

[0119] The method for calculating the number of stored frames s may refer to the above embodiment.

[0120] In step S802, a first timestamp of when the k+sth video frame to be decoded in the video to be decoded is input into the decoder is recorded.

[0121] The method for calculating the first timestamp may refer to the above embodiment.

[0122] In step S803, a second timestamp of when the kth video frame to be decoded in the video to be decoded is input into the decoder is recorded.

[0123] The method for calculating the second timestamp may refer to the above embodiment.

[0124] In step S804, a difference between the first timestamp and the second timestamp is calculated as a first load decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded.

[0125] The method of calculating the first load decoding time may refer to the above embodiment.

[0126] In step S805, a third timestamp of when the decoder decodes the kth video frame to be decoded in the video to be decoded is obtained.

[0127] The method for calculating the third timestamp may refer to the above embodiment.

[0128] In step S806, a fourth timestamp of when the decoder decodes the k-1th video frame to be decoded in the video to be decoded is obtained.

[0129] The method for calculating the fourth timestamp may refer to the above embodiment.

[0130] In step S807, the second payload decoding time is obtained according to the third timestamp and the fourth timestamp.

[0131] The method of calculating the second load decoding time may refer to the above embodiment.

[0132] In step S808, when the decoder is in a low load state, the first load decoding time is used as a target decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded.

[0133] In step S809, when the decoder is in a high load state, the second load decoding time is used as a target decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded.

[0134] The method for calculating the target decoding time can refer to the above embodiment. In the embodiment of the present disclosure, in order to accurately calculate the single-frame decoding time of the decoder, the first load decoding time and the second load decoding time calculated by the following two methods are combined: ① the first timestamp of the incoming frame and the second timestamp of the outgoing frame, ② the frame output speed of the hardware chip of the decoder, and the single-frame decoding time of the decoder is estimated according to the minimum value of these two methods.

[0135] The decoding time detection method provided in the embodiment of the present disclosure handles the decoding frame hoarding situation of the hardware chip, and effectively avoids the deviation caused by uneven frame delivery due to abnormal situations such as network jitter, and can accurately calculate the single-frame decoding time of the hardware chip.

[0136] In an exemplary embodiment, the method may further include: if the video to be decoded is a cloud game video, determining whether to send the cloud game video to a terminal device including the decoder based on the target decoding time of the kth video frame to be decoded in the video to be decoded by the decoder.

[0137] The method provided in the embodiment of the present disclosure can be used to accurately estimate the single-frame decoding time (i.e., target decoding time) of the hardware chip of the decoder, and can be applied to cloud games or other video application scenarios, such as real-time audio and video scenarios such as video calls, interactive live broadcasts, and video conferences. The following is an example of cloud games, but the present disclosure is not limited to this.

[0138] Among them, cloud technology refers to a hosting technology that unifies hardware, software, network and other resources in a wide area network or local area network to realize data computing, storage, processing and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model application. It can form a resource pool, which is used on demand and flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the high development and application of the Internet industry, in the future, each item may have its own identification mark, and all need to be transmitted to the background system for logical processing. Data of different levels will be processed separately. All kinds of industry data require strong system backing support, which can only be achieved through cloud computing.

[0139] Among them, cloud gaming, also known as gaming on demand, is an online gaming technology based on cloud computing technology. Cloud gaming technology enables thin clients with relatively limited graphics processing and data computing capabilities to run high-quality games. In a cloud gaming scenario, the game is not played on the player's game terminal, but on a cloud server, which renders the game scene into a video and audio stream and transmits it to the player's game terminal over the network. The player's game terminal does not need to have powerful graphics computing and data processing capabilities, but only needs to have basic streaming media playback capabilities and the ability to obtain player input commands and send them to the cloud server.

[0140] Fig. 9 A schematic diagram schematically shows the application of the decoding time detection method provided by an embodiment of the present disclosure to a cloud gaming scenario.

[0141] See also Fig. 9 , the terminal device 920 is connected to the cloud server 910 through the network 930, and the network 930 can be a wide area network or a local area network, or a combination of the two.

[0142] The terminal device 920 (running a client, such as a game client, an educational learning client, a search client, etc.) is provided with a hardware chip of a decoder, which can be used to decode the cloud game video to be decoded obtained from the cloud server 910.

[0143] Cloud gaming puts game rendering on the cloud server 910 instead of rendering locally on the user's terminal device 920. Cloud gaming sends the game player's operation instructions directly to the cloud server 910, which completes the image rendering. The rendered image will be encoded into a video stream (cloud game video to be decoded), and then the video stream will be sent to the user's terminal device 920 to complete the decoding and display. When the user plays the game on the terminal device 920, the effect looks the same as local rendering due to the high quality of video encoding and low latency.

[0144] For the user's terminal device 920, cloud games can be played immediately without downloading. In the past, users had to download an installation package on the terminal device 920 to play a game, and the installation package was generally large, which wasted the user's time and occupied the storage space of the terminal device 920. In the past, the cost of trying a game was relatively high, but with cloud games, users can open and play anytime, anywhere, and easily experience various games.

[0145] Secondly, local games need to be rendered locally, which places high requirements on the hardware configuration of the game player's terminal device 920. After adopting cloud games, the rendering is moved to the cloud server 910, which greatly reduces the configuration requirements of the user's terminal device 920. Secondly, cloud games solve the problem of cross-platform compatibility. The video streaming solution used by cloud games solves the problem of intercommunication between terminal devices on various platforms.

[0146] It should be noted that the method provided by the embodiment of the present disclosure can be applied to the hardware capability detection logic of the terminal device of the cloud gaming service, and can also be applied to the real-time statistics of the decoding process. By obtaining the accurate single-frame hardware decoding time, it is determined whether the terminal device supports a high-quality cloud gaming service experience. At the same time, it can also provide more accurate decoding data statistics for the decoder.

[0147] It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principle of the present disclosure, and the embodiments of the present disclosure are not limited in this respect. On the contrary, the embodiments of the present disclosure can be applied to any applicable scenario that requires accurate estimation of the single-frame hardware decoding time of the decoder.

[0148] Fig.10 The flowchart of the decoding method according to an embodiment of the present disclosure is schematically shown. The decoding method provided by the embodiment of the present disclosure can be executed by any electronic device, such as a terminal device, but the present disclosure is not limited thereto. Fig.10 As shown, the decoding method provided by the embodiment of the present disclosure may include the following steps.

[0149] In step S110 , the number s of frames stored in the decoder is determined, where s is an integer greater than or equal to 0.

[0150] In step S120, according to the number of stored frames s, a first load decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded is obtained, where k is a positive integer greater than 1.

[0151] In an exemplary embodiment, obtaining the first load decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded according to the number of stored frames s may include: obtaining the first timestamp of the k+sth video frame to be decoded in the video to be decoded being input into the decoder according to the number of stored frames s; obtaining the second timestamp for the decoder to decode the kth video frame to be decoded in the video to be decoded; and obtaining the first load decoding time according to the first timestamp and the second timestamp.

[0152] In step S130, a second load decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded is obtained according to the kth video frame to be decoded and the k-1th video frame to be decoded in the video to be decoded.

[0153] In step S140, a target decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded is determined according to the first load decoding time and the second load decoding time.

[0154] Fig.10 In the embodiment, the method for specifically detecting and obtaining the target decoding time of the decoder decoding the kth to-be-decoded video frame in the to-be-decoded video can refer to the above Figure 1-9 Description of embodiments.

[0155] In step S1010, the video to be decoded is decoded according to the target decoding time of the kth video frame to be decoded.

[0156] The decoding method provided in the embodiments of the present disclosure can better control the decoding process of the video to be decoded by obtaining the accurate single-frame decoding time of the decoder.

[0157] Fig.11 The block diagram of a decoding time detection device according to an embodiment of the present disclosure is schematically shown. The decoding time detection device provided by the embodiment of the present disclosure can be set in any electronic device, such as a terminal device. Fig.11 As shown, the decoding time detection device 1100 provided in the embodiment of the present disclosure may include: a decoder stack frame number determination unit 1110, a first load decoding time acquisition unit 1120, a second load decoding time acquisition unit 1130 and a target decoding time acquisition unit 1140.

[0158] In the embodiment of the present disclosure, the decoder frame hoarding number determination unit 1110 can be used to determine the decoder's hoarding frame number s, where s is an integer greater than or equal to 0. The first load decoding time acquisition unit 1120 can be used to obtain the first load decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded according to the hoarding frame number s, where k is a positive integer greater than 1. The second load decoding time acquisition unit 1130 can be used to obtain the second load decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded according to the kth video frame to be decoded and the k-1th video frame to be decoded in the video to be decoded. The target decoding time acquisition unit 1140 can be used to determine the target decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded according to the first load decoding time and the second load decoding time.

[0159] The decoding time detection device provided by the embodiment of the present disclosure, on the one hand, by determining the number of frames s of the decoder, the first load decoding time of the decoder decoding the kth video frame to be decoded in the video to be decoded can be obtained according to the determined number of frames s of the decoder, that is, the embodiment of the present disclosure optimizes the detection method of the single frame decoding time of the decoder with the decoding frame hoarding situation, which can make the single frame decoding time estimation of the decoder more accurate; on the other hand, further according to the kth video frame to be decoded and the k-1th video frame to be decoded in the video to be decoded, the second load decoding time of the decoder decoding the kth video frame to be decoded in the video to be decoded is obtained, so as to adapt to the scene such as uneven frame delivery of the decoder caused by factors such as network jitter. At the same time, the embodiment of the present disclosure also finally determines the target decoding time of the decoder decoding the kth video frame to be decoded in the video to be decoded according to the first load decoding time and the second load decoding time determined in the above manner, optimizes the decoder with the decoding frame hoarding situation, and effectively avoids the deviation of the single frame decoding time estimation caused by the uneven frame delivery caused by abnormal situations such as network jitter, so as to accurately detect the single frame decoding time of the decoder. The solution provided by the embodiments of the present disclosure can be applied to, for example, cloud gaming or other video application scenarios where the estimation accuracy of the time required to decode a single frame is relatively high.

[0160] In an exemplary embodiment, the decoder frame hoarding number determination unit 1110 may include: a test code stream obtaining unit, which may be used to obtain a test code stream having the same video size as the video to be decoded; a test video frame input unit, which may be used to input K test video frames in the test code stream to the decoder, where K is a positive integer greater than or equal to 1, and the last first preset number of test video frames in the K test video frames are not instant decoding refresh frames; a test decoding frame counting unit, which may be used to count T test decoding frames output by the decoder when decoding the test code stream, where T is a positive integer greater than or equal to 1; and a hoarding frame number obtaining unit, which may be used to obtain the number of hoarding frames s of the decoder according to the K test video frames and the T test decoding frames.

[0161] In an exemplary embodiment, the decoder frame number determination unit 1110 may include: a preset to-be-decoded video frame obtaining unit, which may be used to obtain the first second preset number of to-be-decoded video frames in the to-be-decoded video; a to-be-decoded video frame input unit, which may be used to obtain K of the second preset number of to-be-decoded video frames input to the decoder at the i-th moment. i Frame video frame to be decoded, K i is a positive integer greater than or equal to 1, i is a positive integer greater than or equal to 1 and less than or equal to the second preset number; the decoded video frame output unit can be used to record the T output of the decoder decoding the second preset number of to-be-decoded video frames at the i-th moment.i Frame decoded video frame, T i is a positive integer greater than or equal to 1; a predetermined time frame number acquisition unit can be used to obtain the number of frames according to the K i The frame to be decoded video frame and the T i The frame decoder decodes the video frame to obtain the number of frames stored by the decoder at the i-th moment; the number of frames stored determination unit can be used to determine the number of frames stored by the decoder s according to the number of frames stored by the decoder at the i-th moment.

[0162] In an exemplary embodiment, the preset to-be-decoded video frame obtaining unit may include: a video frame type obtaining unit, which can be used to obtain the frame type of the to-be-decoded video frame in the to-be-decoded video; an instant decoding refresh frame elimination unit, which can be used to eliminate the to-be-decoded video frames whose frame type is the instant decoding refresh frame and the third preset number of to-be-decoded frames before and after it from the to-be-decoded video; a second number of to-be-decoded video frame selection unit, which is used to select the first second preset number of to-be-decoded video frames from the remaining to-be-decoded video.

[0163] In an exemplary embodiment, the to-be-decoded video frame input unit may include: a slow start unit, which may be used to convert K at a predetermined speed at a jth moment. j The video frame to be decoded is input to the decoder so that there is no video frame to be decoded in the buffer of the decoder. Wherein, j is a positive integer greater than or equal to 1 and less than or equal to i.

[0164] In an exemplary embodiment, the first load decoding time obtaining unit 1120 may include: a first timestamp obtaining unit, which can be used to obtain the first timestamp of the k+sth video frame to be decoded in the video to be decoded input into the decoder according to the number of frames s; a second timestamp obtaining unit, which can be used to obtain the second timestamp of the kth video frame to be decoded in the video to be decoded when the decoder decodes the kth video frame to be decoded in the video to be decoded; a first load decoding time calculation unit, which can be used to obtain the first load decoding time according to the first timestamp and the second timestamp.

[0165] In an exemplary embodiment, the second load decoding time obtaining unit 1130 may include: a third timestamp obtaining unit, which can be used to obtain a third timestamp of the kth video frame to be decoded in the video to be decoded when the decoder decodes it; a fourth timestamp obtaining unit, which can be used to obtain a fourth timestamp of the k-1th video frame to be decoded in the video to be decoded when the decoder decodes it; and a second load decoding time calculation unit, which can be used to obtain the second load decoding time based on the third timestamp and the fourth timestamp.

[0166] In an exemplary embodiment, the target decoding time obtaining unit 1140 may include: a comparison unit, which can be used to compare the first load decoding time and the second load decoding time; a target decoding time determination unit, which can be used to determine the minimum value of the first load decoding time and the second load decoding time as the target decoding time for the decoder to decode the kth video frame to be decoded.

[0167] In an exemplary embodiment, the decoding time detection device 1100 may also include: a cloud gaming performance determination unit, which may be used to determine whether to send the cloud gaming video to a terminal device including the decoder based on the target decoding time of the kth video frame to be decoded in the video to be decoded by the decoder if the video to be decoded is a cloud gaming video.

[0168] The specific implementation of each unit in the decoding time detection device provided by the embodiment of the present disclosure can refer to the content of the above-mentioned decoding time detection method, which will not be repeated here.

[0169] Fig.12 The block diagram of a decoding device according to an embodiment of the present disclosure is schematically shown. The decoding device provided by the embodiment of the present disclosure can be set in any electronic device, such as a terminal device. Fig.12 As shown, the decoding device 1200 provided by the embodiment of the present disclosure may include: a decoder frame number determination unit 1110, a first load decoding time acquisition unit 1120, a second load decoding time acquisition unit 1130, a target decoding time acquisition unit 1140 and a video decoding unit 1210 to be decoded.

[0170] The implementation of the decoder frame number determination unit 1110, the first load decoding time acquisition unit 1120, the second load decoding time acquisition unit 1130, and the target decoding time acquisition unit 1140 can refer to the above Fig.11 Embodiment: The to-be-decoded video decoding unit 1210 may be configured to decode the to-be-decoded video according to a target decoding time of the kth to-be-decoded video frame.

[0171] The specific implementation of each unit in the decoding device provided in the embodiment of the present disclosure can refer to the content of the above-mentioned decoding method, which will not be repeated here.

[0172] It should be noted that although several units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0173] An embodiment of the present disclosure provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the decoding time detection method described in the above embodiment is implemented.

[0174] An embodiment of the present disclosure provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the decoding method described in the above embodiment is implemented.

[0175] An embodiment of the present disclosure provides an electronic device, comprising: at least one processor; a storage device configured to store at least one program, and when the at least one program is executed by the at least one processor, the at least one processor implements the decoding time detection method as described in the above embodiments.

[0176] An embodiment of the present disclosure provides an electronic device, comprising: at least one processor; and a storage device configured to store at least one program, so that when the at least one program is executed by the at least one processor, the at least one processor implements the decoding method described in the above embodiment.

[0177] The embodiment of the present disclosure provides a computer program product or a computer program, which includes a computer instruction, and the computer instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the decoding time detection method provided in the above various optional embodiments.

[0178] The embodiment of the present disclosure provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the decoding method provided in the above various optional embodiments.

[0179] Fig.13 FIG. 1 shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present disclosure. It should be noted that: Fig.13 The electronic device 1300 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0180] like Fig.13As shown, the electronic device 1300 includes a central processing unit (CPU) 1301, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1302 or the program loaded from the storage part 1308 to the random access memory (RAM) 1303. In the RAM 1303, various programs and data required for system operation are also stored. The CPU 1301, ROM 1302 and RAM 1303 are connected to each other through a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0181] The following components are connected to the I / O interface 1305: an input section 1306 including a keyboard, a mouse, etc.; an output section 1307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1308 including a hard disk, etc.; and a communication section 1309 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1309 performs communication processing via a network such as the Internet. A drive 1310 is also connected to the I / O interface 1305 as needed. A removable medium 1311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1310 as needed so that a computer program read therefrom is installed into the storage section 1308 as needed.

[0182] In particular, according to an embodiment of the present disclosure, the process described below with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 1309, and / or installed from a removable medium 1311. When the computer program is executed by the central processing unit (CPU) 1301, various functions defined in the method and / or device of the present application are executed.

[0183] It should be noted that the computer-readable storage medium shown in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with at least one wire, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM (Erasable Programmable Read Only Memory, Erasable Programmable Read Only Memory) or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable storage medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical cable, RF (Radio Frequency), etc., or any suitable combination of the above.

[0184] The flowchart and block diagram in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the method, device and computer program product according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains at least one executable instruction for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0185] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0186] As another aspect, the present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the following embodiment. For example, the electronic device may implement the following Figure 1 or Figure 3 or Figure 5 or Figure 6 or Figure 7 or Figure 8 or Fig.10 The steps shown.

[0187] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0188] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modifications, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure.

[0189] The description and examples are to be regarded as exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims. It should be understood that the present disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A decoding time detection method, characterized in that: include: Determine the number of frames stored by the decoder, s is an integer greater than or equal to 0; According to the number of stored frames s, a first load decoding time of the decoder decoding the kth video frame to be decoded in the video to be decoded is obtained, where k is a positive integer greater than 1; According to the kth video frame to be decoded and the k-1th video frame to be decoded in the video frames to be decoded, obtaining a second load decoding time for the decoder to decode the kth video frame to be decoded in the video frames to be decoded; A target decoding time for the decoder to decode a kth video frame to be decoded in the video to be decoded is determined according to the first load decoding time and the second load decoding time.

2. The method according to claim 1, characterized in that Determine the number of frames stored by the decoder, s, including: Obtaining a test bitstream having the same video size as the video to be decoded; Inputting K test video frames in the test bitstream to the decoder, where K is a positive integer greater than or equal to 1, and the last first preset number of test video frames in the K test video frames are not instant decoding refresh frames; Counting T test decoding frames output by the decoder when decoding the test bitstream, where T is a positive integer greater than or equal to 1; The number of frames stored in the decoder s is obtained according to the K-frame test video frame and the T-frame test decoding frame.

3. The method according to claim 1, characterized in that Determine the number of frames stored by the decoder, s, including: Obtaining a first second preset number of to-be-decoded video frames in the to-be-decoded video; Obtain K of the second preset number of to-be-decoded video frames input to the decoder at the i-th moment i Frame video frame to be decoded, K i is a positive integer greater than or equal to 1, i is a positive integer greater than or equal to 1 and less than or equal to the second preset number; Record the T output by the decoder decoding the second preset number of to-be-decoded video frames at the i-th moment i Frame decoded video frame, T i is a positive integer greater than or equal to 1; According to the K i The frame to be decoded video frame and the T i Decode the video frame by frame to obtain the number of frames stored by the decoder at the i-th moment; The number of frames stored by the decoder s is determined according to the number of frames stored by the decoder at the i-th moment.

4. The method according to claim 3, characterized in that Obtaining the first second preset number of to-be-decoded video frames in the to-be-decoded video, comprising: Obtaining a frame type of a video frame to be decoded in the video to be decoded; Eliminate, from the video to be decoded, the frame type of which is an instant decoding refresh frame and a third preset number of video frames to be decoded before and after the frame type; The first second preset number of video frames to be decoded are selected from the remaining videos to be decoded.

5. The method according to claim 3, characterized in that: Obtain K of the second preset number of to-be-decoded video frames input to the decoder at the i-th moment i Frame Video frame to be decoded, including: At the jth moment, K is moved at a predetermined speed. j Inputting a video frame to be decoded into the decoder so that there is no video frame to be decoded in the buffer of the decoder; Here, j is a positive integer greater than or equal to 1 and less than or equal to i.

6. The method according to claim 1, characterized in that Obtaining, according to the number of stored frames s, a first load decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded, comprising: According to the number of stored frames s, obtaining a first timestamp of a k+s-th video frame to be decoded in the video to be decoded being input into the decoder; Obtaining a second timestamp of the kth video frame to be decoded in the video to be decoded by the decoder; The first load decoding time is obtained according to the first timestamp and the second timestamp.

7. The method according to claim 1, characterized in that Obtaining, according to the kth video frame to be decoded and the k-1th video frame to be decoded in the video frames to be decoded, a second load decoding time for the decoder to decode the kth video frame to be decoded in the video frames to be decoded, comprising: Obtaining a third timestamp when the decoder decodes the kth video frame to be decoded in the video to be decoded; Obtaining a fourth timestamp of the k-1th to-be-decoded video frame in the to-be-decoded video decoded by the decoder; The second payload decoding time is obtained according to the third timestamp and the fourth timestamp.

8. The method according to claim 1, characterized in that Determining, according to the first load decoding time and the second load decoding time, a target decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded, comprising: comparing the first payload decoding time with the second payload decoding time; A minimum value between the first load decoding time and the second load decoding time is determined as a target decoding time for the decoder to decode the kth video frame to be decoded.

9. The method according to claim 1, characterized in that: Also includes: If the video to be decoded is a cloud game video, it is determined whether to send the cloud game video to a terminal device including the decoder according to the target decoding time of the kth video frame to be decoded in the video to be decoded by the decoder.

10. A decoding method, characterized in that: include: Determine the number of frames stored by the decoder, s is an integer greater than or equal to 0; According to the number of stored frames s, a first load decoding time of the decoder decoding the kth video frame to be decoded in the video to be decoded is obtained, where k is a positive integer greater than 1; According to the kth video frame to be decoded and the k-1th video frame to be decoded in the video frames to be decoded, obtaining a second load decoding time for the decoder to decode the kth video frame to be decoded in the video frames to be decoded; Determining a target decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded according to the first load decoding time and the second load decoding time; The video to be decoded is decoded according to the target decoding time of the kth video frame to be decoded.

11. The method according to claim 10, characterized in that Obtaining, according to the number of stored frames s, a first load decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded, comprising: According to the number of stored frames s, obtaining a first timestamp of a k+s-th video frame to be decoded in the video to be decoded being input into the decoder; Obtaining a second timestamp of the kth video frame to be decoded in the video to be decoded by the decoder; The first load decoding time is obtained according to the first timestamp and the second timestamp.

12. A decoding time detection device, characterized in that: include: A decoder frame storage quantity determination unit, used to determine the decoder frame storage quantity s, s is an integer greater than or equal to 0; A first load decoding time obtaining unit, configured to obtain, according to the number of stored frames s, a first load decoding time for the decoder to decode a kth video frame to be decoded in the video to be decoded, where k is a positive integer greater than 1; A second load decoding time obtaining unit, configured to obtain a second load decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded according to the kth video frame to be decoded and the k-1th video frame to be decoded in the video to be decoded; The target decoding time obtaining unit is used to determine the target decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded according to the first load decoding time and the second load decoding time.

13. A decoding device, characterized in that: include: A decoder frame storage quantity determination unit, used to determine the decoder frame storage quantity s, s is an integer greater than or equal to 0; A first load decoding time obtaining unit, configured to obtain, according to the number of stored frames s, a first load decoding time for the decoder to decode a kth video frame to be decoded in the video to be decoded, where k is a positive integer greater than 1; A second load decoding time obtaining unit, configured to obtain a second load decoding time for the decoder to decode the kth video frame to be decoded in the video to be decoded according to the kth video frame to be decoded and the k-1th video frame to be decoded in the video to be decoded; A target decoding time obtaining unit, configured to determine a target decoding time for the decoder to decode a kth video frame to be decoded in the video to be decoded according to the first load decoding time and the second load decoding time; The to-be-decoded video decoding unit is used to decode the to-be-decoded video according to the target decoding time of the kth to-be-decoded video frame.

14. An electronic device, characterized in that: include: at least one processor; A storage device configured to store at least one program, when the at least one program is executed by the at least one processor, enables the at least one processor to implement the decoding time detection method as described in any one of claims 1 to 9 or the decoding method as described in any one of claims 10 to 11.

15. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the decoding time detection method according to any one of claims 1 to 9 or the decoding method according to any one of claims 10 to 11 is implemented.

Images