Decoding processing method, device, computer equipment and storage medium

Abstract

This application proposes a decoding processing method, apparatus, computer device, and storage medium. The decoding processing method includes: during hard decoding of first multimedia data, obtaining the operating state of the decoder, where the operating state includes a working state or an idle state; if the decoder is in the idle state, obtaining second multimedia data, where the second multimedia data is a copy of the first multimedia data; and, in the idle state, using the decoder to decode the second multimedia data. This application can stabilize the performance of the decoder and improve decoding efficiency.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a decoding processing method, a decoding processing device, a computer device, and a computer-readable storage medium. Background Art

[0002] In the multimedia service field, encoded multimedia data must be decoded and then rendered on a page for users to view. However, if the decoder performance is insufficient, the decoding process can be unstable, resulting in video freezes and poor viewing quality.

[0003] To address this issue, a common approach is to use the hardware decoder's Application Program Interface (API) to accelerate the video decoding process. However, even with hardware-accelerated decoding, the high demands of ultra-low latency applications can lead to inherent decoder performance issues (including slow decoding speeds and instability), which can reduce the efficiency of the hardware decoding process. Therefore, stabilizing decoder performance is a pressing technical challenge. Summary of the Invention

[0004] The embodiments of the present application provide a decoding processing method, apparatus, computer device, and storage medium, which can stabilize the performance of the decoder and improve decoding efficiency.

[0005] On the one hand, an embodiment of the present application provides a decoding processing method, the method comprising:

[0006] During hard decoding of the first multimedia data, obtaining an operating state of the decoder, where the operating state includes a working state or an idle state;

[0007] If the decoder is in an idle state, obtaining second multimedia data, where the second multimedia data is obtained by copying the first multimedia data;

[0008] In the idle state, the second multimedia data is decoded by using a decoder.

[0009] In one aspect, an embodiment of the present application provides a decoding processing device, the device comprising:

[0010] an acquiring unit, configured to acquire an operating state of the decoder during hard decoding of the first multimedia data, where the operating state includes a working state or an idle state;

[0011] The acquiring unit is further configured to acquire second multimedia data if the decoder is in an idle state, where the second multimedia data is obtained by copying the first multimedia data;

[0012] The decoding unit is configured to decode the second multimedia data using a decoder in an idle state.

[0013] In one embodiment, the decoding unit is further configured to: suspend decoding processing of the second multimedia data when the running state of the decoder is switched from the idle state to the working state.

[0014] In one embodiment, the first multimedia data refers to a multimedia data stream, the multimedia data stream includes N data packets, each data packet encapsulates M media frames, and N and M are both positive integers; the decoding processing device further includes a processing unit:

[0015] The processing unit is configured to, when the decoder receives an i-th data packet of the multimedia data stream, copy the i-th data packet to obtain a clone of the i-th data packet, wherein the second multimedia data includes the clone of the i-th data packet, where i is a positive integer and i≤N;

[0016] The decoding unit is further configured to: when the decoder is in a working state, decode each media frame in the i-th data packet, and output the decoded media frames in the i-th data packet.

[0017] In one possible implementation, the clone packet of the i-th data packet contains M clone frames, and the M clone frames correspond one-to-one to the M media frames in the i-th data packet; the decoding unit is specifically used to: in an idle state, use a decoder to decode each clone frame in the clone packet of the i-th data packet.

[0018] In a possible implementation, the processing unit is further configured to: when the decoder receives the (i+1)th data packet of the multimedia data stream, copy the (i+1)th data packet to obtain a clone of the (i+1)th data packet;

[0019] The processing unit is further configured to delete the cloned packet of the i-th data packet, and the second multimedia data includes the cloned packet of the i+1-th data packet.

[0020] In a possible implementation, the acquisition unit is further used to: acquire performance parameters of the decoder during hard decoding of the first multimedia data; if the performance parameters of the decoder do not meet the performance index requirements, trigger the step of acquiring the operating status of the decoder.

[0021] In one possible implementation, the performance parameter of the decoder includes a decoding duration of the first multimedia data, and the performance indicator requirement includes a reference decoding frame rate; and the processing unit is further configured to:

[0022] Determine the actual decoding frame rate of the decoder based on the decoding time;

[0023] If the actual decoding frame rate of the decoder is less than the reference decoding frame rate, it is determined that the performance parameters of the decoder do not meet the performance indicator requirements.

[0024] In one possible implementation, the first multimedia data includes multiple media frames, the performance parameter of the decoder includes a decoding duration of each media frame in the first multimedia data, and the performance indicator requirement includes a reference decoding timing range; and the processing unit is further configured to:

[0025] Calculate the decoding speed timing of the decoder based on the decoding duration of each media frame;

[0026] If the decoding speed timing of the decoder exceeds the reference decoding timing range, it is determined that the performance parameters of the decoder do not meet the performance index requirements.

[0027] In a possible implementation, a first thread, a second thread, a third thread, and a fourth thread are run in the decoder, and the first thread, the second thread, the third thread, and the fourth thread are run in parallel with each other;

[0028] The first thread is used to perform hard decoding on the first multimedia data, and the second thread is used to sequentially output the decoded first multimedia data;

[0029] The third thread is used to copy the first multimedia data to obtain second multimedia data;

[0030] The fourth thread is used to obtain the running state of the decoder, and obtain and decode the second multimedia data when the decoder is in an idle state.

[0031] In one possible implementation, the first multimedia data refers to a cloud game video stream; and the processing unit is further configured to:

[0032] Obtain the scene requirements of the cloud game, which are used to indicate the decoding mode of the cloud game video stream;

[0033] If the scenario requirements indicate that the cloud gaming video game adopts the accelerated decoding mode, the running status of the decoder is obtained during the hard decoding of the cloud gaming video stream.

[0034] On the one hand, an embodiment of the present application provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the above-mentioned decoding processing method.

[0035] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is read and executed by a processor of a computer device, the computer device executes the above-mentioned decoding processing method.

[0036] In one aspect, embodiments of the present application provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the above-described decoding method.

[0037] In an embodiment of the present application, during the process of hard decoding the first multimedia data, the operating state of the decoder can be obtained, and the operating state can include an idle state or a working state. If the operating state of the decoder is the idle state, second multimedia data obtained by copying the first multimedia data can be obtained, and the second multimedia data can be decoded by the decoder in the idle state. The present application can utilize the idle state of the decoder to decode the cloned first multimedia data while the decoder is hard decoding the first multimedia data, thereby maintaining a continuous and stable operating frequency of the decoder, thereby stabilizing the performance of the decoder and improving decoding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0039] Figure 1 This is a flowchart of a decoding processing method provided in an embodiment of the present application;

[0040] Figure 2 1 is a schematic diagram of a decoding speed timing of a decoder provided in an embodiment of the present application;

[0041] Figure 3 is a schematic diagram of a reference decoding timing range provided in an embodiment of the present application;

[0042] Figure 4 is a schematic diagram of another reference decoding timing range provided in an embodiment of the present application;

[0043] Figure 5 This is a flowchart of another decoding processing method provided by an embodiment of the present application;

[0044] Figure 6 This is a schematic structural diagram of a decoding processing device provided in an embodiment of the present application;

[0045] Figure 7 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0047] The embodiments of this application propose a decoding processing solution that can stabilize the performance of the decoder, specifically enabling the decoder to maintain a constant decoding frame rate and maintain the stability of the decoding speed timing; it can be applied to business scenarios that require hardware decoding acceleration, such as ultra-high frame rate real-time streaming media transmission and playback scenarios, and cloud gaming decoding and rendering scenarios that require ultra-low latency, etc. Next, the principle of this decoding processing solution is explained accordingly:

[0048] First, the first multimedia data that needs to be hard-decoded is obtained. The first multimedia data can be video data, audio data, cloud game data, etc. Moreover, the first multimedia data can be a multimedia data stream, or any data packet included in a multimedia data stream. Next, the obtained first multimedia data can be copied to obtain the second multimedia data. It can be understood that the first multimedia data contains multiple media frames, each media frame will be copied to obtain a corresponding clone frame, and the second multimedia data contains these clone frames; further, in the process of hard decoding the first multimedia data, the operating status of the decoder can be obtained. The operating status of the decoder can include a working state or an idle state; if the operating state of the decoder obtained is an idle state, the second multimedia data can be decoded and processed by the decoder in the idle state. In this way, the decoder can be ensured to be at a constant operating frequency at all times, thereby improving the performance of the decoder, enabling the decoder to output a constant decoding frame rate, and maintain the stability of the decoding speed timing, thereby effectively improving the decoding efficiency.

[0049] In one possible implementation, the decoding processing solution of this application can be combined with blockchain technology. For example, data such as the first multimedia data and the second multimedia data can be uploaded to a blockchain for storage, ensuring that the data on the blockchain is not easily tampered with. Based on the solution provided in the embodiments of this application, during the hard decoding of the first multimedia data, both the first and second multimedia data can be sent to and stored on the blockchain network. This allows the decoder to retrieve the second multimedia data when the decoder is idle. Furthermore, the decoder can be used to decode the second multimedia data in the idle state.

[0050] The following is an introduction to the technical terms involved in the embodiments of this application:

[0051] Cloud technology is a general term for network technologies, information technologies, integration technologies, management platform technologies, and application technologies based on the cloud computing business model. It can form a resource pool that can be used flexibly and conveniently on demand. Cloud computing technology will become a crucial support. Backend services for technical network systems, such as video websites, image websites, and more portals, require extensive computing and storage resources. With the rapid development and application of the internet industry, every item will likely have its own unique identifier, requiring transmission to backend systems for logical processing. Different levels of data will be processed separately, and data from various industries will require a strong system backend, which can only be achieved through cloud computing.

[0052] At present, cloud technology is mainly divided into cloud basic technology and cloud application. Cloud basic technology can be further divided into: cloud computing, cloud storage, database and big data; cloud application can be further divided into: medical cloud, cloud Internet of Things, cloud security, cloud call, private cloud, public cloud, hybrid cloud, cloud gaming, cloud education, cloud conferencing, cloud social networking and artificial intelligence cloud services.

[0053] From the perspective of basic technology, the decoding and processing method of the present application involves cloud computing under cloud technology; from the perspective of application, the decoding and processing method of the present application involves cloud gaming under cloud technology.

[0054] Cloud computing refers to the delivery and usage model of IT infrastructure, enabling on-demand, scalable access to required resources over the internet. Broadly speaking, cloud computing refers to the delivery and usage model of services, enabling on-demand, scalable access to required services over the internet. These services can be IT-related, software-related, internet-related, or other services. Cloud computing is the product of the convergence of traditional computer and network technologies, including grid computing, distributed computing, parallel computing, utility computing, network storage technologies, virtualization, and load balancing.

[0055] Cloud gaming, also known as gaming on demand, is an online gaming technology based on cloud computing. Cloud gaming enables thin clients with relatively limited graphics and data processing capabilities to run high-quality games. In cloud gaming scenarios, the game runs not on the player's terminal but on a cloud server. The cloud server renders the game scene into a video and audio stream, which is then transmitted to the player's terminal over the network. The player's terminal does not need powerful graphics and data processing capabilities; it only needs basic streaming capabilities and the ability to receive player input and send it to the cloud server.

[0056] The decoding processing solution provided in this application involves large-scale calculations and requires large computing power and storage space. Therefore, in a feasible implementation of this application, a computer device can obtain sufficient computing power and storage space through cloud computing technology.

[0057] Furthermore, the decoding processing solution provided in this application can be applicable to cloud gaming scenarios. The cloud gaming data (such as game image data, game audio data, etc.) in the cloud gaming scenario can be used as the first multimedia data, and then hard decoding acceleration processing can be performed on the cloud gaming data, thereby stabilizing the decoding performance of the decoder of the game client to a large extent, thereby improving the user's gaming experience.

[0058] Regarding the description of the decoding processing scheme mentioned above, the following points need to be explained:

[0059] ① The decoding processing scheme mentioned above can be executed by a computer device, which can be a terminal device or a server. Among them, the terminal device can be a mobile phone, tablet computer, laptop computer, PDA, mobile internet device (MID), vehicle, vehicle-mounted equipment, roadside equipment, aircraft, wearable device, smart TV, etc., such as smart watch, smart bracelet, pedometer, etc., and other devices with decoding processing functions, etc.; the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as basic cloud computing services such as big data and artificial intelligence platforms, etc.

[0060] ② The computer equipment mentioned above needs to be configured with an Intel CPU (Intel Central Processing Unit) integrated graphics card, and the video decoding module of the Intel CPU integrated graphics card used in the business scenario performs video hardware decoding operations.

[0061] ③The decoding processing scheme mentioned above can be combined with blockchain technology. Furthermore, the decoding processing process of the second multimedia data using the decoder in this application can be executed on the blockchain. This can not only ensure the fairness and justice of the decoding processing process, but also make the decoding processing process traceable, thereby improving the security of the decoding processing process.

[0062] ④ The decoding processing solution mentioned above can be applied to the decoding processing module of the client in the cloud gaming scenario, and can also be used in any business scenario that requires ultra-low latency hardware video decoding. The high-performance decoding mode of the decoding processing solution provided by this application improves the hardware decoding speed. Since the decoder is used to decode the cloned multimedia data in the idle state, the decoding speed timing stability can be guaranteed, laying a solid foundation for the ultimate user experience on the product side.

[0063] Based on the above analysis, the following Figure 1 Describe the decoding method of this application. Figure 1 , Figure 1 This is a flowchart of a decoding processing method provided by an embodiment of the present application. The decoding processing method can be executed by a computer device, which can be, for example, a vehicle-mounted device, a smart phone, a tablet computer, a smart wearable device, or other terminal device equipped with a decoder. Figure 1As shown, the decoding processing method may include steps S110 to S140. Among them:

[0064] S110: During hard decoding of the first multimedia data, an operating state of the decoder is obtained, where the operating state includes a working state or an idle state.

[0065] In an embodiment of the present application, the first multimedia data may be any data packet in a multimedia data stream, or the first multimedia data may be a separate data. The first multimedia data may include but is not limited to: video data, audio data, or cloud game data, etc. Moreover, the first multimedia data is data encoded by an encoder, and the data format of the first multimedia data may include but is not limited to: H.264 format (a new generation of video compression format specified by the International Telecommunication Union), MPEG-4 (Moving Pictures Experts Group-4, an international standard dynamic image format) format, etc. The embodiment of the present application does not impose any restrictions on the data format of the first multimedia data.

[0066] Among them, the so-called hard decoding refers to hardware decoding, which is the decoding work of the video through hardware. Hardware decoding is performed by a graphics processing unit (GPU). Using GPU decoding can reduce the workload of the CPU and reduce power consumption. It should be noted that the decoding processing scenario involved in the embodiment of the present application is mainly a hardware decoding (hereinafter referred to as hard decoding) scenario. Therefore, the decoder in the embodiment of the present application can refer to the decoding module in the Intel CPU integrated graphics card in the computer device, and the decoding module can include a video decoding module and an audio decoding module, etc.

[0067] In one possible implementation, after acquiring the first multimedia data, the performance parameters of the decoder are acquired. Then, if it is determined based on the performance parameters of the decoder that the performance of the decoder does not meet the scene requirements, the step of acquiring the running status of the decoder is triggered. In addition, if it is determined based on the performance parameters of the decoder that the performance of the decoder meets the scene requirements, the first multimedia data can be directly decoded and processed by a conventional decoding processing method. Among them, the performance parameters of the decoder may include but are not limited to parameters such as decoding time, decoding speed, and decoding speed timing stability. This means that in an embodiment of the present application, after acquiring the first multimedia data, it can be determined in advance whether the performance of the decoder meets the scene requirements. If the performance of the decoder cannot meet the scene requirements, the decoding processing method provided by the embodiment of the present application can be executed to achieve stable decoder performance in specific scenarios, wherein the specific scenarios include business scenarios with high requirements on decoder performance, such as ultra-high frame rate real-time streaming media transmission and playback scenarios, and ultra-low latency cloud game decoding and rendering scenarios, etc.

[0068] In an embodiment of the present application, the obtained operating state of the decoder may include a working state or an idle state. In one possible implementation, when the operating state of the decoder switches from the idle state to the working state, the decoding process of the second multimedia data is suspended. The so-called working state refers to the state in which the decoder is performing decoding processing; then, the so-called idle state may refer to the state in which the decoder is not performing decoding processing; for example, assuming that the decoding frame rate of the decoder is 60fps (Frames Per Second, i.e., 60 frames of data can be decoded per second), that is, the decoder receives one multimedia data every 16ms, then the time period when the decoder decodes the received multimedia data is in the working state; and the time period when the decoder waits to receive each multimedia data (e.g., 16ms) is in the idle state. It is understandable that when the decoding process being performed by the decoder is suspended or interrupted, the decoder can enter the idle state from the working state; similarly, when the situation of the decoder being suspended or interrupted is resolved or restored, the decoder can enter the working state from the idle state.

[0069] In one possible implementation, the first multimedia data may refer to a multimedia data stream, which may include N data packets, each of which encapsulates M media frames, where N and M are both positive integers. Then, when the decoder receives the i-th data packet of the multimedia data stream, it may copy the i-th data packet to obtain a clone of the i-th data packet. The i-th data packet may be any data packet in the multimedia data stream, i is a positive integer and i≤N. Furthermore, the second multimedia data includes a clone of the i-th data packet. At this time, when the decoder is in a working state, each media frame in the i-th data packet is decoded and the media frames in the i-th data packet after decoding are output.

[0070] In a specific implementation, it is assumed that the multimedia data stream is a video data stream, and the media frames can be image frames, audio frames, or cloud game data frames, etc. Then, when the decoder needs to perform hard decoding on a video data stream, when any data packet in the video data stream is obtained, the obtained data packet can be copied, that is, the decoder performs the cloneVideoPkt operation to obtain a cloned packet (cloned new packet) of the data packet. Then, the second multimedia data includes the copied cloned new packet. When the decoder is in a working state, each media frame included in the obtained data packet can be decoded in sequence, and then each decoded media frame is output frame by frame in sequence. In one possible implementation, each media frame in the data packet can carry a media identifier, and the media identifier of each media frame is used to indicate the position and order of the media frame in the data packet. The decoder can decode each media frame in sequence according to the media identifier carried by each media frame, and output each decoded media frame in sequence according to the media identifier carried by each media frame. Wherein, the media identifier includes but is not limited to: characters, numbers, or strings, etc., and the embodiments of the present application are not limited to this.

[0071] S120: If the decoder is in an idle state, obtain second multimedia data, where the second multimedia data is obtained by copying the first multimedia data.

[0072] In one possible implementation, the first multimedia data may be a multimedia data stream, which may include N data packets, each of which encapsulates M media frames. When a decoder obtains the i-th data packet in the multimedia data stream, it may copy the i-th data packet to obtain a clone of the i-th data packet. The second multimedia data may include the clone of the i-th data packet.

[0073] In another possible implementation, when the decoder receives the i+1th data packet of the multimedia data stream, it copies the i+1th data packet to obtain a clone of the i+1th data packet. Alternatively, the clone of the i-th data packet can be deleted, and the second multimedia data then contains the clone of the i+1th data packet. This means that each time the decoder receives a new data packet, the clone obtained by copying the data packet can replace the clone of the previous data packet. This ensures that the decoder stores the latest clone, thereby reducing the storage burden and alleviating the cache pressure on the memory, thereby further improving the performance of the decoder.

[0074] S130: In an idle state, use a decoder to decode the second multimedia data.

[0075] In one possible implementation, the second multimedia data may include a clone packet of the i-th data packet, the clone packet of the i-th data packet including M clone frames, and the M clone frames correspond one-to-one to the M media frames in the i-th data packet. Decoding the second multimedia data using a decoder in an idle state may refer to decoding each clone frame in the clone packet of the i-th data packet using a decoder in an idle state.

[0076] In the embodiment of the present application, during the hard decoding of the first multimedia data, performance parameters of the decoder may be obtained. If the performance parameters of the decoder do not meet the performance index requirements, the step of obtaining the operating status of the decoder is triggered.

[0077] In one possible implementation, the performance parameters of the decoder include a decoding duration of the first multimedia data, and the performance indicator requirement includes a reference decoding frame rate. Therefore, the actual decoding frame rate of the decoder can be determined based on the decoding duration of the first multimedia data. If the actual decoding frame rate of the decoder is less than the reference decoding frame rate, it is determined that the performance parameters of the decoder do not meet the performance indicator requirement.

[0078] For example, assume that the decoder takes 20 ms to decode the first multimedia data and the reference decoding frame rate is 60 fps. The decoding frame rate refers to the number of media frames that can be decoded in one second. Therefore, the relationship between decoding time and decoding frame rate can be: decoding time t = 1000 / fps (ms). Based on the decoder's 20 ms decoding time for the first multimedia data, the actual decoding frame rate of the decoder can be determined to be 1000 / 20 = 50 fps. Since the actual decoding frame rate (50 fps) of the decoder is less than the reference decoding frame rate (60 fps), the decoder's performance parameters can be determined to not meet the performance index requirements. For another example, assume that the decoder takes 10 ms to decode the first multimedia data and the reference decoding frame rate is 60 fps. Based on the decoder's 10 ms decoding time for the first multimedia data, the actual decoding frame rate of the decoder can be determined to be 1000 / 10 = 100 fps. Since the actual decoding frame rate (100 fps) of the decoder is greater than the reference decoding frame rate (60 fps), the decoder's performance parameters can be determined to meet the performance index requirements.

[0079] In another possible implementation, the first multimedia data includes multiple media frames, the performance parameters of the decoder include the decoding duration of each media frame in the first multimedia data, and the performance indicator requirement includes a reference decoding timing range. The decoding speed timing of the decoder can be calculated based on the decoding duration of each media frame. If the decoding speed timing of the decoder exceeds the reference decoding timing range, it is determined that the performance parameters of the decoder do not meet the performance indicator requirement.

[0080] For example, assuming that the first media data includes n media frames, where n is a positive integer. Then, based on the decoding duration of each media frame, the decoding speed timing of the decoder can be obtained. Figure 2 As shown, Figure 2 Schematic diagram of the decoding speed sequence of a decoder provided in an embodiment of the present application. Figure 2 As shown, the decoding speed timing of the decoder can be composed of the decoding duration of each media frame included in the first multimedia data, such as the decoding duration of the first frame t1 = 16ms, the decoding duration of the second frame t2 = 20ms, the decoding duration of the third frame t3 = 32ms, the decoding duration of the fourth frame t4 = 16ms, the decoding duration of the fifth frame t5 = 44ms, and the decoding duration of the sixth frame t6 = 24ms. Then, the decoding duration of each media frame can constitute the decoding speed timing of the decoder. In addition, the computer device can obtain a reference decoding timing range. Figure 3 As shown, Figure 3This is a schematic diagram of a reference decoding timing range provided by an embodiment of the present application. In one possible implementation, the reference decoding timing range can be determined based on the decoding duration of the first frame. For example, assuming that the decoding duration of the first media frame in the first multimedia data is t1, then the decoding duration range corresponding to the remaining n-1 media frames in the first multimedia data can be: 0.5t1~3t1. This means that all media frames correspond to the same decoding duration range (0.5t1~3t1). For example, if the decoding durations of the remaining n-1 media frames all need to be within the decoding duration range, then the decoding speed timing of the decoder does not exceed the reference decoding timing range, and it can be determined that the performance parameters of the decoder meet the performance index requirements; if the decoding duration of one media frame among the decoding durations of the remaining n-1 media frames does not meet the decoding duration range, then the decoding speed timing of the decoder exceeds the reference decoding timing range, and it can be determined that the performance parameters of the decoder do not meet the performance index requirements. It should be noted that the reference decoding timing range in the embodiment of the present application can include the decoding duration range corresponding to each media frame.

[0081] In another possible implementation, it is also possible to further determine whether the decoding speed timing of the decoder exceeds the reference decoding timing range based on a ratio threshold. For example, a ratio threshold can be set in advance. First, the number of media frames in the n-1 media frames that fail to meet the reference decoding timing range can be determined based on the actual decoding time of the n-1 media frames and the reference decoding time range (0.5t1~3t1). Assume that there are k, k is a positive integer and k≤n-1. Then calculate the ratio k / n-1 of media frames that do not meet the reference duration range, and determine whether the decoding speed timing of the decoder exceeds the reference decoding timing range based on k / n-1 and the ratio threshold. If k / n-1 is less than the ratio threshold, it can be determined that the performance parameters of the decoder meet the performance index requirements; if k / n-1 is greater than or equal to the ratio threshold, it can be determined that the performance parameters of the decoder do not meet the performance index requirements. Among them, the ratio threshold can be adaptively adjusted according to the scene requirements, and the ratio threshold can also be manually set based on experience. The embodiment of the present application does not make specific restrictions on this.

[0082] In another possible implementation, the reference decoding timing range can be determined based on the decoding duration between two adjacent media frames. This means that the decoding duration range corresponding to each media frame may be different. Assume that the decoding duration of the first frame is t1, the decoding duration of the second frame is t2, the decoding duration of the second frame is t3... and the decoding duration of the nth frame is tn. Assume that the decoding duration range (absolute value) between t1 and t2 can be: 0≤t2-t1≤max1, the duration range between t2 and t3 can be: 0≤t3-t2≤max2, and the duration range between t3 and t4 can be: 0≤t4-t3≤max3... Among them, max1, max2 and max3 are different from each other.

[0083] For example, see Figure 4 , Figure 4 This is a schematic diagram of another reference decoding timing range provided by an embodiment of the present application. Figure 2 From the decoding speed timing diagram of the decoder shown, it can be determined that the decoding time difference between the first and second frames is: 20ms-16ms=4ms, the decoding time difference between the second and third frames is: 32ms-20ms=12ms, and so on, the decoding time differences are 16ms, 28ms, 20ms, and so on. Then, assuming that the decoding time range between two adjacent media frames is as follows: Figure 4 As shown, the decoding duration range between the first and second frames is 0-10ms, the decoding duration range between the second and third frames is 0-15ms, and so on. The decoding duration range between the fifth and sixth frames is 0-30ms. Each decoding duration range corresponds to a decoding duration difference (i.e., the difference between the decoding durations of any two adjacent media frames). Then, based on the decoding duration differences between any two adjacent media frames determined above, it is determined whether the decoding duration differences between any two adjacent media frames meet the corresponding decoding duration range. For example, the decoding duration difference between the first and second frames is 4ms, which meets the corresponding decoding duration range (0-10ms). The decoding duration difference between the second and third frames is 12ms, which also meets the corresponding decoding duration range (0-15ms). For another example, the decoding duration difference between the fourth and fifth frames is 28ms, which does not meet the corresponding decoding duration range (0-25ms). Finally, if the decoding time difference between any two adjacent media frames meets the corresponding decoding time range, it can be determined that the performance parameters of the decoder meet the performance index requirements; if there is a decoding time difference between two adjacent media frames that does not meet the corresponding decoding time range, it is determined that the decoding speed timing of the decoder exceeds the reference decoding timing range, that is, it is determined that the performance parameters of the decoder do not meet the performance index requirements.

[0084] In this way, it is possible to determine in advance whether the decoder meets the performance index requirements based on the performance parameters of the decoder. Of course, the performance index requirements may vary in different business scenarios. If the decoder does not meet the performance index requirements, the decoding processing method provided in the embodiments of the present application can be used to optimize the performance of the decoder and improve the working efficiency of the decoder.

[0085] In summary, the embodiments of the present application can be applied to cloud gaming scenarios. That is, if the first multimedia data refers to a cloud gaming video stream, then the scenario requirements for the cloud gaming can be obtained, where the scenario requirements are used to indicate the decoding mode of the cloud gaming video stream. If the scenario requirements indicate that the cloud gaming video game uses an accelerated decoding mode, then during the hard decoding of the cloud gaming video stream, the operating status of the decoder is obtained.

[0086] In one possible implementation, when developing cloud games, game developers can pre-set the game mode of the game to the accelerated decoding mode. Then, in the cloud game scenario, the decoding processing solution provided in the embodiment of the present application can be used to accelerate the hard decoding of the cloud game video stream, thereby stabilizing the decoding performance of the decoder of the game client to a large extent, thereby improving the user's gaming experience.

[0087] In another possible implementation, the cloud gaming scenario requirements can also be determined based on the network environment of the game client running the game. If the network environment of the game client where user A is running is a weak network environment, it can be determined that the cloud gaming scenario requirements indicate that the cloud gaming video stream requires the use of accelerated decoding mode; if the network environment of the game client where user B is running is a strong network environment, it can be determined that the cloud gaming scenario requirements indicate that the cloud gaming video stream does not require the use of accelerated decoding mode.

[0088] In ultra-high frame rate real-time streaming media transmission and playback scenarios, as well as ultra-low latency cloud game decoding and rendering scenarios, the embodiments of the present application can solve the problems of long decoding time and poor timing stability that may occur in scenarios with ultra-low latency requirements for hardware decoding, thereby ensuring smooth and stable video decoding performance and bringing an ultimate user experience to services with ultra-low latency requirements such as cloud games.

[0089] In an embodiment of the present application, during the process of hard decoding the first multimedia data, the operating state of the decoder can be obtained, and the operating state can include an idle state or a working state. If the operating state of the decoder is the idle state, second multimedia data obtained by copying the first multimedia data can be obtained, and the second multimedia data can be decoded by the decoder in the idle state. The present application can utilize the idle state of the decoder to decode the cloned first multimedia data while the decoder is hard decoding the first multimedia data, thereby maintaining a continuous and stable operating frequency of the decoder, thereby stabilizing the performance of the decoder and improving decoding efficiency.

[0090] See Figure 5 , Figure 5 This is a flowchart of another decoding processing method provided by an embodiment of the present application. Figure 5 As shown, the decoding processing method is executed by a decoder, in which a first thread, a second thread, a third thread and a fourth thread are running, and the first thread, the second thread, the third thread and the fourth thread are running in parallel with each other. Figure 1 The decoding processing method described in the embodiment, Figure 5 The first thread (thread 1) shown is used to: hard-decode the first multimedia data, the second thread (thread 2) is used to: output the decoded multimedia data in sequence, the third thread (thread 3) is used to: copy the first multimedia data to obtain the second multimedia data, and the fourth thread (thread 4) is used to: obtain the running status of the decoder, and obtain and decode the second multimedia data when the decoder is in an idle state.

[0091] In the embodiment of the present application, a decoder is used to perform hard decoding on a multimedia data stream as an example for detailed description. Assume that the multimedia data stream includes multiple data packets. Next, in combination with Figure 5 The decoding processing method provided in the embodiment of the present application is further described in detail. Figure 5 The decoding processing method shown includes steps S501 to S509:

[0092] S501: Waiting for a new data packet.

[0093] In one possible implementation, the multimedia data stream is based on streaming transmission. Streaming transmission means that each data packet in the multimedia data stream is transmitted one after another. That is, each time the decoder obtains a data packet in the multimedia data stream, it completes the decoding process for the data packet, waits for the next data packet, and then decodes the next data packet, and so on. The length of time the decoder needs to wait for a new data packet can be determined based on the decoder's decoding frame rate. The decoding time is t = 1000 / fps (ms). For example, if the decoder's decoding frame rate is 60fps, then after obtaining the first data packet, the decoder needs to wait t = 1000 / 60 = 16ms before receiving the next data packet.

[0094] S502: Acquire first multimedia data.

[0095] In this embodiment of the present application, the first multimedia data may be any of the N data packets included in the multimedia data stream, each of which encapsulates M media frames, where N and M are both positive integers. That is, the first multimedia data obtained by thread 1 running in the decoder encapsulates M media frames. The media frames may include, but are not limited to, image frames, audio frames, cloud game video frames, and the like.

[0096] S503: Copy operation.

[0097] In a possible implementation, each time the decoder obtains a new data packet, it may simultaneously start thread 3 to perform a cloneVideoPkt operation (copy operation) on the data packet, thereby obtaining a cloned new packet (second multimedia data).

[0098] S504: Determine whether the state is idle.

[0099] In one possible implementation, thread 4 obtains the operating state of the decoder (including an idle state or a working state) and determines whether the decoder is in the idle state. If the decoder is in the idle state, thread 4 is triggered to execute an operation to obtain the second multimedia data. The idle state may refer to whether the decoder is in a state of waiting to receive a new data packet. If so, the decoder is determined to be in the idle state.

[0100] S505: Acquire second multimedia data.

[0101] In one possible implementation, if the decoder is in an idle state, thread 4 can retrieve second multimedia data. This second multimedia data can also include a clone of any data packet. For example, the second multimedia data retrieved by thread 4 can be a clone of the data packet currently being retrieved by thread 1, or a clone of the previous data packet retrieved by thread 1. Furthermore, the clone of the latest data packet can replace the clone of the previous data packet, thereby reducing the decoder's storage burden.

[0102] S506: Decode the second multimedia data.

[0103] In one possible implementation, in an idle state, thread 4 may execute decodeClonePacket, meaning that thread 4 in the decoder decodes the second multimedia data. The second multimedia data includes M clone frames, each corresponding one-to-one to the M media frames in the first multimedia data. Therefore, decoding the second multimedia data by the decoder may involve sequentially decoding the M clone frames.

[0104] In a possible implementation, if the running state of the decoder is switched from the idle state to the working state, thread 4 suspends decoding processing of the second multimedia data.

[0105] S507: Decode the first multimedia data.

[0106] In one possible implementation, after thread 1 obtains the first multimedia data, it may decode the first multimedia data. The decoder performs a decodeOneFrame operation, i.e., the decoder decodes the first multimedia data, which may refer to the decoder sequentially decoding M media frames.

[0107] S508: Decode and transmit back.

[0108] In a possible implementation, the so-called decode callback may mean that thread 2 sequentially outputs M media frames included in the first multimedia data after decoding.

[0109] S509: Determine whether the reading is completed.

[0110] In one possible implementation, after each data packet is decoded, thread 1 can determine whether the current multimedia data stream has been read. If so, thread 1 determines that the decoding of the multimedia data stream has been completed. If not, thread 1 continues to wait for the next data packet and repeats the above steps until the multimedia data stream is read. If the current data packet is detected to carry an end identifier, thread 1 determines that the multimedia data stream has been read.

[0111] It should be noted that the step numbers in the above steps S501-S509 do not limit the order in which these steps are executed. For example, step S502 and step S503 can be executed simultaneously. For another example, step S506 can be executed before step S507, or after step S507.

[0112] In an embodiment of the present application, during the process of hard decoding the first multimedia data, the operating state of the decoder can be obtained, and the operating state can include an idle state or a working state. If the operating state of the decoder is the idle state, second multimedia data obtained by copying the first multimedia data can be obtained, and the second multimedia data can be decoded by the decoder in the idle state. The present application can utilize the idle state of the decoder to decode the cloned first multimedia data while the decoder is hard decoding the first multimedia data, thereby maintaining a continuous and stable operating frequency of the decoder, thereby stabilizing the performance of the decoder and improving decoding efficiency.

[0113] See Figure 6 , Figure 6 This is a schematic diagram of the structure of a decoding processing device provided in an embodiment of the present application. The decoding processing device 600 can be applied to Figure 1 and Figure 5 The computer device in the corresponding method embodiment. The decoding processing device 600 can be a computer program (including program code) running in a computer device, for example, the decoding processing device 600 is an application software; the device can be used to perform the corresponding steps in the method provided in the embodiment of the present application. The decoding processing device 600 may include:

[0114] An acquiring unit 601 is configured to acquire an operating state of a decoder during hard decoding of the first multimedia data, where the operating state includes a working state or an idle state;

[0115] The acquiring unit 601 is further configured to acquire second multimedia data if the decoder is in an idle state, where the second multimedia data is obtained by copying the first multimedia data;

[0116] The decoding unit 602 is configured to decode the second multimedia data using a decoder in an idle state.

[0117] In a possible implementation, the decoding unit 602 is further configured to: suspend decoding processing of the second multimedia data when the running state of the decoder is switched from the idle state to the working state.

[0118] In one possible implementation, the first multimedia data refers to a multimedia data stream, which includes N data packets, each of which encapsulates M media frames, where N and M are both positive integers. The decoding processing device 600 may further include a processing unit 603:

[0119] The processing unit 603 is configured to, when the decoder receives an i-th data packet of the multimedia data stream, copy the i-th data packet to obtain a clone of the i-th data packet, wherein the second multimedia data includes the clone of the i-th data packet, where i is a positive integer and i≤N;

[0120] The decoding unit 602 is further configured to: when the decoder is in a working state, decode each media frame in the i-th data packet, and output the decoded media frames in the i-th data packet.

[0121] In one possible implementation, the clone packet of the i-th data packet includes M clone frames, and the M clone frames correspond one-to-one to the M media frames in the i-th data packet; the decoding unit 602 is specifically used to: in an idle state, use a decoder to decode each clone frame in the clone packet of the i-th data packet.

[0122] In a possible implementation, the processing unit 603 is further configured to: when the decoder receives the i+1th data packet of the multimedia data stream, copy the i+1th data packet to obtain a clone of the i+1th data packet;

[0123] The processing unit 603 is further configured to delete the cloned packet of the i-th data packet, and the second multimedia data includes the cloned packet of the i+1-th data packet.

[0124] In a possible implementation, the acquisition unit 601 is further used to: obtain the performance parameters of the decoder during hard decoding of the first multimedia data; if the performance parameters of the decoder do not meet the performance index requirements, trigger the step of obtaining the operating status of the decoder.

[0125] In one possible implementation, the performance parameter of the decoder includes a decoding duration of the first multimedia data, and the performance indicator requirement includes a reference decoding frame rate; and the processing unit 603 is further configured to:

[0126] Determine the actual decoding frame rate of the decoder based on the decoding time;

[0127] If the actual decoding frame rate of the decoder is less than the reference decoding frame rate, it is determined that the performance parameters of the decoder do not meet the performance indicator requirements.

[0128] In one possible implementation, the first multimedia data includes multiple media frames, the performance parameter of the decoder includes a decoding duration of each media frame in the first multimedia data, and the performance indicator requirement includes a reference decoding timing range; and the processing unit 603 is further configured to:

[0129] Calculate the decoding speed timing of the decoder based on the decoding duration of each media frame;

[0130] If the decoding speed timing of the decoder exceeds the reference decoding timing range, it is determined that the performance parameters of the decoder do not meet the performance index requirements.

[0131] In a possible implementation, a first thread, a second thread, a third thread, and a fourth thread are run in the decoder, and the first thread, the second thread, the third thread, and the fourth thread are run in parallel with each other;

[0132] The first thread is used to perform hard decoding on the first multimedia data, and the second thread is used to sequentially output the decoded first multimedia data;

[0133] The third thread is used to copy the first multimedia data to obtain second multimedia data;

[0134] The fourth thread is used to obtain the running state of the decoder, and obtain and decode the second multimedia data when the decoder is in an idle state.

[0135] In one possible implementation, the first multimedia data refers to a cloud game video stream; and the processing unit 603 is further configured to:

[0136] Obtain the scene requirements of the cloud game, which are used to indicate the decoding mode of the cloud game video stream;

[0137] If the scenario requirements indicate that the cloud gaming video game adopts the accelerated decoding mode, the running status of the decoder is obtained during the hard decoding of the cloud gaming video stream.

[0138] In an embodiment of the present application, during the process of hard decoding the first multimedia data, the operating state of the decoder can be obtained, and the operating state can include an idle state or a working state. If the operating state of the decoder is the idle state, second multimedia data obtained by copying the first multimedia data can be obtained, and the second multimedia data can be decoded by the decoder in the idle state. The present application can utilize the idle state of the decoder to decode the cloned first multimedia data while the decoder is hard decoding the first multimedia data, thereby maintaining a continuous and stable operating frequency of the decoder, thereby stabilizing the performance of the decoder and improving decoding efficiency.

[0139] See Figure 7 , Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. The computer device 700 is used to execute Figure 1 and Figure 5 The steps performed by a computer device in the corresponding method embodiment are as follows: the computer device 700 includes: one or more processors 710; one or more input devices 720; one or more output devices 730; and a memory 740. The processors 710, input devices 720, output devices 730, and memory 740 are connected via a bus 750. The memory 740 is used to store a computer program, which includes program instructions. The processor 710 is used to call the program instructions stored in the memory 740 to perform the following operations:

[0140] During hard decoding of the first multimedia data, obtaining an operating state of the decoder, where the operating state includes a working state or an idle state;

[0141] If the decoder is in an idle state, obtaining second multimedia data, where the second multimedia data is obtained by copying the first multimedia data;

[0142] In the idle state, the second multimedia data is decoded by using a decoder.

[0143] In a possible implementation, when the running state of the decoder is switched from the idle state to the working state, the processor 710 suspends decoding processing of the second multimedia data.

[0144] In one possible implementation, the first multimedia data refers to a multimedia data stream, where the multimedia data stream includes N data packets, each data packet encapsulates M media frames, and N and M are both positive integers. The processor 710 is further configured to perform the following operations:

[0145] When the decoder receives an i-th data packet of the multimedia data stream, the i-th data packet is copied to obtain a clone packet of the i-th data packet, and the second multimedia data includes the clone packet of the i-th data packet, where i is a positive integer and i≤N;

[0146] When the decoder is in a working state, each media frame in the i-th data packet is decoded and the decoded media frames in the i-th data packet are output.

[0147] In one possible implementation, the clone packet of the i-th data packet includes M clone frames, and the M clone frames correspond one-to-one to the M media frames in the i-th data packet. In an idle state, the processor 710 uses a decoder to decode the second multimedia data, including:

[0148] In the idle state, the decoder is used to decode each clone frame in the clone packet of the i-th data packet.

[0149] In one possible implementation, when the decoder receives the (i+1)th data packet of the multimedia data stream, the processor 710 copies the (i+1)th data packet to obtain a clone of the (i+1)th data packet;

[0150] The processor 710 deletes the cloned packet of the i-th data packet, and the second multimedia data includes the cloned packet of the i+1-th data packet.

[0151] In one possible implementation, during the process of hard decoding the first multimedia data by the processor 710, a performance parameter of the decoder is obtained;

[0152] If the performance parameters of the decoder do not meet the performance index requirements, the step of obtaining the operating status of the decoder is triggered.

[0153] In one possible implementation, the performance parameter of the decoder includes a decoding duration of the first multimedia data, and the performance indicator requirement includes a reference decoding frame rate; and the processor 710 is further configured to perform the following operations:

[0154] Determine the actual decoding frame rate of the decoder based on the decoding time;

[0155] If the actual decoding frame rate of the decoder is less than the reference decoding frame rate, it is determined that the performance parameters of the decoder do not meet the performance indicator requirements.

[0156] In one possible implementation, the first multimedia data includes multiple media frames, the performance parameter of the decoder includes a decoding duration of each media frame in the first multimedia data, and the performance indicator requirement includes a reference decoding timing range; and the processor 710 is further configured to perform the following operations:

[0157] Calculate the decoding speed timing of the decoder based on the decoding duration of each media frame;

[0158] If the decoding speed timing of the decoder exceeds the reference decoding timing range, it is determined that the performance parameters of the decoder do not meet the performance index requirements.

[0159] In a possible implementation, a first thread, a second thread, a third thread, and a fourth thread are run in the decoder, and the first thread, the second thread, the third thread, and the fourth thread are run in parallel with each other;

[0160] The first thread is used to perform hard decoding on the first multimedia data, and the second thread is used to sequentially output the decoded first multimedia data;

[0161] The third thread is used to copy the first multimedia data to obtain second multimedia data;

[0162] The fourth thread is used to obtain the running state of the decoder, and obtain and decode the second multimedia data when the decoder is in an idle state.

[0163] In one possible implementation, the first multimedia data refers to a cloud game video stream; and the processor 710 is further configured to perform the following operations:

[0164] Obtain the scene requirements of the cloud game, which are used to indicate the decoding mode of the cloud game video stream;

[0165] If the scenario requirements indicate that the cloud gaming video game adopts the accelerated decoding mode, the running status of the decoder is obtained during the hard decoding of the cloud gaming video stream.

[0166] In an embodiment of the present application, during the process of hard decoding the first multimedia data, the operating state of the decoder can be obtained, and the operating state can include an idle state or a working state. If the operating state of the decoder is the idle state, second multimedia data obtained by copying the first multimedia data can be obtained, and the second multimedia data can be decoded by the decoder in the idle state. The present application can utilize the idle state of the decoder to decode the cloned first multimedia data while the decoder is hard decoding the first multimedia data, thereby maintaining a continuous and stable operating frequency of the decoder, thereby stabilizing the performance of the decoder and improving decoding efficiency.

[0167] It should be understood that the computer device 700 described in the embodiment of the present application can execute the above Figure 1 and Figure 5 The description of the decoding method in the corresponding embodiment can also be performed as described above. Figure 6 The description of the decoding processing device 600 in the corresponding embodiment will not be repeated here.

[0168] In addition, it should be noted that the embodiment of the present application also provides a computer storage medium, and the computer storage medium stores a computer program, and the computer program includes program instructions. When the processor executes the above program instructions, it can execute the above Figure 1 and Figure 5 The method in the corresponding embodiment will therefore not be described in detail here. For technical details not disclosed in the computer storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application. As an example, the program instructions can be deployed on a computer device, or executed on multiple computer devices located in one location, or executed on multiple computer devices distributed in multiple locations and interconnected by a communication network.

[0169] According to one aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device can perform the above-mentioned Figure 1 and Figure 5 The method in the corresponding embodiment will therefore not be described in detail here.

[0170] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0171] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A decoding processing method, characterized in that: include: During hard decoding of the first multimedia data, obtaining performance parameters of a decoder, the performance parameters of the decoder including a decoding time of the first multimedia data; Determining an actual decoding frame rate of the decoder according to the decoding duration; If the actual decoding frame rate of the decoder is less than the reference decoding frame rate included in the performance index requirement, determining that the performance parameter of the decoder does not meet the performance index requirement, and obtaining the operating state of the decoder, where the operating state includes a working state or an idle state; If the decoder is in an idle state, obtaining second multimedia data, where the second multimedia data is obtained by copying the first multimedia data; In the idle state, the second multimedia data is decoded by using the decoder.

2. The method according to claim 1, wherein The method further comprises: When the running state of the decoder is switched from the idle state to the working state, the decoding process of the second multimedia data is suspended.

3. The method according to claim 1, wherein The first multimedia data refers to a multimedia data stream, the multimedia data stream including N data packets, each data packet encapsulating M media frames, where N and M are both positive integers; The method further comprises: When the decoder receives an i-th data packet of the multimedia data stream, the decoder copies the i-th data packet to obtain a clone of the i-th data packet, wherein the second multimedia data includes the clone of the i-th data packet, where i is a positive integer and i≤N; When the decoder is in a working state, each media frame in the i-th data packet is decoded, and the decoded media frames in the i-th data packet are output.

4. The method according to claim 3, wherein The clone packet of the i-th data packet includes M clone frames, and the M clone frames correspond one-to-one to the M media frames in the i-th data packet; The step of decoding the second multimedia data by using the decoder in the idle state includes: In the idle state, the decoder is used to decode each clone frame in the clone packet of the i-th data packet.

5. The method according to claim 3, wherein The method further comprises: When the decoder receives the (i+1)th data packet of the multimedia data stream, the (i+1)th data packet is copied to obtain a clone of the (i+1)th data packet; The cloned packet of the i-th data packet is deleted, and the second multimedia data includes the cloned packet of the i+1-th data packet.

6. The method according to claim 1, wherein The first multimedia data includes a plurality of media frames, the performance parameter of the decoder includes a decoding duration of each media frame in the first multimedia data, and the performance indicator requirement includes a reference decoding timing range; The method further comprises: Calculating a decoding speed timing of the decoder according to a decoding duration of each media frame; If the decoding speed timing of the decoder exceeds the reference decoding timing range, it is determined that the performance parameter of the decoder does not meet the performance indicator requirement.

7. The method according to claim 1, wherein The decoder runs a first thread, a second thread, a third thread, and a fourth thread, wherein the first thread, the second thread, the third thread, and the fourth thread run in parallel with each other; The first thread is used to perform hard decoding on the first multimedia data, and the second thread is used to sequentially output the decoded first multimedia data; The third thread is used to copy the first multimedia data to obtain the second multimedia data; The fourth thread is used to obtain the running state of the decoder, and to obtain and decode the second multimedia data when the decoder is in an idle state.

8. The method according to any one of claims 1 to 7, wherein The first multimedia data refers to a cloud game video stream; the method further includes: Obtaining a scene requirement for a cloud game, where the scene requirement is used to indicate a decoding mode for the cloud game video stream; If the scenario requirement indicates that the cloud game video stream adopts the accelerated decoding mode, the operating status of the decoder is obtained during the hard decoding of the cloud game video stream.

9. A decoding processing device, characterized in that: include: an acquiring unit, configured to acquire performance parameters of a decoder during hard decoding of the first multimedia data, wherein the performance parameters of the decoder include a decoding time duration of the first multimedia data; The acquisition unit is further configured to determine an actual decoding frame rate of the decoder according to the decoding duration; The acquiring unit is further configured to, if the actual decoding frame rate of the decoder is less than the reference decoding frame rate included in the performance indicator requirement, determine that the performance parameter of the decoder does not meet the performance indicator requirement, and acquire an operating state of the decoder, where the operating state includes a working state or an idle state; The acquiring unit is further configured to acquire second multimedia data if the decoder is in an idle state, where the second multimedia data is obtained by copying the first multimedia data; A decoding unit is configured to decode the second multimedia data using the decoder in the idle state.

10. A computer device, characterized in that: include: a processor adapted to execute a computer program; A computer-readable storage medium having a computer program stored therein, wherein the computer program, when executed by the processor, implements the decoding method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the decoding method according to any one of claims 1 to 8.

12. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to perform the decoding processing method according to any one of claims 1 to 8.

Images