Video stream playback method, device, equipment and storage medium

Abstract

The embodiments of the present invention provide a video stream playback method, apparatus, device, and storage medium, which are applied to a player on a Web side, and include: in response to a video playback request, obtaining a to-be-decoded video stream corresponding to the video playback request, wherein the to-be-decoded video stream includes to-be-decoded video frames and to-be-decoded audio frames; establishing multiple threads based on the to-be-decoded video stream, each thread corresponding to a video decoder; decoding the to-be-decoded video frames by multiple video decoders to obtain video frame data corresponding to the to-be-decoded video stream; decoding the to-be-decoded audio frames by an audio decoder to obtain audio frame data corresponding to the to-be-decoded video stream; and playing the video frame data and audio frame data. This solution uses multiple video decoders to decode the to-be-decoded video frames simultaneously, while complying with the security requirement that each thread can only correspond to one single-thread decoding video decoder, thereby effectively improving the decoding efficiency of the video frames and ensuring the smoothness of the video playback on the Web side.

Description

Technical Field

[0001] The present invention relates to the field of Internet technology, and in particular to a video stream playback method, device, equipment and storage medium. Background Art

[0002] Streaming media refers to a media format that is played on the Internet in a streaming manner, such as a common video stream. In order to reduce the transmission and storage costs of video streams, the original video file corresponding to the video stream is usually encoded to compress the redundant information in the video file; after obtaining the encoded video stream, it is decoded according to its corresponding encoding format, that is, decompressed, to obtain a video file that can be used for display. Among them, commonly used codec formats include: Advanced Video Coding (AVC), also known as H.264, High Efficiency Video Coding (HEVC), also known as H.265, etc. When decoding, the video stream can be hardware decoded through the video acceleration function of the graphics card, or software decoded through a decoder running on the CPU.

[0003] When the web player decodes the video stream, it may not be able to perform hardware decoding because the web does not provide an application programming interface corresponding to the hardware decoding of the corresponding encoding format. It can only use software decoding to decode the video stream. When performing software decoding on the video stream, it includes decoding the video frames in the video stream and decoding the audio frames in the video stream. Usually, the player decodes the audio frames quickly and can meet the needs of various playback scenarios, but the decoding speed of the video frames is slower. When decoding the video frames in the video stream, for security reasons, the server corresponding to the web side generally only allows the call of a single-threaded decoding video decoder in a thread established by the web player. The decoding efficiency of a single single-threaded decoding video decoder is limited and cannot adapt to the decoding requirements of video streams with higher frame rates, affecting the user experience. For example, video freezes may occur in the double-speed playback scenario. Summary of the Invention

[0004] Embodiments of the present invention provide a video stream playback method, apparatus, device, and storage medium for improving the decoding efficiency of a web-side player for video frames in a video stream.

[0005] In a first aspect, an embodiment of the present invention provides a video stream playback method, which is applied to a player on a web side. The method includes:

[0006] In response to a video playback request, obtaining a to-be-decoded video stream corresponding to the video playback request, wherein the to-be-decoded video stream includes a to-be-decoded video frame and a to-be-decoded audio frame;

[0007] Establishing multiple threads according to the video stream to be decoded, each thread corresponding to a video decoder;

[0008] Decoding the to-be-decoded video frames by using a plurality of the video decoders to obtain video frame data corresponding to the to-be-decoded video stream;

[0009] Decoding the audio frame to be decoded by an audio decoder to obtain audio frame data corresponding to the video stream to be decoded;

[0010] The video frame data and the audio frame data are played.

[0011] In a second aspect, an embodiment of the present invention provides a video stream playback device, which is applied to a player on a Web side. The device includes:

[0012] An acquisition module, configured to, in response to a video playback request, acquire a to-be-decoded video stream corresponding to the video playback request;

[0013] The processing module is configured to establish multiple threads according to the video stream to be decoded, each thread corresponding to a video decoder; decode the video frames to be decoded by the multiple video decoders to obtain video frame data corresponding to the video stream to be decoded; and decode the audio frames to be decoded by the audio decoder to obtain audio frame data corresponding to the video stream to be decoded;

[0014] A playing module is used to play the video frame data and the audio frame data.

[0015] In a third aspect, an embodiment of the present invention provides an electronic device comprising: a memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor can at least implement the video stream playback method described in the first aspect.

[0016] In a fourth aspect, an embodiment of the present invention provides a non-transitory machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the video stream playback method described in the first aspect.

[0017] In the solution provided by the embodiment of the present invention, the player on the Web side responds to a video playback request and obtains a video stream to be decoded corresponding to the video playback request, wherein the video stream to be decoded includes video frames to be decoded and audio frames to be decoded; then, based on the video stream to be decoded, multiple threads are established, each thread corresponds to a video decoder, and the video frames to be decoded are decoded by multiple video decoders to obtain video frame data corresponding to the video stream to be decoded; the audio frames to be decoded are decoded by the audio decoder to obtain audio frame data corresponding to the video stream to be decoded; finally, the video frame data and the audio frame data are played. In this solution, the player on the Web side establishes multiple threads so that, while meeting the security requirement that each thread can only correspond to one single-thread decoding video decoder, multiple single-thread decoding video decoders can be used at the same time to decode the video frames to be decoded in the video stream to be decoded, thereby effectively improving the decoding efficiency of the video frames and ensuring the smoothness of the video playback on the Web side. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 A flowchart of a video stream playback method provided by an embodiment of the present invention;

[0020] Figure 2 A schematic diagram of a video stream playback process provided by an embodiment of the present invention;

[0021] Figure 3 A schematic diagram of another video stream playback process provided by an embodiment of the present invention;

[0022] Figure 4 A flowchart of another video stream playback method provided by an embodiment of the present invention;

[0023] Figure 5 A flowchart of another video stream playback method provided by an embodiment of the present invention;

[0024] Figure 6 A schematic diagram of another video stream playback process provided by an embodiment of the present invention;

[0025] Figure 7 A schematic structural diagram of a video stream playback device provided by an embodiment of the present invention;

[0026] Figure 8 For Figure 7A schematic structural diagram of an electronic device corresponding to the video stream playback device provided by the illustrated embodiment. DETAILED DESCRIPTION

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0028] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0029] In addition, the step sequence in the following method embodiments is only an example and not a strict limitation.

[0030] The video streaming playback method provided in the embodiments of the present invention can be performed by an electronic device, which can be a terminal device such as a PC, a laptop, a smartphone, or a server. The server can be a physical server including an independent host, a virtual server, a cloud server, or a server cluster.

[0031] Figure 1 A flowchart of a video streaming playback method provided by an embodiment of the present invention is applied to a server on a Web side, such as Figure 1 As shown, the following steps may be included:

[0032] 101. In response to a video playback request, obtain a to-be-decoded video stream corresponding to the video playback request, where the to-be-decoded video stream includes to-be-decoded video frames and to-be-decoded audio frames.

[0033] 102. Establish multiple threads according to the video stream to be decoded, where each thread corresponds to a video decoder.

[0034] 103. Decode the to-be-decoded video frames using multiple video decoders corresponding to the multiple threads to obtain video frame data corresponding to the to-be-decoded video stream.

[0035] 104. Decode the audio frame to be decoded by an audio decoder to obtain audio frame data corresponding to the video stream to be decoded.

[0036] 105. Play the video frame data and the audio frame data.

[0037] A web-based player refers to a player that runs in a web browser. Depending on the terminal device on which the web browser is installed, web-based players include computer web-based players and mobile web-based players, such as mobile phone web-based players. This embodiment does not limit the type of web-based player. For ease of description, web-based players will be referred to as "players" below.

[0038] It should be noted that, in this embodiment, unless otherwise specified, the video decoders mentioned are all single-threaded decoding video decoders. Single-threaded decoding means that the video decoder performs decoding operations through only one thread when decoding video frames.

[0039] Figure 2 Schematic diagram of a video stream playback process provided by an embodiment of the present invention. Figure 2 As shown, when the player plays a video stream, its process can be summarized as follows: First, the video stream acquisition process, including: responding to the user's video playback request, obtaining a video stream that has been encoded and compressed, that is, the video stream to be decoded, which contains video frames to be decoded and audio frames to be decoded. Among them, after encoding and compression, the video stream to be decoded is usually encapsulated into a corresponding video format for streaming, for example, it can be encapsulated into MP4, FLV, AVI and other formats. Second, the demultiplexing process, including: the player disassembles the obtained encapsulated video stream to be decoded through a demultiplexer, thereby obtaining frames of compressed video frames (i.e., video frames to be decoded) and audio frames (i.e., audio frames to be decoded). Third, the decoding process, including: the player is equipped with an audio decoder and a video decoder, and the audio decoder decodes the audio frames to be decoded to obtain frames of uncompressed audio frame data (hereinafter referred to as audio frame data); the video decoder decodes the video frames to be decoded to obtain frames of uncompressed video frame data (hereinafter referred to as video frame data). When decoding a video frame, the player establishes a thread—that is, starts a worker thread within the player—and calls a video decoder within that thread to decode the video frame. Fourth, the playback process involves transmitting uncompressed video frame data and uncompressed audio frame data to hardware related to video playback, such as a graphics processing unit (GPU), for video playback.

[0040] This program focuses on Figure 2The decoding process of the video frames to be decoded during the video stream playback process shown, and other contents related to video stream playback, such as the demultiplexing process, the decoding process of the audio frames to be decoded, the playback process after decoding, etc., can be referred to related technologies and will not be repeated in this solution.

[0041] In the related art, when decoding video frames, in order to ensure security, the server corresponding to the Web end usually only allows the call of a single-threaded decoding video decoder for a thread established by the player. It is easy to understand that the decoding capability of a single-threaded decoding video decoder is limited. With the development of video encoding technology and the increase in video playback demand, the decoding capability of a single-threaded decoding video decoder cannot adapt to the decoding requirements of some specific encoding formats or special playback scenarios. For example: the decoding capability of the video decoder corresponding to some mobile web players cannot meet the video frame decoding requirements of video streams in the encoding format of H.265, which results in the player being unable to play video streams encoded in the H.265 format; for example: in the fast playback, that is, double-speed playback scenario, the decoding capability of the video decoder corresponding to the player is insufficient, resulting in video playback being stuck, etc., affecting the user experience.

[0042] To solve at least one of the above technical problems, the present invention provides Figure 1 The video stream playback method shown below. Figure 3 right Figure 1 The video streaming playback method shown in the figure is explained. Figure 3 A schematic diagram of another video stream playback process provided by an embodiment of the present invention.

[0043] like Figure 3 As shown, first, in response to a user's video playback request, a video stream to be decoded corresponding to the video playback request is obtained. Then, based on the video stream to be decoded, n threads are established, namely thread 1, thread 2, ..., thread n. Each thread corresponds to a video decoder. For example, thread 1 corresponds to video decoder 1, meaning thread 1 calls video decoder 1. Similarly, thread 2 corresponds to video decoder 2, ..., and thread n corresponds to video decoder n. Then, the n video decoders corresponding to the n threads decode the obtained video frames to be decoded in the video stream to be decoded, obtaining video frame data 1, video frame data 2, ..., and video frame data n. Based on video frame data 1, video frame data 2, ..., and video frame data n, the video frame data corresponding to the video stream to be decoded is determined. The audio frames to be decoded in the video stream to be decoded are decoded by the audio decoder to obtain audio frame data corresponding to the video stream to be decoded. Finally, the video frame data decoded by the n video decoders and the audio frame data decoded by the audio decoder are synchronously played.

[0044] Optionally, the number of threads to be established can be determined based on the frame rate of the video stream to be decoded, where the frame rate is the number of video frames displayed per unit time. For example, when the frame rate of the video stream to be decoded is less than a first threshold, a first number of threads is established; when the frame rate of the video stream to be decoded is greater than or equal to the first threshold and less than a second threshold, a second number of threads is established; when the frame rate of the video stream to be decoded is greater than or equal to the second threshold, a third number of threads is established, and so on. The first and second thresholds, and the corresponding number of threads, can be customized.

[0045] It should be noted that, in this embodiment, the programming language corresponding to the player is different from the programming language corresponding to the decoder. For example, the player is implemented in JavaScript and the decoder is implemented in C++. To ensure that the thread established or started in the player can call the decoder, Webassembly, also known as wasm, is usually used to implement the call of the decoder by the thread established by the player.

[0046] If the player using JavaScript is regarded as the JS side, and the wasm and the decoder using C++ are regarded as the wasm side, then in the related art, only one thread is established on the JS side. For security reasons, only one video decoder can be called on the wasm side. The video stream playback method provided in this embodiment can be understood as establishing multiple threads on the JS side. Although each thread still only calls one video decoder, establishing multiple threads on the JS side can realize parallel calling of multiple video decoders on the wasm side, thereby improving the video decoding performance of the player.

[0047] In summary, when decoding the video stream to be decoded, this solution establishes multiple threads according to the video stream to be decoded, and decodes the video frames to be decoded in the video stream to be decoded through multiple video decoders corresponding to the multiple threads one by one. It takes security into consideration and ensures that each thread only calls one video decoder and the video decoder is single-threaded decoding. By establishing multiple threads, multiple video decoders are enabled to simultaneously decode the video frames to be decoded in the same video stream to be decoded, which improves the video decoding capability of the corresponding decoder of the player, ensures the video decoding efficiency, and enables the web-side player to adapt to various scenarios of video stream playback.

[0048] In actual application, when playing a video stream, the number of threads used to decode the video stream to be decoded can also be determined in combination with the hardware environment corresponding to the player. Figure 4 As shown, Figure 4 A flowchart of another video stream playback method provided by an embodiment of the present invention, which is applied to a player on a web side, may include the following steps:

[0049] 401. In response to a video playback request, obtain a to-be-decoded video stream corresponding to the video playback request, where the to-be-decoded video stream includes to-be-decoded video frames and to-be-decoded audio frames.

[0050] 402. Establish a test thread, where the test thread corresponds to a video decoder; decode a test video frame in a preset video stream to be tested by the video decoder corresponding to the test thread to determine the video decoding capability of the video decoder under the current hardware environment.

[0051] 403. According to the decoding capability of the video decoder in the current hardware environment and the video stream to be decoded, multiple threads are established, each thread corresponding to a video decoder.

[0052] 404. Decode the video frames to be decoded by using multiple video decoders corresponding to the multiple threads to obtain video frame data corresponding to the video stream to be decoded; and decode the audio frames to be decoded by using the audio decoder to obtain audio frame data corresponding to the video stream to be decoded.

[0053] 405. Play the video frame data and the audio frame data.

[0054] The specific implementation process of step 401, step 404 and step 405 can refer to the above embodiment and will not be described in detail in this embodiment.

[0055] It's understood that as a software program running on a CPU, a video decoder's video decoding capabilities are tied to the CPU's operating state. In other words, a video decoder's video decoding capabilities vary with the hardware environment in which it operates. For example, a video decoder running on a particular CPU may exhibit different video decoding capabilities at different times. Similarly, the same video decoder running on different CPUs may exhibit different video decoding capabilities due to differences in the CPUs used.

[0056] In actual applications, when decoding a video frame in a video stream to be decoded is required, if too many threads are established, resources will be wasted; if too few threads are established, the video decoding requirements corresponding to the video stream to be decoded cannot be met.

[0057] In this embodiment, in order to meet the video decoding requirements of the video stream to be decoded without wasting resources, before establishing multiple threads for the video stream to be decoded, a test thread is first established. The test thread calls a video decoder to decode the test video frames in the pre-set test video stream to determine the video decoding capability of the video decoder under the current hardware environment. Then, based on the determined video decoding capability, the number of threads to be established to decode the video frames to be decoded in the video stream to be decoded is determined. The pre-set test video stream can be a video stream containing a set number of test video frames, for example, a video stream containing 15 test video frames. The test video frames have been encoded and compressed and are video frames to be decoded.

[0058] Optionally, multiple video streams to be tested corresponding to multiple encoding formats can be pre-stored. When determining the video decoding capability corresponding to the video decoder under the current hardware environment, a target video stream to be tested with the same encoding format as the video stream to be decoded is determined from the multiple video streams to be tested, and the test video frames in the target video stream to be tested are decoded. Thereafter, the video decoding capability corresponding to the video decoder under the current hardware environment is determined based on the video decoding result of the target video stream to be tested.

[0059] Optionally, the decoding capability of the video decoder may be represented by the number of video frames decoded per unit time; or, the decoding capability may be represented by the number of video frame data obtained by decoding.

[0060] Optionally, the video decoding requirements of the video stream to be decoded can be represented by the frame rate corresponding to the video stream to be decoded. Accordingly, after determining the video decoding capability of the video decoder under the current hardware environment, a matching number of threads can be determined and established based on the video decoding capability of the video decoder under the current hardware environment and the frame rate corresponding to the video stream to be decoded. For example, assuming that the video decoder under the current hardware environment can decode m video frames per unit time, and the frame rate corresponding to the video stream to be decoded is to display 2m video frames per unit time, then two threads can be established to call the two video decoders to decode the video stream to be decoded.

[0061] In this embodiment, when establishing multiple threads for the video stream to be decoded, the video decoding capability of the video decoder under the current hardware environment is tested, and a number of threads that is adapted to the hardware environment and video decoding requirements are established, thereby meeting the decoding requirements of the video stream to be decoded without causing waste of resources.

[0062] Figure 5 A flowchart of another video stream playback method provided in an embodiment of the present invention, which is applied to a player on a web side, may include the following steps:

[0063] 501. In response to a video playback request, obtain a to-be-decoded video stream corresponding to the video playback request, where the to-be-decoded video stream includes to-be-decoded video frames and to-be-decoded audio frames.

[0064] 502. Establish multiple threads according to the video stream to be decoded, where each thread corresponds to a video decoder.

[0065] 503. Determine multiple target queues of video frames to be decoded corresponding to the video streams to be decoded according to the number of threads corresponding to the multiple threads, wherein the multiple target queues of video frames to be decoded correspond to the multiple threads in a one-to-one manner.

[0066] 504 . Decode the to-be-decoded video frames contained in the multiple target to-be-decoded video frame queues respectively by using multiple video decoders to obtain target video frame data corresponding to the multiple target to-be-decoded video frame queues respectively.

[0067] 505. Determine the video frame data corresponding to the video stream to be decoded according to the target video frame data.

[0068] 506. Decode the audio frame to be decoded by an audio decoder to obtain audio frame data corresponding to the video stream to be decoded.

[0069] 507. Play the video frame data and the audio frame data.

[0070] The specific implementation process of step 501, step 502, step 506 and step 507 can refer to the above embodiment and will not be described in detail in this embodiment.

[0071] In practical applications, when a video stream to be decoded is transmitted, the video frames to be decoded are usually transmitted in units of Group of Picture (GOP). A GOP is a group of continuous pictures in the video stream to be decoded, and a GOP contains multiple video frames to be decoded.

[0072] It is understandable that when the player obtains the video stream to be decoded, it actually obtains the picture groups corresponding to the video stream to be decoded one by one. In order to ensure the real-time decoding, the video decoder decodes the currently obtained picture group. When decoding the video, decoding is performed in units of picture groups. Therefore, determining multiple target video frame queues to be decoded corresponding to the video stream to be decoded can also be understood as determining multiple picture group queues to be decoded. Every time the player obtains a picture group, it determines the target video frame queue to be decoded to which it belongs, so that the video decoder can perform video decoding on it. In the specific implementation process, the number of multiple target video frame queues to be decoded matches the number of multiple threads established, each target video frame queue to be decoded corresponds to a thread, and each target video frame queue to be decoded contains multiple video frames to be decoded in units of picture groups.

[0073] In an optional embodiment, multiple video frames to be decoded contained in picture groups in multiple target video frame queues to be decoded are respectively decoded by multiple video decoders to obtain target video frame data corresponding to the multiple target video frame queues to be decoded, including: determining the target video frame queues to which the multiple picture groups contained in the video stream to be decoded respectively belong, and decoding the multiple video frames to be decoded contained in picture groups in the multiple target video frame queues to be decoded by multiple video decoders to obtain target video frame data corresponding to the multiple target video frame queues to be decoded.

[0074] Optionally, after acquiring a certain picture group, the player may determine the target to-be-decoded video frame queue to which the currently acquired picture group belongs according to the number of undecoded picture groups respectively contained in the current plurality of target to-be-decoded video frame queues.

[0075] For ease of understanding, combined Figure 6 Give an example. Figure 6 A schematic diagram of another video stream playback process provided by an embodiment of the present invention is shown as follows: Figure 6 As shown, assuming the video stream to be decoded contains several groups of pictures, namely GOP1, GOP2, GOP3, etc., when the player obtains the video stream to be decoded, it obtains the several groups of pictures in the order of GOP1, GOP2, GOP3, etc. Assuming that two threads are established based on the video stream to be decoded, two corresponding target video frame queues to be decoded are determined, namely target video frame queue 1 to be decoded and target video frame queue 2 to be decoded. Target video frame queue 1 corresponds to thread 1, which calls video decoder 1; target video frame queue 2 corresponds to thread 2, which calls video decoder 2.

[0076] When the player obtains GOP1, since neither the target to-be-decoded video frame queue 1 nor the target to-be-decoded video frame queue 2 contains any undecoded picture groups, GOP1 can be assigned to either the target to-be-decoded video frame queue 1 or the target to-be-decoded video frame queue 2. Figure 6 In the illustrated scenario, GOP1 is assigned to target queue 1 of video frames to be decoded.

[0077] When the player obtains GOP2, since target queue 1 for video frames to be decoded already contains GOP1, GOP2 is assigned to target queue 2 for video frames to be decoded. Subsequently, GOP3 is assigned to target queue 1 for video frames to be decoded, and so on. In other words, after obtaining several groups of pictures corresponding to the video stream to be decoded, the player alternately assigns these groups of pictures to the multiple target queues for video frames to be decoded based on the order in which they were obtained.

[0078] Video decoder 1 decodes GOP1, GOP3, and other groups of pictures in target video frame queue 1 to obtain target video frame data 1. Video decoder 2 decodes GOP2 and other groups of pictures in target video frame queue 2 to obtain target video frame data 2. Subsequently, based on target video frame data 1 and target video frame data 2, the video frame data corresponding to the video stream to be decoded is determined. Finally, the video frame data corresponding to the video stream to be decoded and the audio frame data obtained by decoding the audio frames to be decoded in the video stream to be decoded by the audio decoder are played.

[0079] In this solution, multiple threads are established based on the video stream to be decoded, and multiple target queues of video frames to be decoded corresponding to the video stream to be decoded are determined based on the number of established threads. When decoding the video stream to be decoded, the video decoder corresponding to each thread decodes the picture groups in the corresponding target queue of video frames to be decoded. This enables parallel decoding of multiple picture groups, ensuring video decoding efficiency and enabling the web player to adapt to various video streaming playback scenarios.

[0080] The following describes in detail one or more embodiments of the video stream playback device of the present invention. Those skilled in the art will appreciate that these devices can be constructed using commercially available hardware components and configured according to the steps taught in this solution.

[0081] Figure 7 A schematic diagram of the structure of a video stream playback device provided by an embodiment of the present invention, which is applied to a player on a Web side, such as Figure 7 As shown, the device includes: an acquisition module 11, a processing module 12, and a playing module 13.

[0082] The acquisition module 11 is configured to, in response to a video playback request, acquire a to-be-decoded video stream corresponding to the video playback request, wherein the to-be-decoded video stream includes to-be-decoded video frames and to-be-decoded audio frames.

[0083] The processing module 12 is used to establish multiple threads based on the video stream to be decoded, each thread corresponding to a video decoder; decode the video frames to be decoded by the multiple video decoders to obtain video frame data corresponding to the video stream to be decoded; and decode the audio frames to be decoded by the audio decoder to obtain audio frame data corresponding to the video stream to be decoded.

[0084] The playing module 13 is configured to play the video frame data and the audio frame data.

[0085] Optionally, the processing module 12 is specifically used to establish a test thread, where the test thread corresponds to a video decoder; the test video frame in the preset video stream to be tested is decoded by the video decoder corresponding to the test thread to determine the video decoding capability corresponding to the video decoder under the current hardware environment; and multiple threads are established based on the video decoding capability and the video stream to be decoded.

[0086] Optionally, the processing module 12 is further configured to establish multiple threads according to the video decoding capability and the frame rate corresponding to the video stream to be decoded.

[0087] Optionally, the processing module 12 is further specifically used to determine a plurality of target video frame queues to be decoded corresponding to the video stream to be decoded according to the number of threads corresponding to the plurality of threads, and the plurality of target video frame queues to be decoded correspond one-to-one to the plurality of threads; respectively decode the video frames to be decoded contained in the plurality of target video frame queues to be decoded by a plurality of the video decoders to obtain target video frame data respectively corresponding to the plurality of target video frame queues to be decoded; and determine the video frame data corresponding to the video stream to be decoded according to the target video frame data.

[0088] Optionally, the video stream to be decoded includes multiple picture groups, each picture group contains multiple video frames to be decoded, and the processing module 12 is further specifically used to determine the target video frame queues to which the multiple picture groups belong; and decode the multiple video frames to be decoded contained in the picture groups in the multiple target video frame queues to be decoded by using multiple video decoders to obtain target video frame data corresponding to the multiple target video frame queues to be decoded.

[0089] Optionally, the processing module 12 is further configured to determine the target queue of video frames to be decoded to which the currently acquired group of pictures belongs according to the number of undecoded groups of pictures in the multiple target queues of video frames to be decoded.

[0090] Optionally, the player uses a first programming language, the decoder uses a second programming language, and the thread established by the player calls the decoder through WebAssembly.

[0091] Figure 7 The device shown can execute the steps introduced in the aforementioned embodiments. For detailed execution process and technical effects, please refer to the description in the aforementioned embodiments and will not be repeated here.

[0092] In one possible design, the above Figure 7 The structure of the video stream playing device shown can be implemented as an electronic device, such as Figure 8As shown, the electronic device may include: a memory 21, a processor 22, and a communication interface 23. The memory 21 stores executable code, and when the executable code is executed by the processor 22, the processor 22 can at least implement the video stream playback method provided in the above embodiment.

[0093] In addition, an embodiment of the present invention provides a non-transitory machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the video stream playback method provided in the aforementioned embodiment.

[0094] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Those skilled in the art can understand and implement the present invention without inventive effort.

[0095] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course can also be implemented by a combination of hardware and software. Based on this understanding, the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a computer product. The present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A video stream playback method, characterized in that: Players used on the web include: In response to a video playback request, obtaining a to-be-decoded video stream corresponding to the video playback request, wherein the to-be-decoded video stream includes a to-be-decoded video frame and a to-be-decoded audio frame; Establishing a test thread, where the test thread corresponds to a video decoder; Determine a target video stream to be tested having the same coding format as the video stream to be decoded from multiple video streams to be tested, and perform video decoding on the test video frame in the target video stream to be tested by using the video decoder corresponding to the test thread; Determine the video decoding capability of the video decoder in the current hardware environment based on the video decoding result of the target video stream to be tested; Determining and establishing a matching number of threads based on the video decoding capability and the frame rate of the video stream to be decoded, where each thread corresponds to a video decoder; Decoding the to-be-decoded video frames by using a plurality of the video decoders to obtain video frame data corresponding to the to-be-decoded video stream; Decoding the audio frame to be decoded by an audio decoder to obtain audio frame data corresponding to the video stream to be decoded; The video frame data and the audio frame data are played.

2. The method according to claim 1, characterized in that Decoding the to-be-decoded video frames by using the plurality of video decoders to obtain video frame data corresponding to the to-be-decoded video stream includes: Determining, according to the number of threads corresponding to the multiple threads, a plurality of target to-be-decoded video frame queues corresponding to the to-be-decoded video stream, wherein the plurality of target to-be-decoded video frame queues correspond one-to-one to the plurality of threads; Decoding the to-be-decoded video frames contained in the multiple target to-be-decoded video frame queues respectively by the multiple video decoders to obtain target video frame data corresponding to the multiple target to-be-decoded video frame queues respectively; According to the target video frame data, video frame data corresponding to the video stream to be decoded is determined.

3. The method according to claim 2, characterized in that The to-be-decoded video stream includes a plurality of picture groups, each picture group includes a plurality of the to-be-decoded video frames, and the to-be-decoded video frames included in the plurality of target to-be-decoded video frame queues are respectively decoded by the plurality of video decoders to obtain target video frame data corresponding to the plurality of target to-be-decoded video frame queues, including: Determining a target queue of video frames to be decoded to which the multiple groups of pictures belong; The multiple video frames to be decoded contained in the picture groups in the multiple target video frame queues to be decoded are decoded respectively by the multiple video decoders to obtain target video frame data corresponding to the multiple target video frame queues to be decoded respectively.

4. The method according to claim 3, characterized in that The determining of a target queue of video frames to be decoded to which the multiple groups of pictures belong includes: The target to-be-decoded video frame queue to which the currently acquired picture group belongs is determined according to the number of undecoded picture groups in the plurality of target to-be-decoded video frame queues.

5. The method according to any one of claims 1 to 4, characterized in that The player uses a first programming language, the decoder uses a second programming language, and the thread established by the player calls the decoder through WebAssembly.

6. A video stream playback device, characterized in that: Players used on the web include: An acquisition module, configured to, in response to a video playback request, acquire a to-be-decoded video stream corresponding to the video playback request, wherein the to-be-decoded video stream includes a to-be-decoded video frame and a to-be-decoded audio frame; A processing module is used to establish a test thread, where the test thread corresponds to a video decoder; determine a target video stream to be tested having the same coding format as that corresponding to the video stream to be decoded from multiple video streams to be tested, and perform video decoding on the test video frames in the target video stream to be tested through the video decoder corresponding to the test thread; determine the video decoding capability corresponding to the video decoder under the current hardware environment based on the video decoding result of the target video stream to be tested; determine and establish a matching number of multiple threads based on the video decoding capability and the frame rate corresponding to the video stream to be decoded, where each thread corresponds to a video decoder; decode the video frames to be decoded through the multiple video decoders to obtain video frame data corresponding to the video stream to be decoded; and decode the audio frames to be decoded through the audio decoder to obtain audio frame data corresponding to the video stream to be decoded. A playing module is used to play the video frame data and the audio frame data.

7. An electronic device, characterized in that: include: A memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the video stream playback method according to any one of claims 1 to 5.

8. A non-transitory machine-readable storage medium, characterized in that The non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor is caused to execute the video stream playback method according to any one of claims 1 to 5.

Images