A browser-based video stream playing method and device

Abstract

This application provides a browser-based video streaming playback method and apparatus, comprising: acquiring a first video frame of a video to be played via a long connection, creating a network worker thread for transcoding, then creating a media data source object and creating a source buffer object according to a multipurpose Internet Mail extension type; adding the created source buffer object to a source buffer list; setting the mode of the source buffer object to sequence mode; receiving and merging multiple received video segments to be played into a continuous video segment through the source buffer list; calculating the difference between the current playback time and the end time of the current buffer segment to obtain the buffer difference; determining the number of frame skips for a frame skipping timer based on the buffer difference and a preset threshold, and skipping frames for the continuous video segment. This can solve the problem of deteriorating real-time performance and playback delay during live streaming in a browser player.

Description

Technical Field

[0001] This application relates to the field of video playback technology, and in particular to a browser-based video streaming playback method and apparatus. Background Technology

[0002] Currently, most MSE (Media Source Extensions) players employ a sequential appending + passive playback strategy. This means the player continuously appends segments to the SourceBuffer and relies on the browser's buffering mechanism to progress. However, when there is network latency, cache misalignment, or excessive processing time, the playback timeline gradually lags behind the actual live stream, leading to a gradual deterioration in real-time performance and even playback delays and stuttering. Summary of the Invention

[0003] The purpose of this application is to provide a browser-based video streaming method and apparatus to solve the problems of deteriorated real-time performance and playback delay during live streaming by browser players. The specific technical solution is as follows:

[0004] A first aspect of this application provides a browser-based video stream playback method, including:

[0005] Obtain the first video frame of the video to be played through a long connection; create a network worker thread based on the first video frame to perform transcoding processing; create a media data source object based on the bitstream segment returned by the created network worker thread;

[0006] Bind the created media data source object to the video tag of the video to be played; create a source buffer object according to the Multipurpose Internet Mail Extensions type; add the created source buffer object to the source buffer list; set the mode of the source buffer object to sequence mode;

[0007] Receive multiple video segments to be played through the source buffer list; merge the received multiple video segments to be played into continuous video buffer data, and update the source buffer object;

[0008] When playing the continuous video buffer data, the current playback time and the current buffer segment end time of the continuous video buffer data are obtained; the difference between the current playback time and the current buffer segment end time is calculated to obtain the buffer difference; based on the buffer difference and a preset threshold, the number of frame skips for the frame skip timer is determined; and frame skipping control is performed on the currently playing content based on the determined number of frame skips.

[0009] In one possible implementation, determining the number of frame skips for the frame skipping timer based on the buffer difference and a preset threshold includes:

[0010] When the buffer difference is greater than 1 second and the frame skipping timer has 0 skipping times, the frame skipping timer is reset to the maximum skipping times.

[0011] When the buffer difference is greater than 0.5 seconds and the frame skipping timer is not 0, the playback time is adjusted to the current buffer segment end time - 0.4 seconds, and the frame skipping counter is decremented by 1.

[0012] In one possible implementation, after skipping frames in the consecutive video segments according to a determined number of skips, the method further includes:

[0013] Identify the update information of the source buffer list; identify the deletion time information of the cached video segments; when it is identified that the source buffer list is not being updated and the current time is more than 20 seconds away from the last time the cached video segment was deleted, call the source buffer list to delete the cached video segments whose removal offset is 10 seconds from the current time.

[0014] In one possible implementation, the method further includes:

[0015] When playing the continuous video segments, the system checks whether the number of video segments to be played in the playback buffer is greater than 1 based on the buffer segment index. If the number of video segments to be played in the playback buffer is greater than 1, and the current buffer segment index is incremented, the playback time is set to jump to the start time of the next segment. The buffer deletion offset is reset, and the buffer area data corresponding to the already played video segments to be played is deleted. Playback continues and the buffer segment index is reset.

[0016] In one possible implementation, the method further includes:

[0017] When playing the continuous video segments, if the current time has exceeded the start time of the next video segment to be played, the current cache is deleted and the next video segment to be played begins to play.

[0018] A second aspect of this application provides a browser-based video streaming playback device, comprising:

[0019] The video acquisition module is used to acquire the first video frame of the video to be played through a long connection; create a network worker thread based on the first video frame to perform transcoding processing; and create a media data source object based on the bitstream segment returned by the created network worker thread.

[0020] The video binding module is used to bind the created media data source object to the video tag of the video to be played; create a source buffer object according to the Multipurpose Internet Mail Extensions type; add the created source buffer object to the source buffer list; and set the mode of the source buffer object to sequence mode.

[0021] The video merging module is used to receive multiple video segments to be played through the source buffer list; merge the received multiple video segments to be played into continuous video buffer data, and update the source buffer object;

[0022] The video frame skipping module is used to obtain the current playback time and the current buffer segment end time of the continuous video buffer data when playing the continuous video buffer data; calculate the difference between the current playback time and the current buffer segment end time to obtain the buffer difference; determine the number of frame skipping times of the frame skipping timer based on the buffer difference and a preset threshold; and perform frame skipping control on the currently playing content based on the determined number of frame skipping times.

[0023] In one possible implementation, the video frame skipping module is specifically used to reset the frame skipping timer to the maximum frame skipping count when the buffer difference is greater than 1 second and the frame skipping count of the frame skipping timer is 0; and when the buffer difference is greater than 0.5 seconds and the frame skipping count of the frame skipping timer is not 0, adjust the playback time to the current buffer segment end time - 0.4 seconds, and decrement the frame skipping counter by 1.

[0024] In one possible implementation, the device further includes:

[0025] The cache deletion module is used to identify the update information of the source buffer list; identify the deletion time information of the cached video segments; when it is identified that the source buffer list is not being updated and the current time is more than 20 seconds away from the last time the cached video segment was deleted, the source buffer list is called to delete the cached video segments whose removal offset is 10 seconds from the current time.

[0026] In one possible implementation, the device further includes:

[0027] The cache reset module is used to detect whether the number of video segments to be played in the playback cache is greater than 1 based on the buffer segment index when playing the continuous video segments; if the number of video segments to be played in the playback cache is greater than 1, and if the current buffer segment index is incremented, the playback time is set to jump to the start time of the next segment; the cache deletion offset is reset, and the cache area data corresponding to the already played video segments to be played is deleted; playback continues and the buffer segment index is reset.

[0028] In one possible implementation, the device further includes:

[0029] The segment playback module is used to, when playing the continuous video segments, if the current time has exceeded the start time of the next video segment to be played, delete the current cache and start playing the next video segment to be played.

[0030] Another aspect of the application embodiments also provides an electronic device, including:

[0031] Memory, used to store computer programs;

[0032] The processor, when executing a program stored in memory, implements any of the above-mentioned browser-based video streaming playback methods.

[0033] In another aspect of the embodiments of this application, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements any of the above-described browser-based video streaming playback methods.

[0034] In another aspect of the embodiments of this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the above-described browser-based video streaming playback methods.

[0035] Beneficial effects of the embodiments in this application:

[0036] This application provides a browser-based video streaming playback method and apparatus, comprising: acquiring a first video frame of a video to be played via a long connection; creating a network worker thread based on the first video frame for transcoding processing; creating a media data source object using the bitstream segment returned by the created network worker thread; binding the created media data source object to the video tag of the video to be played; creating a source buffer object according to a multipurpose Internet Mail extension type; adding the created source buffer object to a source buffer list; setting the mode of the source buffer object to sequence mode; receiving multiple video segments to be played through the source buffer list; merging the received multiple video segments to be played into continuous video buffer data and updating the source buffer object; while playing the continuous video buffer data, acquiring the current playback time and the current buffer segment end time of the continuous video buffer data; calculating the difference between the current playback time and the current buffer segment end time to obtain a buffer difference; determining the number of frame skips for a frame skipping timer based on the buffer difference and a preset threshold; and controlling frame skipping of the currently playing content based on the determined number of frame skips. The solution proposed in this application allows for the creation of a network worker thread and a media data source after acquiring the first video frame of the video to be played. This enables the reception and playback of multiple video segments through the media data source. Furthermore, during playback, the number of frame skips for the frame skipping timer is determined based on the buffer difference and a preset threshold. This allows for frame skipping playback based on the determined number of frame skips, thus resolving the issues of degraded real-time performance and playback delays during live streaming by browser players.

[0037] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0039] Figure 1 A flowchart illustrating a browser-based video stream playback method provided in this application embodiment;

[0040] Figure 2 A schematic diagram of a network worker thread provided in an embodiment of this application;

[0041] Figure 3 A schematic diagram illustrating the detection delay provided in an embodiment of this application;

[0042] Figure 4 A schematic diagram illustrating the source of media data provided in the embodiments of this application;

[0043] Figure 5 A schematic diagram of a browser-based video streaming playback device provided in an embodiment of this application;

[0044] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0045] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0046] A first aspect of this application provides a browser-based video streaming playback method, see [link to relevant documentation]. Figure 1 , Figure 1 A flowchart illustrating a browser-based video stream playback method provided in this application includes:

[0047] Step S11: Obtain the first video frame of the video to be played through a long connection; create a network worker thread based on the first video frame for transcoding processing; create a media data source object through the bitstream segment returned by the created network worker thread;

[0048] Step S12: Bind the created media data source object to the video tag of the video to be played; create a source buffer object according to the multipurpose Internet Mail extension type; add the created source buffer object to the source buffer list; set the mode of the source buffer object to sequence mode;

[0049] Step S13: Receive multiple video segments to be played through the source buffer list; merge the received multiple video segments to be played into continuous video buffer data, and update the source buffer object;

[0050] Step S14: When playing the continuous video buffer data, obtain the current playback time and the current buffer segment end time of the continuous video buffer data; calculate the difference between the current playback time and the current buffer segment end time to obtain the buffer difference; determine the number of frame skips for the frame skip timer based on the buffer difference and a preset threshold; and perform frame skip control on the currently playing content based on the determined number of frame skips.

[0051] Corresponding to step S11 above, the method of this embodiment is applied to a browser. Specifically, a long connection can be created through the browser, and the first video frame of the video to be played can be obtained through this long connection. In actual use, this video frame can be a live video frame. When creating a network worker thread based on the first video frame for transcoding processing, see [link to relevant documentation]. Figure 2 This can be done by declaring a long-lived WebSocket connection (a bidirectional network communication protocol) through the browser, waiting to receive the first video frame of the video to be played, as imaging can only occur when the first video frame is received. This video frame is then used to create a WebWorker (network worker thread) for transcoding, avoiding blocking page rendering and JavaScript tasks. The bitstream fragment returned by this WebWorker is then used to create a MediaSource (media data source).

[0052] Corresponding to step S12 above, when binding the created media data source to the video tag of the video to be played, the tag of the video to be played can be obtained and bound to the created media data source. Specifically, a MediaSource object can be created in the browser and bound to the video tag. In actual use, when creating a source buffer object based on the MIME type, and adding the created source buffer object to the source buffer list, a new SourceBuffer object can be created based on the given MIME type using the `mediaSource.addSourceBuffer(media data source.add source buffer)` method, and then appended to the MediaSource's SourceBuffers list. In one example, the MIME type can be video / mp4 (a video format). The mode of the source buffer object is set to sequence mode.

[0053] Corresponding to step S13 above, since live stream video is generally composed of consecutive segments, in this embodiment, multiple video segments to be played are received through the source buffer list, merged into continuous video buffer data, and the source buffer object is updated. Specifically, sourceBuffer.mode can be set to sequence to avoid additional delays caused by segment alignment. In actual use, multiple real-time arriving video / audio segments are merged into a continuous buffer and then updated via appendBuffer to avoid blocking the UI or causing update blocking. In one example, receiving multiple video segments to be played through the source buffer list may include: creating a video element, creating a media source object, a processing function when the media source is opened, creating a media buffer object, creating a WebSocket connection, listening for WebSocket message events, appending received data to the buffer, and setting the media source URL to the src attribute of the video element.

[0054] Corresponding to step S14 above, when playing the continuous video buffer data, the current playback time and the current buffer segment end time of the continuous video buffer data can be obtained. The buffer difference is then calculated by determining the difference between the current playback time and the current buffer segment end time. Based on the buffer difference and a preset threshold, the number of frame skips for the frame skipping timer is determined. Frame skipping is then performed on the continuous video segment according to the determined number of frame skips. Specifically, currentTime (current playback time) and endTime (current buffer segment end time) can be obtained, and their difference is calculated to obtain the buffer difference endTime - currentTime. If the buffer difference is greater than the preset threshold, the frame skipping logic is executed. This frame skipping solves the problem of deteriorated real-time performance and playback delay during live streaming in the browser player.

[0055] As can be seen, the solution of this application can create a network worker thread and a media data source after obtaining the first video frame of the video to be played, so as to further receive and play multiple video segments to be played through the media data source. During the playback, the number of frame skips of the frame skipping timer is determined according to the buffer difference and the preset threshold, so as to play the video by skipping frames according to the determined number of frame skips, thereby solving the problem of deterioration of real-time performance and playback delay during live streaming of the browser player.

[0056] In one possible implementation, determining the frame skipping count of the frame skipping timer based on the buffer difference and a preset threshold includes: when the buffer difference is greater than 1 second and the frame skipping count of the frame skipping timer is 0, resetting the frame skipping count of the frame skipping timer to the maximum frame skipping count; when the buffer difference is greater than 0.5 seconds and the frame skipping count of the frame skipping timer is not 0, adjusting the playback time to the current buffer segment end time - 0.4 seconds, and decrementing the frame skipping counter by 1. Specifically, see... Figure 3 The delay is calculated using (endTime - currentTime), typically within the range of 0-1 seconds. The frame skipping strategy in this application is as follows: if the buffer difference is greater than 1 second and the frame skipping counter is 0, the counter is reset to the maximum number of frame skips; if the buffer difference is greater than 0.5 seconds and the frame skipping counter has not exhausted, the playback time is adjusted to endTime - 0.4 seconds, and the counter is decremented, performing five consecutive frame skips at a time, after which normal playback resumes, as the browser will automatically catch up based on the data. In this application, subtracting 0.4 seconds instead of directly equaling endTime is to reserve sufficient buffer data for MSE (mean squared error). MSE requires a certain amount of buffer data before playback begins, thus ensuring that the frame skipping steps do not affect the smooth playback of the video.

[0057] In one possible implementation, after skipping frames of the consecutive video segments according to a determined number of skips, the method further includes: identifying update information of the source buffer list; identifying cached video segment deletion time information; and when it is identified that the source buffer list is not being updated and the current time is more than 20 seconds since the last cached video segment deletion, calling the source buffer list to delete the cached video segments from the removal offset to the current time - 10 seconds. In one example, in this embodiment, update information of the source buffer list and cached video segment deletion time information can be identified. Therefore, when the source buffer is not being updated and the current time is more than 20 seconds since the last cache deletion, calling sourceBuffer.remove(start, end) (source buffer list) deletes the buffer from removeOffset (removal offset) to currentTime - 10, updating removeOffset to currentTime - 10, thereby controlling the SourceBuffer size and preventing buffer accumulation that could cause playback stuttering.

[0058] In one possible implementation, the method further includes: when playing the continuous video segments, detecting whether the number of video segments to be played in the playback buffer is greater than 1 according to the buffer segment index; if the number of video segments to be played in the playback buffer is greater than 1, and if the current buffer segment index is incremented, setting the playback time to jump to the start time of the next segment; resetting the cache deletion offset and deleting the buffer data corresponding to the already played video segments to be played; continuing playback and resetting the buffer segment index. In this embodiment, multiple video segments can be automatically switched: the system maintains currentSegmentIndex (current buffer segment index) and obtains the next buffer segment. Detecting whether the number of segments in the playback buffer is greater than 1—the current video has been segmented, the segmentation reason being that the segments are not continuous or the buffer is full. In this case, in order to achieve seamless continuous playback of multiple video segments while controlling the buffer size, the buffer is operated as follows: the current segment index is incremented, and the playback time is set to jump to the start time of the next segment. The cache deletion offset is reset, and the buffer data corresponding to the already played video segments to be played is deleted. The video continues to play, and the buffer index and frame skip counter are reset.

[0059] In one possible implementation, the method further includes: when playing the continuous video segments, if the current time has exceeded the start time of the next video segment to be played, deleting the current cache and starting to play the next video segment to be played. Specifically, the steps for automatic switching between multiple video segments may include: time segmentation processing, whether to segment, if the current time has exceeded the start time of the next segment, deleting the current cache, then starting to play the next video segment, and resetting the catch-up count. In practice, reserving approximately 0.3-0.4 seconds of frames can better ensure the smoothness of video playback. Five consecutive frame skips are performed at a time, followed by normal playback. Because browser playback also automatically catches up based on data, clearing already played data and retaining a 10-second cache ensures smooth playback.

[0060] The solution proposed in this application enables a smooth playback mechanism with automatic segment skipping. By detecting the time boundary of the buffer segment, the system automatically jumps to the latest buffer segment when a new segment is added to the current playback area. This seamless switching avoids playback interruptions caused by segment discontinuities or a full buffer, maintaining smooth playback without manual intervention. Furthermore, see [link to related document]. Figure 4 This feature improves playback real-time performance and user experience by setting `sourceBuffer.mode` to `sequence`, avoiding additional latency caused by segment alignment. It automatically deletes expired cache segments, preventing playback latency accumulation due to buffer backlog. Each time a segment is pushed in, the current playback time and buffer end time are checked, automatically shortening the time difference between the playback and streaming ends. Users can watch the latest content almost simultaneously, significantly improving latency perception in live streaming scenarios. Introducing segment skipping counts avoids playback stuttering or errors caused by frequent frame skipping, ensuring sufficient data in the buffer to support continuous playback. Furthermore, it reduces memory usage and system load by immediately clearing old buffer data after segment cutting, reducing memory consumption and SourceBuffer backlog issues. It improves the stability of MSE playback on the browser side, preventing stuttering or crashes after prolonged playback. It also enables efficient buffer merging and `appendBuffer` optimization, as frequent `appendBuffer` can block the UI or cause update blocking. It merges multiple frames into a single buffer before `appendBuffer`. It controls the `updateend` flag to ensure continuous playback and avoid update conflicts. It allows for delayed `append` under specific conditions (debug or snapshot).

[0061] A second aspect of this application provides a browser-based video streaming playback device, see [link to relevant documentation]. Figure 5 , Figure 5 A schematic diagram of a browser-based video streaming playback device provided in this application embodiment includes:

[0062] The video acquisition module 501 is used to acquire the first video frame of the video to be played through a long connection; create a network worker thread based on the first video frame to perform transcoding processing; and create a media data source object through the bitstream segment returned by the created network worker thread.

[0063] The video binding module 502 is used to bind the created media data source object to the video tag of the video to be played; create a source buffer object according to the multipurpose Internet Mail extension type; add the created source buffer object to the source buffer list; and set the mode of the source buffer object to sequence mode.

[0064] The video merging module 503 is used to receive multiple video segments to be played through the source buffer list; merge the received multiple video segments to be played into continuous video buffer data; and update the source buffer object.

[0065] The video frame skipping module 504 is used to obtain the current playback time and the current buffer segment end time of the continuous video buffer data when playing the continuous video buffer data; calculate the difference between the current playback time and the current buffer segment end time to obtain the buffer difference; determine the number of frame skipping times of the frame skipping timer based on the buffer difference and a preset threshold; and perform frame skipping control on the currently playing content based on the determined number of frame skipping times.

[0066] In one possible implementation, the video frame skipping module is specifically used to reset the frame skipping timer to the maximum frame skipping count when the buffer difference is greater than 1 second and the frame skipping count of the frame skipping timer is 0; and when the buffer difference is greater than 0.5 seconds and the frame skipping count of the frame skipping timer is not 0, adjust the playback time to the current buffer segment end time - 0.4 seconds, and decrement the frame skipping counter by 1.

[0067] In one possible implementation, the device further includes:

[0068] The cache deletion module is used to identify the update information of the source buffer list; identify the deletion time information of the cached video segments; when it is identified that the source buffer list is not being updated and the current time is more than 20 seconds away from the last time the cached video segment was deleted, the source buffer list is called to delete the cached video segments whose removal offset is 10 seconds from the current time.

[0069] In one possible implementation, the device further includes:

[0070] The cache reset module is used to detect whether the number of video segments to be played in the playback cache is greater than 1 based on the buffer segment index when playing the continuous video segments; if the number of video segments to be played in the playback cache is greater than 1, and if the current buffer segment index is incremented, the playback time is set to jump to the start time of the next segment; the cache deletion offset is reset, and the cache area data corresponding to the already played video segments to be played is deleted; playback continues and the buffer segment index is reset.

[0071] In one possible implementation, the device further includes:

[0072] The segment playback module is used to, when playing the continuous video segments, if the current time has exceeded the start time of the next video segment to be played, delete the current cache and start playing the next video segment to be played.

[0073] As can be seen, the solution of this application can create a network worker thread and a media data source after obtaining the first video frame of the video to be played, so as to further receive and play multiple video segments to be played through the media data source. During the playback, the number of frame skips of the frame skipping timer is determined according to the buffer difference and the preset threshold, so as to play the video by skipping frames according to the determined number of frame skips, thereby solving the problem of deterioration of real-time performance and playback delay during live streaming of the browser player.

[0074] In another aspect of the embodiments of this application, an electronic device is also provided, see [link to relevant documentation]. Figure 6 ,include:

[0075] Memory 601 is used to store computer programs;

[0076] Processor 602, when executing a program stored in memory, implements:

[0077] Obtain the first video frame of the video to be played through a long connection; create a network worker thread based on the first video frame to perform transcoding processing; create a media data source object based on the bitstream segment returned by the created network worker thread;

[0078] Bind the created media data source object to the video tag of the video to be played; create a source buffer object according to the Multipurpose Internet Mail Extensions type; add the created source buffer object to the source buffer list; set the mode of the source buffer object to sequence mode;

[0079] Receive multiple video segments to be played through the source buffer list; merge the received multiple video segments to be played into continuous video buffer data, and update the source buffer object;

[0080] When playing the continuous video buffer data, the current playback time and the current buffer segment end time of the continuous video buffer data are obtained; the difference between the current playback time and the current buffer segment end time is calculated to obtain the buffer difference; based on the buffer difference and a preset threshold, the number of frame skips for the frame skip timer is determined; and frame skipping control is performed on the currently playing content based on the determined number of frame skips.

[0081] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0082] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0083] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0084] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0085] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements any of the above-described browser-based video streaming methods.

[0086] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, enables the computer to implement any of the above-described browser-based video streaming playback methods.

[0087] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a solid-state drive (SSD), etc.

[0088] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0089] The various embodiments in this specification are described in a related manner. For the same or similar parts between the embodiments, please refer to the corresponding descriptions. Each embodiment focuses on describing the differences from other embodiments. For related parts, please refer to the descriptions in the method embodiments.

[0090] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A browser-based video streaming playback method, characterized in that, include: The first video frame of the video to be played is obtained through a long connection; a network worker thread is created based on the first video frame to perform transcoding processing; Create a media data source object using the bitstream fragment returned by the created network worker thread; Bind the created media data source object to the video tag of the video to be played; Create a source buffer object based on the Multipurpose Internet Mail Extensions type; add the created source buffer object to the source buffer list; set the mode of the source buffer object to sequence mode; Receive multiple video segments to be played through the source buffer list; merge the received multiple video segments to be played into continuous video buffer data, and update the source buffer object; When playing the continuous video buffer data, obtain the current playback time and the current buffer segment end time of the continuous video buffer data; Calculate the difference between the current playback time and the current buffer segment end time to obtain the buffer difference; Based on the buffer difference and the preset threshold, the frame skipping timer is determined to skip a number of frames; based on the determined number of frame skipping, frame skipping control is performed on the currently playing content.

2. The method according to claim 1, characterized in that, The step of determining the number of frame skips for the frame skipping timer based on the buffer difference and a preset threshold includes: When the buffer difference is greater than 1 second and the frame skipping timer has 0 skipping times, the frame skipping timer is reset to the maximum skipping times. When the buffer difference is greater than 0.5 seconds and the frame skipping timer is not 0, the playback time is adjusted to the current buffer segment end time - 0.4 seconds, and the frame skipping counter is decremented by 1.

3. The method according to claim 1, characterized in that, After performing frame skipping on the consecutive video segments according to the determined number of frame skips, the method further includes: Identify the update information of the source buffer list; identify the deletion time information of the cached video segments; when it is identified that the source buffer list is not being updated and the current time is more than 20 seconds away from the last time the cached video segment was deleted, call the source buffer list to delete the cached video segments whose removal offset is 10 seconds from the current time.

4. The method according to claim 1, characterized in that, The method further includes: When playing the continuous video segments, the system checks whether the number of video segments to be played in the playback buffer is greater than 1 based on the buffer segment index. If the number of video segments to be played in the playback buffer is greater than 1, and the current buffer segment index is incremented, the playback time is set to jump to the start time of the next segment. The buffer deletion offset is reset, and the buffer area data corresponding to the already played video segments to be played is deleted. Playback continues and the buffer segment index is reset.

5. The method according to claim 1, characterized in that, The method further includes: When playing the continuous video segments, if the current time has exceeded the start time of the next video segment to be played, the current cache is deleted and the next video segment to be played begins to play.

6. A browser-based video streaming playback device, characterized in that, include: The video acquisition module is used to acquire the first video frame of the video to be played through a long connection; and to create a network worker thread based on the first video frame for transcoding processing. Create a media data source object using the bitstream fragment returned by the created network worker thread; The video binding module is used to bind the created media data source object to the video tag of the video to be played; Create a source buffer object based on the Multipurpose Internet Mail Extensions type; add the created source buffer object to the source buffer list; set the mode of the source buffer object to sequence mode; The video merging module is used to receive multiple video segments to be played through the source buffer list; merge the received multiple video segments to be played into continuous video buffer data, and update the source buffer object; The video frame skipping module is used to obtain the current playback time and the current buffer segment end time of the continuous video buffer data when playing the continuous video buffer data; Calculate the difference between the current playback time and the current buffer segment end time to obtain the buffer difference; Based on the buffer difference and the preset threshold, the frame skipping timer is determined to skip a number of frames; based on the determined number of frame skipping, frame skipping control is performed on the currently playing content.

7. The apparatus according to claim 6, characterized in that, The video frame skipping module is specifically used to reset the frame skipping timer to the maximum frame skipping count when the buffer difference is greater than 1 second and the frame skipping count of the frame skipping timer is 0; and to adjust the playback time to the current buffer segment end time - 0.4 seconds and decrement the frame skipping counter by 1 when the buffer difference is greater than 0.5 seconds and the frame skipping count of the frame skipping timer is not 0.

8. The apparatus according to claim 6, characterized in that, The device further includes: The cache deletion module is used to identify the update information of the source buffer list; identify the deletion time information of the cached video segments; when it is identified that the source buffer list is not being updated and the current time is more than 20 seconds away from the last time the cached video segment was deleted, the source buffer list is called to delete the cached video segments whose removal offset is 10 seconds from the current time.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-5.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains a computer program that, when executed by a processor, implements the method described in any one of claims 1-5.

Images