Video stream playing method and device, equipment and storage medium

Abstract

The invention relates to the technical field of video stream data processing, in particular to a video stream playing method, device and equipment and a storage medium, and the method comprises the steps: setting an interface for requesting a photo; sending a request through the address, and obtaining a photo; detecting the format of the photo; decoding the picture which is detected to be qualified; and performing graph drawing on the photo to generate a video stream, or processing the photo into the video stream by adopting an encoder. According to the method, the photo is acquired from the address, and the format of the photo is detected, so that automatic frame drop and automatic degradation of photo decoding are realized, smooth playing of the video stream is ensured, and the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of video stream data processing, and in particular to a video stream playing method, device, equipment and storage medium. Background Art

[0002] In video data processing, it is sometimes necessary to convert images in the device into video streams for playback. In the prior art, although the traditional method of converting images to video streams can convert images into video streams for playback, the process of converting images to video streams is prone to lag, and the frame rate of the real-time converted video stream is low, resulting in poor viewing experience.

[0003] In summary, there is an urgent need for a video stream playback method, device, equipment and storage medium to solve the problems of the prior art. Summary of the invention

[0004] The purpose of the present invention is to provide a video stream playback method, which processes photo data into a video stream for playback, thereby solving the problem of playback freeze when converting existing pictures into video streams.

[0005] In order to solve the above technical problems, the present invention provides a video stream playing method, comprising:

[0006] Set up the interface for requesting photos;

[0007] Send a request via address to get photos;

[0008] Detect the format of the photo;

[0009] Decode the qualified photos;

[0010] Generate video streams by drawing graphics on photos, or use encoders to process photos into video streams.

[0011] Optionally, setting an interface for requesting photos includes: setting the number of requests per unit time according to fps set by the business.

[0012] Optionally, during the process of sending the request, a timeout check is performed on the request. If the request times out, the connection with the interface is disconnected; otherwise, the photo is obtained.

[0013] Optionally, perform a timeout check on the request. Specifically, initiate two requests in parallel. If the slower request returns first, it times out.

[0014] Optionally, the situation of detecting the format of the photo includes at least one of the following:

[0015] The format of the photo is qualified, and the photo that has passed the inspection is output;

[0016] The format of the photo is unqualified, it is judged as a known exception, and the interface is returned for reconnection;

[0017] The photo format is unqualified and is judged as an unknown exception, so the resources are released.

[0018] Optionally, the qualified photos are decoded to convert the native photos into resources that can be recognized and played by the browser media element.

[0019] Optionally, during the process of drawing graphics for the photo, the drawing frequency is determined according to the frame rate of the video stream.

[0020] In addition, the present invention also provides a video stream playback device, which is used to implement the above-mentioned video stream playback method, and the device includes:

[0021] An interface setting module, used to set the interface for requesting photos;

[0022] The request sending module is used to send a request through an address to obtain photos;

[0023] A photo detection module is used to detect the format of the photo;

[0024] A photo decoding module, used for decoding the qualified photos;

[0025] The video stream generation module is used to process photos and generate video streams.

[0026] In addition, the present invention also provides an electronic device, comprising:

[0027] Memory, used to store computer programs;

[0028] The processor is used to execute the computer program to implement the steps of the video stream playback method as described above.

[0029] In addition, the present invention also provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the steps of the method for playing a video stream as described above are implemented.

[0030] A video stream playback method provided by the present invention can process photo-formatted data collected by a device and convert the photos into video streams. The method of the present invention has network monitoring and automatic frame reduction technology, which can ensure smooth video stream playback and improve user experience.

[0031] The present invention also provides a video stream playback device, an electronic device and a computer-readable storage medium, which also have the above-mentioned beneficial effects and will not be described in detail here. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0033] Figure 1 A flowchart of a video stream playback method provided by an embodiment of the present invention;

[0034] Figure 2 A structural block diagram of a video stream playback device provided by an embodiment of the present invention;

[0035] Figure 3 The present invention provides a structural block diagram of an electronic device. DETAILED DESCRIPTION

[0036] The core of the present invention is to provide a video stream playback method. In the prior art, although the traditional method of converting pictures to video streams can convert pictures into video streams for playback, the process of converting pictures to video streams is prone to lag, and the real-time converted video stream has a low frame rate and poor viewing experience.

[0037] The video stream playback method provided by the present invention includes setting an interface for requesting photos; sending a request through an address to obtain photos; detecting the format of the photos; decoding the photos that pass the detection; drawing graphics on the photos to generate a video stream, or using an encoder to process the photos into a video stream.

[0038] The present invention obtains photos from an address and detects the photo format at the same time, thereby achieving automatic frame reduction and automatic downgrading of photo decoding, ensuring smooth video stream playback and improving user experience.

[0039] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0040] The terms in this embodiment are explained as follows:

[0041] MJPEG (Motion JPEG): MJPEG is a video compression format, the basic idea is to compress each frame of the image independently into JPEG format. This means that MJPEG is an inter-frame compression format, each frame is a complete JPEG image, instead of using redundant information between frames for compression like many modern video encoding formats.

[0042] Uint8Array: A typed array in JavaScript for handling binary data. Uint8Array is one of the typed arrays introduced in ECMAScript 6 (ES6), specifically used to represent an array containing 8-bit unsigned integers, with each element ranging from 0 to 255.

[0043] PNG (Portable Network Graphics): A widely used bitmap image format. Its specific byte sequence starts with the following hexadecimal representation: 89 50 4E 47 0D 0A1A 0A.

[0044] JPEG (Joint Photographic Experts Group): A widely used image compression format, particularly suitable for storing and transmitting digital photographs and other color images. JPEG is a lossy compression format designed to significantly reduce file size with minimal loss of image quality. JPEG files begin with a specific identifier, usually the byte sequence FF D8, indicating the start of the JPEG file.

[0045] Class: es6 programming class, used here for the same management method

[0046] Canvas: A powerful feature in HTML5 that allows graphics and images to be dynamically drawn on web pages through JavaScript. Canvas provides a programmable canvas that developers can use to create various visual effects such as graphics, animations, games, and data visualization.

[0047] ImageDecoder: An API for decoding image data. It can convert image data (such as JPEG, PNG, WebP, etc.) into images that can be displayed in Canvas or other graphics contexts. ImageDecoder allows asynchronous decoding, supports multiple formats, and provides control over the progress of image decoding.

[0048] VideoEncoder: An API for encoding video streams, which can encode raw video data (such as frames) into a specific format (such as H.264, VP8, etc.). This API is suitable for applications that need to compress video data for storage or transmission, such as real-time video processing and streaming media.

[0049] createImageBitmap: A function used to create an image bitmap object. A bitmap can be created from various sources (such as HTMLImageElement, Canvas, Blob, ImageData, etc.). The generated bitmap can be used for efficient drawing to Canvas or other graphics contexts, and supports asynchronous processing to improve performance.

[0050] Blob: An object used to represent an immutable block of raw data, which can be binary data or text data. Blob is often used to process files (such as images, audio, video, etc.) and create URLs on the client side (using URL.createObjectURL) for display in a web page or download.

[0051] AbortController: An API for controlling asynchronous operations that can abort ongoing requests or operations by sending signals. For example, when using the Fetch API, you can use AbortController to abort requests to prevent no longer necessary network requests from consuming resources.

[0052] Fetch: An API for making network requests, built on Promise, that can replace the traditional XMLHttpRequest. Fetch provides a simpler syntax and more powerful functions, supporting cross-domain requests, asynchronous processing, and more flexible response processing.

[0053] Promise: An object used to handle asynchronous operations, which can represent an operation that has not yet been completed but is expected to be completed in the future.

[0054] Business layer: refers to the usage layer of the present invention, usually the business logic layer.

[0055] Please refer to Figure 1 , Figure 1 The present invention provides a flowchart of a video stream playback method.

[0056] See also Figure 1 In an embodiment of the present invention, the video stream playing method includes:

[0057] S1: Set up an interface for requesting photos.

[0058] In some embodiments, setting an interface for requesting photos includes: setting the number of requests per unit time according to the fps set by the business.

[0059] Specifically, the parameters set by the interface also include device id, media element canvas and photo address. In this embodiment, the timeout period is set to 0.5 seconds. If a single request times out, it will be actively disconnected and a new request will be initiated.

[0060] S2: Send a request through the address to obtain the photo.

[0061] In some embodiments, during the process of sending the request, the request is timed out, and if the request times out, the connection with the interface is disconnected, and if the request does not time out, the photo is obtained. The request is timed out, specifically, two requests are initiated in parallel, and if the slower request returns first, it times out.

[0062] Specifically, this embodiment uses fetch to request data, adds network monitoring when requesting, uses promise to process network requests (initiates two requests in parallel, and times out if the slower one returns first), and uses AbortController to terminate timed-out requests.

[0063] S3: Detect the format of the photo.

[0064] It should be noted that this embodiment processes the data obtained by the interface into ArrayBuffer and detects the photo, specifically checks the number of photo bytes and determines the specific data format of the photo. The standard photo formats supported by this embodiment are at least one of jpeg / png / gif / webp.

[0065] In some embodiments, the situation of detecting the format of the photo includes at least one of the following:

[0066] The format of the photo is qualified, and the photo that has passed the inspection is output;

[0067] The format of the photo is unqualified, it is judged as a known exception, and the interface is returned for reconnection;

[0068] The photo format is unqualified and is judged as an unknown exception, so the resources are released.

[0069] It should be noted that in order to solve the problem of video jamming and find a frame rate suitable for the current client network resources, the method of this embodiment performs automatic frame reduction processing and automatically disconnects the request. The method of this embodiment checks according to the standard format of photos to decide whether to reconnect or release resources, thereby improving data processing efficiency.

[0070] S4: Decode the qualified photos.

[0071] In some embodiments, the qualified photos are decoded and the native photos are converted into resources that can be recognized and played by the browser media element.

[0072] Furthermore, decoding the photo specifically converts the hexadecimal data into a resource that can be recognized and played by the browser media element. If the current client supports ImageDecoder, the photo is decoded and generated, otherwise Blob and createImageBitmap are used to generate the photo.

[0073] Specifically, the method of this embodiment uses automatic degradation processing to decode buffer data (native data obtained by the interface through the address).

[0074] S5: Performing graphics drawing on the photo to generate a video stream, or using an encoder to process the photo into a video stream.

[0075] In this embodiment, the decoded photo is obtained and drawn using canvas.

[0076] Alternatively, use VideoEncoder to directly generate video playback. Finally, release resources and free up memory.

[0077] Furthermore, in the process of drawing graphics for the photo, the drawing frequency is determined according to the frame rate of the video stream.

[0078] By applying the method of this embodiment, the photo-format data collected by the device can be processed and the photos can be converted into video streams. The method of this embodiment includes automatic degradation processing, network monitoring, automatic reconnection and automatic frame reduction. The method of this embodiment can ensure smooth playback of video stream data and improve the user's viewing experience.

[0079] A video stream playing device provided by an embodiment of the present invention is introduced below. The video stream playing device described below and the video stream playing method described above can be referred to each other.

[0080] Please refer to Figure 2 , Figure 2 A video stream playback device provided in an embodiment of the present invention is used to implement the above-mentioned video stream playback method, and the device includes:

[0081] The interface setting module 100 is used to set the interface for requesting photos;

[0082] A request sending module 200, used to send a request through an address to obtain a photo;

[0083] The photo detection module 300 is used to detect the format of the photo;

[0084] A photo decoding module 400 is used to decode the qualified photos;

[0085] The video stream generation module 500 is used to process photos and generate video streams.

[0086] The video stream playback device of this embodiment is used to implement the aforementioned video stream playback method. Therefore, the specific implementation of the video stream playback device can be seen in the embodiment of the video stream playback method in the previous text. For example, the interface setting module 100, the request sending module 200, the photo detection module 300, the photo decoding module 400, and the video stream generation module 500 are respectively used to implement steps S1 to S5 in the aforementioned video stream playback method. Therefore, its specific implementation can refer to the description of the corresponding embodiments of each part, which will not be repeated here.

[0087] A video stream playing device provided by an embodiment of the present invention is introduced below. The video stream playing device described below and the video stream playing method and the video stream playing apparatus described above can be referred to each other.

[0088] Please refer to Figure 3 , Figure 3 This is a structural block diagram of a video stream playback device provided by an embodiment of the present invention. Figure 3 , a video stream playing device, comprising:

[0089] A memory 11, used for storing computer programs;

[0090] The processor 12 is used to execute the computer program to implement the steps of the video stream playback method as described above.

[0091] The video stream playback device of this embodiment is used to implement the aforementioned video stream playback method. Therefore, the specific implementation of the video stream playback device can be seen in the embodiment of the video stream playback method in the previous text. Therefore, its specific implementation can refer to the description of the corresponding embodiments of each part, which will not be repeated here.

[0092] The embodiment of the present invention further discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by the processor 12, the steps of the method for playing a video stream as described above are implemented.

[0093] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0094] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0095] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0096] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0097] The above is a detailed introduction to a video stream playback method, a video stream playback device, and a video stream playback system provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, the present invention can also be improved and modified, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A video stream playback method, characterized in that: include: Set up the interface for requesting photos; Send a request via address to get photos; Detect the format of the photo; Decode the qualified photos; Generate video streams by drawing graphics on photos, or use encoders to process photos into video streams.

2. The video stream playing method according to claim 1, characterized in that: Setting the interface for requesting photos includes: setting the number of requests per unit time according to the fps set by the business.

3. The video stream playing method according to claim 2, characterized in that: In the process of sending the request, a timeout detection is performed on the request. If the request times out, the connection with the interface is disconnected. If the request does not time out, the photo is obtained.

4. The video stream playing method according to claim 3, characterized in that: Perform timeout detection on requests. Specifically, initiate two requests in parallel. If the slower request returns first, it will time out.

5. The video stream playing method according to claim 1, characterized in that: The detection of the photo format includes at least one of the following situations: The format of the photo is qualified, and the photo that has passed the inspection is output; The format of the photo is unqualified, it is judged as a known exception, and the interface is returned for reconnection; The photo format is unqualified and is judged as an unknown exception, so the resources are released.

6. The video stream playing method according to claim 1, characterized in that: Decode qualified photos and convert native photos into resources that can be recognized and played by browser media elements.

7. The video stream playing method according to claim 1, characterized in that: In the process of drawing graphics for photos, the drawing frequency is determined according to the frame rate of the video stream.

8. A video stream playback device, characterized in that: Used to implement the video stream playback method according to any one of claims 1 to 7, the device comprises: An interface setting module, used to set the interface for requesting photos; The request sending module is used to send a request through an address to obtain photos; A photo detection module is used to detect the format of the photo; A photo decoding module, used for decoding the qualified photos; The video stream generation module is used to process photos and generate video streams.

9. A video streaming playback device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the method for video stream playback as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store computer programs; wherein, when the computer program is executed by a processor, the steps of the method for video stream playback as described in any one of claims 1 to 7 are implemented.

Images