Method and System for Playing H265 Video in Browser

Abstract

An embodiment of the present invention provides a method and system for playing H265 videos in a browser, belonging to the technical field of video playback. The method includes: obtaining an initial image frame and a to-be-updated image frame after decoding the H265 video; calculating a difference value between the to-be-updated image frame and the initial image frame; sending the initial image frame and the difference value to the browser front end; and drawing a video picture at the browser front end according to the difference value and the initial image frame. Through the above technical solution, the present invention uses the initial image frame and the difference value to draw a video picture, and the video picture can be directly played in the browser, which can effectively reduce the amount of data transmitted; therefore, the amount of data transmitted to the browser is relatively small, and the browser can efficiently play surveillance videos, which has good practicability in the surveillance industry.

Description

Technical Field

[0001] The present invention relates to the technical field of video playback, and particularly to a method for playing H265 videos in a browser, a system for playing H265 videos in a browser, an electronic device, and a computer-readable storage medium. Background Art

[0002] In the monitoring industry, a large number of video files need to be stored. In the prior art, the H265 standard with a higher compression ratio is often selected as the main video coding format, that is, the HEVC (High Efficiency Video Coding) compression standard. The HEVC compression algorithm effectively improves the coding compression ratio by allocating macros according to the characteristics of the video. At the same time, the intra-frame coding of HEVC increases the number of prediction directions to 33, which increases the computational complexity required by the decoder.

[0003] Due to cost reasons, browser manufacturers have not built an H265 decoder into the browser, so that HEVC videos cannot be directly played in the browser. In the prior art, WebAssembly is used to process HEVC videos, and a general H265 decoder can be compiled for browser use. However, since the browser has a multi-process architecture and IPC is also used for communication between processes, network reception is responsible for the network process and then passed to the Render process for rendering. WebAssembly does not support multi-threading well and cannot utilize the multi-core performance of the CPU. WebAssembly does not support CPU multimedia acceleration instructions such as MMX and SSE. Therefore, the image data of HEVC videos needs to be transferred multiple times, and the huge amount of data generated cannot be efficiently transmitted locally. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a method and system for playing H265 videos in a browser to solve the problem that the browser in the prior art cannot efficiently play H265 videos.

[0005] To achieve the above purpose, the embodiments of the present invention provide a method for playing H265 videos in a browser, the method comprising:

[0006] Obtaining an initial image frame and an image frame to be updated after decoding the H265 video;

[0007] Calculating a difference value between the image frame to be updated and the initial image frame;

[0008] Sending the initial image frame and the difference value to the browser front end;

[0009] Drawing a video picture at the browser front end according to the difference value and the initial image frame.

[0010] Preferably, the method further comprises: updating the initial image frame, including:

[0011] Set the update period of the initial image frame;

[0012] Take the first image frame obtained after the update period as the update value of the initial image frame.

[0013] Preferably, the method further includes H265 video decoding, including:

[0014] Obtain the streaming media data of the surveillance camera;

[0015] Parse the streaming media data according to the encapsulation rules of the streaming media to obtain a video stream;

[0016] Read the video stream into a packet to obtain a video frame;

[0017] Call the graphics card decoder to decode the video frame to obtain an image frame.

[0018] Preferably, there are n image frames in each update period, where the first image frame in each update period is the initial image frame, and the remaining image frames are image frames to be updated.

[0019] Preferably, calculating the difference value between the initial image frame and the image frame to be updated includes:

[0020] Mark the initial image frame and the image frame to be updated as pic1 and pic2 respectively;

[0021] Calculate the absolute value of the difference between pic2 and pic1 to obtain the image difference picdiff;

[0022] Convert the image difference picdiff to obtain a grayscale image;

[0023] Perform threshold processing on the grayscale image to obtain a binary image;

[0024] Calculate the difference contour of the binary image;

[0025] Calculate the difference rectangle according to the area of the difference contour;

[0026] Crop the binary image according to the difference rectangle to obtain a difference image and the coordinates corresponding to the difference image, and the difference image and coordinates are used to characterize the difference value.

[0027] Preferably, the method further includes preprocessing the binary image, and the preprocessing includes median filtering and / or morphological filtering of noise.

[0028] Preferably, the method further includes drawing a video picture, including:

[0029] Determine the initial image frame received by the browser front end;

[0030] Based on the SDL API drawing method and with the initial image frame as a reference, image drawing is performed at the browser front end to obtain an initial screen;

[0031] Based on the SDL sprite drawing and with the received difference value as a reference, the initial screen is partially updated to obtain a new screen.

[0032] An embodiment of the present invention also provides a system for playing H265 video in a browser. The system is used to implement the above-mentioned method for playing H265 video in a browser, and the system includes:

[0033] An acquisition module, configured to acquire an initial image frame and an image frame to be updated after decoding the H265 video;

[0034] A calculation module, configured to calculate the difference value between the initial image frame and the image frame to be updated;

[0035] A sending module, configured to send the initial image frame and the difference value to the browser front end;

[0036] A drawing module, configured to draw a video screen at the browser front end according to the difference value and the initial image frame.

[0037] An embodiment of the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned method for playing H265 video in a browser is implemented.

[0038] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the above-mentioned method for playing H265 video in a browser is implemented.

[0039] Through the above technical solutions, the present invention uses the initial image frame and the difference value to draw a video screen, and the video screen can be directly played in the browser, which can effectively reduce the amount of data transmitted; therefore, the amount of data transmitted to the browser is relatively small, and the browser can efficiently play the surveillance video, which has good practicability in the surveillance industry.

[0040] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent specific implementation part. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The drawings are used to provide a further understanding of the embodiments of the present invention, and constitute a part of the specification. Together with the following specific implementation, they are used to explain the embodiments of the present invention, but do not constitute a limitation to the embodiments of the present invention. In the drawings:

[0042] Figure 1It is a flowchart of a method for playing H265 video in a browser provided by an embodiment of the present invention;

[0043] Figure 2 It is a flowchart of an H265 video decoding operation provided by an alternative embodiment of the present invention;

[0044] Figure 3 It is a flowchart of a method for calculating difference values provided by an alternative embodiment of the present invention;

[0045] Figure 4 It is a flowchart of a method for updating an initial image frame provided by an alternative embodiment of the present invention;

[0046] Figure 5 It is a flowchart of an operation for drawing a video picture provided by an alternative embodiment of the present invention;

[0047] Figure 6 It is a block diagram of a system for playing H265 video in a browser provided by an alternative embodiment of the present invention. Detailed implementation manners

[0048] The following will describe in detail the specific implementation manners of the embodiments of the present invention with reference to the accompanying drawings. It should be understood that the specific implementation manners described herein are only for explaining and illustrating the embodiments of the present invention, and are not used to limit the embodiments of the present invention.

[0049] Figure 1 It is a flowchart of a method for playing H265 video in a browser provided by an embodiment of the present invention. As Figure 1 shown, a method for playing H265 video in a browser, the method includes:

[0050] Step S101: Obtain the initial image frame and the image frame to be updated after decoding the H265 video.

[0051] In this embodiment, the H265 video is a surveillance video of a surveillance camera. As Figure 2 shown, as a further optimization of this embodiment, the method further includes that the H265 video decoding includes:

[0052] Step a1: Obtain the streaming media data of the surveillance camera.

[0053] The streaming media data has the following protocols, such as RTSP, RTMP, HTTP-FLV, HTTP-TS and other protocols. Among them, the RTSP and RTMP protocols can carry different encapsulation container formats, and video and audio tracks are encapsulated in the container, which are called video streams and audio streams.

[0054] Step a2: Parse the streaming media data according to the encapsulation rules of the streaming media to obtain the video stream.

[0055] That is, according to the encapsulation rules of the above RTSP and RTMP protocols, the encapsulation container is parsed to parse out the encapsulated video stream and audio stream. In the subsequent steps, only the video stream is optimized, while the audio stream is not optimized. Therefore, the audio stream can be directly processed using existing methods.

[0056] Step a3: Read the video stream into a packet to obtain video frames. Each packet stores a video frame or an audio frame, and then the corresponding decoder is called to decode the video frame and the audio frame respectively. For example, for a video encoded with HEVC and an audio encoded with audio, the HEVC decoder and the audio decoder will be called accordingly. After decoding, the original image (YUV or RGB) and sound (PCM) data are obtained, and then the image is displayed on the screen according to the synchronized time, and the sound is output to the sound card. Finally, what we see is the video.

[0057] Step a4: Call the graphics card decoder to decode the video frame to obtain an image frame.

[0058] In this embodiment, the obtained image frame is the original image captured by the surveillance camera. Calling the graphics card for decoding operations can make full use of the GPU resources of the client, and the CPU occupancy rate is relatively low.

[0059] Step S102: Calculate the difference value between the image frame to be updated and the initial image frame.

[0060] In this embodiment, since the still time of the surveillance video is relatively long and the picture changes less, the initial image frame can be used as the background picture, and then the difference value between the image frame to be updated and the initial image frame is obtained, and the video picture corresponding to the image frame to be updated is synthesized on the basis of the background picture.

[0061] As a further optimization of this embodiment, as Figure 3 shown, in this embodiment, taking one initial image frame and one image frame to be updated as an example, calculating the difference value between the initial image frame and the image frame to be updated includes:

[0062] Step b1: Mark the initial image frame and the image frame to be updated as pic1 and pic2 respectively.

[0063] Step b2: Calculate the absolute value of the difference between pic2 and pic1 to obtain the image difference picdiff.

[0064] Step b3: Convert the image difference picdiff to obtain a grayscale image.

[0065] Step b4: Perform threshold processing on the grayscale image to obtain a binary image; the threshold processing can filter out some small noise points, and it also includes preprocessing the binary image, where the preprocessing includes median filtering and / or morphological filtering of noise points.

[0066] Step b5: Calculate the difference contour of the binary image, and the difference contour is the difference boundary of the image frame to be updated.

[0067] Step b6: Calculate the difference rectangle according to the area of the difference contour, and each difference rectangle represents that the image frame to be updated has a difference area.

[0068] Step b7: Crop the binary image according to the difference rectangle to obtain the difference image and the coordinates corresponding to the difference image, and the difference image and coordinates are used to represent the difference value.

[0069] Within one update cycle, for other image frames to be updated, repeat steps b1 - b7 to calculate the difference value between each image frame to be updated and the initial image frame.

[0070] As a further optimization of this embodiment, as Figure 4 shown, the method further includes: updating the initial image frame, including:

[0071] Step c1: Set the update cycle of the initial image frame. Each update cycle has n image frames. The update cycle can be set according to time or according to the number of frames; for example, the update cycle is every 50 frames, or an interval of 2 s.

[0072] Step c2: Use the first image frame obtained after reaching the update cycle as the update value of the initial image frame; for example, the first image frame is used as the initial image frame, and the second to the fiftieth frames are used as the image frames to be updated. After reaching the update cycle, that is, the fifty - first frame is used as the new initial image frame, and the fifty - second to the one - hundredth frames are used as the image frames to be updated.

[0073] The setting of the update cycle in this embodiment results in a relatively small difference value between the initial image frame and the image frames to be updated within the new cycle, which can reduce the computational amount of the difference value. At the same time, after the monitoring camera rotates by itself, the background picture of the monitoring video will change greatly. Therefore, the length of the update cycle can be adjusted to reduce the computational amount of the difference value, thereby improving the video playback efficiency.

[0074] Step S103: Send the initial image frame and the difference value to the browser front - end; in this embodiment, the initial image frame and the difference value are transmitted to the program of the browser front - end using the Websocket transmission method.

[0075] Step S104: Draw the video picture on the browser front - end according to the difference value and the initial image frame.

[0076] Specifically, as Figure 5 shown, the method further includes drawing a video picture, including:

[0077] Step d1: Determine the initial image frame received by the browser front end, that is, first determine the initial image frame in each update period.

[0078] Step d2: Based on the SDL API drawing method and with the initial image frame as a reference, perform image drawing on the browser front end to obtain an initial picture, and the initial picture is used as the background picture;

[0079] Step d3: Based on the SDL drawing sprite and with the received difference value as a reference, perform partial update on the initial picture to obtain a new picture.

[0080] Repeat step d3. According to the multiple difference values received in each update period (taking 50 frames as an update period as an example), that is, there are 49 difference values in total, and perform 49 updates on the initial picture at the same time to obtain 49 new pictures. One background picture and 49 new pictures are continuously displayed on the browser to obtain a surveillance video.

[0081] The method of the present invention is applicable to browsers such as chormium, chorme, and firefox that support WebAssembly, and is used for efficiently playing videos encoded with HEVC, and has good practicability in the surveillance industry.

[0082] Figure 6 is a block diagram of a playback system of H265 video in a browser provided by an alternative embodiment of the present invention. As Figure 6 shown, the system is used to implement the above-mentioned method for playing H265 video in a browser. The system includes:

[0083] An acquisition module, configured to acquire the initial image frame and the image frame to be updated after decoding the H265 video;

[0084] A calculation module, configured to calculate the difference value between the initial image frame and the image frame to be updated;

[0085] A sending module, configured to send the initial image frame and the difference value to the browser front end;

[0086] A drawing module, configured to draw a video picture on the browser front end according to the difference value and the initial image frame.

[0087] Through the above technical solution, the present invention uses the initial image frame and the difference value to draw a video picture. The video picture can be directly played in the browser, which can effectively reduce the amount of data transmitted; therefore, the amount of data transmitted to the browser is relatively small, and the browser can efficiently play the surveillance video, and has good practicability in the surveillance industry.

[0088] An embodiment of the present invention further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned method for playing H265 video in a browser is implemented.

[0089] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the above-mentioned method for playing H265 video in a browser is implemented.

[0090] The optional embodiments of the present invention have been described in detail above in conjunction with the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details in the above embodiments. Within the technical concept scope of the embodiments of the present invention, various simple modifications can be made to the technical solutions of the embodiments of the present invention, and these simple modifications all fall within the protection scope of the embodiments of the present invention.

[0091] In addition, it should be noted that, among the various specific technical features described in the above specific embodiments, they can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the embodiments of the present invention do not separately describe various possible combination methods.

[0092] Those skilled in the art can understand that all or part of the steps in implementing the above-mentioned embodiment methods can be completed by instructing relevant hardware through a program. The program is stored in a storage medium, including several instructions for causing a single-chip microcomputer, a chip, or a processor to execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk, or an optical disc that can store program codes.

[0093] In addition, any combination can be made between various different embodiments of the present invention as long as it does not violate the idea of the embodiments of the present invention, and it should also be regarded as the content disclosed by the embodiments of the present invention.

Claims

1. A method for playing H.265 videos in a browser, characterized in that, The method includes: Obtaining the initial image frame and the image frame to be updated after H265 video decoding; the update of the initial image frame includes: setting the update period of the initial image frame; using the first image frame obtained after the update period arrives as the update value of the initial image frame; the length of the update period is related to the rotation of the monitoring camera for obtaining the H265 video; Calculating the difference value between the image frame to be updated and the initial image frame, including: respectively marking the initial image frame and the image frame to be updated as pic1 and pic2; calculating the absolute value of the difference between pic2 and pic1 to obtain the image difference picdiff; converting the image difference picdiff to obtain a grayscale image; performing threshold processing on the grayscale image to obtain a binary image; calculating the difference contour of the binary image; calculating a difference rectangle according to the area of the difference contour; cropping the binary image according to the difference rectangle to obtain a difference image and the coordinates corresponding to the difference image, and the image difference and coordinates are used to represent the difference value; Sending the initial image frame and the difference value to the browser front-end; Drawing a video picture according to the difference value and the initial image frame on the browser front-end, including: determining the initial image frame received by the browser front-end; performing image drawing on the browser front-end based on the SDL API drawing method with the initial image frame as a reference to obtain an initial picture; performing local update on the initial picture based on the SDL drawing sprite with the received difference value as a reference to obtain a new picture; repeating the step of performing local update on the initial picture based on the SDL drawing sprite with the received difference value as a reference to obtain a new picture to obtain a monitoring video.

2. The method according to claim 1, characterized in that, The method further includes that H265 video decoding includes: Obtaining the streaming media data of the monitoring camera; Parsing the streaming media data according to the encapsulation rules of the streaming media to obtain a video stream; Reading the video stream into a packet to obtain a video frame; Invoking a graphics card decoder to perform decoding processing on the video frame to obtain an image frame.

3. The method according to claim 2, characterized in that, There are n image frames in each update period, where the first image frame in each update period is the initial image frame, and the remaining image frames are the image frames to be updated.

4. The method according to claim 1, characterized in that, The method further includes preprocessing the binary image, and the preprocessing includes median filtering and / or morphological filtering of noise.

5. A system for playing H.265 videos in a browser, the system being used to implement the method for playing H.265 videos in a browser according to any one of claims 1-4, characterized in that, The system includes: An acquisition module, configured to acquire the initial image frame and the image frame to be updated after H265 video decoding, and the update of the initial image frame includes: setting the update period of the initial image frame; using the first image frame obtained after the update period arrives as the update value of the initial image frame, and the length of the update period is related to the rotation of the monitoring camera for obtaining the H265 video; A calculation module, configured to calculate the difference value between an initial image frame and an image frame to be updated, including: respectively labeling the initial image frame and the image frame to be updated as pic1 and pic2; calculating the absolute value of the difference between pic2 and pic1 to obtain an image difference picdiff; converting the image difference picdiff to obtain a grayscale image; performing threshold processing on the grayscale image to obtain a binary image; calculating the difference contour of the binary image; calculating a difference rectangle according to the area of the difference contour; cropping the binary image according to the difference rectangle to obtain a difference image and the coordinates corresponding to the difference image, where the difference image and the coordinates are used to represent the difference value; A sending module, configured to send the initial image frame and the difference value to the browser front end; A drawing module, configured to draw a video picture on the browser front end according to the difference value and the initial image frame, including: determining the initial image frame received by the browser front end; performing image drawing on the browser front end based on the SDL API drawing method with the initial image frame as a reference to obtain an initial picture; performing local update on the initial picture based on the SDL-drawn sprite with the received difference value as a reference to obtain a new picture.

6. An electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the H265 video playing method in a browser according to any one of claims 1-4.

7. A computer-readable storage medium, on which a computer program is stored, characterized in that, When the program is executed by the processor, it implements the H265 video playing method in a browser according to any one of claims 1-4.

Images