Video playing method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a video playing method and device, computer equipment, a readable storage medium and a program product, and belongs to the technical field of artificial intelligence. The method comprises the following steps: capturing a to-be-hyper-divided video frame image from video stream data of a target video, and adding the to-be-hyper-divided video frame image to a frame processing queue of a browser; the target video is a video to be played; creating a video super-division multi-thread in the browser, and performing image super-division processing on the video frame image in the frame processing queue through at least one sub-thread in the video super-division multi-thread to obtain a super-divided video frame image; adding the super-divided video frame images into a rendering queue in sequence, and rendering the video frame images in the rendering queue through a main thread of the browser; playing the rendered video rendered by the main thread in the browser; the resolution of the rendered video is higher than that of the target video. By adopting the method, the real-time video interaction effect of the browser can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a video playback method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] Image super-resolution technology (SR) is an important technology in computer vision to improve image quality and resolution. It can reconstruct a high-resolution image from a low-resolution image.

[0003] In the relevant technologies of browser real-time video interaction, affected by the performance of the client device, network bandwidth, encoding and decoding performance, the resolution of the video displayed during the browser real-time video interaction is often not high, resulting in poor interactive effect of the browser real-time video interaction.

[0004] Therefore, there is a problem in the related technology that the real-time video interaction effect of the browser is not high. Summary of the invention

[0005] Based on this, it is necessary to provide a video playback method, device, computer equipment, computer-readable storage medium and computer program product that can enhance the real-time video interaction effect of the browser in response to the above technical problems.

[0006] In a first aspect, the present application provides a video playback method, comprising:

[0007] Capturing the video frame image to be super-resolved from the video stream data of the target video, and adding the video frame image to be super-resolved to the frame processing queue of the browser; the target video is the video to be played;

[0008] Creating a video super-resolution multithread in the browser, and performing image super-resolution processing on the video frame images in the frame processing queue through at least one sub-thread in the video super-resolution multithread to obtain the super-resolution video frame images;

[0009] Adding the super-resolved video frame images to a rendering queue in order, and rendering the video frame images in the rendering queue through the main thread of the browser;

[0010] The rendered video obtained by rendering by the main thread is played in the browser; the resolution of the rendered video is higher than the resolution of the target video.

[0011] In one embodiment, capturing the video frame image to be super-resolved from the video stream data of the target video includes:

[0012] In response to the video playback instruction received by the browser, the video stream data of the target video is pulled through the browser;

[0013] A video object is created in the browser, and the video stream data is loaded through the video object, and the video frame image to be super-resolved is screened from the target video.

[0014] In one embodiment, the step of loading the video stream data through the video object and selecting the video frame image to be super-resolved from the target video includes:

[0015] In the process of loading the video stream data through the video object, obtaining a current frame video frame image of the target video;

[0016] In the case where the current video frame image is the first video frame image, using the first video frame image as the video frame image to be super-resolved;

[0017] In the case that the current video frame image is not the first video frame image, determining a previous video frame image of the current video frame image;

[0018] When the difference between the previous video frame image and the current video frame image is greater than a preset difference, the non-first video frame image is used as the video frame image to be super-resolved.

[0019] In one embodiment, the adding the super-resolved video frame images to a rendering queue in order comprises:

[0020] For each of the super-resolved video frame images, obtaining a video frame serial number of each of the super-resolved video frame images in the target video; the video frame serial number is used to represent the position information of the video frame image to be super-resolved corresponding to the super-resolved video frame image in the target video;

[0021] According to the video frame sequence numbers corresponding to the super-resolved video frame images, the super-resolved video frame images are added to the rendering queue in sequence.

[0022] In one embodiment, rendering the video frame images in the rendering queue through the main thread of the browser includes:

[0023] Determine a target video frame image in the rendering queue; the video frame sequence number corresponding to the target video frame image is less than the video frame sequence number corresponding to other video frame images; the other video frame images are video frame images other than the target video frame image in the rendering queue;

[0024] In response to reaching the rendering time of the target video frame image, removing the target video frame image from the rendering queue, and executing a rendering task for the target video frame image;

[0025] Return to the step of determining the target video frame image in the rendering queue until there is no video frame image in the rendering queue.

[0026] In one embodiment, the step of creating a video super-resolution multithreading in the browser includes:

[0027] Obtaining device configuration information of a terminal device running the browser;

[0028] Determine the number of multithreads that matches the device configuration information;

[0029] Video super-resolution multi-threading is created in the browser according to the number of multi-threads.

[0030] In a second aspect, the present application further provides a video playback device, comprising:

[0031] An acquisition module is used to capture the video frame image to be super-resolved from the video stream data of the target video, and add the video frame image to be super-resolved to the frame processing queue of the browser; the target video is the video to be played;

[0032] A super-resolution module is used to create a video super-resolution multi-thread in the browser, and perform image super-resolution processing on the video frame images in the frame processing queue through at least one sub-thread in the video super-resolution multi-thread to obtain the super-resolution video frame images;

[0033] A rendering module, used to add the super-resolved video frame images to a rendering queue in order, and render the video frame images in the rendering queue through the main thread of the browser;

[0034] A playback module is used to play the rendered video obtained by the main thread rendering in the browser; the resolution of the rendered video is higher than the resolution of the target video.

[0035] In a fourth aspect, the present application also provides a computer device, wherein the computer device includes the above-mentioned energy management system.

[0036] In a fifth aspect, the present application also provides a computer-readable storage medium having a computer program stored thereon, and the computer program implements the steps of the above method when executed by a processor.

[0037] In a sixth aspect, the present application also provides a computer program product, including a computer program, which implements the steps of the above method when executed by a processor.

[0038] The above-mentioned video playback method, device, computer equipment, computer-readable storage medium and computer program product capture the video frame images to be super-resolved from the video stream data of the target video, and add the video frame images to be super-resolved to the frame processing queue of the browser; then, by creating a video super-resolution multi-thread in the browser, the video frame images in the frame processing queue are super-resolved through at least one sub-thread in the video super-resolution multi-thread to obtain the super-resolved video frame images; then, by adding the super-resolved video frame images to the rendering queue in sequence, the video frame images in the rendering queue are rendered through the main thread of the browser; and the rendered video obtained by rendering through the main thread is played in the browser; in this way, by using multiple threads in the browser to super-resolve the video frame images of the video stream data, the resolution of the rendered video obtained based on the browser rendering is higher than the resolution of the original video, so that the browser of the device terminal can still display the video screen with the resolution meeting the conditions under the transmission background of low bandwidth and low bit rate, effectively improving the video display effect and video interaction effect in the process of real-time video interaction through the browser. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0040] FIG1 is a diagram of an application environment of a video playback method in one embodiment;

[0041] FIG2 is a schematic diagram of a flow chart of a video playback method in one embodiment;

[0042] FIG3 is a schematic diagram of a process flow of super-resolution processing in one embodiment;

[0043] FIG4 is an architecture diagram of a video playback system in one embodiment;

[0044] FIG5 is a schematic diagram of a flow chart of a video playback method in another embodiment;

[0045] FIG6 is a structural block diagram of a video playback device in one embodiment;

[0046] FIG. 7 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0048] The video playback method provided in the embodiment of the present application can be applied in the application environment shown in FIG1. ​​The first terminal device 102 and the second terminal device 106 communicate with the server 104 through a network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers.

[0049] In actual applications, during the process of real-time browser video interaction between the first terminal device 102 and the second terminal device 106, the second terminal device 106 can perform a streaming operation through the browser, that is, send the video stream data of the target video collected by the second terminal device 106 to the server 104. The first terminal device 102 can perform a stream pulling operation through a browser, that is, the first terminal device 102 pulls the video stream data of the target video through the server 104; then, the first terminal device 102 captures the video frame image to be super-resolution from the video stream data of the target video, and adds the video frame image to be super-resolution to the frame processing queue of the browser; the first terminal device 102 creates a video super-resolution multi-thread in the browser, and the first terminal device 102 performs image super-resolution processing on the video frame image in the frame processing queue through at least one sub-thread in the video super-resolution multi-thread to obtain the super-resolution video frame image; the first terminal device 102 adds the super-resolution video frame image to the rendering queue in sequence, and renders the video frame image in the rendering queue through the main thread of the browser; the first terminal device 102 plays the rendered video obtained by rendering through the main thread in the browser; the resolution of the rendered video is higher than the resolution of the target video. In practical applications, the first terminal device 102 and the second terminal device 106 may be, but are not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices may be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server 104 may be an independent physical server, or a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.

[0050] It is understood that the terms "first", "second", etc. used in this application can be used in this article to describe various technical features, but these technical features are not limited by these terms. These terms are only used to distinguish the first technical feature from another technical feature.

[0051] In an exemplary embodiment, as shown in FIG2 , a video playback method is provided, which is described by taking the method applied to the first terminal device 102 in FIG1 as an example, and includes the following steps S202 to S208 . Among them:

[0052] Step S202: Capture the video frame image to be super-resolved from the video stream data of the target video, and add the video frame image to be super-resolved to the frame processing queue of the browser.

[0053] The target video is the video to be played on the first terminal device.

[0054] The video frame image to be super-resolved may refer to a video frame image to be subjected to image super-resolution processing.

[0055] The frame processing queue may refer to a queue for caching video frame images to be subjected to image super-resolution processing.

[0056] In a specific implementation, during the process of real-time video interaction between the first terminal device and the second terminal device through the browser, the second terminal device can perform a push stream operation through the browser, that is, send the video stream data of the target video collected by the second terminal device to the server. The first terminal device can perform a pull stream operation through the browser, that is, the first terminal device pulls the video stream data of the target video through the server;

[0057] Please refer to FIG. 3 . The first terminal device captures the video frame image to be super-resolved from the video stream data of the target video, and adds the video frame image to be super-resolved to the frame processing queue of the browser.

[0058] Specifically, the first terminal device can filter all video frame images of the target video according to preset filtering conditions, and select video frame images that meet the filtering conditions as video frame images to be super-resolved; the first terminal device creates a frame processing queue in the browser, and stores the video frame images to be super-resolved in the frame processing queue.

[0059] Step S204, creating a video super-resolution multi-thread in the browser, and performing image super-resolution processing on the video frame images in the frame processing queue through at least one sub-thread in the video super-resolution multi-thread to obtain super-resolution video frame images.

[0060] The video super-resolution multithreading may refer to multiple threads for super-resolution processing of a target video. The video super-resolution multithreading may include at least one child thread, and the child thread may be implemented by a web worker thread. The webworker thread is a JavaScript feature that allows background threads to be created in a browser so that scripts can be run without blocking the main thread.

[0061] In a specific implementation, the first terminal device can create at least one web worker thread in the browser, and the main thread of the first terminal device can pass the video frame image in the frame processing queue to an available web worker thread, and the web worker thread receives the incoming video frame image and executes the super-resolution algorithm on the video frame image; wherein the web worker thread can input the incoming video frame image into a pre-trained image super-resolution model; and perform image super-resolution processing on the video frame image through the image super-resolution model to obtain the super-resolution video frame image. Among them, the super-resolution algorithm has a variety of image super-resolution models to choose from. Taking the image super-resolution model as an example of the PP-MSVSR model, if you want to super-resolution the 720p video frame into a 4k effect, you need to set the super-resolution scale of the model to 3. In actual applications, the image super-resolution model is pre-loaded in the browser of the first terminal device.

[0062] Step S206, adding the super-resolved video frame images to the rendering queue in order, and rendering the video frame images in the rendering queue through the main thread of the browser.

[0063] In the specific implementation, please refer to Figure 3 again. After the web worker thread obtains the super-resolution video frame image, each webworker thread can send the super-resolution video frame image back to the main thread of the browser; after the main thread of the browser receives the super-resolution video frame image, it adds the super-resolution video frame image to the rendering queue in order. Then, the sorted video frame images in the rendering queue are rendered by the main thread of the browser. In practical applications, batch processing and data compression can be used to optimize the communication process between the web worker thread and the main thread, reduce the amount of data communication between each sub-thread and the main thread, and reduce communication overhead.

[0064] Step S208, playing the rendered video obtained by the main thread rendering in the browser.

[0065] The resolution of the rendered video is higher than the resolution of the target video. In practical applications, the resolution of the rendered video can be 2K or 4K. The resolution of the target video can be 1080p or 720p.

[0066] In a specific implementation, the first terminal device plays the rendered video obtained by the main thread rendering in the browser, so that the resolution of the video played by the browser is higher than the target video captured by the second terminal device.

[0067] It should be noted that, when the first terminal device and the second terminal device stop performing real-time browser video interaction, the first terminal device can terminate each Web Worker thread in the video super-resolution multi-thread and release related resources.

[0068] In the above-mentioned video playback method, the video frame images to be super-resolved are captured from the video stream data of the target video, and the video frame images to be super-resolved are added to the frame processing queue of the browser; then, by creating a video super-resolution multi-thread in the browser, the video frame images in the frame processing queue are super-resolved through at least one sub-thread in the video super-resolution multi-thread to obtain the super-resolved video frame images; then, by sequentially adding the super-resolved video frame images to the rendering queue, the video frame images in the rendering queue are rendered through the main thread of the browser; and the rendered video obtained by rendering through the main thread is played in the browser; in this way, by using multiple threads in the browser to perform super-resolution processing on the video frame images of the video stream data, the resolution of the rendered video obtained based on the browser rendering is higher than the resolution of the original video, so that the browser of the device terminal can still display the video screen with the resolution meeting the conditions under the transmission background of low bandwidth and low bit rate, effectively improving the video display effect and video interaction effect in the process of real-time video interaction through the browser.

[0069] In an exemplary embodiment, the video frame images to be super-resolved are captured from the video stream data of the target video, including: in response to a video playback instruction received by the browser, the video stream data of the target video is pulled through the browser; a video object is created in the browser, and the video stream data is loaded through the video object to filter the video frame images to be super-resolved from the target video.

[0070] The video play instruction may be an instruction to instruct the first terminal device to play the target video. Specifically, when the first terminal device successfully establishes a connection with the second terminal device and starts real-time browser video interaction, it can be determined that the browser of the first terminal device receives the video play instruction.

[0071] In a specific implementation, the process in which the first terminal device captures the video frame image to be super-resolved from the video stream data of the target video specifically includes: the first terminal device responds to the video playback instruction received by the browser, and sends a stream pull request for the target video to the server through the browser, and the stream pull request is used to instruct the server to send the video stream data of the target video to the first terminal device.

[0072] After the first terminal device receives the video stream data of the target video, the first terminal device can create a browser video object (i.e., video object), load the video stream data through the browser video object, and select hardware or software to decode the video stream data into all video frame images.

[0073] Then, the first terminal device screens all the video frame images of the target video according to the preset screening conditions, and screens out the video frame images that meet the screening conditions as the video frame images to be super-resolved. It should be noted that the process of the first terminal device screening the video frame images to be super-resolved from the target video will be further described below, which will not be described here.

[0074] The technical solution of this embodiment responds to the video playback instruction received by the browser and pulls the video stream data of the target video through the browser; creates a video object in the browser, and loads the video stream data through the video object, and filters the video frame images to be super-resolved from the target video, thereby efficiently capturing the video frame images to be super-resolved from the video stream data of the target video.

[0075] In an exemplary embodiment, video stream data is loaded through a video object, and video frame images to be super-resolved are screened from a target video, including: in the process of loading the video stream data through the video object, a current video frame image of the target video is obtained; when the current video frame image is the first video frame image, the first video frame image is used as the video frame image to be super-resolved;

[0076] When the current video frame image is not the first video frame image, the previous video frame image of the current video frame image is determined; when the difference between the previous video frame image and the current video frame image is greater than the preset difference, the non-first video frame image is used as the video frame image to be super-resolved.

[0077] In a specific implementation, when the first terminal device loads video stream data through a video object and screens video frame images to be super-resolved from the target video, the first terminal device can obtain the current video frame image of the target video and determine whether the current video frame image is the first video frame image or a non-first video frame image; when the current video frame image is the first video frame image, the first terminal device uses the first video frame image as the video frame image to be super-resolved.

[0078] In the case of a non-first frame video frame image, the first terminal device can obtain the previous frame video frame image of the current frame video frame image, and obtain the difference between the previous frame video frame image and the current frame video frame image; determine whether the content of the previous frame video frame image is the same as the current frame video frame image; when the content of the previous frame video frame image is the same as the current frame video frame image, the first terminal device does not use the current frame video frame image as the video frame image to be super-resolved; and uses the next frame video frame image of the current frame video frame image as the current frame video frame image, and returns to execute the step of determining the previous frame video frame image of the current frame video frame image, until the current frame video frame image is the last frame video frame image.

[0079] The technical solution of this embodiment is to obtain the current video frame image of the target video during the process of loading video stream data through the video object. When the content of the previous video frame image is consistent with the current video frame image, the current video frame image will not be added to the frame processing queue as the video frame image to be super-resolved, thereby effectively avoiding the video super-resolution multi-threaded repeated super-resolution operations on the video frame images of the same content, and effectively reducing the data processing amount of the first device terminal.

[0080] In an exemplary embodiment, the super-resolved video frame images are sequentially added to the rendering queue, including: for each super-resolved video frame image, obtaining the video frame sequence number of each super-resolved video frame image in the target video; and adding the super-resolved video frame images to the rendering queue in sequence according to the video frame sequence numbers corresponding to each super-resolved video frame image.

[0081] The video frame sequence number is used to represent the position information of the video frame image to be super-resolved corresponding to the super-resolved video frame image in the target video. The closer the position of the video frame image to be super-resolved corresponding to the super-resolved video frame image in the target video is, the smaller the video frame sequence number corresponding to the super-resolved video frame image is.

[0082] In a specific implementation, when the first terminal device sequentially adds the super-resolved video frame images to the rendering queue, for each super-resolved video frame image, the first terminal device can sequentially add the super-resolved video frame images to the rendering queue according to the video frame sequence number corresponding to each super-resolved video frame image. In practical applications, the first terminal device can use a minimum heap data structure to optimize the sorting and search operations of the rendering queue.

[0083] The technical solution of this embodiment obtains the video frame serial number of each super-resolved video frame image in the target video, and adds the super-resolved video frame images to the rendering queue in sequence according to the video frame serial numbers corresponding to each super-resolved video frame image. This allows the main thread of the browser to effectively render a rendered video consistent with the target video content based on the video frame images in the rendering queue.

[0084] In an exemplary embodiment, rendering video frame images in a rendering queue is performed through a main thread of a browser, including: determining a target video frame image in the rendering queue; in response to reaching a rendering time of the target video frame image, removing the target video frame image from the rendering queue and executing a rendering task for the target video frame image; returning to the step of determining the target video frame image in the rendering queue until no video frame image exists in the rendering queue.

[0085] The video frame sequence number corresponding to the target video frame image is smaller than the video frame sequence numbers corresponding to other video frame images, and the other video frame images are video frame images other than the target video frame image in the rendering queue. In other words, the target video frame image may refer to the earliest video frame image.

[0086] In a specific implementation, when the first terminal device renders the video frame image in the rendering queue through the main thread of the browser, the first terminal device can have a rendering loop in the rendering queue. In each rendering loop, the first terminal device can check the target video frame image in the rendering queue; and determine whether the current time has reached the rendering time of the target video frame image; wherein the rendering time can be calculated based on the frame rate of the target video and the current timestamp. When the current time has reached the rendering time of the target video frame image, the first terminal device can remove the target video frame image from the rendering queue and execute the rendering task for the target video frame image; for example, the first terminal device renders the target video frame image to a graphic presentation element (e.g., a graphic presentation element named <canvas>elements, allowing developers to draw graphics, animations, and other complex graphics effects through JavaScript); specifically, the first terminal device can use webGPU acceleration technology to leverage the capabilities of GPU hardware to improve the rendering processing speed, and the rendering task will be appropriately decomposed into multiple small tasks or use WebWorker's postMessage and onMessage for asynchronous rendering to ensure that the rendering process does not block the main process.

[0087] Then, the first terminal device returns to the step of determining the target video frame image in the rendering queue and starts the next rendering cycle until there is no video frame image in the rendering queue.

[0088] The technical solution of this embodiment is to determine the target video frame image in the rendering queue; in response to reaching the rendering time of the target video frame image, remove the target video frame image from the rendering queue, execute the rendering task of the target video frame image, and return to the step of determining the target video frame image in the rendering queue until there is no video frame image in the rendering queue, thereby realizing real-time rendering of the super-resolved video frame image in a rendering loop, thereby reducing the delay of the rendered video as much as possible.

[0089] In an exemplary embodiment, creating a video super-resolution multithreading in a browser includes:

[0090] Obtain device configuration information of a terminal device running a browser; determine the number of multithreads that matches the device configuration information; and create video super-resolution multithreading in the browser according to the number of multithreads.

[0091] In a specific implementation, when the first terminal device creates a video super-resolution multi-thread in a browser, the first terminal device can obtain its own device configuration information; determine the number of multi-threads that match it according to the device configuration information; in actual applications, the higher the device performance and computing resource idle rate of the first terminal device, the greater the number of multi-threads. Then, the first terminal device can create a video super-resolution multi-thread in the browser according to the number of multi-threads. For example, if the number of multi-threads is 4, the video super-resolution multi-thread created by the first terminal device in the browser includes four sub-threads.

[0092] According to the technical solution of this embodiment, the first terminal device obtains the device configuration information of the terminal device running the browser, determines the number of multi-threads that match the device configuration information, and creates video super-resolution multi-threads in the browser according to the number of multi-threads, thereby dynamically setting the number of threads of the video super-resolution multi-threads in combination with the device performance of the first terminal device, so as to achieve a good video super-resolution effect and maintain good operating performance of the first terminal device.

[0093] To facilitate understanding by those skilled in the art, FIG4 also exemplifies an architecture diagram of a video playback system; wherein, the video playback system includes a service layer, a processing layer and a rendering layer; wherein, the service layer includes a streaming media server, and the streaming media server is used to implement WebRTC (a technology that supports direct audio and video communication between web browsers) push-pull streams and live push-pull streams; the processing layer includes an end-to-end machine learning platform and a web worker thread; the end-to-end machine learning platform is used to load an image super-resolution algorithm; the web worker thread is used to implement multi-threaded processing of super-resolution tasks; the rendering layer uses webGPU acceleration technology to utilize the capabilities of GPU hardware to improve the rendering processing speed, and renders the target video frame image to the graphics presentation element canvas to display the rendered video.

[0094] In another embodiment, as shown in FIG. 5 , a video playback method is provided, which is described by taking the method applied to the terminal in FIG. 1 as an example, and includes the following steps:

[0095] Step S502: Pull the video stream data of the target video through the browser.

[0096] Step S504, creating a video object in the browser, loading video stream data through the video object, filtering the video frame images to be super-resolved from the target video, and adding the video frame images to be super-resolved to the frame processing queue of the browser.

[0097] Step S506, creating a video super-resolution multi-thread in the browser, and performing image super-resolution processing on the video frame images in the frame processing queue through at least one sub-thread in the video super-resolution multi-thread to obtain super-resolution video frame images.

[0098] Step S508, adding the super-resolved video frame images to the rendering queue in order, and rendering the video frame images in the rendering queue through the main thread of the browser.

[0099] Step S510, playing the rendered video obtained by the main thread rendering in the browser.

[0100] It should be noted that the specific limitations of the above steps can refer to the specific limitations of a video playback method above.

[0101] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0102] Based on the same inventive concept, the embodiment of the present application also provides a video playback device for implementing the video playback method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations of one or more video playback device embodiments provided below can refer to the limitations of the video playback method above, and will not be repeated here.

[0103] In an exemplary embodiment, as shown in FIG6 , a video playback device is provided, including:

[0104] The acquisition module 610 is used to capture the video frame image to be super-resolved from the video stream data of the target video, and add the video frame image to be super-resolved to the frame processing queue of the browser; the target video is the video to be played;

[0105] The super-resolution module 620 is used to create a video super-resolution multi-thread in the browser, and perform image super-resolution processing on the video frame images in the frame processing queue through at least one sub-thread in the video super-resolution multi-thread to obtain the super-resolution video frame images;

[0106] A rendering module 630 is used to add the super-resolved video frame images to a rendering queue in order, and render the video frame images in the rendering queue through the main thread of the browser;

[0107] The playing module 640 is used to play the rendered video obtained by the main thread rendering in the browser; the resolution of the rendered video is higher than the resolution of the target video.

[0108] In one embodiment, the acquisition module 610 is specifically used to respond to the video playback instruction received by the browser, pull the video stream data of the target video through the browser; create a video object in the browser, and load the video stream data through the video object, and filter the video frame image to be super-resolved from the target video.

[0109] In one embodiment, the acquisition module 610 is specifically used to acquire the current frame video frame image of the target video during the process of loading the video stream data through the video object; when the current frame video frame image is the first frame video frame image, the first frame video frame image is used as the video frame image to be super-resolved; when the current frame video frame image is not the first frame video frame image, the previous frame video frame image of the current frame video frame image is determined; when the difference between the previous frame video frame image and the current frame video frame image is greater than a preset difference, the non-first frame video frame image is used as the video frame image to be super-resolved.

[0110] In one embodiment, the rendering module 630 is specifically used to obtain, for each of the super-resolved video frame images, a video frame serial number of each of the super-resolved video frame images in the target video; the video frame serial number is used to represent the position information of the video frame image to be super-resolved in the target video corresponding to the super-resolved video frame image; and the super-resolved video frame images are added to the rendering queue in sequence according to the video frame serial numbers corresponding to each of the super-resolved video frame images.

[0111] In one of the embodiments, the rendering module 630 is specifically used to determine a target video frame image in the rendering queue; the video frame serial number corresponding to the target video frame image is less than the video frame serial numbers corresponding to other video frame images; the other video frame images are video frame images other than the target video frame image in the rendering queue; in response to reaching the rendering time of the target video frame image, the target video frame image is removed from the rendering queue, and a rendering task for the target video frame image is executed; and the step of determining the target video frame image in the rendering queue is returned until there is no video frame image in the rendering queue.

[0112] In one embodiment, the super-resolution module 620 is specifically used to obtain device configuration information of a terminal device running the browser; determine the number of multi-threads matching the device configuration information; and create video super-resolution multi-threads in the browser according to the number of multi-threads.

[0113] Each module in the above video playback device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0114] In an exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be shown in FIG7. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface, the display unit, and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a video playback method is implemented. The display unit of the computer device is used to form a visually visible picture, which may be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0115] Those skilled in the art will understand that the structure shown in FIG. 7 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0116] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above-mentioned video playback method. The steps of the video playback method here may be the steps of the video playback method in each of the above-mentioned embodiments.

[0117] In one embodiment, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the processor executes the steps of the above-mentioned video playback method. The steps of the video playback method here can be the steps of the video playback method in each of the above-mentioned embodiments.

[0118] In one embodiment, a computer program product is provided, including a computer program, which, when executed by a processor, causes the processor to execute the steps of the above-mentioned video playback method. The steps of the video playback method here may be the steps of the video playback method in each of the above-mentioned embodiments.

[0119] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0120] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0121] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0122] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.< / canvas>

Claims

1. A video playback method, characterized in that: The method comprises: Capturing the video frame image to be super-resolved from the video stream data of the target video, and adding the video frame image to be super-resolved to the frame processing queue of the browser; Creating a video super-resolution multithread in the browser, and performing image super-resolution processing on the video frame images in the frame processing queue through at least one sub-thread in the video super-resolution multithread to obtain the super-resolution video frame images; Adding the super-resolved video frame images to a rendering queue in order, and rendering the video frame images in the rendering queue through the main thread of the browser; The rendered video obtained by rendering through the main thread is played in the browser.

2. The method according to claim 1, characterized in that The step of capturing the video frame image to be super-resolved from the video stream data of the target video comprises: In response to the video playback instruction received by the browser, the video stream data of the target video is pulled through the browser; A video object is created in the browser, and the video stream data is loaded through the video object, and the video frame image to be super-resolved is screened from the target video.

3. The method according to claim 2, characterized in that The step of loading the video stream data through the video object and selecting the video frame image to be super-resolved from the target video includes: In the process of loading the video stream data through the video object, obtaining a current frame video frame image of the target video; In the case where the current video frame image is the first video frame image, using the first video frame image as the video frame image to be super-resolved; In the case that the current video frame image is not the first video frame image, determining a previous video frame image of the current video frame image; When the difference between the previous video frame image and the current video frame image is greater than a preset difference, the non-first video frame image is used as the video frame image to be super-resolved.

4. The method according to claim 1, characterized in that: The step of adding the super-resolved video frame images to a rendering queue in order comprises: For each of the super-resolved video frame images, obtaining a video frame serial number of each of the super-resolved video frame images in the target video; the video frame serial number is used to represent the position information of the video frame image to be super-resolved corresponding to the super-resolved video frame image in the target video; According to the video frame sequence numbers corresponding to the super-resolved video frame images, the super-resolved video frame images are added to the rendering queue in sequence.

5. The method according to claim 4, characterized in that The rendering of the video frame images in the rendering queue through the main thread of the browser includes: Determine a target video frame image in the rendering queue; the video frame sequence number corresponding to the target video frame image is less than the video frame sequence number corresponding to other video frame images; the other video frame images are video frame images other than the target video frame image in the rendering queue; In response to reaching the rendering time of the target video frame image, removing the target video frame image from the rendering queue, and executing a rendering task for the target video frame image; Return to the step of determining the target video frame image in the rendering queue until there is no video frame image in the rendering queue.

6. The method according to claim 1, characterized in that The step of creating video super-resolution multi-threading in the browser comprises: Obtaining device configuration information of a terminal device running the browser; Determine the number of multithreads that matches the device configuration information; Video super-resolution multi-threading is created in the browser according to the number of multi-threads.

7. A video playback device, characterized in that: The device comprises: An acquisition module is used to capture the video frame image to be super-resolved from the video stream data of the target video, and add the video frame image to be super-resolved to the frame processing queue of the browser; A super-resolution module is used to create a video super-resolution multi-thread in the browser, and perform image super-resolution processing on the video frame images in the frame processing queue through at least one sub-thread in the video super-resolution multi-thread to obtain the super-resolution video frame images; A rendering module, used to add the super-resolved video frame images to a rendering queue in order, and render the video frame images in the rendering queue through the main thread of the browser; The playing module is used to play the rendered video obtained by the main thread rendering in the browser.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images