Video Redirection Method, Virtual Machine, Terminal, Device and Storage Medium

Abstract

An embodiment of the present application provides a video redirection method, virtual machine, terminal, device, and storage medium. The method includes: in response to a web page loading instruction in a first browser, redirecting the page information of the first browser to a second browser; parsing the redirected page information in the second browser to obtain multimedia information in the page information, and redirecting the multimedia information to the terminal through the second browser so that the terminal decodes the multimedia information; in the second browser, rendering and displaying at least other page data in the redirected page information that does not include the multimedia information to obtain a first screen; and transmitting a target screen to the terminal, where the target screen includes the first screen and a second screen, and the second screen is a rendered screen other than web page information in the first browser.

Description

Technical Field

[0001] This application relates to the field of computer application technologies, and in particular, to a video redirection method, a virtual machine, a terminal, a device, and a storage medium. Background Art

[0002] In the virtual desktop infrastructure (VDI) scenario, the cost of graphics card virtualization is very high, and common virtualization servers do not have graphics cards. Web page (H5) video playback requires video decoding, consuming a large amount of resources of the central processing unit (CPU) of the virtualization platform, resulting in a decrease in the VDI concurrency number and increasing the enterprise server cost. Moreover, the frame rate and image quality of H5 video playback cannot reach the playback effect of a physical machine, resulting in a poor experience. Summary of the Invention

[0003] To solve the above technical problems, embodiments of this application provide a video redirection method, a virtual machine, a terminal, a device, and a storage medium.

[0004] Embodiments of this application provide a video redirection method, which is applied to a virtual machine. The method includes:

[0005] In response to a web page loading instruction in a first browser, redirect the page information of the first browser to a second browser;

[0006] Parse the redirected page information in the second browser to obtain multimedia information in the page information, and redirect the multimedia information to a terminal through the second browser, so that the terminal decodes the multimedia information;

[0007] In the second browser, render and display other page data in the redirected page information that at least does not include the multimedia information to obtain a first screen;

[0008] Transmit a target screen to the terminal, where the target screen includes the first screen and a second screen, and the second screen is a rendered screen other than the web page information in the first browser.

[0009] Embodiments of this application provide a video redirection method, which is applied to a terminal. The method includes:

[0010] Receive the multimedia information of the redirected page information in the second browser sent by the virtual machine, and the target screen of the virtual machine; wherein the target screen includes a first screen and a second screen; wherein, the first screen is a screen obtained by rendering other page data in the page information that does not include the multimedia information by the second browser; the second screen is a rendered screen other than the web page information in the first browser;

[0011] Decode the multimedia information, and superimpose the multimedia information on the target screen and display it in the first browser.

[0012] An embodiment of the present application provides a virtual machine, and the virtual machine includes:

[0013] A first redirection module, configured to, in response to a web page loading instruction in the first browser, redirect the page information of the first browser to the second browser;

[0014] A second redirection module, configured to parse the redirected page information in the second browser to obtain the multimedia information in the page information; redirect the multimedia information to the terminal through the second browser, so that the terminal decodes and displays the multimedia information;

[0015] A determination module, configured to render and display other page data in the redirected page information that does not include the multimedia information in the second browser to obtain a first screen;

[0016] A transmission module, configured to transmit the target screen to the terminal, where the target screen includes the first screen and the second screen, and wherein the second screen is a rendered screen other than the web page information in the first browser.

[0017] An embodiment of the present application provides a terminal, and the terminal includes:

[0018] A receiving module, configured to receive the multimedia information of the redirected page information in the second browser sent by the virtual machine, and the target screen of the virtual machine; wherein the target screen includes a first screen and a second screen; wherein, the first screen is a screen obtained by rendering other page data in the page information that does not include the multimedia information by the second browser; the second screen is a rendered screen other than the web page information in the first browser;

[0019] A processing module, configured to decode the multimedia information, and superimpose the multimedia information on the target screen and display it in the first browser.

[0020] An embodiment of the present application provides a video redirection device, which includes a memory and a processor. The memory stores a computer program that can run on the processor, and when the processor executes the program, the steps in the above video redirection method are implemented.

[0021] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above video redirection method are implemented.

[0022] An embodiment of the present application provides a video redirection method, virtual machine, terminal, device and storage medium. By responding to an instruction to load a video in a first browser, the page information of the first browser is redirected to a second browser. The redirected page information is parsed in the second browser to determine the multimedia information in the redirected page information. The multimedia information is redirected to the terminal through the second browser, and other page data in the redirected page information that does not include multimedia information is rendered and displayed in the second browser. Then, the first screen and the second screen, that is, the rendering screen other than the web page information in the first browser, are sent to the terminal, so that the terminal can decode the multimedia information and superimpose and display the decoded multimedia information and the target screen. In this way, when the user uses a standard browser, that is, loads an H5 video in the first browser, the habit of using the browser by the user can be not changed, the page in the first browser is automatically redirected on the virtual machine side, a customized browser, that is, the second browser, is used to parse the redirected page information, the multimedia information in the redirected page information is obtained, and the multimedia information is redirected to the terminal again. The part that consumes the graphics processor during the H5 video page loading process in the virtual machine can be processed on the terminal. In this way, the consumption of CPU resources of the virtualization platform can be reduced, the number of concurrent users can be increased, and the overall experience is the same as that of a physical machine. Description of the Drawings

[0023] Figure 1 It is a schematic flow chart of web page video playback provided by an embodiment of the present application;

[0024] Figure 2 It is a schematic implementation flow chart of the video redirection method provided by an embodiment of the present application;

[0025] Figure 3 It is another schematic implementation flow chart of the video redirection method provided by an embodiment of the present application;

[0026] Figure 4 It is another schematic implementation flow chart of the web page video redirection method provided by an embodiment of the present application;

[0027] Figure 5A schematic diagram of a module for a video redirection process provided by an embodiment of the present application;

[0028] Figure 6 A schematic diagram of the structure of a virtual machine provided by an embodiment of the present application;

[0029] Figure 7 A schematic diagram of the structure of a terminal provided by an embodiment of the present application;

[0030] Figure 8 A schematic diagram of the hardware entity of a video redirection device provided by an embodiment of the present application. Detailed implementation manners

[0031] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be construed as limitations on the present application. All other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present application.

[0032] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0033] In the following description, the terms "first / second / third" are only used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first / second / third" can be interchanged with a specific order or sequence when allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0035] Before further elaborating on the embodiments of the present application, the nouns and terms involved in the embodiments of the present application are described. The nouns and terms involved in the embodiments of the present application are subject to the following explanations.

[0036] H5 video: The video element on a Hyper Text Markup Language (HTML5) page, with the tag <video>, The video formats supported include MP4 (MPEG-4), media files (WebM), audio compression (OGGVobis, Ogg), etc. H5 videos are well-supported in mainstream browsers and mobile devices and can be used in online audio-visual and video live streaming scenarios.

[0037] HLS protocol: The full English name is Http Live Streaming, which is a streaming media transmission protocol based on the Hypertext Transfer Protocol (HTTP) and was proposed by Apple. The protocol splits the video file into multiple small segments, and the segment information is described by an m3u8 format file (an M3U file in UTF-8 encoding format). The client continuously receives these video segments to complete the playback.

[0038] MSE standard: Media Source Extensions, the full English name is Media Source Extensions API, which is a standard implemented by browsers to enhance the H5 video playback ability. Based on MSE, the scripting language (JavaScript) can flexibly control the H5 video playback to implement video playback protocols such as HLS.

[0039] Thin Client: A basic application-free computing dumb terminal in the client-server network system, with a lower cost compared to ordinary terminals. It communicates with the server through some protocols and then accesses the local area network. The thin client transmits its inputs such as the mouse and keyboard to the server for processing, and the server then sends the processing results back to the thin client for display on the terminal.

[0040] To facilitate the understanding of the technical solutions of the embodiments of the present application, the related technologies of the embodiments of the present application are described below.

[0041] The embodiments of the present application provide a web video playback process. Figure 1 For the schematic diagram of the web video playback process provided by the embodiments of the present application, as Figure 1 shown, when playing an H5 video, the H5 video playback process 10 needs to go through the following processes: 11 page loading, 12 video tag parsing, 13 HLS format parsing, 14 H5 video filling, and 15 H5 video playback. That is, when playing an H5 video, the H5 video needs to be redirected to the client for playback. In the related technologies, the H5 redirection solution 100 includes:

[0042] 1. Page redirection 101: According to the rules of the Uniform Resource Locator (URL), the browser page is redirected to the terminal. This implementation is simple, but it requires the terminal to run a complete browser, which places a relatively high pressure on the Central Processing Unit (CPU) of the terminal.

[0043] 2. Tag redirection 102: Identify in the H5 page <video>The label sends the obtained video source to the client H5 player after acquisition. This technology is easy to implement, but it does not support HLS format videos. Currently, the mainstream video sites all use the HLS format, so the compatibility of this solution is poor.

[0044] 3. MSE Hook 103: Hook the MSE API interface through JavaScript to capture video data. The implementation process is complex and can support HLS format videos. However, the subsequent video processing depends on the browser, so a browser also needs to run on the terminal, which puts a relatively large pressure on the terminal CPU.

[0045] 4. Video redirection 104: Through a customized browser, obtain the data after splitting the audio and video, and then transmit it to the client for playback. The client uses the Graphics Processing Unit (GPU) to decode the video, which puts little pressure on the CPU and is suitable for thin client scenarios. However, using a customized browser changes the usage habits and the acceptance level of users is not high.

[0046] Based on this, the embodiments of the present application propose the following technical solutions. In order to be able to understand the features and technical content of the embodiments of the present application in more detail, the implementation of the embodiments of the present application will be described in detail below with reference to the accompanying drawings. The attached drawings are only for reference and explanation purposes and are not used to limit the embodiments of the present application.

[0047] The embodiments of the present application provide a video redirection method, and this video redirection method is applied to a virtual machine. Figure 2 For the implementation flow diagram of the video redirection method provided by the embodiments of the present application, as Figure 2 shown, the video redirection method includes the following steps:

[0048] Step S201: In response to a web page loading instruction in the first browser, redirect the page information of the first browser to the second browser.

[0049] Here, the first browser is a standard browser, which generally refers to a browser that provides excellent support for the CSS2.1 specification defined by the W3C and can perfectly render web pages. Current standard browsers include: Opera 10.10, Safari 4.0.4, Internet Explorer 8, Firefox 3, Chrome, and other browsers. The page information of the first browser includes content such as a navigation bar, a web page screen, a bookmark bar, an address bar, and tabs. The first browser contains user data. A web page loading instruction refers to when a user opens a web page in the first browser within a virtual machine, in response to the user's operation, a corresponding web page loading instruction is generated. The second browser is a customized browser inside the virtual machine. The page information of the second browser is different from that of the first browser. The second browser only displays the web page screen in the page information of the first browser and does not display content such as the bookmark bar and theme in the first browser.

[0050] In some embodiments, the technical solution of the embodiments of the present application can be applied in a VDI scenario. VDI virtualizes the user's desktop by running an operating system on a server in a data center; the user connects to the virtual desktop through a client computing protocol from a client device (a client computer or a home computer device), enabling access to the desktop as if accessing a traditional locally installed desktop.

[0051] In some realizable embodiments, a video to be loaded and played in the first browser of a cloud desktop virtual machine can be redirected to a terminal computer for playback, thereby saving the computing resources of the virtual machine.

[0052] In some realizable embodiments, in a VDI scenario, a user loads an H5 video in the first browser. A browser plugin is set in the first browser of the virtual machine, which can monitor the page loading in the first browser. When the page loading in the first browser is an instruction for web page loading, the page information in the first browser can be detected according to the uniform resource locator. When the network address in the page information of the first browser matches the address in a preset address library, that is, when the network address in the page information of the first browser is a video playback address, the page information of the first browser is automatically redirected to the second browser.

[0053] In some realizable embodiments, when the first browser plugin monitors the page loading situation in the first browser and determines to redirect the web page page information loaded in the first browser, the loading process of the original web page is stopped. At the same time, a web page redirection service is called to start the customized browser inside the virtual machine, that is, the second browser, and redirect the page loaded in the first browser to the second browser, and display the redirected page information in the second browser.

[0054] Step S202: Parse the redirected page information in the second browser to obtain the multimedia information in the page information, and redirect the multimedia information to the terminal through the second browser so that the terminal decodes the multimedia information.

[0055] Here, the multimedia information may be video data and audio data included in the page information of the first browser. The second browser receives the redirected page information and parses the hypertext markup language corresponding to the redirected page information. In one example, in the second browser, an application programming interface of the hypertext markup language control series provided by the second browser, that is, an audio or video operation interface, can be called to create a hypertext transfer protocol request plug-in for the current redirected page, detect the hypertext transfer protocol request on the current redirected page, and in the hypertext transfer protocol request plug-in, filter all hypertext transfer protocol data requests sent from the current redirected page, and separate the video data loading request from them to obtain the multimedia information in the page information.

[0056] In some implementable embodiments, a customized browser, i.e., the second browser, can be implemented based on the open-source browser (Chromium), that is, an audio or video operation interface is set in the Chromium browser, and the web page parsing engine in the Chromium browser can also be adjusted to obtain the second browser. After the second browser is started, the second browser uses the audio or video operation interface to filter all hypertext transfer protocol data requests sent from the current redirected page, separates the video data loading request from them, and obtains the audio data and video data in the page information, that is, the multimedia information.

[0057] In some implementable embodiments, after the multimedia information is obtained, the multimedia information can be redirected to the terminal through the second browser, so that the terminal decodes the multimedia information and displays the decoded multimedia information. The terminal is a physical machine computer terminal used by the user, that is, a computer device connected to the virtual desktop. In some embodiments, a customized browser, i.e., the second browser, can be implemented based on the Chromium browser, and the audio and video parsing engine of the Chromium browser can be modified, so that the processing flow of the audio and video data of the H5 page can be modified, and after the multimedia information is obtained after the second browser parses the page information, the parsed multimedia information is redirected to the terminal again, and the terminal decodes and displays the multimedia information.

[0058] In some realizable embodiments, the virtual machine sends the acquired multimedia information to the VDI client program of the terminal. After the VDI client program decodes the multimedia information using the terminal hardware, video images and audio data are obtained, and the video images are sent to the main module of the VDI client. The main module synthesizes them into a complete screen graphic and displays it on the screen of the terminal.

[0059] Step S203: In the second browser, render and display other page data in the redirected page information that does not include at least the multimedia information to obtain a first screen.

[0060] Here, after redirecting the page information of the first browser to the second browser, only the multimedia data in the page information is redirected to the terminal again. After the multimedia data is redirected to the terminal again, other page data that does not include the video image data, audio data, user bookmarks, and address bar in the redirected page information can be obtained.

[0061] After the second browser parses the redirected page information, obtains the multimedia information in the redirected page information, and sends the multimedia information in the redirected page information to the terminal, other page data that does not include the multimedia information in the redirected page information can be obtained, and the other page data is rendered and displayed in the second browser. That is, the web page screen finally displayed in the second browser does not include the multimedia information and does not include the user's bookmarks and address bar. In this way, the part that consumes the CPU in the H5 video-loading page can be processed inside the virtual machine, and the part that consumes the GPU can be processed on the terminal.

[0062] In the second browser, parse the hypertext transfer protocol document corresponding to the other page data into a node tree, then process the style sheet to form a cascading style sheet model, then merge the node tree and the cascading style sheet model into a rendering tree, and finally draw the nodes of the rendering tree on the screen of the second browser to implement the rendering and display of the other page data and obtain a first screen.

[0063] After the second browser parses the redirected page data, redirects the multimedia information again, and renders and displays at least other page data that does not include the multimedia information in the second browser on the virtual machine server side. This enables the rendering work to be carried out in the second browser of the virtual machine, reducing the load on the CPU of the virtualization server.

[0064] Step S204: Transmit the target screen to the terminal, where the target screen includes the first screen and a second screen, and the second screen is a rendered screen other than the web page information in the first browser.

[0065] Here, after the first redirection of the page information of the first browser, the screen composed of bookmarks, address bar, and navigation bar after the web page information is removed by the first browser is rendered, and the second screen can be obtained.

[0066] After obtaining the target screens, i.e., the first screen and the second screen, the virtual machine can transmit the target screens to the terminal in the following two ways:

[0067] Method 1: Synthesize the first screen and the second screen to obtain the target screen; transmit the target screen to the terminal.

[0068] After obtaining the target screens, i.e., the first screen and the second screen, the first screen and the second screen can be synthesized in the virtual machine first to obtain the first complete screen, i.e., the target screen, and then the target screen is sent to the terminal. Then, the terminal superimposes the decoded multimedia information on the second complete screen and renders the final screen displayed in the first browser of the terminal.

[0069] Method 2: Transmit the first screen and the second screen to the terminal respectively.

[0070] After obtaining the target screens, i.e., the first screen and the second screen, the virtual machine can transmit the first screen and the second screen to the terminal respectively, so that the terminal synthesizes the first screen and the second screen to obtain the second complete screen. Then, the terminal superimposes the decoded multimedia information on the second complete screen and renders the final screen displayed in the first browser of the terminal.

[0071] In some implementable embodiments, after the virtual machine redirects the multimedia information to the terminal using the second browser, the contents of the first browser and the second browser are merged in the virtual machine, and the merged screen of the first browser and the second browser is displayed in the virtual machine, and the screen of the second browser is displayed on top of the first browser.

[0072] In the embodiments of the present application, when the user uses a standard browser, i.e., the first browser, to load an H5 video, without changing the user's browser usage habits, the page in the first browser can be automatically redirected on the virtual machine side, and a customized browser, i.e., the second browser, is used to parse the redirected page information to obtain the multimedia information in the redirected page information, and the multimedia information is redirected to the terminal. In this way, the part that consumes the GPU during the H5 video page loading process in the virtual machine can be processed on the terminal. Thus, the consumption of CPU resources of the virtualization platform can be reduced, the number of concurrent users can be increased, and the overall experience is the same as that of a physical machine.

[0073] In some embodiments, when loading a video in the first browser, the page information of the first browser can be redirected to the second browser through the following steps. That is, the above step S101 can be implemented through steps S1011 and S1012:

[0074] Step S1011: In response to the web page loading instruction, obtain the page information in the first browser.

[0075] Here, a browser plugin is set in the first browser of the virtual machine, which can monitor the loading of the page in the first browser. When the page loading in the first browser is an instruction for web page loading, the browser plugin can intercept and obtain the page information of the web page.

[0076] Step S1012: When the page information meets the preset conditions, redirect the page information to the second browser.

[0077] Here, after obtaining the page information of the first browser, use the uniform resource locator to detect the page information in the first browser. When the page information meets the preset conditions, the page information is redirected to the second browser.

[0078] In some implementable embodiments, when the page information meets the preset conditions, call the web page redirection service, start the second browser, redirect the page information to the second browser, and display the redirected page information in the second browser. Moreover, at the same time, stop the loading process of the original page in the first browser.

[0079] In the embodiments of the present application, when receiving an instruction for web page loading in the first browser, when the content of the page information meets the preset conditions, the page information can be redirected to the second browser, so that the redirected page information can be parsed and redirected again in the second browser, reducing the energy consumption of the virtual machine CPU.

[0080] In some embodiments, after obtaining the page information of the first browser, it can be determined whether to redirect the page information of the first browser to the second browser according to the uniform resource location rule. That is, the above step S1012 can be implemented through the following steps:

[0081] Step 1: Detect the uniform resource locator in the page information.

[0082] Here, the Uniform Resource Locator (URL) is a representation method used to specify the location of information on the World Wide Web service program of the Internet. The URL is a concise representation of the location and access method of resources that can be obtained from the Internet and is the address of standard resources on the Internet. Each file on the Internet has a unique URL, which contains information indicating the location of the file and how the browser should handle it.

[0083] In some implementable embodiments, the first browser is a standard browser, and the page information includes content such as a navigation bar, a web page screen, a bookmark bar, an address bar, and tab pages, and the URL in the address bar in the page information can be detected.

[0084] Step 2: When the URL matches the address in the preset address database, redirect the page information to the second browser.

[0085] Here, the preset address database includes multiple URL addresses that can play videos. For example: URL addresses such as http: / / player.youku.com / player.php / sid / xxxxxxxxx / v.swf, www.iqiyi.com, http: / / cache.tv.qq.com / qqplayerout.swf?vid=o013075230b, etc. After obtaining the URL in the redirected page information, match the domain name system of the URL with the domain name system of each address stored in the preset address library. When the domain name systems match, it indicates that the redirected page information includes video elements, and the page information will be redirected to the second browser.

[0086] In the embodiments of the present application, the URL in the redirected page information can be detected according to the URL rule, so that the page information corresponding to the URL that matches the address in the preset address database can be redirected.

[0087] In some embodiments, after redirecting the page information in the first browser to the second browser, the redirected page information can be parsed in the second browser to obtain the multimedia information in the redirected page information. That is, the above step S102 can be implemented through step S1021 and step S1022:

[0088] Step S1021: In the second browser, use a video parsing engine to parse the redirected page information to obtain audio and video data to be decoded.

[0089] Here, the second browser receives the redirected page information, calls the application interface of the hypertext markup language control series provided by the second browser, parses the hypertext markup language corresponding to the page information, redirects the web page screen of the first browser inside the virtual machine, and displays it on the second browser. The web page screen of the first browser does not include the user's bookmarks and other page information in the first browser page information. The audio-visual data includes video frame data and audio data. The video parsing engine is a web page parsing engine obtained by adjusting the chromium browser. In some embodiments, a custom browser, i.e., the second browser, can be implemented based on the open-source browser (chromium), that is, an audio or video operation interface is set in the chromium browser, and the web page parsing engine in the chromium browser is adjusted to obtain the second browser. In the second browser, all hypertext transfer protocol data requests sent from the current redirected page are filtered using the audio or video operation interface, and the adjusted web page parsing engine is used to separate the video data loading request from the page information to obtain the audio data and video frame data in the page information.

[0090] In some implementable embodiments, after the second browser parses the page information using the video parsing engine, the audio data and video frame data are obtained, and the subsequent process is intercepted, and the audio data and video data are not decoded and played.

[0091] Step S1022: Determine the audio-visual data to be decoded as the multimedia information in the page information.

[0092] In the embodiments of the present application, the multimedia information in the page information of the first browser can be obtained by parsing the redirected page through the video parsing engine in the second browser.

[0093] The embodiments of the present application provide a video redirection method, which is applied to a terminal. Figure 3 Another schematic diagram of the implementation process of the video redirection method provided by the embodiments of the present application is as Figure 3 shown, and the video redirection method includes the following steps:

[0094] Step S301: Receive the multimedia information of the redirected page information in the second browser sent by the virtual machine, and the target screen of the virtual machine.

[0095] Here, the target screen includes a first screen and a second screen; wherein, the first screen is a screen rendered by the second browser for other page data in the page information that does not include the multimedia information; the second screen is a rendered screen other than the web page information in the first browser.

[0096] A computer device for the terminal to connect to a virtual desktop. The VDI client program of the terminal receives the multimedia information of the redirected page information in the second browser sent by the virtual machine, that is, the audio and video data in the redirected page information.

[0097] After the terminal receives the target screen, if the target screen is a composite screen including the first screen and the second screen, the terminal can directly superimpose the target screen on the received redirected multimedia information to form the final screen displayed in the first browser of the terminal.

[0098] After the terminal receives the target screen, if the target screen is the first screen and the second screen, the terminal can first synthesize the first screen and the second screen to obtain the second complete screen, and then superimpose the second complete screen on the received redirected multimedia information to form the final screen displayed in the first browser of the terminal.

[0099] Step S302: Decode the multimedia information, and superimpose the multimedia information on the target screen and display it in the first browser.

[0100] Here, the VDI client program of the terminal uses the terminal hardware to decode the multimedia information. For example, the terminal uses the player module and the graphics card to implement H5 video hardware decoding to obtain the video screen and audio data, and sends the video screen and audio data to the VDI client main module, which is synthesized into a complete screen graphic by the main module and displayed on the screen of the terminal. In some embodiments, the first browser is opened on the display screen of the terminal. After decoding the multimedia information in the redirected page sent by the second browser, the first screen of the first browser and the second screen of the second browser are merged to obtain the desktop cloud screen, and the decoded video screen is displayed on the upper layer of the desktop cloud screen.

[0101] In the embodiments of the present application, the terminal receives the multimedia information of the redirected page information in the second browser and the target screen of the virtual machine, decodes the multimedia information, and synthesizes and displays the decoded multimedia information with the target screen. In this way, the video decoding part of the page loading in the first browser of the virtual machine can be processed on the terminal side, reducing the CPU energy consumption on the virtual machine side, ensuring the compatibility of video formats, supporting low-end terminal access, and saving the terminal cost.

[0102] In some embodiments, after the terminal receives the multimedia information and the target screen redirected from the page information in the first browser sent by the virtual machine through the second browser, the multimedia information and the target screen can be decoded and displayed through steps S3021 and S3022:

[0103] Step S3021: Decode the multimedia information using the terminal hardware to obtain audio-visual data.

[0104] Here, the terminal hardware is the player module and the graphics card of the terminal. Since audio data and video data need to be compressed first when stored and transmitted, otherwise the data volume is too large, and compression requires a certain encoding to store and transmit the highest-quality audio-visual data with the smallest capacity. Therefore, when the data needs to be played, it needs to be decoded by a decoder first. The audio-visual data is the video picture and the audio data.

[0105] In some implementable embodiments, the player module and the graphics card are used to decode video and audio, and the video picture and the audio data are determined.

[0106] In some implementable embodiments, after obtaining the decoded video picture and audio data, the video picture is sent to the VDI client main module of the terminal, and the main module synthesizes it into a complete screen graphic and displays it.

[0107] Step S3022: Use a preset graphics transfer protocol to superimpose the audio-visual data and the target picture and display them on the first browser.

[0108] Here, the preset graphics transfer protocol is at least a VDI-based graphics transfer protocol. On the terminal, a preset graphics transfer protocol can be used to draw and display the desktop, such as drawing pictures and rectangles. Since the multimedia information in the page information of the first browser, that is, the video after redirection, in the terminal's view, the VDI desktop is divided into two parts: the video picture and the non-video picture, that is, the target picture. To draw the desktop and superimpose the audio-visual data and the target picture, a complete display desktop can be displayed on the terminal side.

[0109] By synthesizing the audio-visual data and the target picture, the audio-visual data in the page information of the first browser can be redirected to the terminal and displayed on the terminal's display desktop. In some embodiments, the VDI client main module of the terminal can synthesize the audio-visual data and the display desktop obtained by drawing the target picture to obtain a complete screen graphic and display it on the terminal's display interface.

[0110] In some implementable embodiments, the processes of drawing the desktop and synthesizing the desktop are not perceptible to the user.

[0111] In the embodiments of the present application, the terminal can decode the redirected multimedia information in the first browser in a hard decoding manner, and draw the desktop to fully display the desktop formed by the multimedia and the target screen, which can reduce the energy consumption of the virtual machine server, support the decoding and display of various formats of multimedia information, ensure the compatibility of video redirection, and reduce the terminal access cost.

[0112] The embodiments of the present application provide an HLS format H5 video redirection workflow in a VDI scenario. Figure 4 It is another schematic implementation flowchart of the web page video redirection method provided by the embodiments of the present application. As Figure 4 shown, the specific workflow is as follows:

[0113] 1. Default screen 41

[0114] The default screen of the first browser, that is, the page information of the first browser, can be at least divided into three parts: a navigation bar 401, a page 402, and a video element 403.

[0115] 2. After web page redirection 42

[0116] Inside the virtual machine, call the web page redirection service to redirect the H5 web page to the second browser for display, and obtain the page 404 after web page redirection. The second browser can parse the page data, strip the H5 audio and video data, and then send this data to the terminal.

[0117] 3. Terminal final screen 43

[0118] After the terminal draws the desktop based on the VDI display protocol, it is also necessary to display the H5 video based on the H5 redirection protocol. From the user's perspective, the H5 page is fully displayed.

[0119] Among them, the operations of the default screen 41 and web page redirection are implemented inside the virtual machine 44, and the operations of the terminal final screen are implemented inside the terminal 45.

[0120] The embodiments of the present application provide a module schematic diagram of a video redirection process. Figure 5 It is a module schematic diagram of a video redirection process provided by the embodiments of the present application. As Figure 5 shown, the key modules involved in the H5 video redirection process in the present application are as follows:

[0121] 1. Browser plug-in 501

[0122] The browser plug-in 501 is used to monitor page loading and determine whether the page information of the first browser needs to be redirected based on the uniform resource locator rule. When redirection is required, send the page information of the first browser to the web page redirection service, and then stop the original page loading process.

[0123] 2. Web page redirection service 502

[0124] After the web page redirection service receives the page information, a second browser is started inside the virtual machine to display the redirected page. The second browser can separate the resources that need to be rendered by the CPU and the multimedia information (i.e., audio and video data) that needs to be decoded by the GPU in the page information, and display the separated page inside the virtual machine. The separated audio and video data is sent to the terminal for decoding and display by the terminal.

[0125] 3. H5 redirection client 503

[0126] Here, the H5 redirection client 503 processes the received redirected audio and video data, which can be implemented based on a general player and does not depend on the format of the browser. The H5 redirection client 503 mainly consumes GPU resources, and a low-end integrated graphics card in a thin client scenario can meet the requirements.

[0127] Among them, the browser plugin 501 and the web page redirection service module are located inside the virtual machine 51, while the web page video redirection client 503 is located inside the terminal 52.

[0128] In the embodiment of the present application, since the part of the page information of the first browser, i.e., the H5 page, that consumes the CPU is processed inside the virtual machine, and the part that consumes the GPU is processed by the terminal, it is possible to increase the number of concurrent users while not requiring users to adjust their operation habits, and the overall experience is the same as that of a physical machine.

[0129] The embodiment of the present application provides a virtual machine Figure 6 which is a schematic structural diagram of the virtual machine provided by the embodiment of the present application, as Figure 6 shown, the virtual machine 600 includes:

[0130] A first redirection module 601, configured to redirect the page information of the first browser to a second browser in response to a web page loading instruction in the first browser;

[0131] A second redirection module 602, configured to parse the redirected page information in the second browser to obtain the multimedia information in the page information, and redirect the multimedia information to the terminal through the second browser so that the terminal decodes the multimedia information;

[0132] A determination module 603, configured to render and display at least other page data in the redirected page information that does not include the multimedia information in the second browser to obtain a first screen;

[0133] A transmission module 604 for transmitting a target screen to the terminal, where the target screen includes the first screen and the second screen, and the second screen is a rendering screen in the first browser except for web page information.

[0134] In the virtual machine 600 described above, the first redirection module 601 includes:

[0135] A first acquisition sub-module for acquiring page information in the first browser in response to the web page loading instruction.

[0136] A redirection sub-module for redirecting the page information to the second browser when the page information meets a preset condition.

[0137] In the virtual machine 600 described above, the redirection sub-module includes:

[0138] A detection unit for detecting the uniform resource locator in the page information.

[0139] A redirection unit for redirecting the page information to the second browser when the uniform resource locator matches an address in a preset address database.

[0140] In the virtual machine 600 described above, the second redirection module 602 includes:

[0141] A parsing sub-module for parsing the redirected page information in the second browser using a video parsing engine to obtain audio-visual data to be decoded.

[0142] A first determination sub-module for determining the audio-visual data to be decoded as the multimedia information in the page information.

[0143] In the virtual machine 600 described above, the transmission module 604 includes:

[0144] A second determination sub-module for synthesizing the first screen and the second screen to obtain the target screen.

[0145] A first transmission sub-module for transmitting the target screen to the terminal; or,

[0146] A second transmission sub-module for transmitting the first screen and the second screen to the terminal respectively.

[0147] An embodiment of the present application provides a terminal. Figure 7 It is a structural schematic diagram of the terminal provided by the embodiment of the present application, as Figure 7 shown, the terminal 700 includes:

[0148] A receiving module 701, configured to receive multimedia information of a redirected page in a second browser sent by a virtual machine, and a target screen of the virtual machine; wherein the target screen includes a first screen and a second screen; wherein, the first screen is a screen obtained by rendering other page data in the page information that does not include the multimedia information by the second browser; the second screen is a rendering screen other than web page information in a first browser.

[0149] A processing module 702, configured to decode the multimedia information, and superimpose the multimedia information and the target screen and display the result in the first browser.

[0150] In the above terminal 700, the processing module 702 includes:

[0151] A decoding sub-module, configured to decode the multimedia information by using terminal hardware to obtain audio and video data.

[0152] A processing sub-module, configured to use a preset graphics transfer protocol to superimpose the audio and video data and the target screen and display the result in the first browser.

[0153] Correspondingly, an embodiment of the present application provides a video redirection device 800, such as a computer device. Figure 8 The following is a schematic diagram of the hardware entity of the video redirection device provided by the embodiment of the present application. As Figure 8 shown, the hardware entity of the device 800 includes: a processor 801, a communication interface 802, and a memory 803, wherein:

[0154] The processor 801 generally controls the overall operation of the device 800.

[0155] The communication interface 802 can enable the device 800 to communicate with other terminals or servers through a network.

[0156] The memory 803 is configured to store instructions and applications executable by the processor 801, and can also cache data to be processed or already processed by the processor 801 and each module in the device 800 (for example, image data, audio data, voice communication data, and video communication data), and can be implemented by flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0157] Another embodiment of the present application provides a video redirection device, which includes each module included, as well as each sub-module and each unit included in each module, and can be implemented by a processor in a terminal; of course, it can also be implemented by specific logic circuits; during implementation, the processor can be a CPU, a microprocessor (Microprocessor Unit, MPU), a digital signal processor (Digital Signal Processing, DSP), or a field programmable gate array (Field Programmable Gate Array, FPGA), etc.

[0158] It should be noted that the description of the above terminal embodiments is similar to the description of the above method embodiments and has similar beneficial effects to the method embodiments. For the technical details not disclosed in the terminal embodiments of the present application, please refer to the description of the method embodiments of the present application for understanding.

[0159] Correspondingly, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the video redirection method provided in the above embodiments are implemented.

[0160] It should be pointed out here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments and has similar beneficial effects to the method embodiments. For the technical details not disclosed in the storage medium and device embodiments of the present application, please refer to the description of the method embodiments of the present application for understanding.

[0161] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures, or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, the "in one embodiment" or "in an embodiment" that appears throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the order numbers of the above processes do not mean the order of execution. The execution order of each process should be determined by its function and internal logic and should not constitute any limitation to the implementation process of the embodiments of the present application. The serial numbers of the embodiments of the present application above are only for description and do not represent the advantages or disadvantages of the embodiments.

[0162] It should be noted that, in this document, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element qualified by the phrase "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising such element.

[0163] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the displayed or discussed components can be through some interfaces. The indirect coupling or communication connection of devices or units can be electrical, mechanical, or other forms.

[0164] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they can be located in one place or distributed to multiple network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0165] In addition, each functional unit in the embodiments of the present application can be all integrated in a processing unit, or each unit can be separately a unit alone, or two or more units can be integrated in a unit; the above integrated units can be implemented in the form of hardware, or in the form of hardware plus software functional units.

[0166] Those of ordinary skill in the art can understand that all or part of the steps to implement the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above method embodiments; and the foregoing storage medium includes various media that can store program codes, such as removable storage devices, read-only memory (ROM), magnetic disks, or optical discs.

[0167] Alternatively, if the above integrated units of the present application are implemented in the form of software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present application, in essence or the part that contributes to the related art, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as removable storage devices, ROMs, magnetic disks, or optical discs.

[0168] As described above, only some embodiments of the present application are provided, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.< / video> < / video>

Claims

1. A video redirection method, characterized in that, The method is applied to a virtual machine, and the method includes: In response to a web page loading instruction in a first browser, redirect the page information of the first browser to a second browser; the first browser is a standard browser inside the virtual machine, and the second browser is a customized browser inside the virtual machine, and the web page screen in the page information of the first browser is displayed in the second browser; Parse the redirected page information in the second browser to obtain the multimedia information in the page information, and redirect the multimedia information to the terminal through the second browser so that the terminal decodes the multimedia information; In the second browser, render and display other page data in the redirected page information that does not include the multimedia information at least to obtain a first screen; Transmit a target screen to the terminal, where the target screen includes the first screen and a second screen, and the second screen is a rendered screen other than the web page information in the first browser.

2. The method according to claim 1, characterized in that, The step of, in response to a web page loading instruction in a first browser, redirecting the page information of the first browser to a second browser includes: In response to the web page loading instruction, obtain the page information in the first browser; When the page information meets a preset condition, redirect the page information to the second browser.

3. The method according to claim 2, characterized in that, The step of, when the page information meets a preset condition, redirecting the page information to the second browser includes: Detect the uniform resource locator in the page information; When the uniform resource locator matches an address in a preset address database, redirect the page information to the second browser.

4. The method according to claim 1, characterized in that, The step of, in the second browser, parsing the redirected page information to obtain the multimedia information in the page information includes: In the second browser, use a video parsing engine to parse the redirected page information to obtain audio-video data to be decoded; Determine the audio-video data to be decoded as the multimedia information in the page information.

5. The method according to claim 1, characterized in that, The step of transmitting the target screen to the terminal, where the target screen includes the first screen and a second screen, includes: Combine the first screen and the second screen to obtain the target screen; Transmit the target screen to the terminal; or, Transmit the first screen and the second screen to the terminal respectively.

6. A video redirection method, characterized in that, The method is applied to a terminal, and the method includes: Receive the multimedia information of the redirected page information in the second browser sent by the virtual machine, and the target screen of the virtual machine; where the target screen includes a first screen and a second screen; where, the first screen is a screen rendered by the second browser for other page data in the page information that does not include the multimedia information; the second screen is a rendered screen other than the web page information in the first browser; the first browser is a standard browser inside the virtual machine, and the second browser is a customized browser inside the virtual machine, and the web page screen in the page information of the first browser is displayed in the second browser; Decode the multimedia information, and superimpose the multimedia information on the target screen and then display it on the first browser.

7. The method according to claim 6, wherein decoding the multimedia information and superimposing the multimedia information on the target picture and then displaying the result on the first browser includes: Use the terminal hardware to decode the multimedia information to obtain audio and video data; Use a preset graphics transmission protocol to superimpose the audio and video data and the target screen and display them on the first browser.

8. A virtual machine, characterized in that, The virtual machine includes: A first redirection module, configured to, in response to a web page loading instruction in the first browser, redirect the page information of the first browser to the second browser; A second redirection module, configured to parse the redirected page information in the second browser to obtain the multimedia information in the page information; redirect the multimedia information to the terminal through the second browser, so that the terminal decodes and displays the multimedia information; A determination module, configured to render and display other page data in the redirected page information that does not include the multimedia information in the second browser to obtain a first screen; A transmission module, configured to transmit a target screen to the terminal, where the target screen includes the first screen and a second screen, and the second screen is a rendered screen other than the web page information in the first browser; Wherein, the first browser is a standard browser inside the virtual machine, the second browser is a customized browser inside the virtual machine, and the second browser displays the web page screen in the page information of the first browser.

9. A terminal, characterized in that, The terminal includes: A receiving module, configured to receive the multimedia information of the redirected page information in the second browser sent by the virtual machine, and the target screen of the virtual machine; the target screen includes a first screen and a second screen; the first screen is a screen obtained by rendering other page data in the page information that does not include the multimedia information by the second browser; the second screen is a rendered screen other than the web page information in the first browser; A processing module, configured to decode the multimedia information, and superimpose the multimedia information on the target screen and then display it on the first browser; Wherein, the first browser is a standard browser inside the virtual machine, the second browser is a customized browser inside the virtual machine, and the second browser displays the web page screen in the page information of the first browser.

10. A video redirection device, comprising a memory and a processor, the memory storing a computer program that can run on the processor, characterized in that, When the processor executes the program, it implements the steps in the video redirection method according to any one of claims 1 to 5, or when the processor executes the program, it implements the steps in the video redirection method according to claim 6 or 7.

11. A computer-readable storage medium, having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps in the video redirection method according to any one of claims 1 to 5, or when the computer program is executed by the processor, it implements the steps in the video redirection method according to claim 6 or 7.

Images