A mobile terminal, a display device, and a method for screen mirroring according to a browser

Abstract

The present application provides a mobile terminal, a display device, and a method for screen mirroring according to a browser. The method includes: obtaining the browser address of the browser to be screen mirrored, and sending a web content request for obtaining the web page content of the browser to the display device based on the browser address; receiving the receipt information returned by the display device according to the web content request; parsing the browser type of the browser to be screen mirrored from the receipt information; initiating a screen mirroring connection to the display device according to the browser type; in response to the event of successful screen mirroring connection, sending screen mirroring data to the display device through the browser to be screen mirrored, so that the display device displays the screen mirroring data. For browsers without gesture thread requirements, screen data can be directly recorded based on the browser to be screen mirrored. For browsers with gesture thread requirements, screen data can be recorded based on the confirmed screen mirroring event of the operation control. The present application can execute different screen mirroring methods according to different browser types, thereby improving the compatibility of the browser.

Description

Technical Field

[0001] This application relates to the field of screen mirroring technology, and particularly to a mobile terminal, a display device, and a method for screen mirroring according to a browser. Background Art

[0002] Mirror screen mirroring means that a mobile terminal is connected to a display device via a wired or wireless manner, so that the screen displayed on the mobile terminal is mirrored to the display device, and the display device displays the content corresponding to the screen mirroring data sent by the mobile terminal in real time. When a user performs mirror screen mirroring on the mobile terminal and the display device, the mobile terminal and the display device usually need to be pre-connected to the local area network composed of the same router. During the mirror screen mirroring process, the mobile terminal needs to record the screen of the content displayed on the mobile terminal in real time and forward the recorded screen data to the display device through the router, and the display device performs real-time decoding and display.

[0003] Taking the mobile terminal as a computer and the display device as a TV as an example, the browser web page content of the computer can be mirrored to the TV to achieve browser mirroring from the computer side to the TV side. To achieve browser mirroring, in one implementation, after opening a browser page on the mobile terminal, a screen mirroring request can be automatically sent to the display device. For example, after entering the browser address on the mobile terminal, the mobile terminal directly triggers an operation to send a screen mirroring request to the display device. In this way, after the display device allows screen mirroring, the mobile terminal can send the screen mirroring data to the display device so that the display device displays the screen mirroring data.

[0004] However, in the above implementation, the type of the browser needs to be considered. For example, for a browser that does not have gesture thread requirements such as clicking, inputting, and swiping, the mobile terminal can directly send a screen mirroring request to the display device. For a browser with gesture thread requirements, it is required to apply for screen recording through the gesture thread. If the direct screen mirroring method is still used, an error will occur and the screen mirroring cannot be successfully completed. Therefore, during the browser screen mirroring process, the browser compatibility is poor. Summary of the Invention

[0005] This application provides a mobile terminal, a display device, and a method for screen mirroring according to a browser, which can perform different screen mirroring methods according to different browser types, thereby improving the browser compatibility.

[0006] In a first aspect, some embodiments of this application provide a mobile terminal, including

[0007] a display unit configured to display a terminal interface;

[0008] a communication unit configured to establish a screen mirroring connection with a display device;

[0009] a processor configured to:

[0010] Obtain the browser address of the browser to be screen-cast, and send a web content request for obtaining the browser web content to the display device based on the browser address;

[0011] Receive the receipt information returned by the display device according to the web content request;

[0012] Analyze the receipt information to obtain the browser type of the browser to be screen-cast;

[0013] Initiate a screen-cast connection to the display device according to the browser type;

[0014] In response to the event of successful screen-cast connection, send screen-cast data to the display device through the browser to be screen-cast, so that the display device displays the screen-cast data.

[0015] Based on the above mobile terminal, the first aspect of the present application further provides a method for screen-casting according to a browser, which is applied to a mobile terminal. The mobile terminal includes a display unit, a communication unit and a processor. The method for screen-casting according to a browser includes:

[0016] Obtain the browser address of the browser to be screen-cast input, and initiate a web content request for obtaining the browser web content to the display device based on the browser address;

[0017] Receive the receipt information returned by the display device according to the web content request;

[0018] Analyze the receipt information to obtain the browser type of the browser to be screen-cast included in the receipt information;

[0019] Initiate a screen-cast connection to the display device according to the browser type;

[0020] In response to the event of successful screen-cast connection, send screen-cast data to the display device through the browser to be screen-cast, so that the display device displays the screen-cast data.

[0021] As can be seen from the above technical solutions, the mobile terminal and the method for screen mirroring according to a browser provided in the above embodiments include: obtaining the browser address of the browser to be screen mirrored, and sending a web content request for obtaining the browser web content to a display device based on the browser address; receiving the receipt information returned by the display device according to the web content request; parsing the receipt information to obtain the browser type of the browser to be screen mirrored; initiating a screen mirroring connection to the display device according to the browser type; in response to the event of successful screen mirroring connection, sending screen mirroring data to the display device through the browser to be screen mirrored, so that the display device displays the screen mirroring data. For browsers without gesture thread requirements, screen data can be directly recorded based on the browser to be screen mirrored. For browsers with gesture thread requirements, screen data can be recorded after confirming the screen mirroring event for operation controls, etc. The present application can execute different screen mirroring methods according to different browser types, thereby improving the compatibility of the browser.

[0022] In a second aspect, some embodiments of the present application provide a display device, including:

[0023] A display, configured to display a screen mirroring interface;

[0024] A communicator, configured to establish a screen mirroring connection with a mobile terminal;

[0025] A controller, configured to:

[0026] Receive a web content request sent by the mobile terminal, where the web content request is a request for obtaining browser web content sent by the mobile terminal to the display device based on the browser address;

[0027] Generate receipt information according to the web content request, and send the receipt information to the mobile terminal, where the receipt information includes the browser type;

[0028] Receive the screen mirroring connection sent by the mobile terminal according to the browser type;

[0029] In response to the event of successful screen mirroring connection, receive the screen mirroring data sent by the mobile terminal, and control the display to display the screen mirroring data.

[0030] Based on the above display device, a second aspect of the present application further provides a method for screen mirroring according to a browser, which is applied to the display device. The display device includes a display, a communicator, and a controller. The method for screen mirroring according to the browser includes:

[0031] Receive a web content request sent by the mobile terminal, where the web content request is a request for obtaining browser web content sent by the mobile terminal to the display device based on the browser address;

[0032] Generate a receipt message according to the web page content request, and send the receipt message to the mobile terminal, where the receipt message includes the browser type;

[0033] Receive the screen mirroring connection sent by the mobile terminal according to the browser type;

[0034] In response to the event of successful screen mirroring connection, receive the screen mirroring data sent by the mobile terminal, and control the display to display the screen mirroring data.

[0035] As can be seen from the above technical solutions, the display device and the method for screen mirroring according to the browser provided in the above embodiments can receive the web page content request sent by the mobile terminal, generate a receipt message according to the web page content request, and send the receipt message to the mobile terminal. Among them, the receipt message includes the browser type. After that, the display device can receive the screen mirroring connection sent by the mobile terminal according to the browser type. In response to the event of successful screen mirroring connection, the display device can receive the screen mirroring data sent by the mobile terminal and control the display to display the screen mirroring data. This application can perform different screen mirroring methods according to different browser types of the mobile terminal, and cooperate with the mobile terminal and the display device to improve the compatibility of the browser. Description of the Drawings

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0037] Figure 1 Schematic diagram of the operation scenario between the display device and the control device provided in some embodiments of the present application;

[0038] Figure 2 Provided by some embodiments of the present application Figure 1 Hardware configuration block diagram of the display device in;

[0039] Figure 3 Provided by some embodiments of the present application Figure 1 Hardware configuration block diagram of the control device in;

[0040] Figure 4 Provided by some embodiments of the present application Figure 1 Software configuration schematic diagram of the display device in;

[0041] Figure 5 Schematic diagram of realizing mirror screen projection through a router within the same local area network provided by some embodiments of the present application;

[0042] Figure 6 Schematic flow chart of a method for a display device provided in some embodiments of the present application to perform screen mirroring according to a browser;

[0043] Figure 7 Schematic flow chart of a mobile terminal provided in some embodiments of the present application for parsing receipt information;

[0044] Figure 8 Schematic flow chart of a mobile terminal provided in some embodiments of the present application for initiating a screen mirroring connection to a display device according to a browser type;

[0045] Figure 9 Schematic flow chart of a mobile terminal provided in some embodiments of the present application for identifying the browser type of a browser to be screen mirrored;

[0046] Figure 10 Schematic diagram of the operation control effect of a browser to be screen mirrored of the first browser type provided in some embodiments of the present application for initiating screen mirroring to a display device;

[0047] Figure 11 Schematic flow chart of a mobile terminal provided in some embodiments of the present application for sending screen mirroring data to a display device 200 through a browser to be screen mirrored;

[0048] Figure 12 Schematic flow chart of a mobile terminal provided in some embodiments of the present application for initiating a screen mirroring connection to a display device in another way;

[0049] Figure 13 Schematic flow chart of a mobile terminal provided in some embodiments of the present application for initiating a screen mirroring connection to a display device in yet another way;

[0050] Figure 14 Schematic flow chart of a display device provided in some embodiments of the present application for displaying screen mirroring data;

[0051] Figure 15 Schematic flow chart of the negotiation process between a display device and a mobile terminal provided in some embodiments of the present application. Detailed implementation manners

[0052] To make the objectives, technical solutions, and advantages of some embodiments of the present application clearer, the technical solutions of some embodiments of the present application will be clearly and completely described below in conjunction with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments.

[0053] It should be noted that the brief description of terms in some embodiments of the present application is only for the convenience of understanding the following described embodiments, rather than intending to limit the embodiments of some embodiments of the present application. Unless otherwise specified, these terms should be understood in their ordinary and common meanings.

[0054] In some embodiments of the present application, the terms "first", "second", "third", etc. in the specification, claims and the above-mentioned drawings are used to distinguish similar or like objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that such terms can be interchanged under appropriate circumstances.

[0055] The terms "comprising" and "having" and any variations thereof are intended to cover but not exclusively include. For example, a product or device comprising a series of components does not necessarily have to be limited to all the components clearly listed, but may include other components not clearly listed or inherent to these products or devices.

[0056] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or a combination of hardware or / and software code that can perform functions related to that element.

[0057] Figure 1 It is a schematic diagram of the operation scenario between the display device and the control device provided for some embodiments of the present application. As Figure 1 shown, the user can operate the display device 200 through the mobile terminal 300 and the control device 100.

[0058] In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through network communication protocols to achieve the purpose of one-to-one control operation and data communication. It can also transmit the audio and video content displayed on the mobile terminal 300 to the display device 200 to achieve the synchronous display function.

[0059] As Figure 1 also shown, the display device 200 also conducts data communication with the server 400 through various communication methods. The display device 200 is allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN) and other networks.

[0060] In addition to providing the function of receiving broadcast television, the display device 200 can also additionally provide an intelligent network television function with computer support functions, including but not limited to, network television, smart television, Internet Protocol Television (IPTV), etc.

[0061] Figure 2 Provided for some embodiments of the present application Figure 1 is the hardware configuration block diagram of the display device 200 in

[0062] In some embodiments, the display device 200 includes at least one of a tuner-demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0063] In some embodiments, the detector 230 is configured to collect signals of the external environment or for interacting with the outside.

[0064] In some embodiments, the display 260 includes a display screen assembly for presenting a picture, and a driving assembly for driving image display, which is configured to receive an image signal output from the controller, and is for displaying video content, image content, a menu manipulation interface, a user manipulation UI interface, and the like.

[0065] In some embodiments, the communicator 220 is a component for communicating with an external device or a server 400 according to various communication protocol types.

[0066] In some embodiments, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0067] In some embodiments, if a user inputs a user command on a graphical user interface (GUI) that can be displayed on the display 260, the user input interface receives the user input command through the graphical user interface (GUI).

[0068] In some embodiments, the user interface 280 is an interface that can be used to receive control inputs.

[0069] Figure 3 For some embodiments of the present application Figure 1 is a hardware configuration block diagram of a control device. As Figure 3 shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0070] The control device 100 is configured to control the display device 200, and can receive input operation instructions of a user, and convert the operation instructions into instructions recognizable and responsive by the display device 200, playing an intermediary role in the interaction between the user and the display device 200.

[0071] In some embodiments, the control device 100 can be an intelligent device. For example, the control device 100 can install various applications for controlling the display device 200 according to user requirements.

[0072] In some embodiments, as Figure 1As shown, after installing the application of the control and display device 200, the mobile terminal 300 or other intelligent electronic devices can perform functions similar to those of controlling the device 100.

[0073] The controller 110 includes a processor unit 112, a RAM 113, a ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between internal components and the data processing functions between the external and internal.

[0074] Under the control of the controller 110, the communication interface 130 realizes the communication of control signals and data signals with the display device 200. The communication interface 130 may include at least one of other near - field communication modules such as a WiFi chip 131, a Bluetooth module 132, an NFC module 133, etc.

[0075] The user input / output interface 140, where the input interface includes at least one of a microphone 141, a touchpad 142, a sensor 143, a key 144, and other input interfaces.

[0076] In some embodiments, the control device 100 includes at least one of the communication interface 130 and the input / output interface 140. The communication interface 130 configured in the control device 100, such as modules like WiFi, Bluetooth, NFC, etc., can encode user input instructions through the WiFi protocol, or the Bluetooth protocol, or the NFC protocol and send them to the display device 200.

[0077] The memory 190 is used to store various operating programs, data, and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0078] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0079] Figure 4 For some embodiments of this application Figure 1 The schematic diagram of the software configuration in the display device. In some embodiments, the system is divided into four layers, from top to bottom are the application layer (abbreviated as "application layer"), the application framework layer (abbreviated as "framework layer"), the system library layer (abbreviated as "system runtime library layer"), and the kernel layer.

[0080] In some embodiments, at least one application runs in the application layer. These applications can be window programs, system setting programs, clock programs, camera applications, etc. that come with the operating system; they can also be application programs developed by third-party developers.

[0081] The framework layer provides application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that decides the actions of the applications in the application layer.

[0082] As Figure 4 shown, in some embodiments of the present application, the application framework layer includes Managers, Content Provider, View System, etc. Among them, the Managers include at least one of the following modules: The Activity Manager is used to interact with all the activities running in the system; the Location Manager is used to provide access to the system location service for system services or applications; the Package Manager is used to retrieve various information related to the application packages currently installed on the device; the Notification Manager is used to control the display and clearing of notification messages; the WindowManager is used to manage icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0083] In some embodiments, the Activity Manager is used to: manage the life cycles of various applications and the normal navigation back function.

[0084] In some embodiments, the Window Manager is used to manage all window programs.

[0085] In some embodiments, the system runtime layer provides support for the upper layer, i.e., the framework layer. When the framework layer is accessed, the operating system runs the C / C++ libraries included in the system runtime layer to implement the functions to be achieved by the framework layer.

[0086] In some embodiments, the kernel layer is the layer between hardware and software. As Figure 4 shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor drivers (such as fingerprint sensors, temperature sensors, touch sensors, pressure sensors, etc.).

[0087] In some embodiments, the kernel layer further includes a power driver module for power management.

[0088] In some embodiments, Figure 4 the software programs and / or modules corresponding to the software architecture in Figure 2 or Figure 3 are stored in the first memory or the second memory as shown.

[0089] Based on the above display device 200 and mobile terminal 300, a mirror screen projection connection can be established to enable the display device 200 to play and display the content in the mobile terminal 300. Mirror screen projection means that the mobile terminal 300 is connected to the display device 200 in a wired or wireless manner, so that the screen displayed in the mobile terminal 300 is projected onto the display device 200, and the display device 200 displays the content corresponding to the screen projection data sent by the mobile terminal 300 in real time.

[0090] When the user performs mirror screen projection on the mobile terminal 300 and the display device 200, it is usually necessary to pre-set the mobile terminal 300 and the display device 200 within the same local area network. For example, taking the mobile terminal as a computer and the display device as a TV, when the user performs mirror screen projection on the computer and the TV, it is usually necessary to pre-connect the computer and the TV within the local area network composed of the same router. Figure 5 FIG. is a schematic diagram of realizing mirror screen projection through a router within the same local area network provided by some embodiments of the present application. As Figure 5 shown, during the mirror screen projection process, the computer can record the screen in real time and forward the recorded screen data to the TV, and the TV performs real-time decoding and display. In this way, the browser web page content of the computer can be projected onto the TV to achieve browser projection from the computer side to the TV side.

[0091] To realize browser projection between the mobile terminal 300 and the display device 200, in some embodiments, after the browser page is opened on the mobile terminal 300, a screen projection can be automatically initiated to the display device 200. After the browser address is input on the mobile terminal 300, the mobile terminal 300 can directly trigger the operation of initiating screen projection to the display device 200. In this way, after the display device 200 allows screen projection, the mobile terminal 300 can send the screen projection data to the display device 200 so that the display device 200 displays the screen projection data. For example, on the computer side, a web page is opened through the browser. The web page will send a web page acquisition request to the TV and apply for the permission of screen recording through the browser. After obtaining the recording permission, the computer side can obtain the video stream and send the video stream to the TV. After receiving the video stream, the TV can play the video stream.

[0092] However, in the above implementation, the type of browser needs to be taken into account. In some embodiments, browsers can be classified into browsers with gesture thread requirements and browsers without gesture thread requirements according to the classification method of whether they need to execute screen mirroring through gesture threads such as clicking, inputting, and swiping. For example, a gesture thread can include threads corresponding to user operations such as clicking, inputting, and swiping. That is to say, during the browser screen mirroring process, when a browser with gesture requirements applies for screen mirroring permissions such as screen recording, it needs to be initiated during user operations such as clicking, inputting, and swiping. In this way, for browsers without gesture thread requirements, the mobile terminal 300 can directly initiate screen mirroring to the display device 200. For example, after entering the browser address and opening a web page, the web page can directly perform screen mirroring to the display device 200. For browsers with gesture thread requirements, however, it is necessary to apply for screen recording through the gesture thread, that is, screen mirroring needs to be triggered by operations such as clicking and inputting. If the method of directly performing screen mirroring by opening the web page is still adopted, it will cause an error during the screen mirroring process and the screen mirroring cannot be successfully completed.

[0093] For example, after some browsers open a web page, an authorization window for obtaining web page content will automatically pop up. After clicking OK in the authorization window, screen mirroring can be initiated. When the web page is opened, a screen mirroring connection can be initiated to the TV. After the connection is successful, the user can be applied for screen recording permissions. During this process, automatic screen mirroring can be performed without clicking operation buttons and controls. However, some browsers do not allow the logic of automatically applying for screen recording permissions when the browser is opened. In this scenario, when the automatic application method is still adopted, an error prompt will pop up, which is used to indicate that the application for permissions needs to be implemented through gesture operations such as clicking, swiping, and inputting. Therefore, for browsers with gesture thread requirements, the compatibility is poor during the browser screen mirroring process.

[0094] To implement browser screen mirroring between the mobile terminal 300 and the display device 200, some embodiments of the present application provide a mobile terminal 300, including a display unit 310, a communication unit 320, and a processor 330. Among them, the display unit 310 is configured to display a terminal interface, and the communication unit 320 is configured to establish a screen mirroring connection with the display device 200. The present application can execute different screen mirroring methods according to different browser types, thereby improving the compatibility of the browser.

[0095] To facilitate the understanding of the technical solutions in some embodiments of the present application, the following will describe each step in detail in combination with some specific embodiments and drawings. Figure 6 It is a schematic flowchart of a method for a display device provided in some embodiments of the present application to perform screen mirroring according to a browser. As Figure 6 shown, the processor 330 in some embodiments of the present application is configured to execute the following steps S1 - S5, and the specific content is as follows:

[0096] Step S1: The mobile terminal 300 obtains the browser address of the browser to be screen-cast, and based on the browser address, sends a web content request for obtaining the browser web content to the display device 200.

[0097] When performing screen-cast control on a display device 200 such as a TV through a browser web page, it is necessary to obtain a TV device on the same network. After determining the device that can be screen-cast, then perform screen-cast interaction. Exemplarily, after logging in to a browser web page through a computer, when wanting to screen-cast this web page to a TV on the same network, it is necessary to obtain the device information of these TVs to be screen-cast, such as the device list of the display devices to be discovered on the same network.

[0098] After obtaining the device list of the display devices to be discovered on the same network, the content of the browser web page can be screen-cast to any display device 200 on the same network. When performing browser screen-casting, the mobile terminal 300 can first obtain the browser address of the browser to be screen-cast. For example, still taking the mobile terminal 300 as a computer, the user can input the browser address based on the browser of the computer, and after the input is completed, based on the browser address, send a web content request for obtaining the browser web content to the display device 200. This web content request is the request sent by the mobile terminal 300 to the display device 200 for obtaining the browser web content based on the browser address. After step S1 is completed, the following step S2 can be executed.

[0099] Step S2: The mobile terminal 300 receives the receipt information returned by the display device 200 according to the web content request.

[0100] In some embodiments, the browser address input on the computer side points to the display device 200. A Hypertext Transfer Protocol Secure (HTTPS) server can be run on the display device 200 side. After receiving the web content request sent by the mobile terminal 300, the display device 200 can request a web page based on the HTTPS server, generate receipt information according to the web content request, and at the same time return the receipt information to the mobile terminal 300. In this way, the mobile terminal 300 can execute the next process based on this receipt information. After step S2 is completed, the following step S3 can be executed.

[0101] Step S3: The mobile terminal 300 parses the receipt information to obtain the browser type of the browser to be screen-cast.

[0102] After the display device 200 returns the receipt information to the mobile terminal 300, the mobile terminal 300 can parse this receipt information. Figure 7 It is a schematic flow diagram of the mobile terminal parsing receipt information provided by some embodiments of this application, asFigure 7 As shown, the mobile terminal 300 can parse the browser page display content of the browser to be screen-cast and the logic script for the browser to be screen-cast to the display device 200 from the receipt information. Among them, the browser page display content is the page content presented on the browser web page, which may include relevant pictures, text and other description information. The logic script is a script that executes different logic steps at different stages of screen-casting, that is, different scripts are executed in different steps. Exemplarily, a part of the script is executed when the browser web page is opened, and a part is executed during the operation event of clicking a control or button, etc.

[0103] After parsing the browser page display content and the logic script of the browser to be screen-cast from the receipt information, the mobile terminal 300 can control the display unit 310 to display the browser page according to the browser page display content and detect the browser type of the browser to be screen-cast based on the logic script and the browser page. That is to say, in the embodiments of the present application, the browser type can be judged based on the logic script. Exemplarily, usually the same script runs in the same step. Taking the opening of a browser web page as an example, during the process of judging the browser type, if the script runs in browser A, the logic script will be automatically executed after the web page is opened, and the current browser type can be judged as browser A through the logic script; if the same script runs in browser B, the current browser can be judged as browser B through the logic script. After step S3 is executed, the following step S4 can be executed.

[0104] Step S4: The mobile terminal 300 initiates a screen-casting connection to the display device 200 according to the browser type.

[0105] To solve the browser compatibility problem, after judging the browser type based on the logic script, the mobile terminal 300 can initiate a screen-casting connection to the display device 200 according to the browser type. Figure 8 This is a schematic flow diagram of the mobile terminal provided in some embodiments of the present application for initiating a screen-casting connection to the display device according to the browser type, as Figure 8As shown, before initiating a screen mirroring connection, the mobile terminal needs to first identify the browser type of the browser to be screen mirrored. In some embodiments, according to whether an operation of initiating screen mirroring through a gesture thread is required, the browser types can be divided into a first browser type that requires a gesture thread and a second browser type that does not require a gesture thread. It can be understood that the first browser type needs to trigger screen mirroring through a gesture thread, while the second browser type does not need to trigger screen mirroring through a gesture thread. However, it should be noted that for the second browser type, it only does not need to be triggered through a gesture thread. If it is triggered through a gesture thread, the screen mirroring process can also be executed. For the first browser type, it must be triggered through a gesture thread. If the steps of the gesture thread are skipped and screen mirroring is directly performed, it cannot be achieved. For example, an error may be prompted and the screen mirroring may fail, etc.

[0106] To identify the browser type of the browser to be screen mirrored, the mobile terminal 300 can preset a browser type list in advance and determine the specific type of the browser to be screen mirrored through this browser list. Figure 9 The following is a schematic flowchart of the mobile terminal executing the operation of identifying the browser type of the browser to be screen mirrored provided by some embodiments of the present application. As Figure 9 shown, the mobile terminal 300 can preset a first browser type list in advance. All browser-related information that requires a gesture thread can be stored in this list. For example, it can include the browser name, browser version number, browser gesture thread identification code, etc. Exemplarily, the browser gesture thread identification code can be set to true and false. True represents that the browser requires a gesture thread, and false represents that the browser does not require a gesture thread. In some embodiments, the browser gesture thread identification code can also be represented by 0 and 1, etc. The present application does not make specific limitations on this.

[0107] After the first browser type list is formulated, when the web page of the browser to be screen mirrored is opened, the mobile terminal 300 can read the browser type corresponding to the browser to be screen mirrored. After detection, if the first browser type list contains this browser type, that is, the browser type of the browser to be screen mirrored is in the first browser type list, it can be determined that the browser type of the browser to be screen mirrored is the first browser type. Exemplarily, the first browser type can be a browser type that requires a gesture thread. For a browser that requires a gesture thread, the mobile terminal can perform network connection and instruction negotiation with the display device 200 when the web page of the browser to be screen mirrored is opened, but does not perform screen recording and transmission operations for the time being. Buttons and controls for performing screen mirroring can be displayed on the web page of the browser to be screen mirrored, and the screen mirroring operation can be initiated based on the determination event of this button or control.

[0108] Conversely, if the first browser type list does not include the browser type, that is, the browser type of the browser to be screen-cast is not in the first browser type list, the browser type of the browser to be screen-cast can be determined as the second browser type. Exemplarily, the second browser type can be a browser type without gesture thread requirements. For a browser type without gesture thread requirements, a screen-cast connection can be directly initiated to the display device 200 through the browser. For example, the mobile terminal 300 can initiate a network connection to the display device 200 to negotiate instructions with the display device 200, and then directly perform screen recording and transmission to implement the screen-cast operation between the mobile terminal 300 and the display device 200.

[0109] Continue to refer to Figure 8 , after the browser type of the browser to be screen-cast is determined, if the browser type is the first browser type, the mobile terminal 300 can control the display unit 310 to display an operation control for initiating a screen-cast to the display device 200, and perform a screen-cast connection to the display device 200 based on the operation control. Figure 10 Schematic diagram of the effect of the operation control for a browser to be screen-cast of the first browser type provided by some embodiments of the present application. As Figure 10 shown, for a browser to be screen-cast of the first browser type, an operation control 101 for initiating a screen-cast to the display device 200 can be set in the browser to be screen-cast. For example, the name of the operation control 101 can be Start Screen-Cast. At the same time, in order to facilitate the monitoring of the operation control, a listening process can be set for the control. In this way, only after a gesture thread is generated by the operation control, the operation of initiating a screen-cast to the display device 200 is performed, otherwise the screen-cast operation is not initiated. If the browser type is the second browser type, the browser to be screen-cast does not require a gesture thread, and the mobile terminal 300 can directly initiate a screen-cast connection to the display device 200 through the browser to be screen-cast.

[0110] Exemplarily, when initiating a screen-cast connection to the display device 200 through a browser to be screen-cast of the first browser type, in response to the user clicking the operation control of Start Screen-Cast, an application for the screen recording permission will be initiated to the display device 200. After the display device 200 permits, the mobile terminal 300 starts to record the screen image. It can be understood that if the display device 200 does not permit, the mobile terminal 300 will not perform operations such as screen image recording.

[0111] In some embodiments, when the mobile terminal 300 applies for screen mirroring to the display device 200, it can initiate instruction negotiation with the display device 200. At the same time, information such as the device name and browser model of the mobile terminal 300 can be sent to the display device 200. The display device 200 will return the result of whether screen mirroring is allowed. If the result returned by the display device 200 is that screen mirroring is allowed, the mobile terminal 300 can negotiate the screen mirroring data content with the display device 200. Exemplarily, the browser to be screen-mirrored can traverse the preset media format list, select the supported media formats, and then send media format information such as the supported resolution, frame rate, and supported video stream to the display device 200. After the sending is completed, the media format negotiated and returned by the display device 200 is received. After step S4 is executed, the following step S5 can be executed.

[0112] Step S5: In response to the event of successful screen mirroring connection, the mobile terminal 300 sends screen mirroring data to the display device 200 through the browser to be screen-mirrored, so that the display device 200 can display the screen mirroring data.

[0113] After the mobile terminal 300 initiates a screen mirroring connection to the display device 200 according to the browser type, in response to the event of successful screen mirroring connection, the mobile terminal 300 can send screen mirroring data to the display device 200 through the browser to be screen-mirrored, so that the display device 200 can display the screen mirroring data.

[0114] Figure 11 The flowchart of the mobile terminal provided by some embodiments of the present application for sending screen mirroring data to the display device 200 through the browser to be screen-mirrored is as follows Figure 11 As shown, when the mobile terminal 300 generates screen mirroring data, for browser types without gesture thread requirements, screen data can be directly recorded based on the browser to be screen-mirrored. Or, for browser types with gesture thread requirements, screen data can be recorded based on the confirmed screen mirroring event of the operation control. After the screen data recording is completed, the screen data can be encoded to generate screen mirroring data. Finally, the mobile terminal 300 sends the screen mirroring data to the display device 200. In this way, after receiving the screen mirroring data, the display device 200 can control the display 260 to display the screen mirroring data.

[0115] As can be seen from the above technical solutions, the mobile terminal 300 provided in the above embodiments can obtain the browser address of the browser to be screen-cast, and send a web content request for obtaining the web content of the browser to the display device 200 based on the browser address; receive the receipt information returned by the display device 200 according to the web content request; parse the receipt information to obtain the browser type of the browser to be screen-cast; initiate a screen-cast connection to the display device 200 according to the browser type; in response to the event of successful screen-cast connection, send screen-cast data to the display device 200 through the browser to be screen-cast, so that the display device 200 displays the screen-cast data. Through the above mobile terminal 300, for browser types without gesture thread requirements, screen data can be directly recorded based on the browser to be screen-cast, or for browser types with gesture thread requirements, screen data can be recorded based on the confirmed screen-cast event of the operation control. After the screen data recording is completed, the screen data can be encoded to generate screen-cast data. The above mobile terminal 300 can execute different screen-cast methods according to different browser types, thereby improving the compatibility of the browser.

[0116] Figure 12 The following is a schematic flowchart of a mobile terminal provided in some embodiments of the present application to initiate a screen-cast connection to a display device in another way. As Figure 12 shown, in the above embodiments, the mobile terminal 300 determines whether to automatically initiate browser screen-casting or initiate screen-casting through an operation control by pre-defining a first browser type list and reading the browser type corresponding to the browser to be screen-cast. However, in some embodiments, the browser will be upgraded during actual use, and the corresponding screen-cast method will also change. Exemplarily, a browser that originally did not require a gesture thread newly requires a gesture thread, or a browser that originally required a gesture thread is modified to not require a gesture thread. However, the pre-specified first browser type list is fixed. When the browser type changes, the method of matching according to the first browser type will no longer be applicable.

[0117] To be compatible with the situation where the browser type changes, in some embodiments, the mobile terminal 300 can obtain the model information and version information of the browser to be screen-cast. In response to the start event of the browser to be screen-cast, the mobile terminal 300 can upload the model information and version information of the browser to be screen-cast to the server 400. Among them, the server 400 stores a first identifier for indicating the requirement of a gesture thread and a second identifier for indicating no requirement of a gesture thread. After that, the mobile terminal 300 can detect the corresponding identifier of the browser to be screen-cast according to the model information and version information. If the identifier corresponding to the browser to be screen-cast is the first identifier, the mobile terminal 300 can control the display unit 310 to display an operation control for initiating screen-casting to the display device 200, and execute a screen-casting connection to the display device 200 based on the operation control; if the identifier corresponding to the browser to be screen-cast is the second identifier, the mobile terminal 300 can initiate a screen-casting connection to the display device 200 through the browser to be screen-cast.

[0118] Exemplarily, different version information and model information can be configured for the browser in the server 400, and the version information and model information are continuously updated. In this way, when the mobile terminal 300 initiates screen-casting, it can obtain the model information and version information of the browser to be screen-cast. For example, the model information and version information of the browser to be screen-cast are (Browser A, 90), and the information of (Browser A, 90) is sent to the server 400. After detection, the information about Browser A stored in the server 400 is (Browser A, 90, false), where false can be used to represent the identifier of no requirement of a gesture thread. Then, the server 400 can return the information of (Browser A, 90, false) to the mobile terminal 300. After receiving the feedback information, the mobile terminal 300 can directly initiate a screen-casting connection to the display device 200 through the browser to be screen-cast.

[0119] It can be understood that the browser type will change. When it is detected that the information of Browser A stored in the server 400 becomes (Browser A, 90, true), then true represents that the browser to be screen-cast has a requirement for a gesture thread. At this time, the mobile terminal 300 can control the display unit 310 to display an operation control for initiating screen-casting to the display device 200, and then initiate a screen-casting connection to the display device 200 based on the click operation on the operation control.

[0120] Figure 13 The flowchart shows another way for the mobile terminal provided in some embodiments of the present application to initiate a screen-casting connection to a display device, as Figure 13As shown, in some other embodiments, the mobile terminal 300 can also initiate a screen mirroring connection to the display device 200 in the following manner. In response to the opening event of the browser to be screen mirrored, the mobile terminal 300 can send a negotiation instruction to the display device 200, initiate a screen mirroring connection to the display device 200 according to the negotiation instruction, and monitor the screen mirroring result of the screen mirroring connection. If the screen mirroring result does not contain error information, the mobile terminal 300 can record screen data based on the browser to be screen mirrored to generate screen mirroring data, and send the screen mirroring data to the display device 200 so that the display device 200 can display the screen mirroring data. If the screen mirroring result contains error information, the mobile terminal 300 can identify the error category of the error information. When it is detected that the error category is a preset category, the mobile terminal 300 can control the display unit 310 to display an operation control for initiating screen mirroring to the display device 200, and initiate a screen mirroring connection to the display device 200 based on the operation control; in response to the event of successful screen mirroring connection, send the screen mirroring data to the display device 200 through the browser to be screen mirrored so that the display device 200 can display the screen mirroring data.

[0121] Exemplarily, when the mobile terminal 300 opens the web page of the browser to be screen mirrored, it can establish a network connection with the display device 200 and perform network connection and instruction negotiation. After successful negotiation, the mobile terminal 300 can automatically initiate screen mirroring to the display device 200 while monitoring the result of the screen mirroring. If the screen mirroring result does not contain error information, it indicates that the browser to be screen mirrored supports automatic initiation of screen mirroring. Then the mobile terminal 300 can start screen data, encoding, etc. to generate screen mirroring data, and send the screen mirroring data to the display device 200 so that the display device 200 can display the screen mirroring data sent by the mobile terminal 300. On the contrary, if the screen mirroring result contains error information, the category of the error information can be identified. For example, the preset category of the error information can be that when directly initiating screen mirroring in a browser with a gesture thread requirement, an error message will be prompted. In this case, if it is such an error, the mobile terminal 300 can control the display unit 310 to display an operation control for initiating screen mirroring to the display device 200, and initiate a screen mirroring connection to the display device 200 based on the operation control. At the same time, in response to the event of successful screen mirroring connection, the mobile terminal 300 can send the screen mirroring data to the display device 200 through the browser to be screen mirrored so that the display device 200 can display the screen mirroring data. In some embodiments, if the error information is not the preset category, a specific error message prompt can be displayed.

[0122] Some embodiments of the present application also provide a method for screen mirroring according to a browser. This method can be applied to the mobile terminal 300, and the mobile terminal 300 includes a display unit 310, a communication unit 320, and a processor 330. In some embodiments, the method for screen mirroring according to a browser can include the following content:

[0123] The mobile terminal 300 obtains the browser address of the browser to be screen-cast, and sends a web content request for obtaining the browser web content to the display device 200 based on the browser address;

[0124] The mobile terminal 300 receives the receipt information returned by the display device 200 according to the web content request;

[0125] The mobile terminal 300 analyzes the receipt information to obtain the browser type of the browser to be screen-cast;

[0126] The mobile terminal 300 initiates a screen-cast connection to the display device 200 according to the browser type;

[0127] In response to the event of successful screen-cast connection, the mobile terminal 300 sends screen-cast data to the display device 200 through the browser to be screen-cast, so that the display device 200 displays the screen-cast data.

[0128] It can be seen from the above technical solutions that for the method of screen-casting according to the browser provided in the above embodiments, for browser types without gesture thread requirements, screen data can be directly recorded based on the browser to be screen-cast, or for browser types with gesture thread requirements, screen data can be recorded based on the confirmed screen-cast event of the operation control. After the screen data recording is completed, the screen data can be encoded to generate screen-cast data. The present application can execute different screen-casting methods according to different browser types, thereby improving the compatibility of the browser.

[0129] Some embodiments of the present application further provide a display device 200. In some embodiments, the display device 200 may include a display 260, a communicator 220, and a controller 250. Figure 14 It is a schematic flowchart of the display device provided by some embodiments of the present application for executing the display of screen-cast data. As Figure 14 shown, wherein the display 260 is configured to display the screen-cast interface, the communicator 220 is configured to establish a screen-cast connection with the mobile terminal 300, and the controller 250 is configured to execute the following process:

[0130] The display device 200 receives the web content request sent by the mobile terminal 300, where the web content request is a request for obtaining the browser web content sent by the mobile terminal 300 to the display device 200 based on the browser address;

[0131] The display device 200 generates receipt information according to the web content request and sends the receipt information to the mobile terminal 300, and the receipt information includes the browser type;

[0132] The display device 200 receives the screen-cast connection sent by the mobile terminal 300 according to the browser type;

[0133] In response to the event of successful screen mirroring connection, the display device 200 receives the screen mirroring data sent by the mobile terminal 300, and controls the display 260 to display the screen mirroring data.

[0134] As can be seen from the above technical solutions, the display device 200 provided in the above embodiments can receive the web content request sent by the mobile terminal 300, generate a receipt message according to the web content request, and send the receipt message to the mobile terminal 300. Among them, the receipt message includes the browser type. After that, the display device 200 can receive the screen mirroring connection sent by the mobile terminal 300 according to the browser type. In response to the event of successful screen mirroring connection, the display device 200 can receive the screen mirroring data sent by the mobile terminal 300 and control the display 260 to display the screen mirroring data. This application can execute different screen mirroring methods according to different browser types of the mobile terminal 300, and cooperate with the mobile terminal 300 and the display device 200 to improve the compatibility of the browser.

[0135] Figure 15 It is a schematic diagram of the negotiation process between the display device and the mobile terminal provided in some embodiments of the present application. As Figure 15 shown, in some embodiments, the display device 200 can receive the negotiation instruction sent by the mobile terminal 300 and screen the data format of the data to be screen mirrored according to the negotiation instruction. Exemplarily, the browser to be screen mirrored can traverse the preset media format list, select the supported media formats, and then send the media format information such as the supported resolution, frame rate, and supported video stream to the display device 200. After the sending is completed, after the display device 200 receives the format information sent by the mobile terminal 300, the display device 200 can select the appropriate media resolution, frame rate, format, etc. information and return it to the mobile terminal 300, so that the mobile terminal 300 can generate screen mirroring data based on the data format for the data to be screen mirrored.

[0136] In some embodiments, when selecting the media format, the display device 200 can select according to the following rules. For example, the media format needs to select the optimal playback format supported by the display device 200. The resolution and frame rate can select high resolution and high frame rate when the network is good. When the network is poor, if image quality is prioritized, high resolution and low frame rate can be selected. If smoothness is prioritized, low resolution and high frame rate can be selected, etc.

[0137] For the same and similar parts among the various embodiments in this specification, reference can be made to each other, and details will not be repeated here.

[0138] Those skilled in the art can clearly understand that the technologies in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solutions in the embodiments of the present invention, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of the various embodiments or some parts of the embodiments of the present invention.

[0139] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the various embodiments of the present application.

[0140] For the sake of convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussions are not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. According to the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A mobile terminal, characterized in that, Including: A display unit configured to display a terminal interface; A communication unit configured to establish a screen mirroring connection with a display device; A processor configured to: Obtain the browser address of the browser to be screen mirrored, and send a web content request for obtaining the browser web content to the display device based on the browser address; Receive the receipt information returned by the display device according to the web content request; Analyze the receipt information to obtain the browser type of the browser to be screen mirrored; the browser type includes a first browser type with gesture thread requirements and a second browser type without gesture thread requirements; Initiate a screen mirroring connection to the display device according to the browser type; In response to an event of successful screen mirroring connection, send screen mirroring data to the display device through the browser to be screen mirrored, so that the display device displays the screen mirroring data.

2. The mobile terminal according to claim 1, wherein The step in which the processor executes analyzing the receipt information to obtain the browser type of the browser to be screen mirrored is further configured to: Analyze the browser page display content of the browser to be screen mirrored and the logic script for the browser to be screen mirrored to perform screen mirroring to the display device from the receipt information; Control the display unit to display the browser page according to the browser page display content; Detect the browser type of the browser to be screen mirrored based on the logic script and the browser page.

3. The mobile terminal according to claim 1, characterized in that The step in which the processor executes initiating a screen mirroring connection to the display device according to the browser type is further configured to: Identify the browser type of the browser to be screen mirrored; If the browser type is the first browser type, control the display unit to display an operation control for initiating screen mirroring to the display device, and perform a screen mirroring connection to the display device based on the operation control; If the browser type is the second browser type, initiate a screen mirroring connection to the display device through the browser to be screen mirrored.

4. The mobile terminal according to claim 3, wherein The step in which the processor executes identifying the browser type of the browser to be screen mirrored is further configured to: Formulate a list of the first browser types; Read the browser type corresponding to the browser to be screen mirrored; If the list of the first browser types includes the browser type, determine that the browser type is the first browser type; If the list of the first browser types does not include the browser type, determine that the browser type is the second browser type.

5. The mobile terminal according to claim 3, characterized in that The step in which the processor executes sending screen mirroring data to the display device through the browser to be screen mirrored is further configured to: Record screen data based on the browser to be screen mirrored, or record the screen data based on a confirmed screen mirroring event for the operation control; Encode the screen data to generate screen mirroring data; Send the screen mirroring data to the display device.

6. The mobile terminal according to claim 3, characterized in that The processor is further configured to: Obtain the model information and version information of the browser to be screen mirrored; In response to the startup event of the browser to be screen mirrored, upload the model information and the version information to a server, and the server stores a first identifier for indicating a gesture thread requirement and a second identifier for indicating no gesture thread requirement; Detect the identifier corresponding to the browser to be screen-cast according to the model information and the version information; If the identifier corresponding to the browser to be screen-cast is the first identifier, control the display unit to display an operation control for initiating screen-casting to the display device, and perform a screen-casting connection to the display device based on the operation control; If the identifier corresponding to the browser to be screen-cast is the second identifier, initiate a screen-casting connection to the display device through the browser to be screen-cast.

7. The mobile terminal according to claim 1, wherein The processor is further configured to: In response to an event of opening the browser to be screen-cast, send a negotiation instruction to the display device; Initiate a screen-casting connection to the display device according to the negotiation instruction, and monitor the screen-casting result of the screen-casting connection; If the screen-casting result does not contain error information, record screen data based on the browser to be screen-cast to generate screen-casting data, and send the screen-casting data to the display device so that the display device displays the screen-casting data; If the screen-casting result contains error information, identify the error category of the error information, and if it is detected that the error category is a preset category, control the display unit to display an operation control for initiating screen-casting to the display device, and initiate a screen-casting connection to the display device based on the operation control; In response to the event of successful screen-casting connection, send screen-casting data to the display device through the browser to be screen-cast so that the display device displays the screen-casting data.

8. A method for screen mirroring according to a browser, characterized in that, Applied to a mobile terminal, the mobile terminal includes a display unit, a communication unit and a processor, and the method for screen-casting according to the browser includes: Obtain the browser address of the browser to be screen-cast, and send a web content request for obtaining the browser web content to the display device based on the browser address; Receive the receipt information returned by the display device according to the web content request; Analyze the receipt information to obtain the browser type of the browser to be screen-cast; the browser type includes a first browser type with a gesture thread requirement and a second browser type without a gesture thread requirement; Initiate a screen-casting connection to the display device according to the browser type; In response to the event of successful screen-casting connection, send screen-casting data to the display device through the browser to be screen-cast so that the display device displays the screen-casting data.

9. A display device, characterized in that, Includes: A display configured to display a screen-casting interface; A communicator configured to establish a screen-casting connection with the mobile terminal; A controller configured to: Receive a web content request sent by the mobile terminal, where the web content request is a request sent by the mobile terminal to the display device based on the browser address to obtain the browser web content; Generate receipt information according to the web content request, and send the receipt information to the mobile terminal, where the receipt information includes the browser type; the browser type includes a first browser type with a gesture thread requirement and a second browser type without a gesture thread requirement; Receive the screen-casting connection sent by the mobile terminal according to the browser type; In response to the event of successful screen mirroring connection, receive the screen mirroring data sent by the mobile terminal, and control the display to display the screen mirroring data.

10. The display device according to claim 9, wherein The controller is further configured to: Receive the negotiation instruction sent by the mobile terminal; Filter the data format of the data to be screen mirrored according to the negotiation instruction; Send the data format to the mobile terminal so that the mobile terminal generates the screen mirroring data based on the data to be screen mirrored according to the data format.

Images