Display equipment and switching method of two-way window

Abstract

The invention provides a display device and a double-channel window switching method, and the method comprises the steps: obtaining an anchor point coordinate and key information of a key instruction in response to the key instruction input by a user based on a floating window; determining an operation area of the key instruction according to the anchor point coordinates; if the operation area is a first display area, intercepting a preset number of first area images in the first target range, and when the image contents in the preset number of first area images are the same, generating a first window switching instruction and mapping the anchor point to a non-projection area according to the instruction; and if the operation area is a non-screen projection area, intercepting a preset number of second area images in a second target range, and when the image contents of the preset number of second area images are the same, generating a second window switching instruction and mapping the anchor point to the first display area according to the instruction. By detecting the state change of the boundary of the operation area where the anchor point is located, window switching is executed, and the problem that windows cannot be switched in the two-way control process is solved.

Description

Technical Field

[0001] This application relates to the technical field of display devices, and in particular, to a display device and a method for switching dual windows. Background Art

[0002] A display device is an intelligent device that can present a user interface and support user interaction. Taking a smart TV as an example, a smart TV is a display device based on Internet application technology, with an open operating system and a chip, and has an open application platform, enabling two-way human-computer interaction functions. A smart TV can integrate various functions such as audio and video, entertainment, and data to meet the diverse and personalized needs of users. For example, users can watch various movies, TV dramas, variety shows, etc. on a smart TV, and can also connect terminal devices such as mobile phones and tablets to the smart TV to achieve functions such as screen mirroring and screen projection.

[0003] To meet the user's demand for multi-device synchronous control, due to the improvement of device chips, performance, etc., some display devices support the function of dual-channel synchronous control. Among them, one channel can be connected to a High Definition Multimedia Interface (HDMI) device for reverse control of the same screen. For example, the HDMI device can be a computer device, and the other channel is used to connect to a mobile terminal device such as a mobile phone or a tablet to achieve the direction control function during screen projection. During the dual-channel control process, users can project the media playback screen of terminal devices such as computers, mobile phones, and tablets or the screen projection screen of mobile terminals onto the screen of the display device. When users perform operations on the screen, such as handwriting, adding annotations, panning, etc., these operations can also be real-time fed back to the terminal device or the mobile terminal to achieve the effect of two-way interaction and improve the user experience.

[0004] However, when implementing the dual-channel control function, since the screen projection screen and the media playback screen may be presented in different windows respectively. For example, the screen projection screen is presented in the upper playback window, and the media playback screen is presented in the lower playback window. Therefore, the upper playback window can obtain the input key commands, while the lower playback window cannot obtain the input key commands. Therefore, when implementing the dual-channel control function, the two windows cannot be switched. Summary of the Invention

[0005] Some embodiments of this application provide a display device and a method for switching dual windows to solve the problem that windows cannot be switched during dual-channel control.

[0006] In a first aspect, some embodiments of this application provide a display device, including:

[0007] A display configured to display a user interface; the user interface includes a floating window and a media playback window, the floating window is used to display the screen mirroring image of a first terminal device; the media playback window is used to display the media playback image of a second terminal device; the floating window is disposed on top of the media playback window;

[0008] A communicator configured to establish communication connections with the first terminal device and the second terminal device respectively;

[0009] A controller configured to:

[0010] In response to a key instruction input by a user based on the floating window, obtain the anchor coordinates and key information of the key instruction; the floating window includes a first display area for displaying the screen mirroring image; the screen mirroring display height of the screen mirroring image is equal to the screen height of the display; the non-screen mirroring area other than the first display area in the floating window is a transparent view; the media playback window includes a second display area for displaying the media playback image, and the second display area is calculated according to the screen size and the first display area;

[0011] Determine the operation area of the key instruction according to the anchor coordinates; the operation area is the first display area or the non-screen mirroring area;

[0012] If the operation area is the first display area, intercept a preset number of first area images within a first target range. When the image contents in the preset number of first area images are the same, generate a first window switching instruction, and map the anchor to the non-screen mirroring area according to the first window switching instruction;

[0013] If the operation area is the non-screen mirroring area, intercept a preset number of second area images within a second target range. When the image contents in the preset number of second area images are the same, generate a second window switching instruction, and map the anchor to the first display area according to the second window switching instruction.

[0014] In a second aspect, some embodiments of the present application provide a method for switching dual windows, which can be applied to the display device in the first aspect. The display device includes a display, a communicator, and a controller. The method for switching dual windows includes:

[0015] In response to a key instruction input by a user based on the floating window, obtain the anchor coordinates and key information of the key instruction; the floating window includes a first display area for displaying the screen mirroring image; the screen mirroring display height of the screen mirroring image is equal to the screen height of the display; the non-screen mirroring area outside the first display area in the floating window is a transparent view; the media playback window includes a second display area for displaying the media playback image, and the second display area is calculated according to the screen size and the first display area;

[0016] Determine the operation area of the key instruction according to the anchor coordinates; the operation area is the first display area or the non-screen mirroring area;

[0017] If the operation area is the first display area, intercept a preset number of first area images within a first target range. When the image content in the preset number of first area images is the same, generate a first window switching instruction, and map the anchor to the non-screen mirroring area according to the first window switching instruction;

[0018] If the operation area is the non-screen mirroring area, intercept a preset number of second area images within a second target range. When the image content in the preset number of second area images is the same, generate a second window switching instruction, and map the anchor to the first display area according to the second window switching instruction.

[0019] As can be seen from the above technical solutions, some embodiments of the present application provide a display device and a method for switching dual windows. In response to a key instruction input by a user based on a floating window, obtain the anchor coordinates and key information of the key instruction; determine the operation area of the key instruction according to the anchor coordinates; if the operation area is the first display area, intercept a preset number of first area images within a first target range. When the image content in the preset number of first area images is the same, generate a first window switching instruction and map the anchor to the non-screen mirroring area according to this instruction; if the operation area is the non-screen mirroring area, intercept a preset number of second area images within a second target range. When the image content in the preset number of second area images is the same, generate a second window switching instruction and map the anchor to the first display area according to this instruction. By detecting the state change of the boundary of the operation area where the anchor is located, perform window switching to solve the problem of unable to switch windows during the dual-channel control process. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] To more clearly illustrate the technical solutions in some embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required for the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0021] Figure 1 Schematic diagram of the operation scenario between the display device and the control device provided in some embodiments of the present application;

[0022] Figure 2 Hardware configuration block diagram of the display device provided in some embodiments of the present application;

[0023] Figure 3 Hardware configuration block diagram of the control device provided in some embodiments of the present application;

[0024] Figure 4 Schematic diagram of the software configuration in the display device provided in some embodiments of the present application;

[0025] Figure 5 Schematic diagram of the process of the display device executing the method for switching dual windows provided in some embodiments of the present application;

[0026] Figure 6 Schematic diagram of the layer stacking scenario of the floating window and the media playback window provided in some embodiments of the present application;

[0027] Figure 7 Schematic diagram of the distribution of the floating window provided in some embodiments of the present application;

[0028] Figure 8 Schematic diagram of the timing of the display device executing the method for controlling dual windows provided in some embodiments of the present application;

[0029] Figure 9 Schematic diagram of the effect of the display device executing the dual window control mode provided in some embodiments of the present application;

[0030] Figure 10 Schematic diagram of the range of the floating window provided in some embodiments of the present application;

[0031] Figure 11 Schematic diagram of the range of the second display area provided in some embodiments of the present application;

[0032] Figure 12 Schematic diagram of the coordinate range of the second display area provided in some embodiments of the present application;

[0033] Figure 13 Schematic diagram of the scenario of the anchor point adjustment interface provided in some embodiments of the present application;

[0034] Figure 14 Scenario schematic diagram of the first target range provided for some embodiments of the present application;

[0035] Figure 15 Scenario schematic diagram of the second target range provided for some embodiments of the present application. Detailed implementation manners

[0036] The embodiments will be described in detail below, and the examples are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following embodiments do not represent all implementation manners consistent with the present application. They are merely examples of systems and methods consistent with some aspects of the present application detailed in the claims.

[0037] It should be noted that the brief description of the terms in the present application is only for facilitating the understanding of the following described implementation manners, rather than intending to limit the implementation manners of the present application. Unless otherwise specified, these terms should be understood in their ordinary and common meanings.

[0038] The terms "first", "second", "third", etc. in the specification, claims and the above-mentioned drawings of the present application are used to distinguish similar or same-kind objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that such used terms can be interchanged under appropriate circumstances.

[0039] The terms "comprising" and "having" and any variations thereof are intended to cover but not exclude inclusion. For example, a product or device comprising a series of components does not necessarily have to be limited to all the clearly listed components, but may include other components not clearly listed or inherent to these products or devices.

[0040] The term "module" refers to any known or later-developed hardware, software, firmware, artificial intelligence, fuzzy logic or a combination of hardware or / and software code that can perform functions related to the element.

[0041] Figure 1 Schematic diagram of the operation scenario between the display device and the control device provided for some embodiments of the present application. As Figure 1 shown, the user can operate the display device 200 through touch operations, the mobile terminal 300 and the control device 100. Among them, the control device 100 can be a remote control, a stylus, etc.

[0042] In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and establish connection communication through network communication protocols to achieve one-to-one control operations and data communication purposes. It is also possible to transmit the audio and video content displayed on the mobile terminal 300 to the display device 200 to achieve synchronous display functions.

[0043] As Figure 1 also shown, the display device 200 also conducts data communication with the server 400 through various communication methods. The display device 200 is allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0044] In addition to providing a broadcast receiving television function, the display device 200 can also additionally provide an intelligent network television function with computer support functions, including but not limited to, network television, smart television, Internet Protocol Television (IPTV), etc.

[0045] Figure 2 For some embodiments of this application Figure 1 is the hardware configuration block diagram of the display device 200 in

[0046] In some embodiments, the display device 200 includes at least one of a tuner demodulator 210, a communicator 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0047] In some embodiments, the detector 230 is used to collect signals from the external environment or interact with the outside. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of environmental light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environmental scenes, user attributes, or user interaction gestures. Or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0048] In some embodiments, the display 260 includes a display screen component for presenting a picture, and a driving component for driving image display, which is used to receive the image signal output from the controller and display video content, image content, and menu manipulation interfaces, as well as user manipulation UI interfaces, etc.

[0049] In some embodiments, the communicator 220 is a component for communicating with external devices or servers 400 according to various communication protocol types. The display device 200 may be provided with multiple communicators 220 according to different supported communication methods. For example, when the display device 200 supports wireless network communication, the display device 200 may be provided with a communicator 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communicator 220 including a Bluetooth function.

[0050] The communicator 220 can enable the display device 200 to communicate with external devices or servers 400 through wireless or wired connection methods. Among them, the wired connection can connect the display device 200 with external devices through components such as data lines and interfaces. The wireless connection can connect the display device 200 with external devices through wireless signals or wireless networks. The display device 200 can directly establish a connection relationship with external devices, or can indirectly establish a connection relationship through gateways, routers, connection devices, etc.

[0051] In some embodiments, the display device 200 can also support establishing communication connection relationships with multiple external devices simultaneously. For example, the external devices can be terminal devices 500 such as mobile phones and tablets. The first terminal device 510 and the second terminal device 520 can establish a communication connection with the display device 200 by accessing the wireless local area network where the display device 200 is located. Multiple external devices can also connect to the display device 200 through different types of connection methods. For example, the first terminal device 510 connects to the display device 200 through a wireless local area network; the second terminal device 520 connects to the display device 200 through Bluetooth.

[0052] To achieve the communication connection between the display device 200 and the terminal device 500, communicators supporting the same communication type are respectively provided on the display device 200 and the terminal device 500. For example, when the display device 200 is provided with a communicator 220 including a WiFi module, the terminal device 500 should also be provided with a communicator including a WiFi module. And, while having communicators of the same type, a specific communication transmission protocol is also required between the display device 200 and the terminal device 500 for data transfer. For example, WiFi protocol, Bluetooth connection protocol, ZigBee protocol, NFC protocol, etc.

[0053] In some embodiments, the controller 250 includes a processor, a video processor, an audio processor, a graphics processor, RAM, ROM, a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0054] In some embodiments, the controller 250 and the tuner demodulator 210 may be located in different split devices, that is, the tuner demodulator 210 may also be in an external device of the main device where the controller 250 is located, such as an external set-top box, etc.

[0055] In some embodiments, the user may input a user command in a graphical user interface (GUI) displayed on the display 260, and then the user input interface receives the user input command through the graphical user interface (GUI).

[0056] In some embodiments, the user interface 280 is an interface that can be used to receive control inputs.

[0057] Figure 3 For some embodiments provided by this application Figure 1 The hardware configuration block diagram of the control device in. As Figure 3 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0058] The control device 100 is configured to control the display device 200, and can receive input operation instructions from the user, and convert the operation instructions into instructions recognizable and responsive by the display device 200, playing an intermediary role in the interaction between the user and the display device 200.

[0059] In some embodiments, the control device 100 may be an intelligent device. For example: The control device 100 can install various applications for controlling the display device 200 according to user needs.

[0060] In some embodiments, as Figure 1 As shown, after installing an application for controlling the display device 200, the mobile terminal 300 or other intelligent electronic devices can perform functions similar to those of the control device 100.

[0061] The controller 110 includes a processor 112, a RAM 113, a ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between internal components and the data processing functions between the external and internal.

[0062] Under the control of the controller 110, the communication interface 130 realizes the communication of control signals and data signals with the display device 200. The communication interface 130 may include at least one of a WiFi chip 131, a Bluetooth module 132, an NFC module 133, and other short-range communication modules.

[0063] User input / output interface 140, where the input interface includes at least one of a microphone 141, a touchpad 142, a sensor 143, a button 144, and other input interfaces.

[0064] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The communication interface 130 is configured in the control device 100, such as modules like WiFi, Bluetooth, NFC, etc., which can encode user input instructions through the Wi-Fi protocol, or the Bluetooth protocol, or the NFC protocol and send them to the display device 200.

[0065] The memory 190 is used to store various operating programs, data, and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0066] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0067] Such as Figure 4 , in some embodiments, the system is divided into four layers, from top to bottom are the application layer (abbreviated as "application layer"), the application framework layer (abbreviated as "framework layer"), the Android runtime and system library layer (abbreviated as "system runtime library layer"), and the kernel layer.

[0068] In some embodiments, at least one application program runs in the application layer. These application programs can be window programs, system setting programs, or clock programs, etc. that come with the operating system; or they can be application programs developed by third-party developers. In specific implementations, the application program packages in the application layer are not limited to the above examples.

[0069] The framework layer provides an application programming interface (API) and a programming framework for the application programs. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center, which decides the actions of the application programs in the application layer. The application programs can access the resources in the system and obtain the services of the system through the API interface during execution.

[0070] Such as Figure 4As shown in the figure, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc. Among them, the view system can design and implement the interface and interaction of the application. The view system includes lists, grids, text boxes, buttons, etc. The managers include at least one of the following modules: The Activity Manager is used to interact with all the activities running in the system; the Location Manager is used to provide access to the system location service for system services or applications; the Package Manager is used to retrieve various information related to the application packages currently installed on the device; the Notification Manager is used to control the display and clearing of notification messages; the Window Manager is used to manage the icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0071] In some embodiments, the Activity Manager is used to manage the life cycle of each application and the usual navigation back function, such as controlling the exit, opening, and backward movement of the application. The Window Manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, taking screenshots, and controlling the change of the display window (such as shrinking the display window, jittering the display, distorting the display, etc.).

[0072] In some embodiments, the system runtime layer provides support for the upper layer, i.e., the framework layer. When the framework layer is used, the Android operating system will run the C / C++ libraries included in the system runtime layer to implement the functions to be achieved by the framework layer.

[0073] In some embodiments, the kernel layer is the layer between hardware and software. As Figure 4 shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0074] The above embodiments illustrate the hardware / software architecture and function implementation of a display device 200 and other contents. The display device 200 is an intelligent device capable of presenting a user interface and supporting user interaction. Taking a smart TV as an example, a smart TV is a display device based on Internet application technology, with an open operating system and a chip, and has an open application platform, which can realize two-way human-computer interaction functions. A smart TV can integrate various functions such as audio-visual, entertainment, and data to meet the diverse and personalized needs of users. For example, users can watch various movies, TV dramas, variety shows, etc. on the smart TV, and can also connect terminal devices such as mobile phones and tablets to the smart TV to realize functions such as screen mirroring and screen projection.

[0075] To meet the user's demand for multi-device synchronous control, with the improvement of device chips, performance, etc., some display devices already have the function of dual-channel synchronous control. The emergence of the dual-channel synchronous control function can more conveniently realize the collaborative work of multiple devices, improving the device working efficiency and user experience.

[0076] In dual-channel synchronous control, one channel can be connected to a High Definition Multimedia Interface (HDMI) device for reverse control of the same screen. Among them, HDMI is a fully digital audio-visual interface that can transmit uncompressed video data and multi-channel audio signals. In dual-channel synchronous control, one HDMI interface is used to connect high-definition multimedia devices such as computer devices to realize the reverse control function of the same screen. For example, users can project the content on the computer screen onto the display device and operate on the display device. For example, handwriting, adding annotations, etc., and these operations will be real-time feedback to the computer screen, thus realizing two-way interaction. That is to say, during the dual-channel control process, users can project the media resource playback screen of terminal devices 500 such as computers, mobile phones, and tablets or the screen projection screen of mobile terminals onto the screen of the display device 200. In this way, users can view the content of multiple devices on one screen, which is convenient for comparison and analysis. At the same time, when users perform operations on the screen, such as handwriting, adding annotations, panning, etc., these operations can also be real-time feedback to the terminal device or mobile terminal to achieve the effect of two-way interaction.

[0077] The implementation of the dual-channel synchronous control function not only improves the work efficiency of users, but also brings a more rich entertainment experience to users. For example, in an educational scenario, a teacher can, through the dual-channel synchronous control function, project the courseware content on the computer screen onto the screen of the display device 200, and at the same time control the direction and size of the projected screen through the mobile terminal 300, facilitating classroom teaching. In a home entertainment scenario, a user can, through the dual-channel synchronous control function, project the screen content of devices such as a TV box and a game console onto the screen of the display device 200, and at the same time control the direction and size of the projected screen through a mobile terminal 300 such as a mobile phone, realizing a more immersive movie-watching and gaming experience.

[0078] However, when implementing the dual-channel control function, since the projected screen and the media playback screen may be presented in different windows respectively. For example, the projected screen is presented in the upper playback window, and the media playback screen is presented in the lower playback window. Therefore, the upper playback window can obtain the input key commands, while the lower playback window cannot obtain the input key commands. Thus, when implementing the dual-channel control function, the switching between the two windows cannot be achieved. For example, when the anchor point is within the range of the projected screen, the adjustable range of the anchor point is only the area range corresponding to the projected screen, and the anchor point cannot be adjusted to the area range of the media playback screen.

[0079] To solve the problem of unable to switch windows during the dual-channel control process, some embodiments of the present application provide a display device 200. The display device 200 includes a display 260, a communicator 220, and a controller 250. Among them, the display 260 is configured to display a user interface, and the user interface includes a floating window and a media playback window. The floating window is used to display the projected screen of the first terminal device; the media playback window is used to display the media playback screen of the second terminal device; the floating window is set above the media playback window; the communicator 220 is configured to establish communication connections with the first terminal device 510 and the second terminal device 520 respectively, and the controller 250 is configured to execute the switching method of the dual-channel windows. The display device 200 can determine whether there is still a movable area range by identifying the boundary range of the first display area or the non-projected area where the anchor point is located. If there is no movable area range, it means that the anchor point has reached the edge of the boundary. If the user wants to execute the window switching, after generating the window switching instruction, execute the window switching, thereby solving the problem of unable to switch windows during the dual-channel control process.

[0080] To facilitate the understanding of the technical solutions in some embodiments of the present application, the following will describe each step in detail in combination with some specific embodiments and drawings. Figure 5 Schematic flow chart of the method for the display device provided in some embodiments of the present application to execute the switching of dual-channel windows, asFigure 5 As shown, when the display device 200 performs the switching of the dual-channel window, the following steps S1 - S4 may be included, and the specific content is as follows:

[0081] Step S1: In response to a key instruction input by the user based on the floating window, obtain the anchor coordinates and key information of the key instruction.

[0082] Figure 6 Schematic diagram of the layer stacking scenario of the floating window and the media playback window provided by some embodiments of the present application. As Figure 6 shown, two windows may be set in the display device 200, namely the floating window and the media playback window. Among them, the floating window is set on the upper layer of the media playback window. The floating window and the media playback window can be used for different application displays respectively.

[0083] Figure 7 Schematic diagram of the distribution of the floating window provided by some embodiments of the present application. As Figure 7 shown, in some embodiments, the floating window includes a first display area, that is, the screen mirroring area. The first display area is used to display the screen mirroring picture, and the screen mirroring display height of the screen mirroring picture is equal to the screen height of the display 260. Combining Figure 8 , the non-screen mirroring area outside the first display area in the floating window is a transparent view. For example, this transparent view can be set through the screen mirroring application. The media playback window includes a second display area, and the second display area is used to display the media playback picture. The media playback picture may include a movie picture, a game picture, a web page picture, etc. Among them, the second display area is calculated according to the screen size and the first display area, and the determination method of the second display area will be introduced in detail later.

[0084] In order to implement the function of the display device 200 to execute dual-channel window control, in some embodiments, multiple functional modules or applications may be deployed in the display device 200. Figure 8 Schematic diagram of the timing of the method for the display device to execute dual-channel window control provided by some embodiments of the present application. As Figure 8 shown, a screen mirroring application, a live TV application, a platform module, and a middleware module may be deployed in the display device 200. At the same time, the display device 200 can also establish a communication connection relationship with an external device such as a second terminal device 520 and a screen mirroring device such as a first terminal device 510. Among them, the screen mirroring application can be used to display the screen mirroring picture and send touch events, relevant control information, etc. to the first terminal device 510. The live TV application can be used to display the media playback picture and send touch events, relevant control information, etc. to the second terminal device 520. The platform module can be used to receive events or messages sent by the screen mirroring application, the live TV application, etc., and the middleware module is used to execute media picture playback, set the size and position of the media picture, etc.

[0085] In some embodiments, the floating window can serve the screen mirroring application. Combining Figure 7 , that is, the screen mirroring area and the non-screen mirroring area can serve the screen mirroring application. The screen mirroring application can display the screen mirroring image of the first terminal device 510 in the form of a window and in the form of a floating window. After the screen mirroring application is started, the screen mirroring image can be located on the left or right side of the display device 200 such as a TV. Since it is located on the upper layer, the floating window will obtain all user key operations. At the same time, the non-screen mirroring area is set to a transparent view, so that the media playback image played in the media playback window located below it will not be affected, that is, it will not be blocked.

[0086] It should be noted that in the embodiments of the present application, since the floating window is at the top layer, it can obtain all input operation instructions and operation events. And the screen mirroring application serves the floating window. Therefore, the screen mirroring application can forward relevant instructions and operation events to other applications or modules. For example, it can be forwarded to the live TV application, the platform module, and the middleware module. After receiving the instructions and events, these applications or modules will execute their specific contents.

[0087] In some embodiments, the display device 200 further includes a device interface 240, and the device interface 240 is configured to connect an external device. In response to the plugging and unplugging signal of the external device accessing the device interface 240, the display device 200 can switch the playback channel to the data channel corresponding to the device interface 240 and obtain the media playback image through the data channel. In response to the screen mirroring instruction for displaying the screen mirroring image, the display device 200 can generate a window switching instruction, and according to the window switching instruction, control the display 260 to display the screen mirroring image in the first display area of the floating window and display the media playback image in the second display area of the media playback window.

[0088] Exemplarily, the external device can be the second terminal device 520. When the user connects the second terminal device 520 to the TV, in response to the plugging and unplugging signal accessing the device interface 240, the display device 200 can switch to the data channel corresponding to the device interface 240 through the live TV application. Combining Figure 8 , taking the user switching to the HDMI channel as an example, the switching instruction can be sent to the live TV application, and the live TV application can send the event of switching to the HDMI channel to the platform module, and the platform module then sends it to the middleware module. After receiving it, the middleware module can obtain the media playback image. When detecting the screen mirroring instruction of the first terminal device 510, the TV will generate a window switching instruction and switch to the dual-window control mode. Figure 9 This is a schematic diagram of the effect of the display device executing the dual-window control mode provided by some embodiments of the present application. Combining Figure 8 With Figure 9, in the dual-window control mode, the projection display area of the floating window shows the projection screen, and the second display area of the media playback window shows the media playback screen. Among them, the range of the second display area is smaller than the non-projection area. Below, the setting methods for the first display area and the second display area will be introduced in detail.

[0089] When the display device 200 sets the first display area, it can first obtain the screen height and screen width of the display 260, then set the projection display height of the projection screen to the screen height, and then obtain the original projection screen height and the original projection screen width of the projection screen, and calculate the projection scaling ratio according to the ratio of the original projection screen height to the screen height. Then, calculate the projection display width of the projection screen in the first display area according to the ratio of the original projection screen width to the projection scaling ratio, and finally set the first display area according to the projection display height and the projection display width.

[0090] Exemplarily, when the user watches the media playback screen in the live TV application and in response to the projection instruction, the display device 200 can first obtain the screen height and screen width of the display 260. After the projection starts, after receiving the projection protocol from the first terminal device 510, the display device 200 can set the projection display height of the projection screen to the screen height of the TV. Assume that W1 is the screen width of the TV, H1 is the screen height of the TV, W2 is the original projection screen width of the projection screen, and H2 is the original projection screen height of the projection screen. Then the projection scaling ratio Z1 = H2 / H1, that is, the ratio of the original projection screen height to the screen height. Then the projection display width of the projection screen in the first display area, that is, the projection display width W3 shown on the TV side, is W3 = W2 / Z1, and the projection display height H3 = H1. In this way, after the projection display height H3 and the projection display width W3 are calculated, the first display area can be set.

[0091] In order to obtain the operation instructions executed by the user, the projection application can be displayed on the TV screen in the form of a floating window, and the window can be set to obtain the focus of the system to receive the user's key operations. Figure 10 The following is a schematic diagram of the range of the floating window provided by some embodiments of the present application. As Figure 10 shown, in some embodiments, the width of the floating window is the screen width W1 of the TV, and the height of the floating window is the screen height H1 of the TV. On the right side of the TV is the projection screen with a projection display width of W3 and a projection display height of H1, that is, the first display area. On the left side of the projection screen is the non-projection area. The width of the non-projection area W4 = W1 - W3, and the height of the non-projection area is still H1. The non-projection area is the range of the transparent view.

[0092] When the display device 200 sets the second display area, it can first determine the non-casting area according to the screen size and the first display area. Here, the screen size includes the screen height and the screen width. Then, the non-casting area is set as a transparent view, and the media playback window and the non-casting area are aligned. After alignment, the playback picture width of the media playback picture is set to the width of the non-casting area, and the original media picture height and the original media picture width of the media playback picture are obtained. Then, the display scaling ratio is calculated according to the ratio of the original media picture width to the playback picture width, and the playback picture height of the media playback picture is calculated according to the ratio of the original media picture height to the display scaling ratio. Finally, the second display area is determined according to the playback picture height and the playback picture width.

[0093] Exemplarily, after the casting screen is displayed in the first display area, the non-casting area can be determined according to the screen size and the first display area. Refer to Figure 8 , the casting application can send the display state of the casting screen, the sizes W4 and H1 of the non-casting area to the live TV application in the form of a notification. After receiving the notification, the live TV application sets the non-casting area as a transparent view and scales the media playback picture by aligning the media playback window and the non-casting area in terms of width, that is, calculates the second display area. The playback picture width HW1 of the media playback picture is set to the width W4 of the non-casting area. Assume that the original media picture width of the media playback picture is HOW1, the original media picture height is HOH1, and the playback picture width of the media playback picture is HW1, HW1 = W4. Then, the display scaling ratio HZ1 of the media playback picture is calculated according to the ratio of the original media picture width to the playback picture width, that is, HZ1 = HOW1 / W4. When the casting screen is set on the right side of the screen, the media playback picture can be set on the left side of the screen, and its position can be set as needed. After the display scaling ratio is determined, the playback picture height HH1 of the media playback picture can be calculated according to the ratio of the original media picture height HOH1 to the display scaling ratio HZ1, that is, HH1 = H0H1 / HZ1. After calculating the above values, the live TV application can send the data to the platform module, and the platform module can then send it to the middleware module, and the second display area can be set according to the values in the middleware module. It can also set the second display area through the live TV application, and the functions of specific applications can be set according to actual needs.

[0094] Figure 11 The following is a schematic diagram of the range of the second display area provided by some embodiments of the present application. As Figure 11As shown in the figure, after the height HH1 and width HW1 of the playback screen are determined, the second display area can be determined. It can be seen that since the non-screen mirroring area is a transparent view, the content displayed in the second display area will not be blocked. In this way, when the screen mirroring screen is displayed in the first display area and the media playback screen is displayed in the second display area, the effect of simultaneous display of two windows can be presented.

[0095] To improve the display effect of the user interface, when the width HW1 of the playback screen in the second display area is equal to the width W4 of the non-screen mirroring area, the media playback screen can be centered and displayed in the non-screen mirroring area. Figure 12 Schematic diagram of the coordinate range of the second display area provided by some embodiments of the present application, as Figure 12 shown. In some embodiments, taking the upper left corner coordinates of the screen as (0, 0) as an example, the floating window range from the upper left corner to the lower right corner is the screen range, which is a rectangular area of (0, 0) to (W1, H1). The coordinate range of the screen mirroring screen display from the upper left corner to the lower right corner is a rectangular area of (W4, 0) to (W1, H1). The setting method of the second display area can be that the distance from the top of the second display area to the top of the screen is the same as the distance from the bottom of the second display area to the bottom of the screen, that is, there are black borders of the same width at the top and bottom of the second display area. This black border is the black border formed after the media playback screen is scaled, so as to ensure the centered display of the second display area. Correspondingly, when the height of the playback screen is HH1, its coordinate range from the upper left corner to the lower right corner is a rectangular area of (0, (H1 - HH1) / 2) to (W4, (H1 / 2 + HH1 / 2)). In this way, the setting of the second display area in the media playback window is completed.

[0096] Continue to refer to Figure 8 , after the first display area of the floating window and the second display area of the media playback window are set, when the user inputs a key command based on the floating window, for example, when inputting a key command through a remote control, the upper-layer floating window can obtain the anchor point coordinates and key information of the key command through the screen mirroring application, and perform subsequent operations based on the anchor point coordinates and key information. After step S1 is completed, the following step S2 can be executed.

[0097] Step S2: Determine the operation area of the key command according to the anchor point coordinates; the operation area is the first display area or the non-screen mirroring area.

[0098] When the user inputs a key command, since the floating window and the media playback window serve different applications respectively. For example, the floating window serves the screen mirroring application, and the media playback window serves the live TV application. Then the floating window at the upper layer can obtain the input key command and can detect the operation behavior and operation path of the key command. For example, whether the key command operates on the screen mirroring area or the non-screen mirroring area, and whether the operation event is a press event, a slide event or a long press event, etc. And it can distribute to different applications according to different operation areas. For example, for the operation on the screen mirroring area, it can be distributed to the screen mirroring application. For the operation on the non-screen mirroring area, it can be first distributed from the screen mirroring application to the live TV application, and then the live TV application sends it to the corresponding terminal device. Furthermore, it is distributed to different terminal devices through different applications, achieving the effect of reverse control of the terminal device.

[0099] Exemplarily, the screen mirroring area belongs to the call range of the screen mirroring application. The screen mirroring application is responsible for displaying the screen mirroring picture of the first terminal device 510 and can distribute the touch operation corresponding to the key command to the first terminal device 510, thereby realizing the reverse control of the first terminal device 510. And the non-screen mirroring area belongs to the call range of the live TV application. The live TV application is responsible for displaying the media playback picture of the second terminal device 520 and can distribute the touch operation corresponding to the key command to the second terminal device 520, thereby realizing the reverse control of the second terminal device 520.

[0100] In some embodiments, the operation area is the first display area or the non-screen mirroring area. When the display device 200 determines the operation area of the key command according to the anchor point coordinates, it can divide the first display area into a first operation area and divide the non-screen mirroring area into a second operation area. Then it can detect the falling range of the anchor point coordinates. If the anchor point coordinates fall in the first display area, it can determine that the operation area of the key command is the first operation area. If the anchor point coordinates fall in the non-screen mirroring area, it can determine that the operation area of the key command is the second operation area.

[0101] Exemplarily, the anchor coordinates can be regarded as a virtual reference point, which can represent the position of the user's finger or other input device on the screen. By detecting the falling range of the anchor coordinates, the display device 200 can determine in which operation area the user intends to execute the key instruction. If the anchor coordinates fall within the screen mirroring area, then the display device 200 can determine that the operation area of the key instruction is the first operation area, which means that the user intends to execute the key instruction within the first display area. If the anchor coordinates fall within the non-screen mirroring area, then the display device 200 can determine that the operation area of the key instruction is the second operation area. This means that the user intends to execute the key instruction within the non-screen mirroring area. In this way, the operation area of the key instruction can be determined according to the anchor coordinates. After step S2 is completed, the following step S3 can be executed.

[0102] Step S3: If the operation area is the first display area, intercept a preset number of first area images within the first target range. When the image contents in the preset number of first area images are the same, generate a first window switching instruction, and map the anchor to the non-screen mirroring area according to the first window switching instruction.

[0103] In some embodiments, the anchor coordinates can be adjusted. When entering the user interface, the anchor coordinates can have a default initial anchor position. When the anchor coordinates are within the screen mirroring area, that is, within the rectangular area of (W4, 0) to (W1, H1), if the user adjusts the anchor coordinates through the menu key, the adjustable range of the anchor is (W4, 0) to (W1, H1). That is to say, without switching the window, the adjustable range of its anchor is always (W4, 0) to (W1, H1). When switching the window, for example, switching from the first display area of the floating window to the second display area of the media playback window, the adjustable range of the anchor is the rectangular area of (0, (H1 - HH1) / 2) to (W4, (H1 / 2 + HH1 / 2)). Therefore, if you want to adjust the anchor coordinates in different windows, you need to perform the window switching operation.

[0104] To adjust the anchor coordinates, the display device 200 can obtain the anchor positioning instruction and determine the anchor coordinates of the target anchor position according to the obtained anchor positioning instruction and the key instruction input by the user. First, the display device 200 can obtain the anchor positioning instruction, extract the anchor positioning information in the anchor positioning instruction, and determine the initial anchor position in the user interface according to the anchor positioning information. Then obtain the key instruction input by the user, extract the key information in the key instruction, and then obtain the movement information of the anchor according to the key information, and determine the target anchor position according to the movement information, as well as determine the anchor coordinates of the target anchor position.

[0105] Exemplarily, the anchor positioning instruction can be an instruction input by the user through the remote control. The TV will parse this instruction and extract the anchor positioning information therefrom. The anchor positioning information may include the initial position, moving direction, distance, etc. of the target anchor. According to this information, the display device 200 can determine an initial anchor position on the user interface. Next, the display device 200 needs to obtain the key instruction input by the user. This instruction can be an instruction input by the user pressing a certain key on the remote control. The display device 200 will parse the instruction and extract the key information therefrom. Among them, the key information can include the moving direction, distance, etc. of the user instruction. According to this information, the display device 200 can calculate the moving information of the anchor and then determine the target anchor position based on the moving information.

[0106] Figure 13 Schematic diagram of the anchor adjustment interface scenario provided by some embodiments of the present application, as Figure 13 shown, in some embodiments, when entering the anchor adjustment mode, an explicit circular mark will be displayed at the anchor coordinate position on the screen, and this circular mark can move following the direction keys of the remote control. For example, the circular mark in the solid line part is the initial anchor position. After adjustment, when the remote control moves downward, the target anchor position will become the circular mark in the dashed line part, that is, the anchor will move to the target anchor position at the dashed line position.

[0107] In order to accelerate the adjustment speed of the anchor, in some embodiments, the display device 200 can adopt an acceleration strategy in the following manner. In response to the operation of long pressing the direction key, the display device 200 can gradually increase the moving speed of the anchor. For example, after recognizing that the user long presses the direction key of the remote control for 3 seconds, the moving speed of the anchor can gradually increase from 10 pixels per second to 100 pixels per second. The specific increased gradient can be set according to actual needs, and the present application does not make specific limitations.

[0108] It can be understood that after adjusting the anchor, the target anchor position will become the starting coordinate position of subsequent events. In some embodiments, the subsequent event can be the key information of the key instruction. Then the display device 200 can generate control information according to the operation area and the key information, and send the control information to the first terminal device 510 or the second terminal device 520. This control information is used to reversely control the first terminal device 510 or the second terminal device 520.

[0109] To perform the window switching operation, after determining the operation area of the key command according to the anchor coordinates, the display device 200 may execute different processes according to different operation areas. In some embodiments, if the operation area is the first display area, the display device 200 may intercept a preset number of first area images within the first target range. When the image contents in the preset number of first area images are the same, a first window switching command is generated, and the anchor is mapped to the non-cast screen area according to the first window switching command.

[0110] In some embodiments, the above key command may be a command for long-pressing a key. For example, it may be a command issued by the user by long-pressing the direction key on the remote control. Then, when switching windows, it may be a key command for long-pressing the left or right key. Exemplarily, Figure 14 The following is a schematic diagram of the scenario of the first target range provided by some embodiments of the present application. As Figure 14 shown, the first display area is the cast screen area. When the user long-presses the left key in the right cast screen area, combined with Figure 8 , based on the user's key behavior, when the screen mirroring application recognizes the long press, it can calculate the horizontal area of the first target range as shown in Figure 14 . Then, the image contents in the horizontal area range can be compared. If the image contents intercepted multiple times do not change, it indicates that the anchor may have moved to the left edge of the first display area, that is, the intersection line position with the non-cast screen area, indicating that the user may need to perform window switching at this time. That is to say, the display device 200 can determine whether to perform window switching by detecting the state change of the boundary of the operation area where the anchor is located.

[0111] In some embodiments, the display device 200 may obtain the vertical coordinate value in the anchor coordinates, set the pixel interval value, and calculate the vertical coordinate range value according to the vertical coordinate value and the pixel interval value. At the same time, it may obtain the first area coordinate range of the first display area, and determine the first target range according to the first area coordinate range and the vertical coordinate range value. It may also obtain the second area coordinate range of the non-cast screen area, and determine the second target range according to the second area coordinate range and the vertical coordinate range value.

[0112] Exemplarily, continuing to combine Figure 14 , the vertical coordinate value in the anchor coordinates is MH1. Taking the pixel interval value as 5 pixels as an example, the vertical coordinate range value is MH1 - 5 to MH1 + 5. The first area coordinate range of the first display area is (W4, 0) to (W1, H1), then the first target range is the intersection of the rectangular area (W4, 0) to (W1, H1) and the vertical coordinate range value MH1 - 5 to MH1 + 5, that is, the rectangular area (W4, MH1 - 5) to (W1, MH1 + 5).Figure 15 Schematic diagram of the scenario of the second target range provided by some embodiments of the present application, as Figure 15 shown. Based on the same judgment method as above, when the operation area is a non-screen mirroring area, the second target range is a rectangular area of (0, MH1 - 5) to (W4, MH1 + 5).

[0113] In order to determine whether the image content in the first area image is the same, in some embodiments, the display device 200 can periodically capture the pixel values within the first target range through the platform interface to obtain multiple first area images. Then, the first boundary line frame in the first area image can be extracted, and the first position and the first size of the boundary line in the first boundary line frame can be obtained. After comparison, if the first positions and the first sizes in multiple first area images are the same, it can be determined that the image content in the first area images is the same. If the first positions and / or the first sizes in multiple first area images are different, it can be determined that the image content in the first area images is different.

[0114] Exemplarily, multiple images within the first target range, that is, multiple first area images, can be obtained through the platform interface. During the process of the user long-pressing the left mouse button, the first area image can be intercepted every 600 ms and recorded in the memory of the screen mirroring application respectively. After each image is obtained, the boundary line frame in the image can be extracted, such as the boundary line frame of a control or other line frames. Taking three interceptions as an example, if the image content of the third time is the same as the position and size information of the boundary line frames such as straight lines and curves in the previous two times, the screen mirroring application can determine that the content in the first area image is the same. At this time, a first window switching instruction can be generated, and the anchor point can be switched from the screen mirroring area to the non-screen mirroring area according to the first window switching instruction. After step S3 is completed, the following step S4 can be executed.

[0115] Step S4: If the operation area is a non-screen mirroring area, intercept a preset number of second area images within the second target range. When the image content of the preset number of second area images is the same, generate a second window switching instruction, and map the anchor point to the first display area according to the second window switching instruction.

[0116] In order to determine whether the image content in the second area image is the same, in some embodiments, the display device 200 can periodically capture the pixel values within the second target range through the platform interface to obtain multiple second area images, extract the second boundary line frame in the second area image, and then obtain the second position and the second size of the boundary line in the second boundary line frame. After comparison, if the second positions and the second sizes in multiple second area images are the same, it can be determined that the image content in the first area image is the same. If the second positions and / or the second sizes in multiple second area images are different, it can be determined that the image content in the first area image is different.

[0117] Exemplarily, multiple images in the second target range, i.e., multiple second area images, can be obtained through the platform interface. During the process of the user long-pressing the right mouse button, the second area images can be intercepted once every 600 ms and recorded in the memory of the screen mirroring application respectively. After each image is obtained, the boundary wireframes in the image can be extracted, such as the boundary wireframes of controls or other wireframes. Taking three interceptions as an example, if the content of the third image is the same as the positions and size information of the boundary wireframes (such as straight lines and curves) of the previous two images, the screen mirroring application can determine that the content in the second area images is the same. At this time, a second window switching instruction can be generated, and the anchor point can be mapped to the first display area according to the second window switching instruction.

[0118] In this way, by identifying the boundary range of the first display area or the non-screen mirroring area where the anchor point is located, it can be determined whether there is still a movable area range. If there is no movable area range, it means that the anchor point has reached the edge of the boundary. If the user wants to perform window switching, the window can be switched after generating the window switching instruction, thus solving the problem of unable to switch windows during the dual-channel control process.

[0119] In order to identify the operation area where the anchor point is located and facilitate the user to view, in some embodiments, in response to the mapping completion event of the position of the anchor point to the first display area, the display device 200 can generate a first control identification instruction and identify the first outer border of the first display area. Then, according to the first control identification instruction, a first identification frame is added to the first outer border, and the first identification frame is used to identify that the first display area is in a controlled state. In response to the mapping completion event of the position of the anchor point to the non-screen mirroring area, the display device 200 can generate a second control identification instruction and identify the second outer border of the non-screen mirroring area. Then, according to the second control identification instruction, a second identification frame is added to the second outer border, and the second identification frame is used to identify that the non-screen mirroring area is in a controlled state.

[0120] At the same time, when the operation area loses the anchor point control, the display device 200 can also remove the corresponding identification frame. In some embodiments, the display device 200 can identify the adjustable range of the anchor point. If the adjustable range is the rectangular coordinate range of the first display area, the display device 200 can identify the second identification frame. And when there is a second identification frame, a second identification frame removal instruction is generated, and the second identification frame is removed according to the second identification frame removal instruction. If the adjustable range is the rectangular coordinate range of the non-screen mirroring area, the display device 200 can identify the first identification frame. And when there is a first identification frame, a first identification frame removal instruction is generated, and the first identification frame is removed according to the first identification frame removal instruction.

[0121] Exemplarily, after the anchor point switches from the non-screen mirroring area to the first display area, the anchor point of the first display area, i.e., the screen mirroring area, can be set to the anchor point coordinates when the previous screen mirroring area lost control, and the adjustable range of the anchor point changes to a rectangular area from (W4, 0) to (W1, H1). At this time, the screen mirroring application can notify the live TV application of the status change of the screen mirroring area. For example, when the screen mirroring area is in a controlled state, after receiving this notification, the live TV application can remove the original identification frame of the non-screen mirroring area and add a corresponding identification frame to the screen mirroring area. Similarly, after the anchor point switches from the first display area to the non-screen mirroring area, the anchor point of the non-screen mirroring area, i.e., the non-screen mirroring area, can be set to the anchor point coordinates when the previous non-screen mirroring area lost control, and the corresponding adjustable range of the anchor point changes to a rectangular area from (0, (H1 - HH1) / 2) to (W4, (H1 / 2 + HH1 / 2)). At this time, the live TV application can notify the screen mirroring application of the status change of the non-screen mirroring area. For example, when the non-screen mirroring area is in a controlled state, after receiving this notification, the screen mirroring application can remove the original identification frame of the screen mirroring area and, at the same time, add a corresponding identification frame to the non-screen mirroring area.

[0122] As can be seen from the above technical solutions, the above embodiment provides a display device 200 that, in response to a key instruction input by a user based on a floating window, obtains the anchor point coordinates and key information of the key instruction; determines the operation area of the key instruction according to the anchor point coordinates; if the operation area is the first display area, intercepts a preset number of first area images within the first target range, and when the image content in the preset number of first area images is the same, generates a first window switching instruction, and maps the anchor point to the non-screen mirroring area according to the first window switching instruction; if the operation area is the non-screen mirroring area, intercepts a preset number of second area images within the second target range, and when the image content in the preset number of second area images is the same, generates a second window switching instruction, and maps the anchor point to the first display area according to the second window switching instruction. The display device can determine whether there is still a movable area range by identifying the boundary range of the first display area or the non-screen mirroring area where the anchor point is located. If there is no movable area range, it means that the anchor point has reached the edge of the boundary. If the user wants to perform window switching, after generating the window switching instruction, perform window switching, thereby solving the problem of inability to switch windows during the dual-channel control process.

[0123] Based on the above display device 200, some embodiments of the present application further provide a method for switching dual-channel windows, which can be applied to the display device 200 in the above embodiment. The display device 200 includes a display 260, a communicator 220, and a controller 250. In some embodiments, the method for switching dual-channel windows may include the following:

[0124] In response to a key instruction input by a user based on the floating window, obtain the anchor coordinates and key information of the key instruction; the floating window includes a first display area for displaying the screen mirroring image; the screen display height of the screen mirroring image is equal to the screen height of the display; the non-screen mirroring area outside the first display area in the floating window is a transparent view; the media playback window includes a second display area for displaying the media playback image, and the second display area is calculated according to the screen size and the first display area;

[0125] Determine the operation area of the key instruction according to the anchor coordinates; the operation area is the first display area or the non-screen mirroring area;

[0126] If the operation area is the first display area, intercept a preset number of first area images within a first target range. When the image contents in the preset number of first area images are the same, generate a first window switching instruction, and map the anchor to the non-screen mirroring area according to the first window switching instruction;

[0127] If the operation area is the non-screen mirroring area, intercept a preset number of second area images within a second target range. When the image contents in the preset number of second area images are the same, generate a second window switching instruction, and map the anchor to the first display area according to the second window switching instruction.

[0128] As can be seen from the above technical solutions, the above embodiments provide a method for switching dual windows. By identifying the boundary range of the first display area or the non-screen mirroring area where the anchor is located, it is possible to determine whether there is still a movable area range. If there is no movable area range, it means that the anchor has reached the edge of the boundary. If the user wants to execute window switching, after generating the window switching instruction, execute the window switching, thereby solving the problem that the window cannot be switched during the dual-channel control process.

[0129] For the same and similar parts between the various embodiments in this specification, they can be referred to each other and will not be elaborated here.

[0130] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be realized by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solutions in the embodiments of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods in the various embodiments or some parts of the embodiments of the present invention.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the various embodiments of the present application.

[0132] For ease of explanation, the above description has been presented in connection with specific embodiments. However, the above exemplary discussions are not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Many modifications and variations are possible in light of the above teaching. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations suitable for specific use considerations.

Claims

1. A display device, characterized in that, Comprising: A display configured to display a user interface; the user interface includes a floating window and a media playback window, and the floating window is used to display the screen mirroring image of a first terminal device; The media playback window is used to display the media playback image of a second terminal device; The floating window is disposed on the upper layer of the media playback window; A communicator configured to establish communication connections with the first terminal device and the second terminal device respectively; A controller configured to: In response to a key instruction input by a user based on the floating window, obtain the anchor coordinates and key information of the key instruction; the floating window includes a first display area for displaying the screen mirroring image; the screen mirroring display height of the screen mirroring image is equal to the screen height of the display; the non-screen mirroring area other than the first display area in the floating window is a transparent view; the media playback window includes a second display area for displaying the media playback image, and the second display area is calculated according to the screen size and the first display area; Determine the operation area of the key instruction according to the anchor coordinates; The operation area is the first display area or the non-screen mirroring area; If the operation area is the first display area, intercept a preset number of first area images within a first target range. When the image contents in the preset number of first area images are the same, generate a first window switching instruction, and map the anchor to the non-screen mirroring area according to the first window switching instruction; If the operation area is the non-screen mirroring area, intercept a preset number of second area images within a second target range. When the image contents in the preset number of second area images are the same, generate a second window switching instruction, and map the anchor to the first display area according to the second window switching instruction.

2. The display device according to claim 1, wherein The controller is further configured to: Obtain the vertical coordinate value in the anchor coordinates; Set a pixel interval value, and calculate a vertical coordinate range value according to the vertical coordinate value and the pixel interval value; Obtain the first area coordinate range of the first display area, and determine the first target range according to the first area coordinate range and the vertical coordinate range value; Obtain the second area coordinate range of the non-screen mirroring area, and determine the second target range according to the second area coordinate range and the vertical coordinate range value.

3. The display device according to claim 2, wherein The controller is further configured to: Periodically capture the pixel values within the first target range through a platform interface to obtain multiple first area images; Extract the first boundary line frame in the first area image; Obtain the first position and first size of the boundary line in the first boundary line frame; If the first positions and the first sizes in multiple first area images are all the same, determine that the image contents in the first area images are the same; If the first positions and / or the first sizes in multiple first area images are different, determine that the image contents in the first area images are different.

4. The display device according to claim 2, wherein, The controller is further configured to: Periodically capture pixel values within the second target range through the platform interface to obtain multiple images of the second region; Extract the second boundary line frame in the second region image; Obtain the second position and second size of the boundary line in the second boundary line frame; If the second positions and the second sizes in multiple images of the second region are the same, determine that the image content in the first region image is the same; If the second positions and / or the second sizes in multiple images of the second region are different, determine that the image content in the first region image is different.

5. The display device according to claim 2, wherein The controller is further configured to: Generate a first control identification instruction in response to the mapping completion event of the position of the anchor point to the first display area; Identify the first outer border of the first display area; Add a first identification frame to the first outer border according to the first control identification instruction, and the first identification frame is used to identify that the first display area is in a controlled state; Generate a second control identification instruction in response to the mapping completion event of the position of the anchor point to the non-screen mirroring area; Identify the second outer border of the non-screen mirroring area; Add a second identification frame to the second outer border according to the second control identification instruction, and the second identification frame is used to identify that the non-screen mirroring area is in a controlled state.

6. The display device according to claim 5, wherein The controller is further configured to: Identify the adjustable range of the anchor point; If the adjustable range is the rectangular coordinate range of the first display area, identify the second identification frame, and when the second identification frame exists, generate a second identification frame removal instruction and remove the second identification frame according to the second identification frame removal instruction; If the adjustable range is the rectangular coordinate range of the non-screen mirroring area, identify the first identification frame, and when the first identification frame exists, generate a first identification frame removal instruction and remove the first identification frame according to the first identification frame removal instruction.

7. The display device according to claim 2, wherein The controller is further configured to: Obtain the screen height and screen width of the display; Set the screen mirroring display height of the screen mirroring picture to the screen height; Obtain the original screen mirroring picture height and original screen mirroring picture width of the screen mirroring picture; Calculate the screen mirroring scaling ratio according to the ratio of the original screen mirroring picture height to the screen height; Calculate the screen mirroring display width of the screen mirroring picture in the first display area according to the ratio of the original screen mirroring picture width to the screen mirroring scaling ratio; Set the first display area according to the screen mirroring display height and the screen mirroring display width.

8. The display device according to claim 7, characterized in that The controller is further configured to: Determine the non-screen mirroring area according to the screen size and the first display area; the screen size includes the screen height and the screen width; Set the non-screen mirroring area as a transparent view and align the media playback window and the non-screen mirroring area; Set the playback picture width of the media playback picture to the width of the non-screen mirroring area; Obtain the original media picture height and original media picture width of the media playback picture; Calculate the display scaling ratio according to the ratio of the original media picture width to the playback picture width; Calculate the playback screen height of the media asset based on the ratio of the original media asset screen height to the display zoom ratio; Determine a second display area based on the playback screen height and the playback screen width.

9. The display device according to claim 8, wherein, The step in which the controller executes to determine the operation area of the key command according to the anchor coordinates is further configured to: Divide the first display area into a first operation area; Divide the non-casting area into a second operation area; If the anchor coordinates fall within the first display area, determine that the operation area of the key command is the first operation area; If the anchor coordinates fall within the non-casting area, determine that the operation area of the key command is the second operation area.

10. A method for switching between dual channels, characterized in that, Applied to the display device according to any one of claims 1-9, the display device includes a display, a communicator and a controller, and the method for switching dual-channel windows includes: In response to a key command input by the user based on the floating window, obtain the anchor coordinates and key information of the key command; the floating window includes a first display area for displaying the cast screen; the cast display height of the cast screen is equal to the screen height of the display; the non-casting area outside the first display area in the floating window is a transparent view; the media asset playback window includes a second display area for displaying the media asset playback screen, and the second display area is calculated according to the screen size and the first display area; Determine the operation area of the key command according to the anchor coordinates; the operation area is the first display area or the non-casting area; If the operation area is the first display area, intercept a preset number of first area images within a first target range. When the image content in the preset number of first area images is the same, generate a first window switching command, and map the anchor to the non-casting area according to the first window switching command; If the operation area is the non-casting area, intercept a preset number of second area images within a second target range. When the image content in the preset number of second area images is the same, generate a second window switching command, and map the anchor to the first display area according to the second window switching command.

Images