Display device and view display method

Abstract

The invention provides a display device and a view display method, and the method can respond to a display instruction indicating to display a target view on a second display, obtain the display resolution of the second display, and calculate view parameters according to the display resolution. Wherein the second display is an external display of the display equipment, the display resolution is obtained based on a display identification mark of the second display, and the view parameter is used for setting a display scale of the target view on the second user interface; the display scale of the target view on the second user interface is associated with the display scale of the target view on the first user interface. And creating a target view based on the view parameters, and adding the target view to the second user interface. According to the method, when the displays with different resolutions are connected, the view parameters of the views can be adjusted according to the resolutions of the displays, and the problem that the display scales of the views are different due to different resolutions of screens in a double-screen system is solved.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and a view display method. Background Art

[0002] Display devices refer to terminal devices that can output specific display images, which can be terminal devices such as smart TVs, communication terminals, smart advertising screens, projectors, etc. Taking smart TVs as an example, smart TVs are based on Internet application technology, have open operating systems and chips, have open application platforms, can realize two-way human-computer interaction functions, and are TV products that integrate multiple functions such as audio and video, entertainment, and data to meet the diverse and personalized needs of users.

[0003] The display device can form a dual-screen system by connecting an external monitor to its own monitor to achieve a dual-screen display mode. In some interactive scenarios, the display device supports dual-screen extended display, where the main screen (the display of the display device itself) is connected to the secondary screen (the external monitor) to achieve dual-screen display. Sidebar shortcut operation buttons can be displayed on both sides of the secondary screen to quickly operate the content displayed on the secondary screen, allowing users to operate the applications and display content on both screens more intuitively and quickly.

[0004] However, the above interaction is limited by the resolution of the main screen and the secondary screen. If the resolutions of the main screen and the secondary screen are different, the view added to the secondary screen will be displayed on the secondary screen according to the resolution of the main screen, making the proportion of the view displayed on the secondary screen inconsistent, affecting the user experience. Summary of the invention

[0005] The present application provides a display device and a dual-screen display method to solve the problem of different display ratios of views caused by different resolutions of a main screen and a secondary screen in a dual-screen system.

[0006] In a first aspect, the present application provides a display device, including a first display, a device interface, and a controller. The first display is configured to display a first user interface, the first user interface includes a target view; the device interface is configured to connect to a second display, the second display is used to display a second user interface, and the display resolution of the first display is different from the display resolution of the second display; the controller is configured to execute the following program steps:

[0007] In response to a display instruction instructing to display a target view on the second display, acquiring a display resolution of the second display, where the display resolution is obtained based on a display identification identifier of the second display;

[0008] Calculating view parameters according to the display resolution, the view parameters being used to set a display ratio of the target view on the second user interface; the display ratio of the target view on the second user interface is associated with the display ratio of the target view on the first user interface;

[0009] The target view is created based on the view parameters, and the target view is added to the second user interface.

[0010] In a second aspect, the present application further provides a view display method, which is applied to the above-mentioned display device, and the method includes:

[0011] In response to a display instruction instructing to display a target view on a second display, acquiring a display resolution of the second display, where the display resolution is obtained based on a display identification identifier of the second display;

[0012] Calculating view parameters according to the display resolution, the view parameters are used to set a display ratio of the target view on a second user interface; the display ratio of the target view on the second user interface is associated with the display ratio of the target view on a first user interface, the first user interface is an interface displayed on the first display, and the second user interface is an interface displayed on the second display;

[0013] The target view is created based on the view parameters, and the target view is added to the second user interface.

[0014] It can be seen from the above technical solutions that the present application provides a display device and a view display method, which can obtain the display resolution of the second display in response to a display instruction indicating that the target view is displayed on the second display, and calculate the view parameters according to the display resolution. Among them, the second display is a display external to the display device, the display resolution is obtained based on the display identification identifier of the second display, and the view parameters are used to set the display ratio of the target view on the second user interface, and the display ratio of the target view on the second user interface is associated with the display ratio of the target view on the first user interface. Then, the target view is created based on the view parameters, and the target view is added to the second user interface. The method can adjust the view parameters of the view according to the resolution of the display when connecting displays with different resolutions, so as to solve the problem that the display ratio of the view is different due to different screen resolutions in the dual-screen system. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 A schematic diagram of a usage scenario of a display device provided in some embodiments of the present application;

[0017] Figure 2 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0018] Figure 3 A schematic diagram of the connection relationship of external terminal devices provided in some embodiments of the present application;

[0019] Figure 4 A schematic diagram of the hardware configuration of a control device provided in some embodiments of the present application;

[0020] Figure 5 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0021] Figure 6 A structural diagram of a dual-screen display system provided for some embodiments of the present application;

[0022] Figure 7 A diagram showing the relationship between touch data transmission in a dual-screen display system provided in some embodiments of the present application;

[0023] Figure 8 A schematic diagram of a dual-screen display effect provided in some embodiments of the present application;

[0024] Fig. 9 A schematic diagram of a sidebar tool provided in some embodiments of the present application;

[0025] Fig.10 A schematic diagram of the display effect of the sidebar on the main screen and the secondary screen with different resolutions provided in some embodiments of the present application;

[0026] Fig.11 A schematic diagram of a flow chart of a view display method provided in some embodiments of the present application;

[0027] Fig.12 A schematic diagram of a process for calculating view parameters provided in some embodiments of the present application;

[0028] Fig.13 A schematic diagram of a process for distinguishing and displaying identification marks provided in some embodiments of the present application;

[0029] Fig.14 A schematic diagram of the display effect of the first display and the second display displaying the sidebar provided in some embodiments of the present application;

[0030] Fig.15 A schematic diagram of the display effect of the second display displaying the extended menu provided in some embodiments of the present application;

[0031] Fig.16 A schematic diagram of a first display and a second display displaying different application windows provided in some embodiments of the present application;

[0032] Fig.17 A schematic diagram of a process of creating a first application set and a second application set provided in some embodiments of the present application;

[0033] Fig.18 A schematic diagram of an application list, a first application set, and a second application set provided for some embodiments of the present application;

[0034] Fig.19 A schematic diagram of a multi-screen control process provided by some embodiments of the present application;

[0035] Fig. 20 A schematic diagram of an exit process of an application whose application window is displayed on a first display provided in some embodiments of the present application;

[0036] Fig.21 A schematic diagram of the exit process of an application whose application window is displayed on a second display provided in some embodiments of the present application. DETAILED DESCRIPTION

[0037] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0038] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0039] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0040] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0041] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0042] Figure 1 This is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG. 1 , a user can operate the display device 200 through the mobile terminal 300 and the control device 100 .

[0043] In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

[0044] like Figure 1 As also shown in FIG. 4 , the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0045] In addition to providing the broadcast receiving television function, the display device 200 may also provide an intelligent network television function with computer support functions, including but not limited to network television, smart television, Internet Protocol television (IPTV), etc.

[0046] Figure 2 Some embodiments of the present application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0047] In some embodiments, the display device 200 includes at least one of a tuner 210 , a communicator 220 , a detector 230 , a device interface 240 , a controller 250 , a display 260 , an audio output interface 270 , a memory, a power supply, and a communication device interface 280 .

[0048] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0049] In some embodiments, the display 260 includes a display screen component for presenting images, and a driving component for driving image display, which is used to receive image signals output from a controller, display video content, image content, and a menu control interface component and a user control UI interface, etc.

[0050] The display 260 may be a liquid crystal display, an OLED display, or a projection display, and may also be a projection device and a projection screen.

[0051] In some embodiments, the display device 200 may also support touch interaction, that is, a touch interaction module may be provided on the display device 200, and the touch interaction module may be a built-in module or an external module. For example, in order to implement touch interaction operations, a touch panel (TP) may be provided on the display 260 of the display device 200, and the touch panel may receive touch signals input by fingers, contacts, styluses, etc., and execute touch interaction responses according to a pre-set interaction strategy. Of course, the display device 200 may also be a touch-integrated screen.

[0052] In some embodiments, the communicator 220 is a component for communicating with an external device or a server 400 according to various communication protocol types.

[0053] In some embodiments, the device interface 240 may include, but is not limited to, any one or more of the following interfaces: a high-definition multimedia interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input / output interface formed by the above multiple interfaces.

[0054] For example, in order to enrich the display types of the display device 200, the HDMI interface can connect devices with video and audio input / output functions, and implement the corresponding audio and video output / input functions through the HDMI driver in the hardware layer. The USB interface can connect devices with signal and data transmission functions and interactive functions, and access signals, transmit data, and perform specific interactive actions based on the USB driver in the hardware layer.

[0055] In some embodiments, the display device 200 can support multiple interface accesses and perform corresponding functions according to the specific device type accessed. For example, the display device 200 can be connected to another display or terminal device 500 via an HDMI interface to form display functions such as dual screen, split screen, screen transmission, conference, whiteboard, NFC, etc. The display device 200 can also be connected to the terminal device 500 via a USB interface, and activate the open pluggable specification (OPS) channel to connect the terminal device 500. When the terminal device 500 is connected through the OPS channel, since the display device 200 has an independent operating system (system A), and the terminal device 500 can also have an independent operating system (system B), a dual system mode can be formed.

[0056] In some embodiments, the display device 200 and / or the connected terminal device 500 may both support touch interaction. When the display device 200 and the connected terminal device 500 both support touch interaction operations, a data transmission and conversion module may be provided in the display device 200 to transmit and convert touch data generated during the touch interaction process, so as to achieve channel reverse control, that is, to control the display device 200 through the touch-interactive terminal device 500. For example, Figure 3 As shown, a switch module can be set in the touch interaction system, and the TP module with touch function can be connected to the main control system (system A) of the display device 200 through the USB interface, and at the same time, connected to the terminal device 500 (system B), the external host, and the type-c device. In this way, the OPS channel formed by the HDMI interface can transmit display content data, and the touch data can be transmitted through the conversion module and the USB interface.

[0057] In some embodiments, the controller 250 includes a processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0058] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0059] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0060] In some embodiments, the communication device interface 280 is an interface that can be used to receive control input.

[0061] Figure 4 Some embodiments of the present application provide Figure 1 The hardware configuration diagram of the control device in the figure. Figure 3 As shown, the control device 100 includes a controller 110 , a communication interface 130 , a user input / output interface, a memory 190 , and a power supply 180 .

[0062] The control device 100 is configured to control the display device 200 , and can receive user input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200 .

[0063] In some embodiments, the control device 100 may be a smart device, for example, the control device 100 may be installed with various applications for controlling the display device 200 according to user needs.

[0064] In some embodiments, Figure 1 As shown, the mobile terminal 300 or other intelligent electronic devices can play a similar function as the control device 100 after installing the application for controlling the display device 200 .

[0065] The controller 110 includes a processor 112, a RAM 113, a ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between the internal components and the external and internal data processing functions.

[0066] The communication interface 130 implements communication of control signals and data signals with the display device 200 under the control of the controller 110. The communication interface 130 may include at least one of other near field communication modules such as a WiFi chip 131, a Bluetooth module 132, and an NFC module 133.

[0067] The user input / output interface 140 , wherein the input interface includes at least one of other input interfaces such as a microphone 141 , a touch panel 142 , a sensor 143 , and a button 144 .

[0068] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with a communication interface 130, such as a WiFi, Bluetooth, NFC or other module, which can encode the user input command through the WiFi protocol, Bluetooth protocol, or NFC protocol and send it to the display device 200.

[0069] The memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0070] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0071] Figure 5 Some embodiments of the present application provide Figure 1 Schematic diagram of software configuration in the display device. In some embodiments, the system of the display device 200 can be divided into three layers, namely, application layer, middleware layer and hardware layer from top to bottom.

[0072] The application layer mainly includes commonly used applications on TV and application frameworks. Common applications are mainly applications developed based on browsers, such as HTML5 APPs and native APPs.

[0073] The Application Framework is a complete program model that has all the basic functions required by standard application software, such as file access, data exchange, etc., as well as the user interfaces of these functions (toolbars, status bars, menus, dialog boxes).

[0074] Native apps can support online or offline, message push or local resource access.

[0075] The middleware layer includes various TV protocols, multimedia protocols, system components and other middleware. The middleware can use the basic services (functions) provided by the system software to connect various parts of the application system or different applications on the network, and can achieve the purpose of resource sharing and function sharing.

[0076] The hardware layer mainly includes the hardware abstraction layer (HAL) interface, hardware and drivers. The HAL interface is a unified interface for all TV chips to connect, and the specific logic is implemented by each chip. Drivers mainly include: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver.

[0077] In some embodiments, the display device 200 can connect an external display via HDMI output to form a dual-screen system with the display 260 itself to achieve a dual-screen display mode. Figure 6 As shown, the dual-screen system may include an image generation (ImagePr oducer) module, an image rendering (surface flinger) module, a synthesis (Composer) module, a video process (Video Processing Unit hardware, VPU hardware) module, a main screen (panel screen) and a secondary screen (HDMI screen).

[0078] During the display process, the media player (media player), camera peripheral (camera per view) and graphics interface (openGL ES) of the image generation module can generate image content and pass the image content to the image rendering module. The image rendering module includes a primary display unit (primary display) and an extended display unit (external display). The primary display unit is used to control the display content of the main screen, that is, the main display unit renders the image content through the user interaction layer (UI layer) or the video layer (video layer) according to the image content. In order to obtain a better display effect, the image content of the UI layer can be rendered by the GPU component, so as to form a picture frame for display in the synthesis module. Among them, the synthesis module can form the picture content by software or hardware. For example, the synthesis module can be a hardware synthesis (Hardware Composer, HWC) module.

[0079] In the synthesis module, the picture frames can be stored through the frame buffer and the video buffer, and the picture content can be formed in the corresponding layers through the video process module. That is, the image frames generated by the video playback process form the picture content through the video plane, and the image frames of other image pictures (such as touch tracks, graphics, etc.) form the picture content through the primary OSD plane. Finally, the display device 200 forms the final picture through the blending unit of the video process module and displays it on the main screen.

[0080] The extended display unit is used to control the display content of the secondary screen, that is, after the image generation module passes the image content to the image rendering module, the extended display unit also renders the image content through the UI layer or the video layer. According to the display process of the main screen, the image frame is formed by the image rendering module, the synthesis module and the video process module respectively, and displayed on the secondary screen.

[0081] In some embodiments, in order to obtain a better picture display effect, an additional extended display layer (extemal OSD plane) can be set in the video process module of the display device 200 on the basis of the main OSD layer and the video layer. The extended display layer can be configured with different layer characteristics according to specific display effect requirements. For example, the bitmap refresh mode of the extended display layer can be modified so that the extended display layer is refreshed in real time according to the region, thereby improving the display speed of the graphics. At this time, the extended display layer can be used as an acceleration layer in scenes with high dynamic display effects.

[0082] Taking the display device 200 running a whiteboard drawing application as an example, in order to improve the hand tracking of the line drawing process, after detecting the user's touch operation, the touch track can be drawn in real time on the extended display layer (acceleration layer). The display level of the acceleration layer can be higher than other layers, and when the user completes the touch input, the line drawn by the acceleration layer is updated to other layers, thereby reducing input delay and improving the real-time display effect of the whiteboard application drawing process.

[0083] In some embodiments, the extended display layer can also work together with other graphics to improve display quality. For example, the acceleration layer can respond to the touch events input by the user at the same time as the main OSD layer, and draw graphics separately. However, since the acceleration layer is displayed above the main OSD layer, the graphics drawn by the acceleration layer can block the graphics drawn by the OSD layer to maintain real-time response speed. After the user inputs the touch interaction action, the content displayed by the acceleration layer can be released directly, and the OSD layer directly forms the drawn graphics, so there is no need for the acceleration layer to refresh the drawn graphics to the OSD layer, reducing the graphics mixing time. It not only improves the real-time display effect of the graphics drawing process, but also improves the final presentation quality of the graphics drawing.

[0084] Based on the above dual-screen display system, the user can control the display device 200 to execute different display strategies as needed. For example, the main screen is connected to the secondary screen through HDMIOUT to realize dual-screen display, and the secondary screen integrates a touch component to realize touch-control of the secondary screen content. In addition, the main screen and the secondary screen are also connected via USB, and the USB connection channel can be used to transmit touch data, that is, the secondary screen touch is connected to the USB of the main screen. After the connection is completed, the display device 200 can control the display of the main screen and the secondary screen. Among them, the main screen and the secondary screen can support the same display, that is, the display 260 (main screen) of the display device 200 and the external display display the same content. The main screen and the secondary screen can also support different displays, that is, the main screen and the secondary screen display different contents. Different display is also called extended display, and the main scenarios include: the physical signal source screen is displayed on the secondary screen; the player screen is displayed on the secondary screen; the wireless screen transmission screen displays two screen transmission screens; the conference application displays the shared content on the secondary screen, etc.

[0085] When the secondary screen connected to the display device 200 also supports touch interaction operations, the user can also reversely control the display device 200 or the external terminal device 500 by inputting touch operations on the secondary screen. That is, in some embodiments, after the user inputs a touch interaction action on the secondary screen, the secondary screen will pass the touch data in the touch event to the display device 200 through the USB channel. When the display device 200 is also connected to the external terminal device 500, the touch data can be forwarded to the USB channel connected to the terminal device 500 through the conversion module built into the display device 200, so that the terminal device 500 responds to the touch event input by the user on the secondary screen.

[0086] For example, Figure 7As shown, the user can connect a display with a touch function to the HDMI1 interface of the display device 200 based on system A as the secondary screen of the display device 200. At the same time, the OPS computer based on system B is connected to the HDMI2 interface of the display device 200 as the external terminal device 500 of the display device 200. In addition, the display device 200 is also connected to the secondary screen device through the USB1 interface and the OPS computer through the USB2 interface to form an interactive system that can support the reverse control function. The display 260 of the display device 200 serves as the main screen. During the display process, the main screen and the secondary screen can respectively display the screen content output by the OPS computer in the same display or different display modes.

[0087] When the user inputs a touch operation on the main screen, the display device 200 can transmit the touch data generated by the touch operation to the OPS computer through the USB2 interface channel, and the OPS computer responds to the click touch data. When the user touches and clicks the interactive control on the secondary screen, the generated click touch data can be transmitted to the display device 200 through the USB1 interface channel, and then transmitted to the USB2 interface channel through the conversion module (switch) inside the display device 200, and then transmitted to the OPS computer, and the OPS computer responds to the click touch data.

[0088] It should be noted that the touch reverse control function can generate different interaction logics according to different application scenarios. That is, the reverse control can include the following scenarios:

[0089] There is only a main screen and a secondary screen, that is, the display device 200 is connected to an external display to access the secondary screen, and the display device 200's own display 260 is used as the main screen. In this scenario, the display device 200 needs to distinguish between the touch of the whole device and the external USB touch, configure the touch of the whole device as internal, and configure the external USB touch as external to achieve dual-screen touch control, and the application dynamically adjusts the display according to whether the extended mode is turned on.

[0090] OPS device, main screen and sub-screen, that is, the display device 200 is connected to an external monitor and an OPS device at the same time, the external monitor is used as the sub-screen, the display 260 built into the display device 200 is used as the main screen, and the OPS device is used as the signal source. In this scenario, it is necessary to distinguish the display screen of the OPS signal. When the OPS signal is displayed on the sub-screen, the USB needs to be switched to the OPS channel to realize the reverse control function of the sub-screen to the OPS channel, and the main screen's external touch function needs to be turned off. When the OPS signal is displayed on the main screen, the main screen touch needs to be turned on and switched to the OPS device. Because the USB channel has been switched to the OPS device, the touch of the sub-screen will also be recognized and responded to by the OPS.

[0091] For this scenario, the touch component needs to support the function of turning off the current channel output through an external channel. When switching from USB to OPS channel, it is necessary to determine whether USB touch is connected. If USB touch is connected, the output must be turned off first, and then switched to OPS channel. The hardware design of the above touch reverse control system can control all USB3.0 interfaces through the same switch.

[0092] The following describes the display process based on the above-mentioned dual-screen system through specific examples. It should be understood that the following display process is only an example and does not limit the specific display method of the dual-screen system. Other display methods and interface contents that can be associated with by technical personnel in this field based on the following examples all belong to the protection scope of this application.

[0093] like Figure 8 As shown, according to the above dual-screen system connection method, after connecting to the external screen via an HDMI cable, the dual-screen mode can be started. In the dual-screen mode, the display device 200 can set the display mode of the secondary screen, and the display mode defaults to the extended mode, that is, the secondary screen displays the content of the main screen in an extended manner, which is equivalent to increasing the display area of ​​the main screen. Therefore, the wallpaper displayed on the secondary screen can automatically change with the wallpaper displayed on the main screen.

[0094] After starting the dual-screen mode, the main screen and the secondary screen can display the user interface according to the set display mode. In order to facilitate page control, views can be displayed on the main screen and the secondary screen respectively. In the display device 200, a view is an element of the user interface that can be used to display data, layouts, and controls, such as buttons, text boxes, lists, cards, etc. For example, a view can be as follows: Fig. 9 The sidebar and its associated extended menu shown in FIG. That is, the interface displayed on the main screen and the secondary screen may also include a sidebar, which can be used to control the content displayed on another screen under one screen environment. For example, Fig. 9 As shown, the sidebar is an arrow icon displayed on both sides of the interface, and the user can click the icon to call out the extended menu. The extended menu can include quick function options such as "Back", "Home", "Settings", and "Switch". The user can click any option icon to control the dual-screen display process.

[0095] In some embodiments, the sidebar can control its display process through an independent application, that is, the display device 200 can run a sidebar application. After the sidebar application is started, the sidebar view can be laid out in the two screens respectively to display the sidebar on the two screens. And the sidebar application can detect the status of the two screens in real time, and adjust the display form of the sidebar and the content of the extended menu in real time in combination with the detected status. For example, the sidebar can be hidden and displayed according to whether the user inputs an operation. When it is detected that the user performs a touch operation on the main screen or the secondary screen, the sidebar is displayed; after the user does not operate for 3s, the sidebar is hidden. For another example, the sidebar application can detect the display resolution of the main screen and the secondary screen in real time, and adjust the display size of the sidebar control in real time according to the display resolution. For another example, the display device 200 can detect the applications running on the main screen and the secondary screen respectively, and control the application that responds to the operation according to the application focus state, so that the touch operation acts on the appropriate application to improve the user experience.

[0096] It should be noted that the sidebar application can also dynamically adjust the display mode according to the interaction forms supported by the main screen and the secondary screen. For example, when the main screen supports touch interaction operations and the secondary screen does not support touch interaction operations, the sidebar application can control the display of the sidebar controls only in the main screen user interface; conversely, when the main screen does not support touch interaction operations and the secondary screen supports touch interaction operations, the sidebar application can control the display of the sidebar controls only in the secondary screen user interface. When both the main screen and the secondary screen support touch interaction operations, the sidebar application can control the display of the sidebar controls in the user interfaces of both the main screen and the secondary screen.

[0097] Since the sidebar application is an independently running application, cross-screen control can be achieved based on the sidebar application. For example, in the sidebar-associated extended menu displayed on the main screen, a touch lock for locking or unlocking the touch function of the secondary screen may be included. The user can click the touch lock in the sidebar extended menu displayed on the main screen to lock or unlock the touch function of the secondary screen. Conversely, a control for locking or unlocking the touch function of the main screen may also be provided in the sidebar extended menu displayed on the secondary screen, so that the user can control the touch interaction function of the main screen through the secondary screen.

[0098] Among them, in order to realize the locking and unlocking of the touch interaction function of the main screen or the secondary screen, the display device 200 can set a control program in the application layer, and control the connection relationship of the hardware layer through the control program of the application layer. For example, when the user sets the touch function of the secondary screen to "lock" through the sidebar application, the display device 200 can control the hardware layer to disconnect the USB connection channel between the secondary screen and the display device 200 when it detects that the touch lock function control is activated, such as "external screen touch lock = 1", so that the touch interaction operation data received by the secondary screen cannot be transmitted to the display device 200 through the USB channel, so as to achieve the purpose of locking the touch function of the secondary screen. For another example, the application program set in the application layer of the display device 200 can act on the conversion module built into the display device 200, that is, when it is detected that the touch lock function control is activated, the control conversion module no longer forwards the touch data transmitted by the secondary screen, so that the touch function of the secondary screen can also be locked.

[0099] The sidebar application can also support operations such as adjusting, switching, and closing the content displayed on the main screen and the secondary screen. For example, the user can adjust the display mode of the main screen and the secondary screen through the sidebar, that is, in the extended screen mode, the user can adjust the main screen and the secondary screen to the copy mode with the same display content by clicking the "copy mode" control in the sidebar extended menu. In order to implement the copy mode, the display device 200 can modify the signal source output form of the display device 200 to copy when it detects that the display mode parameter is modified to the parameter value corresponding to the copy mode, such as "display mode = 01 (copy mode)", that is, copy the image frame in the frame buffer, and send it to the main screen and the secondary screen for display synchronously.

[0100] In the extended mode, the display device 200 can control the user interface to be displayed on the main screen or on the secondary screen according to the user settings and the application display requirements. For example, when the user controls to start application A, the screen ID to be displayed can be set for application A. When the screen ID to be displayed is set to panel screen, it means that its corresponding interface is displayed on the main screen; when the screen ID to be displayed is set to HDMI screen, it means that its corresponding interface is displayed on the secondary screen.

[0101] In some embodiments, the display device 200 can also automatically set the screen ID of the application display according to the application parameters. For example, according to the startup order of the applications in the display device 200, the application that is started first is displayed on the main screen, and the application that is started later is displayed on the secondary screen. Then, when the application is started, the display device 200 can detect the number of applications that have been started in addition to the application that is currently being started. When the number of applications started is 0, the screen ID of the display is set to panel screen; when the number of applications started is 1, the screen ID of the display is set to HDMI screen. Since the display device 200 can run multiple applications at the same time, the screen ID of the display can also be set according to the odd and even numbers of the application startup order. That is, the application with an odd startup order is displayed on the main screen, and the screen ID of the display is set to panel screen; the application with an even startup order is displayed on the secondary screen, and the screen ID of the display is set to HDMI screen.

[0102] In some embodiments, the display device 200 can construct an application status table to record relevant information of the application displayed on the main screen or the secondary screen. The recorded information can be recorded and stored through the application status table, and other applications can obtain the specific display parameters of the application by calling and reading the application status table when needed. For example, the sidebar application can obtain which screen the application is displayed on by reading the application status table, and can change the display mode by modifying the application status table. That is, the user can specify the screen to display the user interface through the extended menu of the sidebar. When the user clicks the "main screen" control in the sidebar extended menu of the secondary screen, the sidebar application can modify the display screen ID of the secondary screen application to panelscreen, and update the application status table to adjust the application interface displayed on the secondary screen to the main screen for display.

[0103] The sidebar application can also support the dual-screen swap display function, that is, there is a "swap" control in the extended menu of the sidebar. When the user clicks the "swap" control, the sidebar application can modify the display screen ID of the main screen display application A to hdmis creen, and modify the screen display channel through the hardware layer to control the display of the interface of application A on the secondary screen; at the same time, modify the display screen ID of the secondary screen display application B to panel screen to control the display of the interface of application B on the main screen.

[0104] The content displayed on the main screen and the secondary screen can also be adjusted according to the signal source input. For example, when the display device 200 is not connected to the signal source and the main screen displays the home page, the secondary screen can display the home page wallpaper according to the home page content. When the display device 200 is connected to the signal source and the display screen ID of the signal source is set to HDMI screen, the content corresponding to the signal source (including the no signal interface) is displayed on the secondary screen.

[0105] The user can also control the exit of the dual-screen mode through the sidebar tool. For example, when the user controls the display device 200 to stop running the application B corresponding to the secondary screen by clicking the "Back" control in the secondary screen sidebar multiple times, the display device 200 can exit the dual-screen mode. For another example, the extended menu of the sidebar includes a "Close" control. When the user clicks the "Close" control, the application displayed on the corresponding screen can be controlled to be closed.

[0106] When the display device 200 exits the dual-screen mode, the remaining user interface can be displayed on one of the main screen or the secondary screen. For example, since the display device 200 has its own display, it is more convenient for data transmission and interface rendering. Therefore, when the user closes the display application A through the sidebar of the main screen, the application B displayed on the secondary screen can be automatically adjusted to the main screen for display.

[0107] It should be noted that in the above example, the display device 200 is used as the signal input source, while in actual applications, the signal input source can be the display device 200 or the terminal device 500 externally connected to the display device 200. In the case where the signal input source is the external terminal device 500, the processes of adjusting, switching, and displaying the screen in the above example can be executed by the terminal device 500, or by the display device 200, or by both the display device 200 and the terminal device 500.

[0108] In the dual-screen mode of the display device 200. For the copy mode, since the display content of the main screen is the same as the display content of the secondary screen, when the display content of the main screen is mapped to the secondary screen, it can be scaled proportionally according to the display size of the secondary screen and displayed. In other words, the first user interface displayed in the main screen can be understood as a picture, and the picture is sent to the secondary screen, and the secondary screen will scale the picture proportionally according to the display size of the secondary screen to form a second user interface for display.

[0109] For extended mode, the display content of the main screen is different from that of the secondary screen. In order to independently control the applications and display content of the main screen and the secondary screen, the target view in the main screen needs to be mapped to the secondary screen. The target view is the view that the main screen maps to the secondary screen and can be controlled independently on the secondary screen. For example, Fig. 9 The sidebar shown with its associated expanded menu.

[0110] However, in the dual-screen interaction scenario of the display device 200 in the extended mode, the interaction between the main screen and the secondary screen is limited by the display resolution of the main screen and the secondary screen. Since the view has a fixed display size, if the display resolutions of the main screen and the secondary screen are different, when mapping the view from the main screen to the secondary screen, the view mapped to the secondary screen will be set according to the display resolution of the main screen, so that the view ratio of the view displayed on the secondary screen is different from that of the view displayed on the main screen, affecting the user experience.

[0111] For example, Fig. 9 For example, the sidebar shown in the figure has a display size of 100 pixels. Fig.10 As shown, a schematic diagram of the display effect of the sidebar on the main screen and sub-screen with different resolutions provided by an embodiment of the present application. The display 260 provided by the display device 200 serves as the main screen, and the external display serves as the sub-screen. Among them, the main screen is a screen with a 4K resolution, and the sub-screen is a screen with a 2K resolution. When the sidebar is displayed on the main screen, the display ratio on the main screen is 1 / 40. When it is mapped to the sub-screen for display, since it is set according to the display resolution of the main screen, when it is displayed on the sub-screen, the display ratio on the sub-screen is 1 / 20. That is, the main screen and the sub-screen appear as follows: Fig.10 As shown in the display effect, the size of the sidebar application displayed on the secondary screen is 2 times larger than the size displayed on the main screen.

[0112] In addition, for sidebar application associations such as Fig. 9 In the scenario of the extended menu shown, when the user calls out the extended menu, the extended menu will also be set according to the display resolution of the main screen, so that the view ratio of the extended menu displayed on the secondary screen is different from that displayed on the main screen.

[0113] In order to solve the above-mentioned problems, some embodiments of the present application provide a view display method, which is applied to a display device 200 and is used to adjust the display ratio of a view on a screen with different resolutions. Among them, the display device 200 that can apply the view display method includes: a first display, a device interface 240 and a controller 250. Among them, the first display is the display 260 (main screen) of the display device 200 itself, which is configured to display a first user interface, and the first user interface includes a target view; the device interface 240 is configured to connect to a second display (secondary screen) for displaying a second user interface; the display resolution of the first display is different from the display resolution of the second display. The controller 250 is configured to execute the above-mentioned view display method, such as Fig.11 As shown, Fig.11 The flowchart of the view display method provided in the embodiment of the present application specifically includes the following contents:

[0114] S101: In response to a display instruction instructing to display a target view on a second display, acquiring a display resolution of the second display.

[0115] The display device 200 uses an external display as a second display through the device interface 240, and the display 260 provided by the display device 200 is used as a first display, and dual-screen display is realized through the first display and the second display. During dual-screen display, in order to solve the problem that the view ratio of the view displayed on the first display is different from that on the second display due to the different display resolutions of the first display and the second display, the display device 200 can obtain the display resolution of the second display in response to the display instruction indicating that the target view is displayed on the second display, and adjust the view parameters of the target view according to the display resolution, so that the display ratio of the target view on the first display is the same as that on the second display.

[0116] Since the dual-screen display has a problem of different view ratios in the extended mode, the display device 200 can detect the dual-screen mode of the display device 200, and when it is detected that the dual-screen mode is the extended mode, a display instruction indicating that the target view is displayed on the second display is generated. That is, the dual-screen mode of the display device 200 is associated with the display instruction indicating that the target view is displayed on the second display, and when the dual-screen mode is switched to the extended mode, a display instruction indicating that the target view is displayed on the second display is generated to map the target view in the first display to the second display.

[0117] The dual-screen mode can be switched manually by the user. The display device 200 can be pre-set with a dual-screen mode switching control for the user to switch the dual-screen mode of the display device 200. For example, after the display device 200 is turned on, the sidebar application is started to control the first display to display the following Fig. 9 The sidebar shown in the figure can be called out by the user by clicking the sidebar icon. The extended menu can include quick function options such as "return", "home", "settings", "switch", and "mode". The user can click the "mode" option icon to enter the mode switching menu. Then the user clicks the "extended mode" or "copy mode" option icon to set the dual-screen mode of the display device 200 to the extended mode or the copy mode.

[0118] In addition, the dual-screen mode of the display device 200 can also default to the extended mode. When the second display is connected to the device interface 240, the display device 200 can detect the plug-in and unplug signal generated when the second display is connected to the device interface 240, triggering the operation of mapping the target view in the first display to the second display, and generating a display instruction instructing to display the target view on the second display.

[0119] In some embodiments, Fig.12As shown, it is a schematic diagram of the process of calculating view parameters provided by an embodiment of the present application. The display resolution can be obtained based on the display identification data (Display Identification Data, DisplayId) of the second display. The display identification data is a data type that identifies the display. It is assigned to the display by the operating system when the driver is initialized. It is an identifier (id) used by the DisplayManagerService in the display device 200 system to manage the screen. Through the display identification data, the screen information of the corresponding display can be obtained.

[0120] When the second display is connected to the device interface 240, the display device 200 can obtain the plug-in signal generated when the second display is connected to the device interface 240, and in response to the plug-in signal, the display identification identifier (DisplayId) of the second display can be obtained. Based on the display identification identifier, a display identification object (Display) can be obtained. The display identification object is used to represent the second display.

[0121] After obtaining the display identification object representing the second display, the context object of the display identification object can be obtained, and the display resolution of the second display can be read from the context object. The context object is used to represent the application environment information of the second display, which includes all screen information of the second display.

[0122] It is understandable that in the display device 200 system, Context is an abstract class that can provide an interface for global information (such as application environment information). When obtaining a resource file, the display device 200 can obtain the resource data of the application, such as pictures, strings, colors, sizes, etc., through the context.get Resource algorithm, and use it for the user interface of the application. In this embodiment, when the target view of the first display is mapped to the second display, the proportion of the target view displayed on the second display is different from that displayed on the first display because the obtained resource data such as pictures and sizes are obtained in the context object of the first display, which is not compatible with the second display. For example, a picture that is normally displayed in full screen on the first display with a resolution of 4K can only display 1 / 4 of the picture when it is placed on the first display with a resolution of 2K.

[0123] Therefore, the display device 200 can obtain the context object of the second display, use the context object for introduction, and adjust the view parameters of the target view mapped from the first display to the second display, so that the target view is displayed in a normal proportion in the second display.

[0124] In some embodiments, in order to improve performance, the display device 200 can create and maintain a database for saving the current dual-screen mode of the display device 200. When the dual-screen mode of the display device 200 changes, such as when the user switches the dual-screen mode through the sidebar, the system and application of the display device 200 can monitor the changes in this database to make corresponding changes to the dual-screen mode.

[0125] That is, the display device 200 can create a database for storing the display identification identifier of the second display. When the second display is connected to the device interface 240, in response to the plug-in signal generated when the second display is connected to the device interface 240, the display identification identifier of the second display is obtained, and the display identification identifier is stored in the database. When the dual-screen mode of the display device 200 is switched to the extended mode, a display instruction instructing to display the target view on the second display is generated, and in response to the display instruction, the display identification identifier of the second display is read in the database, and the display resolution of the second display is obtained according to the display identification identifier of the second display.

[0126] In addition, when the display device 200 is connected to multiple external displays, that is, the number of second displays is multiple, the second display to be mapped, that is, the target second display, can be selected by the user, and the display identification identifier of the target second display can be obtained in the database, and the display resolution of the target second display can be obtained based on the display identification identifier, and the view parameters of the target view mapped to the target second display can be adjusted based on the display resolution.

[0127] In some embodiments, Fig.13 As shown, it is a schematic diagram of the process of distinguishing display identification identifiers provided in an embodiment of the present application. In order to distinguish the first display and the second display, the display device 200 can distinguish according to the attribute value of the display identification identifier, and the attribute value includes a first attribute value and a second attribute value. The display device 200 can obtain the display identification identifier loaded by the display device 200 and traverse the loaded display identification identifiers. If the display identification identifier is the first attribute value, the display identification identifier is marked as the display identification identifier of the first display. If the display identification identifier is the second attribute value, the display identification identifier is marked as the display identification identifier of the second display.

[0128] For example, the attribute value of the display identification identifier of the first display is 0, and the attribute value of the display identification identifier of the second display is 1. The display device 200 can obtain the array of DisplayIds of all currently connected displays through the DisplayManager.getDisplays algorithm. Traverse the array and mark the DisplayId of 0 as the DisplayId of the first display, and mark the DisplayId of 1 as the DisplayId of the second display.

[0129] When distinguishing the display identification mark, the distinction can be made when the second display connection device interface 240 is monitored, that is, the display device 200 can respond to the plug-in signal generated by the second display connection device interface 240 to perform the above-mentioned step of distinguishing the display identification mark. It can also be distinguished when the display device 200 is turned on, that is, the display device 200 can respond to the power-on instruction of the display device 200 to perform the above-mentioned step of distinguishing the display identification mark.

[0130] In some embodiments, the different display ratios of the mapped views are caused by the different display resolutions of the dual screens. Therefore, after the display device 200 obtains the display resolution of the second display according to the display identification identifier of the second display, it can obtain the display resolution of the first display. If the display resolution of the first display is the same as the display resolution of the second display, the target view of the first display can be directly mapped to the second display. If the display resolution of the first display is different from the display resolution of the second display, the subsequent steps of adjusting the view parameters of the target view according to the display resolution of the second display are performed.

[0131] S102: Calculate view parameters according to display resolution.

[0132] After obtaining the display resolution of the second display, the view parameters of the target view can be adjusted according to the display resolution. The view parameters are used to set the display ratio of the target view on the second user interface. The display ratio of the target view on the second user interface is associated with the display ratio of the target view on the first user interface.

[0133] Since the display size of the target view is a fixed value, that is, the physical pixels occupied are fixed, for example, Fig. 9 Take the sidebar shown as an example, whose display size is 100 pixels. The display resolutions of the first display and the second display are different, and the physical pixels occupied by the display on the first display and the second display are both 100 pixels, resulting in an uncoordinated display ratio of the views on the first display and the second display. Therefore, the physical pixels occupied by the target view can be scaled according to the acquired display resolution of the second display, so as to calculate the view parameters so that the display ratio of the target view on the first display is the same as that on the second display.

[0134] When calculating the view parameters, the display device 200 may obtain the abstract pixel and position parameter of the target view, wherein the abstract pixel is a density-independent pixel (dp) or a scale-independent pixel (sp), and the position parameter is a preset position relative to the screen border.

[0135] It is understandable that the display position of the target view can be automatically adjusted by the display device 200 according to the preset position parameters and the screen boundaries of different screens, and will not be affected by the display resolutions of the first display and the second display. That is, the display device 200 can set the display position of the target view on the second user interface according to the preset position parameters of the target view.

[0136] For the display size of the target view, the display resolution of the second display needs to be considered. Therefore, after the display device 200 obtains the display resolution of the second display and the abstract pixels of the target view, the abstract pixels can be converted into physical pixels according to the display resolution. The physical pixels are the pixels displayed on the screen of the target view, which determine the display size of the target view on the second user interface.

[0137] Finally, view parameters are generated according to the position parameters and the physical pixels, so as to create a target view mapped to the second display.

[0138] In this embodiment, abstract pixels are introduced to describe the display size of the target view. Since both abstract pixels and physical pixels are pixel units for describing views, they can be converted according to a preset unit conversion method. Therefore, when mapping the target view of the first display to the second display, the abstract pixels can be converted into physical pixels of the corresponding display resolution according to the display resolution of the second display.

[0139] For example, taking the sidebar as an example, a 50dp sidebar has a display size of 100px when displayed on a first display with a 4K resolution, and a 50dp sidebar has a display size of 50px when displayed on a second display with a 2K resolution, so that the display ratio of the sidebar on the first display is the same as the display ratio of the target view on the second display.

[0140] S103: Create a target view based on the view parameters, and add the target view to the second user interface.

[0141] After obtaining the view parameters, a target view to be displayed on the second display can be created according to the view parameters, and the target view can be added to the second user interface of the second display. The display device 200 can implement the creation and addition of the target view through the window management service (WindowManager), wherein the window management service is used to manage the state and properties of the window, view addition, deletion, update, window order, message collection and processing, etc.

[0142] Therefore, the display device 200 can obtain the window management service of the second display according to the display identification identifier of the second display, create a layout parameter class (LayoutParams) in the window management service, and define view parameters in the layout parameter class, thereby introducing the view parameters into the window management service of the second display.

[0143] When creating and adding a target view, the display device 200 may call a view adding algorithm (WindowManger.addView) in the window management service of the second display, use the view adding algorithm to create a target view based on view parameters, and add the target view to the second user interface.

[0144] The extended menu associated with the target view can also be displayed based on the above-mentioned creation and addition steps, that is, the display device 200 can respond to the display instruction of the extended menu of the target view, call the view adding algorithm in the window management service of the second display, use the view adding algorithm, create an extended menu based on the view parameters, and add the extended menu to the second user interface.

[0145] For example, taking the sidebar application as an example, the first display has a 4K resolution and the second display has a 2K resolution. After the display device 200 obtains the DisplayId of the second display, it obtains the WindowManager of the second display according to the DisplayId of the second display, and then sets the view parameters of the sidebar in the second display in the LayoutParams of the WindowManager. Finally, WindowManger.addView is used to add the sidebar to the second user interface, so that the first display and the second display appear as follows: Fig.14 The interface effect shown.

[0146] When the user clicks the arrow icon corresponding to the sidebar through the second display to input a display instruction for the extended menu of the sidebar. In response to the display instruction, the display device 200 can obtain the WindowManager of the second display, and then set the view parameters of the extended menu in the second display in the LayoutParams of the WindowManager. Finally, WindowManger.addView is used to add the extended menu to the second user interface, so that the first display and the second display are displayed as follows: Fig.15 The interface effect shown.

[0147] Based on the above view display method. In some embodiments of the present application, a display device 200 is also provided, including: a first display, a device interface 240 and a controller 250. The first display is configured to display a first user interface, the first user interface includes a target view; the device interface 240 is configured to connect to a second display, the second display is used to display a second user interface, and the display resolution of the first display is different from the display resolution of the second display; the controller 250 is configured to execute the following program steps:

[0148] In response to a display instruction instructing to display the target view on the second display, a display resolution of the second display is acquired.

[0149] The display resolution is obtained based on the display identification mark of the second display.

[0150] Calculates view parameters based on display resolution.

[0151] The view parameter is used to set the display ratio of the target view on the second user interface. The display ratio of the target view on the second user interface is associated with the display ratio of the target view on the first user interface.

[0152] A target view is created based on the view parameters, and the target view is added to the second user interface.

[0153] The same and similar parts between the various embodiments in this specification can be referenced to each other and will not be described again here.

[0154] It can be seen from the above technical solutions that the display device and view display method provided in the above embodiments can obtain the display resolution of the second display in response to the display instruction indicating that the target view is displayed on the second display, and calculate the view parameters according to the display resolution. Among them, the second display is a display external to the display device, the display resolution is obtained based on the display identification identifier of the second display, and the view parameters are used to set the display ratio of the target view on the second user interface, and the display ratio of the target view on the second user interface is associated with the display ratio of the target view on the first user interface. Then, the target view is created based on the view parameters, and the target view is added to the second user interface. The method can adjust the view parameters of the view according to the resolution of the display when connecting displays with different resolutions, so as to solve the problem that the display ratio of the view is different due to different screen resolutions in the dual-screen system.

[0155] When the display device 200 is connected to the second display through the device interface 240 and the dual-screen mode is the extended mode, the application can be started on the first display or the second display according to the user settings and application needs, that is, the application window corresponding to the application is displayed on the first display or the second display, so that the user can operate different applications through the first display and the second display to interact. For the convenience of user use, a sidebar can be displayed on the first display and the second display based on the above view display method to provide users with a shortcut operation menu for operating different displays.

[0156] by Fig. 9 Taking the sidebar shown as an example, the user can click the arrow icon in the sidebar to call out the extended menu. The extended menu may include quick function options such as "Back", "Home", "Settings", and "Switch". The user can click any option icon to control the dual-screen display process. For example, the user can enter an application return instruction through the sidebar to control the exit of the application. The user clicks the "Back" icon in the sidebar displayed on the first display to exit the application of the top application window displayed on the first display.

[0157] However, since both the first display and the second display are controlled by the display device 200 system, when a user inputs an application return instruction through the sidebar displayed on the first display and the second display, the display device 200 will exit the applications in the order in which they are started, and cannot control the exit of the applications corresponding to the application windows displayed on each display, resulting in application exit errors.

[0158] For example, Fig.16 As shown, it is a schematic diagram of the first display and the second display provided in the embodiment of the application showing different application windows. The user controls the display device 200 to start application A, application B and application C in sequence. Among them, the application windows of application A and application C are displayed on the first display, and the application window of application B is displayed on the second display. According to the native logic of Android, when the user enters the application return instruction through the sidebar, the display device 200 will exit application C, application B and application A in sequence according to the startup order of the applications.

[0159] When the user clicks the "Back" icon in the sidebar displayed on the first display, the display device 200 exits application C according to the startup sequence of the applications. When the user clicks the "Back" icon in the sidebar displayed on the second display, the purpose is to exit application B, but according to the native logic of Android, the display device 200 will also exit application C according to the startup sequence of the applications.

[0160] Therefore, in order to solve the above problems, some embodiments of the present application provide a multi-screen control method, which is applied to a display device 200 and is used to control the corresponding application of the display to exit according to the application return instruction input in the user interface of different displays. That is, if the application return instruction is input in the first display, the application response of the first display is controlled, and if the application return instruction is input in the second display, the application response of the second display is controlled.

[0161] The display device 200 to which the multi-screen control method can be applied includes: a first display, a device interface 240 and a controller 250. The first display is the display 260 of the display device 200 itself, which is configured to display a first user interface, which includes an application window of at least one application; the device interface 240 is configured to connect to a second display for displaying a second user interface, which includes an application window of at least one application; the controller 250 is configured to execute the multi-screen control method, which specifically includes the following contents:

[0162] The display device 200 can maintain two sets, namely a first application set and a second application set. The first application set includes applications whose application windows are displayed on the first display, and the second application set includes applications whose application windows are displayed on the second display. Applications whose application windows are displayed on the first display and applications whose application windows are displayed on the second display are distinguished by the first application set and the second application set. When the user inputs an application return instruction through the first user interface, the corresponding application is controlled to exit according to the first application set, and when the user inputs an application return instruction through the second user interface, the corresponding application is controlled to exit according to the second application set.

[0163] The application list in the system stack of the display device 200 records the currently started application and the order in which the application is started, that is, the application list includes the application whose application window is displayed on the first display, the application whose application window is displayed on the second display, and the list order. The list order is used to characterize the order in which the applications are started. The display device 200 can read the display identification identifier (DisplayId) of the first display and the display identification identifier (DisplayId) of the second display, and generate the first application set and the second application set according to the display identification identifier of the first display, the display identification identifier of the second display and the application list.

[0164] That is, in some embodiments, such as Fig.17As shown, it is a schematic diagram of the process of creating a first application set and a second application set provided by an embodiment of the present application. The display device 200 can obtain the application list in the current system stack of the display device 200, obtain the display identification identifier of the first display and the display identification identifier of the second display, and create a first application set and a second application set. In the application list, query the first application associated with the display identification identifier of the first display, and add the first application to the first application set. In the application list, query the second application associated with the display identification identifier of the second display, and add the second application to the second application set.

[0165] Since the application list also includes the startup order of applications, in order to facilitate subsequent query of applications in the top application window of the user interface, the display device 200 can obtain the list order of the application list and sort the applications in the first application set and the second application set according to the list order.

[0166] For example, Fig.18 As shown, it is a schematic diagram of an application list, a first application set, and a second application set provided in an embodiment of the present application. The display device 200 starts application A, application B, application C, and application D in sequence. Among them, the application windows of application A and application C are displayed on the first display, and the application windows of application B and application D are displayed on the second display. The application that is started first in the application list in the current system stack of the display device 200 is displayed at the last position, that is, the list order is application D, application C, application B, application A. According to the list order, the applications in the first application set and the second application set are sorted, and it can be obtained that the application order of the first application set is application C, application A, and the application order of the second application set is application D, application B.

[0167] In the extended mode, the display device 200 can set screen identifiers for different applications according to user settings and application display needs, and then control the application window to be displayed on the first display or on the second display. That is, the display device 200 can detect the screen identifier of the target application in response to the startup instruction of the target application. If it is a screen identifier indicating display on the first display, the application window of the target application is displayed on the first display. If it is a screen identifier indicating display on the second display, the application window of the target application is displayed on the second display. Therefore, when the display device 200 distinguishes between the first application and the second application, it can be determined according to the screen identifier of the application.

[0168] In some embodiments, the screen identifiers set for different applications may be display identifiers (DisplayId) of displays. That is, if the display identifier set for an application is the display identifier of a first display, it means that the application window of the application is displayed on the first display. If the display identifier set for an application is the display identifier of a second display, it means that the application window of the application is displayed on the second display.

[0169] Therefore, when distinguishing the first application from the second application, the display device 200 can traverse the application list and obtain the application information (RunningTaskInfo) of the application, wherein the RunningTaskInfo includes relevant information of the application. The key field is read in the application information, and the key field is used to characterize the field describing the display identification identifier, and the display identification identifier set for the application is queried according to the key field.

[0170] After obtaining the display identification identifier set for the application, if the display identification identifier set for the application is the same as the display identification identifier of the first display, the application is marked as the first application; if the display identification identifier set for the application is the same as the display identification identifier of the second display, the application is marked as the second application.

[0171] For example, the DisplayId of the first display is 0, and the DisplayId of the second display is 1. When traversing the application list, if the DisplayId of the traversed application is 0, it means that its application window is displayed on the first display and is marked as the first application. If the DisplayId of the traversed application is 1, it means that its application window is displayed on the second display and is marked as the second application.

[0172] After the first application set and the second application set are created, they can be changed according to the subsequent startup and exit of the application of the display device 200. When the display device 200 displays a new application window on the first display or the second display, the application corresponding to the new application window is added to the first application set or the second application set. Similarly, when the display device 200 exits the first application or the second application, the first application is removed from the first application set or the second application is removed from the second application set.

[0173] After the first application set and the second application set are established, the display device 200 can control the exit of applications on different displays according to the first application set and the second application set, such as Fig.19 As shown, Fig.19 The flowchart of the multi-screen control provided in the embodiment of the present application is as follows. For the case where the application return instruction is input through the first user interface, the controller 250 is configured to execute the following program steps:

[0174] S201: In response to an application return instruction input through a first user interface, obtain a first application set.

[0175] S202: Query a first target application in a first application set.

[0176] After the first application set is acquired, a first target application is searched in the first application set, wherein the first target application is an application in the topmost application window of the first user interface.

[0177] Since the first application set and the second application set include an application sequence, the display device 200 may mark the first or last application as the first target application according to the application sequence in the first application set.

[0178] It is understandable that the first target application is marked as the first target application, indicating that the application that is launched first is displayed at the last position. Fig.18 Taking the application sequence shown as an example, the first application (application C) in the first application set is marked as the first target application.

[0179] S203: Control the first target application to respond to the application return instruction.

[0180] After the first target application is found, the display device 200 may control the first target application to respond to the application return instruction and exit the first target application.

[0181] In the native logic of the Android system, the response instruction is controlled by whether the application has focus. In other words, the application with focus responds to the instruction issued by the system. Therefore, after the first target application is queried, the system focus can be allocated to the first target application, and the first target application with system focus can be controlled to respond to the application return instruction and exit the application window displayed on the first display.

[0182] The allocation of system focus can be achieved through the activity task manager (ActivityTaskManage). Through the activity task manager in the display device 200 system, the original focus application currently having the focus mark is detected, the focus mark of the original focus application is deleted, and the focus mark is set for the first target application. Fig.18 Taking the application list shown as an example, according to the Android native logic, the original focus application is the last launched application D. The user inputs an application return instruction through the user interface of the first display, the purpose is to exit application C. Therefore, the system focus of application D can be cancelled and the system focus can be allocated to application C.

[0183] In the display device 200 system, each application has an application ID (TaskID), which is a unique identifier assigned to the application by the system. The display device 200 can obtain the application ID of the first target application, call the ActivityTaskManage.setfocusedtask algorithm in the activity task manager, and set a focus mark for the application ID of the first target application, so that the first target application obtains the system focus.

[0184] Similarly, for the case where the application return instruction is input through the second user interface, such as Fig.19 As shown, the controller 250 is configured to perform the following program steps:

[0185] S301: In response to an application return instruction input through a second user interface, obtain a second application set.

[0186] The second application set includes applications whose application windows are displayed on the second display, and the second application set is generated by reading a display identification identifier of the second display.

[0187] S302: Searching for a second target application in a second application set.

[0188] The second target application is the application of the topmost application window of the second user interface.

[0189] S303: Control the second target application to respond to the application return instruction.

[0190] For example, the second display shows Fig. 9 The user clicks the arrow icon of the sidebar to call out the extended menu, and clicks the "return" icon in the extended menu to send an application return instruction to the display device 200. In response to the application return instruction, the display device 200 obtains the pre-established second application set. Fig.18 Taking the second application set shown as an example, the second application set includes application D and application B in the order of applications. The application order indicates that the application started first is displayed at the end. The display device 200 can query the second application set to find that the application in the top application window of the second user interface is application D. Then, through the activity task manager (ActivityTaskManage), the system focus is allocated to application D, so that application D with system focus responds to the application return instruction and exits the application window displayed on the second display.

[0191] After the application window of application D exits, the user clicks the "Return" icon in the extended menu again. The display device 200 can follow the above steps to query in the second application set that the application in the top-level application window of the second user interface is application B, and then allocate system focus to application B, so that application B with system focus responds to the application return instruction and exits the application window displayed on the second display.

[0192] The above embodiment is to realize the application return instruction input in the user interface of different displays by maintaining two lists, and control the corresponding application exit of the display. In order to reduce maintenance costs, the embodiment of the present application also provides a dual-screen control method, which is applied to the above display device 200, such as Fig. 20 As shown, Fig. 20 The schematic diagram of the exit process of an application whose application window is displayed on the first display provided in the embodiment of the present application specifically includes the following contents for the case where an application return instruction is input through the first user interface:

[0193] S401: In response to an application return instruction input through a first user interface, obtaining a display identification identifier of a first display.

[0194] S402: Obtain a list of applications in the current system stack of the display device.

[0195] The application list includes applications whose application windows are displayed on the first display and applications whose application windows are displayed on the second display, and the list order of the application list represents the startup order of the applications.

[0196] S403: Traverse the application list in the order of the list.

[0197] S404: If the display identification identifier of the traversed target application is the same as the display identification identifier of the first display, allocate system focus to the target application.

[0198] S405: Control the target application with focus to respond to the application return instruction.

[0199] The display device 200 can obtain the display identification identifier of the display according to the application return instruction input in the user interface of the different display, search the application of the top-level application window associated with the display identification identifier in the application list in the current system stack of the display device 200 according to the display identification identifier, and control the exit of the application.

[0200] For example, the user inputs an application return instruction through the sidebar displayed on the first display. In response to the application return instruction, the display device 200 obtains the display identification identifier (DisplayId) of the first display, and obtains the application list in the system stack through the ActivityManager.getRunningTasks algorithm in the system, and the list sequence represents the startup sequence of the application. The application list is traversed in the list order to obtain the application information (RunningTaskInfo) of the application, and the display identification identifier (DisplayId) set for the application is read in the application information, which indicates the display that displays the application window of the application.

[0201] by Fig.18 Take the application list shown as an example. The application list includes application D, application C, application B, and application A in the order of the list. The order of the list means that the first application started is displayed at the end. Traverse the application list in the order of the list, and the first application is application D. Compare the DisplayId of application D with the DisplayId of the first display. The two are different, indicating that the application window of application D is not displayed on the first display. The second application is application C. Compare the DisplayId of application C with the DisplayId of the first display. The two are the same, indicating that the application window of application C is the top application window of the first display.

[0202] According to the native logic of Android, the application currently having the system focus is the last launched application D. Therefore, through the activity task manager (ActivityTaskManage) of the display device 200 system, the system focus of application D can be cancelled and the system focus can be allocated to application C. Therefore, application C having the system focus responds to the application return instruction and exits the application window displayed on the first display.

[0203] Similarly, for the case where the application return instruction is input through the second user interface, such as Fig.21 As shown, Fig.21 The schematic diagram of the exit process of an application whose application window is displayed on a second display provided in an embodiment of the present application specifically includes the following contents:

[0204] S501: In response to an application return instruction input through a second user interface, obtaining a display identification identifier of a second display.

[0205] S502: Obtain a list of applications in the current system stack of the display device.

[0206] The application list includes applications whose application windows are displayed on the first display and applications whose application windows are displayed on the second display, and the list order of the application list represents the startup order of the applications.

[0207] S503: Traverse the application list in the order of the list.

[0208] S504: If the display identification identifier of the traversed target application is the same as the display identification identifier of the second display, allocate system focus to the target application.

[0209] S505: Control the target application with focus to respond to the application return instruction.

[0210] For the case of inputting the application return instruction through the second user interface, the steps of inputting the application return instruction through the first user interface may be referred to, and will not be described in detail here.

[0211] Based on the above multi-screen control method. In some embodiments of the present application, a display device 200 is also provided, including: a first display, a device interface 240 and a controller 250. The first display is configured to display a first user interface, the first user interface includes an application window of at least one application; the device interface 240 is configured to connect to a second display, the second display is used to display a second user interface, the second user interface includes an application window of at least one application; the controller 250 is configured to perform the following program steps:

[0212] S201: In response to an application return instruction input through the first user interface, obtain a first application set.

[0213] The first application set includes applications whose application windows are displayed on the first display, and the first application set is generated by reading a display identification identifier of the first display.

[0214] S201: Query a first target application in a first application set.

[0215] The first target application is the application in the topmost application window of the first user interface.

[0216] S201: Control the first target application to respond to the application return instruction.

[0217] The controller 250 is also configured to perform the following program steps:

[0218] S301: In response to an application return instruction input through a second user interface, obtain a second application set.

[0219] The second application set includes applications whose application windows are displayed on the second display, and the second application set is generated by reading a display identification identifier of the second display.

[0220] S302: Searching for a second target application in a second application set.

[0221] The second target application is the application of the topmost application window of the second user interface.

[0222] S303: Control the second target application to respond to the application return instruction.

[0223] The controller 250 is also configured to perform the following program steps:

[0224] S401: In response to an application return instruction input through a first user interface, obtaining a display identification identifier of a first display.

[0225] S402: Obtain a list of applications in the current system stack of the display device.

[0226] The application list includes applications whose application windows are displayed on the first display and applications whose application windows are displayed on the second display, and the list order of the application list represents the startup order of the applications.

[0227] S403: Traverse the application list in the order of the list.

[0228] S404: If the display identification identifier of the traversed target application is the same as the display identification identifier of the first display, allocate system focus to the target application.

[0229] S405: Control the target application with focus to respond to the application return instruction.

[0230] The controller 250 is also configured to perform the following program steps:

[0231] S501: In response to an application return instruction input through a second user interface, obtaining a display identification identifier of a second display.

[0232] S502: Obtain a list of applications in the current system stack of the display device.

[0233] The application list includes applications whose application windows are displayed on the first display and applications whose application windows are displayed on the second display, and the list order of the application list represents the startup order of the applications.

[0234] S503: Traverse the application list in the order of the list.

[0235] S504: If the display identification identifier of the traversed target application is the same as the display identification identifier of the second display, allocate system focus to the target application.

[0236] S505: Control the target application with focus to respond to the application return instruction.

[0237] The same and similar parts between the various embodiments in this specification can be referenced to each other and will not be described again here.

[0238] It can be seen from the above technical solutions that the display device and multi-screen control method provided in the above embodiments can respond to the application return instruction input through the first user interface, obtain the first application set, query the first target application in the first application set, and exit the first target application. Among them, the first application set includes applications whose application windows are displayed on the first display, and the first application set is generated by reading the display identification identifier of the first display, and the first target application is the application of the top-level application window of the first user interface. The method can control the exit of the corresponding application of the display according to the application return instruction input in the user interface of different displays, and solve the problem of returning an error when the application returns in the multi-screen system.

[0239] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention or certain parts of the embodiments.

[0240] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0241] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that: include: A first display configured to display a first user interface, wherein the first user interface includes a target view; a device interface configured to connect to a second display, the second display being used to display a second user interface, the display resolution of the first display being different from the display resolution of the second display; The controller is configured as: In response to a display instruction instructing to display a target view on the second display, acquiring a display resolution of the second display, where the display resolution is obtained based on a display identification identifier of the second display; Calculating view parameters according to the display resolution, where the view parameters are used to set a display ratio of the target view on the second user interface; The display ratio of the target view on the second user interface is associated with the display ratio of the target view on the first user interface; The target view is created based on the view parameters, and the target view is added to the second user interface.

2. The display device according to claim 1, characterized in that The controller is also configured to: Acquire a plug-in / plug-out signal generated when the second display is connected to the device interface; In response to the plug-in / plug-out signal, acquiring a display identification mark of the second display; Acquire a display identification object based on the display identification identifier, where the display identification object is used to represent the second display; Acquire a context object of the display identification object, where the context object is used to represent application environment information of the second display; The display resolution is read in the context object.

3. The display device according to claim 1, characterized in that The controller is also configured to: Creating a database, the database being used to store a display identification mark of the second display; In response to a plug-in / plug-out signal generated when the second display is connected to the device interface, acquiring a display identification identifier of the second display, and storing the display identification identifier in the database; In response to a display instruction instructing to display a target view on the second display, the display identification identifier is read from the database, and the display resolution is acquired according to the display identification identifier.

4. The display device according to claim 1, characterized in that The controller is also configured to: Detecting a dual-screen mode of the display device; When it is detected that the dual-screen mode is the extended mode, a display instruction instructing to display the target view on the second display is generated.

5. The display device according to claim 1, characterized in that The controller is also configured to: In response to a power-on instruction of the display device, obtaining a display identification mark loaded by the display device; Traversing the loaded display identification marks; If the display identification identifier is a first attribute value, marking the display identification identifier as a display identification identifier of the first display; If the display identification identifier is the second attribute value, the display identification identifier is marked as the display identification identifier of the second display.

6. The display device according to claim 1, characterized in that The controller is also configured to: Acquire the window management service of the second display according to the display identification identifier of the second display; Creating a layout parameter class in the window management service; The view parameters are defined in the layout parameter class.

7. The display device according to claim 6, characterized in that The controller is also configured to: Calling the view adding algorithm in the window management service; Using the view adding algorithm, the target view is created based on the view parameters, and the target view is added to the second user interface.

8. The display device according to claim 6, characterized in that The controller is also configured to: In response to a display instruction of the extended menu of the target view, calling a view adding algorithm in the window management service; Using the view adding algorithm, an extended menu is created based on the view parameters, and the extended menu is added to the second user interface.

9. The display device according to claim 1, characterized in that: The controller performs the step of calculating the view parameter according to the display resolution, and is further configured to: Acquire an abstract pixel and a position parameter of the target view, wherein the abstract pixel is a density-independent pixel or a scale-independent pixel, and the position parameter is a preset position relative to a screen boundary; According to the display resolution, convert the abstract pixel into a physical pixel, where the physical pixel is a pixel displayed on the screen of the target view; The view parameter is generated according to the position parameter and the physical pixel.

10. A view display method, characterized in that: Applied to a display device, the display device includes a first display, a device interface and a controller, the device interface is connected to a second display, the display resolution of the first display is different from the display resolution of the second display, the first display is used to display a first user interface, the first user interface includes a target view, and the second display is used to display a second user interface, the method includes: In response to a display instruction instructing to display a target view on a second display, acquiring a display resolution of the second display, where the display resolution is obtained based on a display identification identifier of the second display; Calculating view parameters according to the display resolution, the view parameters being used to set a display ratio of the target view on the second user interface; the display ratio of the target view on the second user interface is associated with the display ratio of the target view on the first user interface; The target view is created based on the view parameters, and the target view is added to the second user interface.

Images