Display device and view display method

Abstract

This application provides a display device and a view display method. The method, in response to a display command instructing the display of a target view on a second display, obtains the display resolution of the second display and calculates view parameters based on the display resolution. The second display is an external display connected to the display device. The display resolution is obtained based on the display identification identifier of the second display. The view parameters are used to set the display ratio of the target view on the second user interface, and the display ratio of the target view on the second user interface is associated with the display ratio of the target view on the first user interface. The method then creates the target view based on the view parameters and adds the target view to the second user interface. This method can adjust the view parameters according to the resolution of the connected displays of different resolutions, solving the problem of different display ratios of views caused by different screen resolutions in a dual-screen system.

Description

Technical Field

[0001] This application relates to the field of display device technology, and in particular to a display device and a view display method. Background Technology

[0002] Display devices refer to terminal devices capable of outputting specific display images, such as smart TVs, communication terminals, smart advertising screens, and projectors. Taking smart TVs as an example, smart TVs are television products based on Internet application technologies, possessing open operating systems and chips, and having open application platforms. They enable two-way human-computer interaction and integrate multiple functions such as audio-visual, entertainment, and data to meet diverse and personalized user needs.

[0003] Display devices can form a dual-screen system by connecting an external monitor to their own monitor, enabling dual-screen display mode. In some interactive scenarios, display devices support dual-screen extended display, where the main screen (the display device's own monitor) is connected to the secondary screen (an external monitor) to achieve dual-screen display. Sidebar shortcut operation buttons can be displayed on both sides of the secondary screen to quickly operate the content displayed on the secondary screen, allowing users to more intuitively and quickly operate the applications and displayed content on both screens.

[0004] However, the above interaction is limited by the resolution of the main screen and the secondary screen. If the resolution of the main screen and the secondary screen are different, the view added on the secondary screen will be displayed on the secondary screen according to the resolution of the main screen, which makes the proportion of the view displayed on the secondary screen inconsistent and affects the user experience. Summary of the Invention

[0005] This application provides a display device and a dual-screen display method to solve the problem of different display ratios of views caused by different resolutions of the main and secondary screens in a dual-screen system.

[0006] In a first aspect, this application provides a display device, including a first display, a device interface, and a controller. The first display is configured to display a first user interface, the first user interface including a target view; the device interface is configured to connect to a second display, the second display being used to display a second user interface, the display resolution of the first display and the display resolution of the second display being different; the controller is configured to execute the following program steps:

[0007] In response to a display instruction instructing the display of a target view on the second display, the display resolution of the second display is obtained, the display resolution being obtained based on the display identification identifier of the second display;

[0008] View parameters are calculated based on the display resolution, and the view parameters are used to set the display ratio of the target view on the second user interface; the display ratio of the target view on the second user interface is related to the display ratio of the target view on the first user interface;

[0009] The target view is created based on the view parameters, and the target view is added to the second user interface.

[0010] Secondly, this application also provides a view display method applied to the above-mentioned display device, the method comprising:

[0011] In response to a display instruction instructing the display of a target view on a second display, the display resolution of the second display is obtained, the display resolution being obtained based on the display identification identifier of the second display;

[0012] View parameters are calculated based on the display resolution. These view parameters are used to set the display ratio of the target view on the second user interface. The display ratio of the target view on the second user interface is related to the display ratio of the target view on the first user interface. The first user interface is the interface displayed on the first display, and the second user interface is the interface displayed on the second display.

[0013] The target view is created based on the view parameters, and the target view is added to the second user interface.

[0014] As can be seen from the above technical solutions, this application provides a display device and a view display method. The method can, in response to a display command instructing the display of a target view on a second display, obtain the display resolution of the second display and calculate view parameters based on the display resolution. The second display is an external display connected to the display device, and the display resolution is obtained based on the display identification identifier of the second display. The view parameters are used to set the display ratio of the target view on the second user interface, and the display ratio of the target view on the second user interface is associated with the display ratio of the target view on the first user interface. The method then creates the target view based on the view parameters and adds the target view to the second user interface. This method can adjust the view parameters according to the resolution of the display when connecting displays of different resolutions, solving the problem of different display ratios of views caused by different screen resolutions in a dual-screen system. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A schematic diagram illustrating the usage scenarios of the display device provided in some embodiments of this application;

[0017] Figure 2 This is a schematic diagram of the hardware configuration of a display device provided in some embodiments of this application;

[0018] Figure 3 This is a schematic diagram illustrating the connection relationship of external terminal devices provided in some embodiments of this application;

[0019] Figure 4 This is a schematic diagram of the hardware configuration of the control device provided in some embodiments of this application;

[0020] Figure 5 This is a schematic diagram of the software configuration of a display device provided in some embodiments of this application;

[0021] Figure 6 This application provides structural diagrams of a dual-screen display system for some embodiments.

[0022] Figure 7 This application provides a diagram illustrating the touch data transmission relationship of a dual-screen display system in some embodiments.

[0023] Figure 8 These are schematic diagrams illustrating the dual-screen display effect provided in some embodiments of this application;

[0024] Figure 9 Schematic diagram of sidebar tools provided for some embodiments of this application;

[0025] Figure 10 These are schematic diagrams illustrating the display effects of the sidebar on main screens and secondary screens with different resolutions, provided in some embodiments of this application.

[0026] Figure 11 A flowchart illustrating a view display method provided in some embodiments of this application;

[0027] Figure 12 A flowchart illustrating the calculation of view parameters provided in some embodiments of this application;

[0028] Figure 13 A flowchart illustrating the process of distinguishing and displaying identification marks provided in some embodiments of this application;

[0029] Figure 14 A schematic diagram illustrating the display effect of the sidebar on the first and second displays provided in some embodiments of this application;

[0030] Figure 15 A schematic diagram illustrating the display effect of an extended menu on a second display provided in some embodiments of this application;

[0031] Figure 16 A schematic diagram illustrating the display of different application windows on a first display and a second display, provided for some embodiments of this application;

[0032] Figure 17 A schematic diagram illustrating the process of creating a first application set and a second application set, provided for some embodiments of this application;

[0033] Figure 18 A schematic diagram illustrating an application list, a first application set, and a second application set provided for some embodiments of this application;

[0034] Figure 19 A schematic diagram illustrating the multi-screen control process provided in some embodiments of this application;

[0035] Figure 20 A schematic diagram illustrating the exit process of an application whose application window is displayed on a first display, as provided in some embodiments of this application;

[0036] Figure 21 This is a schematic diagram illustrating the exit process of an application whose application window is displayed on a second display, as provided in some embodiments of this application. Detailed Implementation

[0037] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application as detailed in the claims.

[0038] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0039] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0040] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0041] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0042] Figure 1 This is a schematic diagram illustrating an operational scenario between a display device and a control device provided in some embodiments of this application. For example... Figure 1 As shown, the user can operate the display device 200 through the mobile terminal 300 and the control device 100.

[0043] In some embodiments, the mobile terminal 300 may install software applications with the display device 200 to establish a connection and communication via a network communication protocol, thereby achieving one-to-one control operation and data communication. Alternatively, audio and video content displayed on the mobile terminal 300 can be transmitted to the display device 200 to achieve synchronous display.

[0044] like Figure 1 The diagram also shows that the display device 200 communicates with the server 400 via various communication methods. This allows the display device 200 to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0045] In addition to providing broadcast television reception functions, the display device 200 can also be equipped with intelligent network television functions that provide computer support, including but not limited to network television, smart television, Internet Protocol television (IPTV), etc.

[0046] Figure 2 Provided for some embodiments of this application Figure 1 Hardware configuration block diagram of display device 200.

[0047] In some embodiments, the display device 200 includes at least one of a tuner 210, a communicator 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a communication device interface 280.

[0048] In some embodiments, detector 230 is used to acquire signals from the external environment or to interact with the outside world. For example, detector 230 includes a light receiver, a sensor for acquiring ambient light intensity; or, detector 230 includes an image acquisition device, such as a camera, which can be used to acquire external environmental scenes, user attributes, or user interaction gestures; or, detector 230 includes a sound acquisition device, such as a microphone, for receiving external sounds.

[0049] In some embodiments, the display 260 includes a display screen component for presenting an image, a driving component for driving image display, a component for receiving image signals output from a controller, and a user control UI interface, etc.

[0050] The display 260 can be an LCD display, an OLED display, or a projection display, and can also be a projection device and a projection screen.

[0051] In some embodiments, the display device 200 may also support touch interaction, that is, a touch interaction module may be provided on the display device 200, which may be a built-in module or an external module. For example, to realize touch interaction operation, the display 260 of the display device 200 may be provided with a touch panel (TP), which can receive touch signals input from fingers, touch points, styluses, etc., and execute touch interaction responses according to a pre-set interaction strategy. Of course, the display device 200 may also be a touch-screen integrated display.

[0052] In some embodiments, the communicator 220 is a component for communicating with external devices or the server 400 according to various communication protocol types.

[0053] In some embodiments, the device interface 240 may include, but is not limited to, one or more interfaces such as: High Definition Multimedia Interface (HDMI), analog or data high-definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, etc. It may also be a composite input / output interface formed by multiple interfaces mentioned above.

[0054] For example, to enrich the display types of display device 200, the HDMI interface can connect to devices with video and audio input / output functions, and implement the corresponding audio and video output / input functions through the HDMI driver in the hardware layer. The USB interface can connect to devices with signal and data transmission functions and interactive functions, and, based on the USB driver in the hardware layer, access signals, transmit data, and perform specific interactive actions.

[0055] In some embodiments, the display device 200 can support multiple interface connections and perform corresponding functions according to the specific device type connected. For example, the display device 200 can connect to another monitor or terminal device 500 via an HDMI interface to form dual-screen, split-screen, screen sharing, conferencing, whiteboard, NFC, and other display functions. The display device 200 can also connect to the terminal device 500 via a USB interface and activate the Open Pluggable Specification (OPS) channel to connect the external terminal device 500. When connecting the terminal device 500 via the OPS channel, since the display device 200 has an independent operating system (System A) and the terminal device 500 can also have an independent operating system (System B), a dual-system mode can be formed.

[0056] In some embodiments, both the display device 200 and / or the connected terminal device 500 can support touch interaction. When both the display device 200 and the connected terminal device 500 support touch interaction, a data transmission and conversion module can be set in the display device 200 to transmit and convert the touch data generated during the touch interaction process, thereby achieving channel reverse control, that is, controlling the display device 200 through the touch-interactive terminal device 500. For example, as... Figure 3 As shown, a switch module can be set up in the touch interaction system. The TP module with touch functionality can be connected to the main control system (system A) of the display device 200 via a USB interface, and can also be connected to the terminal device 500 (system B), an external host, and a type-C device. In this way, the OPS channel formed by the HDMI interface can transmit display content data, and the switch module and USB interface can transmit touch data.

[0057] In some embodiments, the controller 250 includes a processor, a video processor, an audio processor, a graphics processor, a ROM, and a first to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0058] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0059] In some embodiments, a user can input user commands through a graphical user interface (GUI) displayed on a display 260, and the user input interface receives user input commands through the graphical user interface (GUI).

[0060] In some embodiments, the communication device interface 280 is an interface that can be used to receive control input.

[0061] Figure 4 Provided for some embodiments of this application Figure 1 Hardware configuration block diagram of the central control device. (Example) Figure 3 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface, a memory 190, and a power supply 180.

[0062] The control device 100 is configured to control the display device 200, and to receive user input operation commands and convert the operation commands into commands that the display device 200 can recognize and respond to, thus acting as an intermediary for interaction between the user and the display device 200.

[0063] In some embodiments, the control device 100 may be an intelligent device. For example, the control device 100 may be equipped with various applications for controlling the display device 200 according to user needs.

[0064] In some embodiments, such as Figure 1 As shown, the mobile terminal 300 or other smart electronic devices can perform similar functions to the control device 100 after installing the application of the control display device 200.

[0065] The controller 110 includes a processor 112, RAM 113, ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation of the control device 100, as well as the communication and cooperation between internal components and the external and internal data processing functions.

[0066] Under the control of the controller 110, the communication interface 130 enables communication of control signals and data signals with the display device 200. The communication interface 130 may include at least one of other near-field communication modules such as WiFi chip 131, Bluetooth module 132, and NFC module 133.

[0067] User input / output interface 140, wherein the input interface includes at least one of other input interfaces such as microphone 141, touchpad 142, sensor 143, button 144, etc.

[0068] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with the communication interface 130, such as a WiFi, Bluetooth, or NFC module, which can encode user input commands via the WiFi, Bluetooth, or NFC protocol and send them to the display device 200.

[0069] The memory 190 is used to store various operating programs, data, and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can also store various control signal instructions input by the user.

[0070] The power supply 180 is used to provide operating power support for the various components of the control device 100 under the control of the controller.

[0071] Figure 5 Provided for some embodiments of this application Figure 1 The diagram shows the software configuration of the display device. In some embodiments, the system of the display device 200 can be divided into three layers, from top to bottom: the application layer, the middleware layer, and the hardware layer.

[0072] The application layer mainly includes commonly used applications on TVs, as well as the application framework. The commonly used applications are mainly browser-based applications, such as HTML5 apps, and native apps.

[0073] An application framework is a complete program model that has all the basic functions required by standard application software, such as file access, data exchange, and the user interface for these functions (toolbar, status bar, menu, dialog box).

[0074] Native apps can support online or offline access, push notifications, or access to local resources.

[0075] The middleware layer includes various television protocols, multimedia protocols, and system components. Middleware can use the basic services (functions) provided by system software to connect different parts of application systems or different applications on the network, achieving resource sharing and function sharing.

[0076] The hardware layer mainly includes the Hardware Abstraction Layer (HAL) interface, hardware, and drivers. The HAL interface serves as a unified interface for all TV chips, with the specific logic implemented by each individual chip. Drivers primarily include: audio drivers, display drivers, Bluetooth drivers, camera drivers, Wi-Fi drivers, USB drivers, HDMI drivers, sensor drivers (such as fingerprint sensors, temperature sensors, and pressure sensors), and power drivers.

[0077] In some embodiments, the display device 200 can connect an external monitor via HDMI output to form a dual-screen system with its own display 260, thereby achieving a dual-screen display mode. For example... Figure 6 As shown, a dual-screen system may include an image generation (ImageProducer) module, an image rendering (surface rendering) module, a compositing (Composer) module, a video processing unit (VPU hardware) module, as well as a main screen (panel screen) and a secondary screen (HDMI screen).

[0078] During the display process, the media player, camera view, and OpenGL ES graphics interface of the image generation module generate image content and pass it to the image rendering module. The image rendering module includes a primary display unit and an external display unit. The primary display unit controls the content displayed on the main screen; that is, it renders the image content through either the user interface layer (UI layer) or the video layer. To achieve better display effects, the image content in the UI layer can be rendered using GPU components, thus forming the display frames in the compositing module. The compositing module can form the image content using software or hardware; for example, it can be a hardware compositing (HWC) module.

[0079] The compositing module stores image frames using frame buffers and video buffers, and the video processing module forms the image content on the corresponding layers. Specifically, image frames generated during video playback are processed through the video plane to form the image content, while image frames from other images (such as touch traces and graphics) are processed through the primary OSD plane. Finally, the display device 200 uses the blending unit of the video processing module to create the final image, which is then displayed on the main screen.

[0080] The extended display unit controls the content displayed on the secondary screen. After the image generation module passes the image content to the image rendering module, the extended display unit also renders the image content through the UI layer or video layer. Following the main screen's display process, image frames are formed through the image rendering module, compositing module, and video processing module, and then displayed on the secondary screen.

[0081] In some embodiments, to achieve better image display, an extended OSD plane can be additionally set in the video processing module of the display device 200, on top of the main OSD layer and the video layer. This extended OSD plane can be configured with different layer characteristics according to specific display requirements. For example, the bitmap refresh mode of the extended OSD plane can be modified to allow it to refresh in real time by region, thereby improving the display speed of graphics. In this case, the extended OSD plane can be used as an acceleration layer in scenarios with high dynamic range display effects.

[0082] Taking a whiteboard drawing application running on a display device 200 as an example, to improve the responsiveness of the line drawing process, the touch trajectory can be drawn in real time on an extended display layer (acceleration layer) after the user's touch operation is detected. The display level of the acceleration layer can be higher than other layers, and when the user completes the touch input, the lines drawn by the acceleration layer are updated to other layers, thereby reducing input latency and improving the real-time display effect of the whiteboard application drawing process.

[0083] In some embodiments, the extended display layer can also work in conjunction with other graphics to improve display quality. For example, the acceleration layer can respond to user touch events simultaneously with the main OSD layer and draw graphics separately. However, since the acceleration layer is displayed above the main OSD layer, the graphics drawn by the acceleration layer can obscure the graphics drawn by the OSD layer, maintaining real-time response speed. After the user finishes inputting the touch interaction, the content displayed by the acceleration layer can be directly released, and the OSD layer directly forms the drawn graphics, thus eliminating the need for the acceleration layer to refresh the drawn graphics to the OSD layer, reducing graphics blending time. This improves both the real-time display effect of the graphics drawing process and the final presentation quality of the graphics.

[0084] Based on the aforementioned dual-screen display system, users can control the display device 200 to execute different display strategies as needed. For example, the main screen can be connected to the secondary screen via HDMI OUT to achieve dual-screen display. The secondary screen integrates touch components, enabling touch control of its content. Furthermore, the main and secondary screens are connected via USB, and the USB connection channel can be used to transmit touch data; that is, the secondary screen's touch interface connects to the main screen's USB port. After connection, the display device 200 can control the display on both the main and secondary screens. The main and secondary screens can support simultaneous display, meaning the content displayed on the display device 200's monitor 260 (main screen) is the same as that displayed on an external monitor. They can also support different displays, meaning the content displayed on the main and secondary screens is different. Differential display, also known as extended display, is mainly used in scenarios such as: displaying the physical signal source image on the secondary screen; displaying the player image on the secondary screen; displaying two screen sharing images wirelessly; and displaying shared content on the secondary screen in conferencing applications.

[0085] When the secondary screen connected to the display device 200 also supports touch interaction, the user can control the display device 200 or the external terminal device 500 by inputting touch operations on the secondary screen. Specifically, in some embodiments, after the user inputs a touch interaction action on the secondary screen, the secondary screen transmits the touch data from the touch event to the display device 200 via a USB channel. When the display device 200 is also connected to the external terminal device 500, the touch data can be forwarded to the USB channel connected to the terminal device 500 via a built-in conversion module, thereby enabling the terminal device 500 to respond to the touch events input by the user on the secondary screen.

[0086] For example, such as Figure 7As shown, a user can connect a touch-enabled monitor to the HDMI1 interface of the display device 200 based on System A, using it as a secondary screen for the display device 200. Simultaneously, an OPS computer based on System B can be connected to the HDMI2 interface of the display device 200, serving as an external terminal device 500 for the display device 200. Furthermore, the display device 200 also connects to the secondary screen device via USB1 and to the OPS computer via USB2, forming an interactive system capable of supporting reverse control functionality. The monitor 260 of the display device 200 serves as the primary screen. During display, the primary and secondary screens can display the content output by the OPS computer simultaneously or separately.

[0087] When a user inputs touch data on the main screen, the display device 200 transmits the resulting touch data to the OPS computer via the USB2 interface channel, whereby the OPS computer responds to the touch data. Conversely, when a user touches or clicks interactive controls on the secondary screen, the resulting touch data is transmitted to the display device 200 via the USB1 interface channel, then through a switch module within the display device 200 to the USB2 interface channel, and finally to the OPS computer, where it responds to the touch data.

[0088] It should be noted that the touch-to-reverse-control function can generate different interaction logics depending on the application scenario. Specifically, reverse control can include the following scenarios:

[0089] This scenario involves a primary and secondary screen, where display device 200 is connected to an external monitor as the secondary screen, and the built-in monitor 260 of display device 200 serves as the primary screen. In this scenario, display device 200 needs to distinguish between the physical touchscreen and the external USB touchscreen, configuring the physical touchscreen as internal and the external USB touchscreen as external to achieve dual-screen touch control. The application dynamically adjusts the display based on whether the extended mode is enabled.

[0090] In this scenario, the display device 200 is simultaneously connected to an external monitor and an OPS device. The external monitor acts as the secondary screen, while the built-in monitor 260 of the display device 200 serves as the primary screen, and the OPS device acts as the signal source. In this scenario, it's necessary to differentiate the display screens showing the OPS signal. When the OPS signal is displayed on the secondary screen, the USB needs to be switched to the OPS channel to enable the secondary screen to control the OPS channel. Simultaneously, the primary screen's external touch functionality needs to be disabled. When the OPS signal is displayed on the primary screen, the primary screen's touch functionality needs to be enabled and switched to the OPS device. Because the USB channel has been switched to the OPS device, the secondary screen's touch functionality will also be recognized and responded to by the OPS.

[0091] For this scenario, the touch component needs to support disabling the current channel's output via an external channel. When switching from USB to the OPS channel, it needs to determine if USB Touch is connected. If USB Touch is connected, the output must be disabled before switching to the OPS channel. The hardware design of the aforementioned touch-controlled reverse system allows all USB 3.0 ports to be controlled by a single switch.

[0092] The following describes the display process based on the above dual-screen system through specific examples. It should be understood that the following display process is only an example and does not limit the specific display method of the dual-screen system. Other display methods and interface content that can be conceived by those skilled in the art based on the following examples are all within the protection scope of this application.

[0093] like Figure 8 As shown, following the dual-screen system connection method described above, after connecting to the external screen via an HDMI cable, dual-screen mode can be activated. In dual-screen mode, the display device 200 can set the display mode of the secondary screen. The default display mode is extended mode, meaning the secondary screen expands the display of the main screen's content, effectively increasing the main screen's display area. Therefore, the wallpaper displayed on the secondary screen can automatically change to match the wallpaper displayed on the main screen's homepage.

[0094] After activating dual-screen mode, the main screen and secondary screen can display the user interface according to the set display mode. For easier page control, views can be displayed on both the main screen and secondary screen. In the display device 200, a view is an element of the user interface that can be used to display data, layouts, and controls such as buttons, text boxes, lists, and cards. For example, a view can be like... Figure 9 The sidebar and its associated extended menu are shown. That is, the interface displayed on the main screen and the secondary screen can also include a sidebar, which can be used to control the content displayed on the other screen within one screen environment. For example, as... Figure 9 As shown, the sidebar consists of arrow icons displayed on both sides of the interface. Users can click these icons to bring up an extended menu. The extended menu can include shortcut options such as "Back," "Home," "Settings," and "Switch," allowing users to control the dual-screen display process by clicking any option icon.

[0095] In some embodiments, the sidebar's display process can be controlled by an independent application. That is, a sidebar application can run on the display device 200. After the sidebar application starts, the sidebar view can be laid out on both screens to display the sidebar on each screen. Furthermore, the sidebar application can detect the state of both screens in real time and adjust the sidebar's display format and the content of the extended menu accordingly. For example, the sidebar can be hidden and shown based on user input. When a user touches the main screen or secondary screen, the sidebar is displayed; after 3 seconds of inactivity, the sidebar is hidden. Another example is that the sidebar application can detect the display resolution of the main and secondary screens in real time and adjust the display size of the sidebar controls accordingly. Yet another example is that the display device 200 can detect the applications running on the main and secondary screens respectively and control the application responding to the operation based on the application's focus state, so that the touch operation is applied to the appropriate application, improving the user experience.

[0096] It's worth noting that the sidebar application can dynamically adjust its display based on the interaction methods supported by the main and secondary screens. For example, when the main screen supports touch interaction but the secondary screen does not, the sidebar application can control the display of sidebar controls only on the main screen's user interface; conversely, when the main screen does not support touch interaction but the secondary screen does, the sidebar application can control the display of sidebar controls only on the secondary screen's user interface. When both the main and secondary screens support touch interaction, the sidebar application can control the display of sidebar controls on both the main and secondary screens' user interfaces simultaneously.

[0097] Because the sidebar application runs independently, cross-screen control can be achieved based on it. For example, the extended menu associated with the sidebar displayed on the main screen can include a touch lock for locking or unlocking the secondary screen's touch functionality. Users can click the touch lock in the extended menu on the main screen to lock or unlock the secondary screen's touch functionality. Conversely, the extended menu on the secondary screen can also include controls for locking or unlocking the main screen's touch functionality, allowing users to control the main screen's touch interaction functions from the secondary screen.

[0098] To lock and unlock the touch interaction functions of the main screen or secondary screen, the display device 200 can set a control program at the application layer and control the connection relationship at the hardware layer through the application layer control program. For example, when the user sets the secondary screen's touch function to "lock" through the sidebar application, the display device 200 can, upon detecting that the touch lock function control is activated (e.g., "external screen touch lock=1"), control the hardware layer to disconnect the USB connection channel between the secondary screen and the display device 200. This prevents the touch interaction operation data received by the secondary screen from being transmitted to the display device 200 through the USB channel, thus locking the secondary screen's touch function. Alternatively, the application set at the application layer of the display device 200 can act on the built-in conversion module of the display device 200. That is, when the touch lock function control is detected to be activated, the conversion module can be controlled to stop forwarding the touch data transmitted from the secondary screen, thus also locking the secondary screen's touch function.

[0099] The sidebar application also supports operations such as adjusting, switching, and closing the content displayed on the main screen and secondary screen. For example, users can adjust the display mode of the main screen and secondary screen through the sidebar. In extended screen mode, users can click the "Copy Mode" control in the sidebar extended menu to adjust the main screen and secondary screen to a copy mode with the same display content. To implement the copy mode, when the display mode parameter is detected to be changed to the parameter value corresponding to the copy mode, such as "display mode = 01 (copy mode)", the display device 200 can change the signal source output form of the display device 200 to copy, that is, copy the image frames in the frame buffer and send them synchronously to the main screen and secondary screen for display.

[0100] In extended mode, the display device 200 can control the user interface to be displayed on the main screen or the secondary screen according to user settings and application display needs. For example, when the user launches application A, a screen ID can be set for application A. When the screen ID is set to "panel screen", it means that the corresponding interface is displayed on the main screen; when the screen ID is set to "HDMI screen", it means that the corresponding interface is displayed on the secondary screen.

[0101] In some embodiments, the display device 200 can also automatically set the screen ID of the application display according to application parameters. For example, based on the startup order of applications in the display device 200, the application that starts first is displayed on the main screen, and the application that starts later is displayed on the secondary screen. When an application starts, the display device 200 can detect the number of applications currently running besides the one currently starting. When the number of running applications is 0, the screen ID is set to "panel screen"; when the number of running applications is 1, the screen ID is set to "HDMI screen". Since the display device 200 can run multiple applications simultaneously, the screen ID can also be set according to the odd or even number of the application startup order. That is, applications with an odd startup order are displayed on the main screen, and the screen ID is set to "panel screen"; applications with an even startup order are displayed on the secondary screen, and the screen ID is set to "HDMI screen".

[0102] In some embodiments, the display device 200 can construct an application state table to record relevant information about the application's display on the main screen or secondary screen. The recorded information can be stored and accessed through the application state table, allowing other applications to retrieve specific display parameters when needed. For example, a sidebar application can retrieve which screen the application is displayed on by reading the application state table, and can change the display method by modifying the application state table. Specifically, the user can specify the screen to display the user interface through the sidebar's extended menu. When the user clicks the "Main Screen" control in the secondary screen's sidebar extended menu, the sidebar application can modify the secondary screen application's display screen ID to "panelscreen" and update the application state table, causing the application interface displayed on the secondary screen to be moved to the main screen for display.

[0103] The sidebar application also supports dual-screen switching functionality, with a "Switch" control in the sidebar's extended menu. When the user clicks the "Switch" control, the sidebar application can change the display screen ID of application A on the main screen to hdmiscreen and modify the screen display channel at the hardware level to control the display of application A's interface on the secondary screen; simultaneously, it can change the display screen ID of application B on the secondary screen to panel screen to control the display of application B's interface on the main screen.

[0104] The content displayed on the main screen and secondary screen can also be adjusted according to the signal source input. For example, when the display device 200 is not connected to a signal source and the main screen displays the homepage, the secondary screen can display the homepage wallpaper according to the homepage content. When the display device 200 is connected to a signal source and the display screen ID of the signal source is set to HDMI screen, the content corresponding to the signal source (including the no-signal interface) is displayed on the secondary screen.

[0105] Users can also exit dual-screen mode via sidebar tools. For example, by repeatedly clicking the "Back" control in the secondary screen's sidebar, the user can control the display device 200 to stop running the corresponding application B on the secondary screen, thus exiting dual-screen mode. Another example is the "Close" control included in the sidebar's extended menu; clicking the "Close" control can close the application displayed on the corresponding screen.

[0106] When display device 200 exits dual-screen mode, the remaining user interface can be displayed on either the main screen or the secondary screen. For example, since the display device 200's built-in monitor is more convenient for data transfer and interface rendering, when the user closes application A via the sidebar of the main screen, application B displayed on the secondary screen can automatically adjust to the main screen for display.

[0107] It should be noted that in the above example, the display device 200 is used as the signal input source. In actual applications, the signal input source can be the display device 200 or the terminal device 500 connected to the display device 200. When the signal input source is the external terminal device 500, the processes of adjusting, switching, and displaying the screen in the above example can be performed by the terminal device 500, by the display device 200, or by both the display device 200 and the terminal device 500.

[0108] In the dual-screen mode of the display device 200, for the copy mode, since the content displayed on the main screen is the same as that displayed on the secondary screen, when mapping the content of the main screen to the secondary screen, it can be scaled proportionally according to the display size of the secondary screen. In other words, the first user interface displayed on the main screen can be understood as an image. This image is sent to the secondary screen, and the secondary screen will scale the image proportionally according to its display size to form the second user interface for display.

[0109] In extended mode, the content displayed on the main screen and the secondary screen are different. To independently control the applications and display content on the main screen and the secondary screen, a target view in the main screen needs to be mapped to the secondary screen. The target view is the view mapped from the main screen to the secondary screen and can be controlled independently on the secondary screen. For example, Figure 9 The sidebar shown and its associated extended menu.

[0110] However, in the dual-screen interaction scenario of the display device 200 in extended mode, the interaction between the main screen and the secondary screen is limited by their respective display resolutions. Since the view has a fixed display size, if the main screen and the secondary screen have different display resolutions, when mapping the view from the main screen to the secondary screen, the view mapped to the secondary screen will be set according to the main screen's display resolution. This results in a different view ratio displayed on the secondary screen compared to the main screen, affecting the user experience.

[0111] For example, with Figure 9 The sidebar shown is an example, with a display size of 100 pixels. Figure 10 The diagram illustrates the display effect of a sidebar on a main screen and a secondary screen with different resolutions, as provided in this embodiment. The display device 200's built-in monitor 260 serves as the main screen, and an external monitor serves as the secondary screen. The main screen is a 4K resolution screen, and the secondary screen is a 2K resolution screen. When the sidebar is displayed on the main screen, its display ratio is 1 / 40. When mapped to the secondary screen, because it is set according to the main screen's display resolution, its display ratio on the secondary screen is 1 / 20. That is, the main screen and the secondary screen present... Figure 10 The display effect shown shows that the sidebar is twice the size displayed on the secondary screen compared to the size displayed on the main screen.

[0112] In addition, for sidebar application associations such as Figure 9 In the scenario shown, when a user calls up the extended menu, the extended menu will also be set according to the display resolution of the main screen, resulting in the view ratio of the extended menu on the secondary screen being different from the view ratio displayed on the main screen.

[0113] To address the aforementioned problems, some embodiments of this application provide a view display method applied to a display device 200 for adjusting the display ratio of a view on screens of different resolutions. The display device 200, to which the view display method can be applied, includes a first display, a device interface 240, and a controller 250. The first display is the display 260 (main screen) of the display device 200 itself, configured to display a first user interface, which includes a target view. The device interface 240 is configured to connect to a second display (secondary screen) for displaying a second user interface. The display resolution of the first display and the second display are different. The controller 250 is configured to execute the aforementioned view display method, such as... Figure 11 As shown, Figure 11 A flowchart illustrating the view display method provided in this application embodiment specifically includes the following:

[0114] S101: In response to a display instruction indicating that a target view is to be displayed on a second display, obtain the display resolution of the second display.

[0115] Display device 200 connects to an external monitor via device interface 240 as a second monitor, and its built-in monitor 260 serves as the first monitor, achieving dual-screen display. In dual-screen display, to address the issue of different view ratios on the first and second monitors due to their different resolutions, display device 200 can respond to a display command instructing the second monitor to display a target view, obtain the second monitor's resolution, and adjust the target view's parameters accordingly, thereby ensuring the target view's display ratio on the first and second monitors is the same.

[0116] Since there is a difference in view ratio when using dual-screen display in extended mode, the display device 200 can detect the dual-screen mode. When the dual-screen mode is detected to be extended mode, a display instruction is generated to indicate that the target view is displayed on the second display. In other words, the dual-screen mode of the display device 200 is associated with the display instruction indicating that the target view is displayed on the second display. When the dual-screen mode switches to extended mode, a display instruction is generated to indicate that the target view is displayed on the second display, thus mapping the target view from the first display to the second display.

[0117] The dual-screen mode can be switched manually by the user. The display device 200 can have a pre-installed dual-screen mode switching control for users to switch between dual-screen modes. For example, after the display device 200 is powered on, launching the sidebar application controls the first display to show... Figure 9 The sidebar shown allows users to tap the icon to access an extended menu. This menu includes shortcuts such as "Back," "Home," "Settings," "Switch," and "Mode." Users can tap the "Mode" icon to enter the mode switching menu. Then, tapping the "Extended Mode" or "Duplicate Mode" icon will set the dual-screen mode of display device 200 to extended or duplicate mode.

[0118] In addition, the dual-screen mode of the display device 200 can also be set to extended mode by default. When the second display is connected to the device interface 240, the display device 200 can detect the plug-in signal generated when the second display is connected to the device interface 240, trigger the operation of mapping the target view in the first display to the second display, and generate a display instruction to display the target view on the second display.

[0119] In some embodiments, such as Figure 12The diagram shown illustrates the flowchart for calculating view parameters according to an embodiment of this application. The display resolution can be obtained based on the display identification data (DisplayId) of the second display. The display identification data is a data type that identifies the display; it is assigned to the display by the operating system during driver initialization and is the identifier (id) used by the DisplayManagerService in the display device 200 system to manage the screen. Through this display identification data, the screen information of the corresponding display can be obtained.

[0120] When the second display is connected to the device interface 240, the display device 200 can receive the plug-in / plug-out signal generated when the second display is connected to the device interface 240. In response to the plug-in / plug-out signal, it can obtain the display identification identifier (DisplayID) of the second display. Based on the display identification identifier, it can obtain the display identification object (Display). The display identification object is used to represent the second display.

[0121] After obtaining the display identification object representing the second display, the context object of the display identification object can be obtained, and the display resolution of the second display can be read from the context object. The context object is used to characterize the application environment information of the second display, and it includes all screen information of the second display.

[0122] Understandably, in the display device 200 system, Context is an abstract class that provides an interface for global information (such as application environment information). When acquiring resource files, the display device 200 can use the `context.getResource` algorithm to obtain application resource data, such as images, strings, colors, and dimensions, and use it in the application's user interface. In this embodiment, when mapping the target view of the first display to the second display, the ratio of the target view displayed on the second display differs from that on the first display because the acquired resource data, such as images and dimensions, is obtained from the context object of the first display, which is not compatible with the second display. For example, an image that displays normally in full screen on a 4K resolution first display will only display 1 / 4 of the image on a 2K resolution first display.

[0123] Therefore, the display device 200 can obtain the context object of the second display, use the context object to import, and adjust the view parameters of the target view mapped from the first display to the second display so that the target view is displayed in the second display at a normal scale.

[0124] In some embodiments, to improve performance, the display device 200 may create and maintain a database to store the current dual-screen mode of the display device 200. When the dual-screen mode of the display device 200 changes, such as when the user switches the dual-screen mode through the sidebar, the system and applications of the display device 200 can listen to the changes in this database and make corresponding changes to the dual-screen mode.

[0125] In other words, the display device 200 can create a database to store the display identification identifier of the second display. When the second display is connected to the device interface 240, in response to the plugging / unplugging signal generated when the second display is connected to the device interface 240, the display identification identifier of the second display is obtained and stored in the database. When the dual-screen mode of the display device 200 is switched to extended mode, a display command is generated indicating that a target view is displayed on the second display. In response to the display command, the display identification identifier of the second display is read from the database, and the display resolution of the second display is obtained based on the display identification identifier of the second display.

[0126] Furthermore, when multiple displays are connected to the display device 200, i.e., there are multiple second displays, the display identification identifier of the target second display can be obtained from the database according to the second display to be mapped by the user, i.e., the target second display, and the display resolution of the target second display can be obtained according to the display identification identifier. The view parameters of the target view mapped to the target second display can be adjusted according to the display resolution.

[0127] In some embodiments, such as Figure 13 The diagram illustrates a flowchart of a display identification identifier process provided in this embodiment. To distinguish between a first display and a second display, the display device 200 can differentiate them based on the attribute values ​​of the display identification identifier, including a first attribute value and a second attribute value. The display device 200 can acquire the display identification identifiers loaded by itself and iterate through the loaded display identification identifiers. If the display identification identifier has a first attribute value, it is marked as the display identification identifier of the first display. If the display identification identifier has a second attribute value, it is marked as the display identification identifier of the second display.

[0128] For example, the display identification identifier of the first display has an attribute value of 0, and the display identification identifier of the second display has an attribute value of 1. The display device 200 can obtain an array of DisplayIds of all currently connected displays using the DisplayManager.getDisplays algorithm. It iterates through this array, marking those with DisplayId = 0 as the DisplayId of the first display, and those with DisplayId = 1 as the DisplayId of the second display.

[0129] When distinguishing the display identification mark, it can be done when the second display connection device interface 240 is detected, that is, the display device 200 can respond to the plugging / unplugging signal generated when the second display connection device interface 240 is detected and perform the above-mentioned steps to distinguish the display identification mark. Alternatively, it can be done when the display device 200 is powered on, that is, the display device 200 can respond to the power-on command of the display device 200 and perform the above-mentioned steps to distinguish the display identification mark.

[0130] In some embodiments, the different display ratios of the mapped views are due to the different display resolutions of the two screens. Therefore, after obtaining the display resolution of the second display based on the display identification identifier of the second display, the display device 200 can obtain the display resolution of the first display. If the display resolution of the first display is the same as that of the second display, the target view of the first display can be directly mapped to the second display. If the display resolution of the first display is different from that of the second display, the subsequent step of adjusting the view parameters of the target view according to the display resolution of the second display is then performed.

[0131] S102: Calculate view parameters based on display resolution.

[0132] After obtaining the display resolution of the second monitor, the view parameters of the target view can be adjusted according to the display resolution. These view parameters are used to set the display ratio of the target view on the second user interface. The display ratio of the target view on the second user interface is related to the display ratio of the target view on the first user interface.

[0133] Since the display size of the target view is a fixed value, that is, the physical pixels it occupies are fixed, for example, with Figure 9 Taking the sidebar shown as an example, its display size is 100 pixels. Since the first and second displays have different resolutions, the target view occupies 100 physical pixels on both displays, resulting in an inconsistency in the view's aspect ratio on both screens. Therefore, based on the obtained display resolution of the second display, the physical pixels occupied by the target view can be scaled to calculate view parameters that ensure the target view has the same aspect ratio on both screens.

[0134] When calculating view parameters, the display device 200 can obtain the abstract pixels and position parameters of the target view. The abstract pixels are either density-independent pixels (dp) or scale-independent pixels (sp), and the position parameters are preset positions relative to the screen boundaries.

[0135] It is understandable that the display device 200 can automatically adjust the display position of the target view based on preset position parameters and the screen boundaries of different screens, and is not affected by the display resolution of the first and second displays. That is, the display device 200 can set the display position of the target view on the second user interface according to preset target view position parameters.

[0136] The display size of the target view needs to be considered in relation to the display resolution of the second monitor. Therefore, after obtaining the display resolution of the second monitor and the abstract pixels of the target view, the display device 200 can convert the abstract pixels into physical pixels based on the display resolution. The physical pixels are the pixels displayed on the screen that determine the display size of the target view on the second user interface.

[0137] Finally, view parameters are generated based on position parameters and physical pixels to create a target view mapped to a second display.

[0138] In this embodiment, abstract pixels are introduced to describe the display size of the target view. Since both abstract pixels and physical pixels are pixel units describing the view, they can be converted according to a preset unit conversion method. Therefore, when mapping the target view of the first display to the second display, the abstract pixels can be converted into physical pixels corresponding to the display resolution of the second display.

[0139] For example, taking a sidebar as an example, a 50dp sidebar has a display size of 100px when displayed on a 4K resolution first monitor, and a 50dp sidebar has a display size of 50px when displayed on a 2K resolution second monitor, so that the display ratio of the sidebar on the first monitor is the same as the display ratio of the target view on the second monitor.

[0140] S103: Create a target view based on view parameters, and add the target view to the second user interface.

[0141] After obtaining the view parameters, a target view to be displayed on the second monitor can be created based on the view parameters, and the target view can be added to the second user interface of the second monitor. The display device 200 can achieve the creation and addition of the target view through the WindowManager service, which is used to manage the state and attributes of windows, the addition, deletion, and updating of views, window order, message collection and processing, etc.

[0142] Therefore, the display device 200 can obtain the window management service of the second display according to the display identification identifier of the second display. A layout parameter class (LayoutParams) is created in the window management service, and view parameters are defined in the layout parameter class, thereby introducing the view parameters into the window management service of the second display.

[0143] When creating and adding a target view, the display device 200 can call the view adding algorithm (WindowManger.addView) in the window management service of the second display, use the view adding algorithm to create the target view based on the view parameters, and add the target view to the second user interface.

[0144] For the extended menu associated with the target view, it can also be displayed based on the above creation and addition steps. That is, the display device 200 can respond to the display instruction of the extended menu of the target view, call the view addition algorithm in the window management service of the second display, use the view addition algorithm to create the extended menu based on the view parameters, and add the extended menu to the second user interface.

[0145] For example, taking a sidebar application as an example, the first monitor has a 4K resolution, and the second monitor has a 2K resolution. After obtaining the DisplayId of the second monitor, the display device 200 obtains the WindowManager of the second monitor based on the DisplayId, and then sets the view parameters of the sidebar on the second monitor in the LayoutParams of the WindowManager. Finally, WindowManager.addView is used to add the sidebar to the second user interface, so that the first and second monitors display as shown in the image. Figure 14 The interface shown is as follows.

[0146] When a user clicks the arrow icon corresponding to the sidebar on the second display to input a command to display the extended menu of the sidebar, the display device 200 can obtain the WindowManager of the second display and then set the view parameters of the extended menu in the WindowManager's LayoutParams. Finally, WindowManager.addView is used to add the extended menu to the second user interface, so that the first and second displays appear as shown. Figure 15 The interface shown is as follows.

[0147] Based on the above-described view display method, a display device 200 is also provided in some embodiments of this application, including: a first display, a device interface 240, and a controller 250. The first display is configured to display a first user interface, which includes a target view; the device interface 240 is configured to connect to a second display, which displays a second user interface; the display resolution of the first display and the display resolution of the second display are different; the controller 250 is configured to execute the following program steps:

[0148] In response to a display instruction instructing the display of a target view on a second display, the display resolution of the second display is obtained.

[0149] The display resolution is obtained based on the display identification mark of the second display.

[0150] Calculate view parameters based on display resolution.

[0151] The view parameter is used to set the display scale of the target view on the second user interface. The display scale of the target view on the second user interface is related to the display scale of the target view on the first user interface.

[0152] Create a target view based on view parameters, and add the target view to the second user interface.

[0153] The same or similar parts among the various embodiments in this specification can be referred to mutually, and will not be repeated here.

[0154] As can be seen from the above technical solutions, the display device and view display method provided in the above embodiments can, in response to a display command instructing the display of a target view on a second display, obtain the display resolution of the second display and calculate view parameters based on the display resolution. The second display is an external display connected to the display device, and the display resolution is obtained based on the display identification identifier of the second display. The view parameters are used to set the display ratio of the target view on the second user interface, and the display ratio of the target view on the second user interface is associated with the display ratio of the target view on the first user interface. The target view is then created based on the view parameters and added to the second user interface. This method can adjust the view parameters according to the resolution of the display when connecting displays of different resolutions, solving the problem of different display ratios of views caused by different screen resolutions in a dual-screen system.

[0155] When the display device 200 is connected to the second display via the device interface 240 and the dual-screen mode is in extended mode, the application can be launched on either the first or second display according to user settings and application needs. That is, the application window corresponding to the application is displayed on either the first or second display, allowing the user to interact with different applications through the first and second displays respectively. For user convenience, sidebars can be displayed on both the first and second displays based on the above view display method, providing users with quick operation menus for operating different displays.

[0156] by Figure 9 Taking the sidebar shown as an example, users can click the arrow icon in the sidebar to bring up an extended menu. The extended menu may include shortcut options such as "Back," "Home," "Settings," and "Switch," allowing users to control the dual-screen display process by clicking any option icon. For instance, users can enter a command to return to an application via the sidebar to exit that application. Clicking the "Back" icon in the sidebar displayed on the first screen will exit the application in the topmost application window shown on the first screen.

[0157] However, since both the first and second displays are controlled by the display device 200 system, when the user inputs the application return command through the sidebars displayed on the first and second displays, the display device 200 will exit the applications sequentially according to the order in which they were started. It cannot control the exit of the application windows displayed on each display, resulting in application exit errors.

[0158] For example, such as Figure 16 The diagram illustrates the first and second displays showing different application windows, as provided in the embodiment of the application. The user controls the display device 200 to sequentially launch application A, application B, and application C. The application windows of application A and application C are displayed on the first display, while the application window of application B is displayed on the second display. Following native Android logic, when the user inputs an application return command via the sidebar, the display device 200 will exit application C, application B, and application A sequentially, according to the application launch order.

[0159] When the user clicks the "back" icon in the sidebar displayed on the first screen, display device 200 exits application C according to the application launch order. However, when the user clicks the "back" icon in the sidebar displayed on the second screen, intending to exit application B, according to Android's native logic, display device 200 will also exit application C according to the application launch order.

[0160] Therefore, to solve the above problems, some embodiments of this application provide a multi-screen control method applied to a display device 200, used to control the corresponding application to exit on a display based on an application return command input on the user interface of different displays. That is, if an application return command is input on the first display, the application response on the first display is controlled; if an application return command is input on the second display, the application response on the second display is controlled.

[0161] The display device 200 capable of applying the multi-screen control method includes: a first display, a device interface 240, and a controller 250. The first display is the display 260 of the display device 200 itself, configured to display a first user interface, which includes an application window of at least one application. The device interface 240 is configured to connect to a second display for displaying a second user interface, which also includes an application window of at least one application. The controller 250 is configured to execute the multi-screen control method, specifically including the following:

[0162] The display device 200 can maintain two sets: a first application set and a second application set. The first application set includes applications whose application windows are displayed on the first display, and the second application set includes applications whose application windows are displayed on the second display. The first and second application sets are used to distinguish between applications whose application windows are displayed on the first and second displays. When the user inputs an application return command through the first user interface, the corresponding application exits according to the first application set; similarly, when the user inputs an application return command through the second user interface, the corresponding application exits according to the second application set.

[0163] The application list in the system stack of display device 200 records the currently launched applications and their launch order. Specifically, the application list includes applications whose windows are displayed on the first display, applications whose windows are displayed on the second display, and the list order. The list order represents the launch order of the applications. Display device 200 can read the display identification ID (DisplayId) of the first display and the display identification ID (DisplayId) of the second display, and generate a first application set and a second application set based on the display identification IDs of the first and second displays and the application list.

[0164] That is, in some embodiments, such as Figure 17The diagram illustrates the process of creating a first application set and a second application set according to an embodiment of this application. The display device 200 can obtain the application list in its current system stack, obtain the display identification identifiers of the first and second displays, and create the first and second application sets. It queries the application list for the first application associated with the display identification identifier of the first display and adds it to the first application set. Similarly, it queries the application list for the second application associated with the display identification identifier of the second display and adds it to the second application set.

[0165] Since the application list also includes the application launch order, in order to facilitate subsequent querying of the applications in the topmost application window of the user interface, the display device 200 can obtain the list order of the application list and sort the applications in the first application set and the second application set according to the list order.

[0166] For example, such as Figure 18 The diagram illustrates an application list, a first application set, and a second application set provided in an embodiment of this application. The display device 200 sequentially launches applications A, B, C, and D. The application windows of applications A and C are displayed on the first display, while the application windows of applications B and D are displayed on the second display. The application launched first in the application list in the current system stack of the display device 200 is displayed last, i.e., the list order is application D, application C, application B, application A. By sorting the applications in the first and second application sets according to this list order, the application order of the first application set is application C, application A, and the application order of the second application set is application D, application B.

[0167] In extended mode, the display device 200 can set screen identifiers for different applications according to user settings and application display needs, thereby controlling whether the application window is displayed on the first display or the second display. That is, the display device 200 can respond to the launch command of the target application, detect the screen identifier of the target application, and if it is a screen identifier indicating display on the first display, then the application window of the target application is displayed on the first display; if it is a screen identifier indicating display on the second display, then the application window of the target application is displayed on the second display. Therefore, the display device 200 can distinguish between the first application and the second application based on the application's screen identifier.

[0168] In some embodiments, the screen identifier set for different applications can be the display identification identifier (DisplayId) of the display. That is, if the display identification identifier set for an application is the display identification identifier of the first display, it indicates that the application window of that application is displayed on the first display. If the display identification identifier set for an application is the display identification identifier of the second display, it indicates that the application window of that application is displayed on the second display.

[0169] Therefore, when distinguishing between the first and second applications, the display device 200 can traverse the application list to obtain the application information (RunningTaskInfo), where RunningTaskInfo includes relevant application information. It then reads key fields from the application information; these key fields characterize the fields describing the display identification identifier, and queries the display identification identifier set for the application based on these key fields.

[0170] After obtaining the display identification identifier set for the application, if the display identification identifier set for the application is the same as the display identification identifier of the first display, the application is marked as the first application; if the display identification identifier set for the application is the same as the display identification identifier of the second display, the application is marked as the second application.

[0171] For example, the DisplayId of the first display is 0, and the DisplayId of the second display is 1. When iterating through the application list, if an application's DisplayId is 0, it means its application window is displayed on the first display and is marked as the first application. If an application's DisplayId is 1, it means its application window is displayed on the second display and is marked as the second application.

[0172] After creating the first and second application sets, changes can be made based on the subsequent startup and exit of applications on the display device 200. When the display device 200 displays a new application window on the first or second display, the application corresponding to the new application window is added to the first or second application set. Similarly, when the display device 200 exits the first or second application, the first application is removed from the first application set or the second application is removed from the second application set.

[0173] After establishing the first application set and the second application set, the display device 200 can control the exit of applications on different displays according to the first application set and the second application set, such as... Figure 19 As shown, Figure 19 This is a flowchart illustrating multi-screen control provided in an embodiment of this application. In the case of an application return command input via the first user interface, the controller 250 is configured to execute the following program steps:

[0174] S201: In response to an application return instruction input through the first user interface, obtain the first application set.

[0175] S202: Query the first target application in the first application set.

[0176] After obtaining the first set of applications, query the first target application within the first set of applications. The first target application is the application in the topmost application window of the first user interface.

[0177] Since the first application set and the second application set include an application order, the display device 200 can mark the first or last application as the first target application according to the application order in the first application set.

[0178] Understanding this, marking the first element as the primary target application means that the application launched first will be displayed last. Marking the last element as the primary target application means that the application launched first will be displayed first. For example, using... Figure 18 Taking the application order shown as an example, the first application (application C) in the first application set is marked as the first target application.

[0179] S203: Control the first target application to respond to the application return command.

[0180] After the first target application is found, the display device 200 can control the first target application to respond to the application's return command and exit the first target application.

[0181] In the native logic of the Android system, the response to commands is controlled by whether an application has focus. That is, the application with focus responds to commands issued by the system. Therefore, after finding the first target application, system focus can be assigned to the first target application, and the first target application with system focus can be controlled to respond to the application's return command and exit the application window displayed on the first display.

[0182] The allocation of system focus can be achieved through the ActivityTaskManage system. Using the ActivityTaskManage system on display device 200, the system can detect the currently focused application, remove the focus tag from the original focused application, and set the focus tag for the first target application. For example, with... Figure 18 Taking the application list shown as an example, according to Android's native logic, the original focused application is the last launched application, D. The user enters a return command through the user interface on the first screen, aiming to exit application C. Therefore, the system focus can be removed from application D and assigned to application C.

[0183] In the display device 200 system, each application has an application identifier (TaskID), which is a unique identifier assigned to the application by the system. The display device 200 can obtain the application identifier of the first target application, call the ActivityTaskManage.setfocusedtask algorithm in the ActivityTaskManage, and set the focus flag for the application identifier of the first target application so that the first target application can obtain system focus.

[0184] Similarly, for application return commands input through a second user interface, such as... Figure 19 As shown, controller 250 is configured to execute the following program steps:

[0185] S301: In response to an application return command input through a second user interface, obtain the second application set.

[0186] The second application set includes applications whose application windows are displayed on the second display. The second application set is generated by reading the display identification identifier of the second display.

[0187] S302: Query the second target application in the second application set.

[0188] The second target application is the application window at the top of the second user interface.

[0189] S303: Controls the second target application to respond to the application return command.

[0190] For example, the second display shows something like Figure 9 As shown in the sidebar, clicking the arrow icon in the sidebar brings up an expanded menu, and clicking the "Back" icon in the expanded menu sends an application return command to the display device 200. In response to this application return command, the display device 200 retrieves a pre-established second set of applications. Figure 18 Taking the second application set shown as an example, this second application set includes application D and application B in application order. This application order indicates that the application that is launched first is displayed last. The display device 200 can find that the application window at the top of the second user interface is application D in the second application set. Then, through the ActivityTaskManage, it assigns system focus to application D, so that application D with system focus responds to the application return command and exits the application window displayed on the second display.

[0191] After the application window of application D exits, the user clicks the "back" icon in the extended menu again. The display device 200 can then follow the above steps to find the application of the topmost application window of the second user interface as application B in the second application set, and then assign system focus to application B, so that application B with system focus responds to the application return command and exits the application window displayed on the second display.

[0192] The above embodiments maintain two lists to control the exit of the corresponding application on different displays by controlling the user interface input of application return commands on those displays. To reduce maintenance costs, this application also provides a dual-screen control method applied to the aforementioned display device 200, such as... Figure 20 As shown, Figure 20 This is a schematic diagram illustrating the exit process of an application whose application window is displayed on a first display screen, as provided in this application embodiment. Specifically, for the case of an application return command input through the first user interface, the following content is included:

[0193] S401: In response to an application return command input through the first user interface, obtain the display identification identifier of the first display.

[0194] S402: Get the list of applications in the current system stack of the display device.

[0195] The application list includes applications whose application windows are displayed on the first display and applications whose application windows are displayed on the second display. The order of the application list represents the launch order of the applications.

[0196] S403: Iterate through the application list in list order.

[0197] S404: If the display identification identifier of the target application being traversed is the same as the display identification identifier of the first display, assign system focus to the target application.

[0198] S405: Controls the target application with focus to respond to the application return command.

[0199] The display device 200 can obtain the display identification identifier of the display based on the application return command entered in the user interface of different displays, and find the application of the topmost application window associated with the display identification identifier in the application list in the current system stack of the display device 200 based on the display identification identifier, and control the application to exit.

[0200] For example, a user inputs an application return command via a sidebar displayed on the first display. In response to this application return command, the display device 200 obtains the display identification identifier (DisplayId) of the first display and retrieves a list of applications in the system stack using the ActivityManager.getRunningTasks algorithm. The order of this list represents the launch order of the applications. The application list is traversed in order to obtain the application information (RunningTaskInfo) and the display identification identifier (DisplayId) set for the application is read from the application information, which indicates the display on which the application window of the application is displayed.

[0201] by Figure 18 Taking the application list shown as an example, the list includes applications D, C, B, and A in list order. This list order indicates that the first application launched is displayed last. Traversing the application list in list order, the first application encountered is application D. Comparing the DisplayId of application D with the DisplayId of the first display, they are different, indicating that application D's application window is not displayed on the first display. Traversing to the second application, application C, comparing its DisplayId with the DisplayId of the first display, they are the same, indicating that application C's application window is the topmost application window on the first display.

[0202] According to Android's native logic, the application currently holding system focus is the last launched application, D. Therefore, through the ActivityTaskManage of the display device 200 system, system focus can be removed from application D and assigned to application C. This allows application C, which currently holds system focus, to respond to the application return command and exit the application window displayed on the first display.

[0203] Similarly, for application return commands input through a second user interface, such as... Figure 21 As shown, Figure 21 The schematic diagram of the exit process of an application whose application window is displayed on a second display according to an embodiment of this application specifically includes the following:

[0204] S501: In response to an application return command input through the second user interface, obtain the display identification identifier of the second display.

[0205] S502: Get the list of applications in the current system stack of the display device.

[0206] The application list includes applications whose application windows are displayed on the first display and applications whose application windows are displayed on the second display. The order of the application list represents the launch order of the applications.

[0207] S503: Traverse the application list in list order.

[0208] S504: If the display identification identifier of the target application being traversed is the same as the display identification identifier of the second display, assign system focus to the target application.

[0209] S505: Controls the target application with focus to respond to the application return command.

[0210] For application return commands entered through the second user interface, the steps for application return commands entered through the first user interface can be referred to, and will not be repeated here.

[0211] Based on the above multi-screen control method, a display device 200 is also provided in some embodiments of this application, including: a first display, a device interface 240, and a controller 250. The first display is configured to display a first user interface, which includes an application window of at least one application; the device interface 240 is configured to connect to a second display, which displays a second user interface, which also includes an application window of at least one application; the controller 250 is configured to execute the following program steps:

[0212] S201: In response to an application return instruction input through the first user interface, obtain the first application set.

[0213] The first application set includes applications whose application windows are displayed on the first display, and the first application set is generated by reading the display identification identifier of the first display.

[0214] S201: Query the first target application in the first application set.

[0215] The first target application is the application window at the top of the first user interface.

[0216] S201: Control the first target application to respond to the application return instruction.

[0217] Controller 250 is also configured to perform the following procedure steps:

[0218] S301: In response to an application return command input through a second user interface, obtain the second application set.

[0219] The second application set includes applications whose application windows are displayed on the second display. The second application set is generated by reading the display identification identifier of the second display.

[0220] S302: Query the second target application in the second application set.

[0221] The second target application is the application window at the top of the second user interface.

[0222] S303: Controls the second target application to respond to the application return command.

[0223] Controller 250 is also configured to perform the following procedure steps:

[0224] S401: In response to an application return command input through the first user interface, obtain the display identification identifier of the first display.

[0225] S402: Get the list of applications in the current system stack of the display device.

[0226] The application list includes applications whose application windows are displayed on the first display and applications whose application windows are displayed on the second display. The order of the application list represents the launch order of the applications.

[0227] S403: Iterate through the application list in list order.

[0228] S404: If the display identification identifier of the target application being traversed is the same as the display identification identifier of the first display, assign system focus to the target application.

[0229] S405: Controls the target application with focus to respond to the application return command.

[0230] Controller 250 is also configured to perform the following procedure steps:

[0231] S501: In response to an application return command input through the second user interface, obtain the display identification identifier of the second display.

[0232] S502: Get the list of applications in the current system stack of the display device.

[0233] The application list includes applications whose application windows are displayed on the first display and applications whose application windows are displayed on the second display. The order of the application list represents the launch order of the applications.

[0234] S503: Traverse the application list in list order.

[0235] S504: If the display identification identifier of the target application being traversed is the same as the display identification identifier of the second display, assign system focus to the target application.

[0236] S505: Controls the target application with focus to respond to the application return command.

[0237] The same or similar parts among the various embodiments in this specification can be referred to mutually, and will not be repeated here.

[0238] As can be seen from the above technical solutions, the display device and multi-screen control method provided in the above embodiments can, in response to an application return command input through a first user interface, obtain a first application set, query a first target application in the first application set, and exit the first target application. The first application set includes applications whose application windows are displayed on the first display, and is generated by reading the display identification identifier of the first display. The first target application is the application in the topmost application window of the first user interface. The method can control the corresponding application on a display to exit based on application return commands input through user interfaces on different displays, thus solving the problem of incorrect application return in a multi-screen system.

[0239] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or certain parts of the embodiments of the present invention.

[0240] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0241] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the described embodiments and various different variations of embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that, include: A first display is configured to display a first user interface, the first user interface including a target view; The device interface is configured to connect to a second display, which is used to display a second user interface, and the display resolution of the first display is different from that of the second display; The controller is configured as follows: In response to a display instruction indicating that a target view is displayed on the second display, the display resolution of the second display is obtained, the display resolution being obtained based on a display identification identifier of the second display, and the target view being a view that is mapped from the first display to the second display and can be independently controlled by the second display; Obtain the abstract pixels and position parameters of the target view, wherein the abstract pixels are density-independent pixels or scaling-independent pixels, and the position parameters are preset positions relative to the screen boundary; Based on the display resolution, the abstract pixels are converted into physical pixels, where the physical pixels are the pixels of the target view displayed on the screen; View parameters are generated based on the position parameters and the physical pixels, and the view parameters are used to set the display ratio of the target view on the second user interface; The target view is displayed at the same scale on the second user interface as the target view is displayed on the first user interface; The target view is created based on the view parameters, and the target view is added to the second user interface.

2. The display device according to claim 1, characterized in that, The controller is also configured to: Acquire the plugging / unplugging signal generated when the second display is connected to the device interface; In response to the plugging / unplugging signal, the display identification identifier of the second display is obtained; A display identification object is obtained based on the display identification identifier, and the display identification object is used to represent the second display; Obtain the context object of the display recognition object, the context object being used to characterize the application environment information of the second display; The display resolution is read from the context object.

3. The display device according to claim 1, characterized in that, The controller is also configured to: Create a database for storing the display identification identifier of the second display; In response to a plugging / unplugging signal generated when the second display is connected to the device interface, the display identification identifier of the second display is acquired and stored in the database; In response to a display instruction instructing the display of a target view on the second display, the display identification identifier is read from the database, and the display resolution is obtained based on the display identification identifier.

4. The display device according to claim 1, characterized in that, The controller is also configured to: Detect the dual-screen mode of the display device; When the dual-screen mode is detected to be an extended mode, a display instruction is generated to indicate that the target view is displayed on the second display.

5. The display device according to claim 1, characterized in that, The controller is also configured to: In response to the power-on command of the display device, obtain the display identification identifier loaded by the display device; Iterate through the loaded display identification identifiers; If the display identification identifier is a first attribute value, then the display identification identifier is marked as the display identification identifier of the first display; If the display identification identifier is a second attribute value, then the display identification identifier is marked as the display identification identifier of the second display.

6. The display device according to claim 1, characterized in that, The controller is also configured to: Obtain the window management service of the second display according to the display identification identifier of the second display; Create a layout parameter class in the window management service; Define the view parameters in the layout parameter class.

7. The display device according to claim 6, characterized in that, The controller is also configured to: Invoke the view addition algorithm in the window management service; Using the view addition algorithm, the target view is created based on the view parameters, and the target view is added to the second user interface.

8. The display device according to claim 6, characterized in that, The controller is also configured to: In response to the command to display the extended menu of the target view, the view addition algorithm in the window management service is invoked; Using the view-adding algorithm, an extended menu is created based on the view parameters, and the extended menu is added to the second user interface.

9. A view display method, characterized in that, An application to a display device, the display device including a first display, a device interface, and a controller, the device interface being connected to a second display, the first display having a different display resolution than the second display, the first display being used to display a first user interface including a target view, and the second display being used to display a second user interface, the method comprising: In response to a display instruction instructing the display of a target view on a second display, the display resolution of the second display is obtained, the display resolution being obtained based on a display identification identifier of the second display, and the target view being a view mapped from the first display to the second display and which can be independently controlled by the second display; Obtain the abstract pixels and position parameters of the target view, wherein the abstract pixels are density-independent pixels or scaling-independent pixels, and the position parameters are preset positions relative to the screen boundary; Based on the display resolution, the abstract pixels are converted into physical pixels, where the physical pixels are the pixels of the target view displayed on the screen; View parameters are generated based on the position parameters and the physical pixels. The view parameters are used to set the display ratio of the target view on the second user interface. The display ratio of the target view on the second user interface is the same as the display ratio of the target view on the first user interface. The target view is created based on the view parameters, and the target view is added to the second user interface.

Images