Display equipment and split-screen display method

Abstract

The invention provides a display device and a split-screen display method. The display device includes a display, a user interface, a communicator, a memory, and a processor connected to the display, the user interface, the communicator, and the memory, the processor configured to respond to a first selection input for a first application and a second application received through a user input interface, a display is controlled to display a selection result of the first application and a selection result of the second application, a window arrangement mode selection control and a split-screen control are displayed, and the first application and the second application support split-screen display; in response to a second selection input for the window arrangement mode selection control received through the user input interface, determining an arrangement mode of the first split screen area and the second split screen area; and in response to a first control input for the split-screen control received through the user input interface, controlling the display to display an interface of the first application in the first split-screen area and display an interface of the second application in the second split-screen area.

Description

Display device and split-screen display method

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This disclosure claims priority to the Chinese patent applications filed with the China Patent Office on December 30, 2022, with application number 202211721340.4, filed with the China Patent Office on April 14, 2023, with application number 202310402904.6, and filed with the China Patent Office on December 30, 2022, with application number 202211737411.X, the entire contents of which are incorporated by reference into this disclosure. Technical Field

[0003] The present disclosure relates to display technology, and more particularly to a display device and a split-screen display method. Background Art

[0004] Currently, the software system of smart devices supports split-screen display of applications, but the applications for split-screen display can only be obtained from the recently accessed application list, and the order in which the applications are loaded in the recently accessed application list is fixed, which greatly limits the scope of applications for split-screen display. Users are also unable to split-screen display applications as they wish, which is not friendly to the user's interactive experience.

[0005] Summary of the Invention

[0006] An embodiment of the present disclosure provides a display device, including:

[0007] a display configured to display images and / or a user interface;

[0008] A user interface configured to receive instructions from a user;

[0009] a communicator configured to communicate with an external device according to a predetermined protocol;

[0010] a memory configured to store computer instructions and data associated with the display device;

[0011] At least one processor, connected to the display, user interface, communicator and memory, is configured to execute computer instructions to cause the display device to perform:

[0012] In response to a first selection input for a first application and a second application received through the user input interface, controlling the display to display a selection result of the first application and a selection result of the second application, and presenting a window arrangement selection control and a split-screen control, wherein the first application and the second application support split-screen display;

[0013] In response to a second selection input of a window arrangement selection control received through the user input interface, determining an arrangement of the first split screen area and the second split screen area;

[0014] In response to a first control input for the split-screen control received through the user input interface, the display is controlled to display the interface of the first application in the first split-screen area and to display the interface of the second application in the second split-screen area.

[0015] The present disclosure also provides a split-screen display method, including:

[0016] In response to a first selection input for a first application and a second application, displaying a selection result of the first application and a selection result of the second application, and presenting a window arrangement selection control and a split-screen control, wherein the first application and the second application support split-screen display;

[0017] In response to a second selection input of the window arrangement selection control, determining an arrangement of the first split screen area and the second split screen area;

[0018] In response to a first control input to the split-screen control, the interface of the first application is displayed in the first split-screen area, and the interface of the second application is displayed in the second split-screen area. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] FIG1 illustrates an operation scenario between a display device and a control apparatus according to some embodiments;

[0020] FIG2 is a block diagram of a hardware configuration of a control device 100 according to some embodiments;

[0021] FIG3 is a block diagram of a hardware configuration of a display device 200 according to some embodiments;

[0022] FIG4 is a diagram illustrating a software configuration in the display device 200 according to some embodiments;

[0023] FIG5 is a flowchart of a split-screen display method according to some embodiments;

[0024] FIG6 is a first schematic diagram of an interface of a display device according to some embodiments;

[0025] FIG7 is a second schematic diagram of an interface of a display device according to some embodiments;

[0026] FIG8 is a third schematic diagram of an interface of a display device according to some embodiments;

[0027] FIG9 is a fourth schematic diagram of an interface of a display device according to some embodiments;

[0028] FIG10 is a flowchart of a split-screen display method according to some embodiments;

[0029] FIG11 is a schematic diagram of a preset loading interface according to some embodiments;

[0030] FIG12 is a fifth schematic diagram of an interface of a display device according to some embodiments;

[0031] FIG13 is a flow chart of a key input processing method according to some embodiments;

[0032] FIG14 is a flow chart of a method for processing an application startup instruction according to some embodiments;

[0033] FIG15 is another flow chart of a split-screen display method according to some embodiments;

[0034] FIG16 is a flowchart illustrating a display device displaying a first application and a second application in split screen according to some embodiments;

[0035] FIG17 is a flowchart illustrating a display device displaying a first application and a second application in split screen according to some embodiments;

[0036] FIG18 is a flowchart of Android boot-up according to some embodiments;

[0037] FIG19 is a schematic diagram of a split-screen operation process provided by SystemUI according to some embodiments;

[0038] FIG20 is a flow chart illustrating a display device displaying a focus frame according to some embodiments;

[0039] FIG21 is a schematic diagram of a system architecture of a display device according to some embodiments;

[0040] FIG22 is a flowchart illustrating a split-screen display of a first application and a second application on a television according to some embodiments;

[0041] FIG23 is another flowchart illustrating a split-screen display of a first application and a second application on a television according to some embodiments;

[0042] FIG24 is another flowchart illustrating a split-screen display of a first application and a second application on a television according to some embodiments;

[0043] FIG25 is another flowchart illustrating a split-screen display of a first application and a second application on a television according to some embodiments;

[0044] FIG26 is a flow chart illustrating a display device responding to a mouse operation according to some embodiments;

[0045] 27 is a flow chart illustrating a display device zooming a focus window using a zoom button according to some embodiments;

[0046] 28 is a flow chart illustrating a display device zooming a focus window using a zoom button according to some embodiments;

[0047] 29 is a flow chart illustrating a display device scaling a focus window using a spacer according to some embodiments;

[0048] 30 is a flow chart illustrating a display device scaling a focus window using a spacer according to some embodiments;

[0049] FIG31 is a flowchart of a display device performing timing detection on an application stack according to some embodiments;

[0050] FIG32 is a flowchart illustrating a display device switching a focus window according to some embodiments;

[0051] FIG33 is a flowchart of a display device switching a focus window according to some embodiments;

[0052] FIG34 is a flow chart of a display device displaying a message prompt box according to some embodiments;

[0053] FIG35 is a schematic diagram of a display device displaying a message prompt box according to some embodiments;

[0054] FIG36 is another flow chart of a split-screen display method according to some embodiments;

[0055] FIG37 is a schematic diagram of a user interface of a display according to some embodiments;

[0056] FIG38 is a schematic diagram of another user interface displayed on a display according to some embodiments;

[0057] FIG39 is a schematic diagram of another user interface displayed on a display according to some embodiments;

[0058] FIG40 is a schematic diagram of another user interface displayed on a display according to some embodiments;

[0059] Figure 41 is a flowchart of controlling two display windows to play the current video and the local video according to some embodiments. DETAILED DESCRIPTION

[0060] FIG1 is a schematic diagram of an operation scenario between a display device and a control device according to an embodiment. As shown in FIG1 , a user can operate a display device 200 through a smart device 300 or a control device 100 .

[0061] In some embodiments, the control device 100 may be a remote controller. Communication between the remote controller and the display device may include infrared protocol communication, Bluetooth protocol communication, or other short-range communication methods, and the display device 200 may be controlled wirelessly or wired. The user may control the display device 200 by inputting user commands through buttons on the remote controller, voice input, control panel input, and the like.

[0062] In some embodiments, a smart device 300 (such as a mobile terminal, tablet computer, computer, laptop computer, etc.) can also be used to control the display device 200. For example, the display device 200 can be controlled using an application running on the smart device.

[0063] In some embodiments, the display device may not use the aforementioned smart device or control device to receive instructions, but may receive user control through touch or gestures.

[0064] In some embodiments, the display device 200 can also be controlled in a manner other than the control device 100 and the smart device 300. For example, the user's voice command control can be directly received through a module for obtaining voice commands configured inside the display device 200, or the user's voice command control can be received through a voice control device set outside the display device 200.

[0065] In some embodiments, the display device 200 also communicates data with the server 400. The display device 200 may be connected to a local area network (LAN), a wireless local area network (WLAN), or other networks. The server 400 may provide various content and interactions to the display device 200. The server 400 may be a single cluster or multiple clusters, and may include one or more types of servers.

[0066] Figure 2 illustrates a block diagram of the configuration of a control device 100 according to an exemplary embodiment. As shown in Figure 2 , the control device 100 includes a processor 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 receives user input commands and converts them into commands that the display device 200 can recognize and respond to, acting as an intermediary for interaction between the user and the display device 200.

[0067] As shown in FIG3 , the display apparatus 200 includes at least one of a tuner and demodulator 210 , a communicator 220 , a detector 230 , an external device interface 240 , a processor 250 , a display 260 , an audio output interface 270 , a memory, a power supply, and a user interface.

[0068] In some embodiments, the processor includes a processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and first to nth interfaces for input / output.

[0069] The display 260 includes a display screen component for presenting images, a driving component for driving image display, a component for receiving image signals output from a processor, and a component for displaying video content, image content, and a menu control interface and a user control UI interface.

[0070] The display 260 may be a liquid crystal display, an OLED display, or a projection display, and may also be a projection device and a projection screen.

[0071] Communicator 220 is a component used to communicate with external devices or servers using various communication protocols. For example, the communicator may include at least one of a Wi-Fi module, a Bluetooth module, a wired Ethernet module, or other network communication protocol chip or a near-field communication protocol chip, as well as an infrared receiver. Display device 200 can use communicator 220 to send and receive control signals and data signals with external control device 100 or server 400.

[0072] The user interface can be used to receive control signals from the control device 100 (such as an infrared remote controller, etc.).

[0073] Detector 230 is used to collect signals from the external environment or external interactions. For example, detector 230 includes a light receiver, a sensor for collecting ambient light intensity; or detector 230 includes an image collector, such as a camera, for collecting external environmental scenes, user attributes, or user interaction gestures; or detector 230 includes a sound collector, such as a microphone, for receiving external sounds.

[0074] The external device interface 240 may include, but is not limited to, any one or more of the following: a high-definition multimedia interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input / output interface formed by multiple of the above interfaces.

[0075] The tuner-demodulator 210 receives broadcast television signals via a wired or wireless reception method, and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

[0076] In some embodiments, the processor 250 and the tuner / demodulator 210 may be located in different separate devices, that is, the tuner / demodulator 210 may also be located in an external device of the main device where the processor 250 is located, such as an external set-top box.

[0077] Processor 250 controls the operation of the display device and responds to user operations through various software control programs stored in memory. Processor 250 controls the overall operation of display device 200. For example, in response to receiving a user command to select a UI object to be displayed on display 260, processor 250 may perform operations related to the object selected by the user command.

[0078] In some embodiments, the processor includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), RAM Random Access Memory (RAM), ROM (Read-Only Memory, ROM), a first interface to an nth interface for input / output, a communication bus (Bus), etc.

[0079] The user may input a user command through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user may input a user command through a specific voice or gesture, and the user input interface may recognize the voice or gesture through a sensor to receive the user input command.

[0080] A user interface is the medium for interaction and information exchange between an application or operating system and the user. It converts information between its internal form and a user-friendly format. A common user interface is the graphical user interface (GUI), which refers to a graphical user interface related to computer operations. It can be an icon, window, control, or other interface element displayed on an electronic device's display. Controls can include visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.

[0081] Referring to Figure 4, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (referred to as the "application layer"), the application framework layer (referred to as the "framework layer"), the Android runtime (Android runtime) and system library layer (referred to as the "system runtime library layer"), and the kernel layer.

[0082] In some embodiments, at least one application runs in the application layer. These applications can be window programs, system settings programs, clock programs, etc. that come with the operating system, or applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0083] The framework layer provides applications with an application programming interface (API) and programming framework. The application framework layer includes predefined functions. The application framework layer acts as a processing center, determining the actions taken by applications in the application layer. Through the API, applications can access system resources and services during execution.

[0084] As shown in Figure 4, the application framework layer in the embodiment of the present disclosure includes managers, content providers, etc., wherein the manager includes at least one of the following modules: an activity manager for interacting with all activities running in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to the application packages currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0085] In some embodiments, the activity manager is used to manage the lifecycle of each application and common navigation back functions, such as controlling the exit, opening, and backing of an application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, taking screenshots, and controlling display window changes (such as shrinking the display window, shaking the display, distorting the display, etc.).

[0086] In some embodiments, the system runtime layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system will run the C / C++ library contained in the system runtime layer to implement the functions to be implemented by the framework layer.

[0087] In some embodiments, the kernel layer is a layer between hardware and software. As shown in FIG4 , the kernel layer includes at least one of the following drivers: an audio driver, a display driver, a Bluetooth driver, a camera driver, a Wi-Fi driver, a USB driver, an HDMI driver, a sensor driver (such as a fingerprint sensor, a temperature sensor, a pressure sensor, etc.), and a power driver.

[0088] FIG5 is a flow chart of a split-screen display method provided by an embodiment of the present disclosure, the method comprising:

[0089] S501: Receive a first selection input for a first application and a second application.

[0090] The various inputs described in the embodiments of the present disclosure (such as the first selection input, the second selection input, the first control input, the second control input, etc.) can all be the user's input for the corresponding application identifier, or the user's input for the corresponding operation control. The input method can be click input, voice input, etc.

[0091] Among them, the click input can include single-click input or double-click input. The click input can be a finger click input, or a click input through an input device such as a mouse, or a click input issued by a remote control or other control device. The embodiments of the present disclosure do not limit this.

[0092] The application identifier can be understood as an icon of an application, or any other control that can represent an application, such as the application identifier of the first application, the application identifier of the second application, etc. in the embodiment of the present disclosure.

[0093] S502: In response to a first selection input, display a selection result of the first application and a selection result of the second application.

[0094] For example, FIG6 shows a first schematic diagram of an interface of a display device provided by an embodiment of the present disclosure. FIG6 illustrates an application selection interface, which includes application identifiers for multiple applications supported by the current platform and available for user selection, representing Application 1, Application 2, and so on. The application selection interface also includes a focus frame 61, which is used to represent the application currently preselected by the user. For example, in the interface shown in FIG6, the application currently preselected by the user is Application 2.

[0095] For example, FIG7 shows a second schematic diagram of an interface of a display device provided by an embodiment of the present disclosure. FIG7 , based on the application selection interface shown in FIG6 , shows the selection results of the first application and the second application based on the user's first selection input. As shown in FIG7 , the first focus frame 71 and the second focus frame 72 respectively represent the user's selection results for the first application and the second application. The current user's selection results for the first application and the second application are application 3 and application 5, respectively.

[0096] In some embodiments, in response to a first selection input for a first application and a second application received through the user input interface, the first focus frame is moved to the location of the application identifier of the first application, and the second focus frame is moved to the location of the application identifier of the second application; the color of the first focus frame and / or the second focus frame is changed, and the first focus frame after the color change is used to indicate the user's selection result for the first application, and the second focus frame after the color change is used to indicate the user's selection result for the second application.

[0097] Split-screen display requires the user to select at least two applications. In some embodiments, the user first selects the first application through the first focus frame, and then selects the second application through the second focus frame. When the user needs to open a split-screen application, the position of the first focus frame in the display device is controlled; when the first focus frame moves to the identification position of the first application that the user needs to select, the user's selection result for the first application is identified by changing the color of the first focus frame. Similarly, the position of the second focus frame in the display device is controlled; when the second focus frame moves to the identification position of the second application that the user needs to select, the user's selection result for the second application is identified by changing the color of the second focus frame.

[0098] In some embodiments, the shape of the focus frame may be changed to indicate the user's selection result for the first application or the second application, which is not limited in this embodiment of the present disclosure.

[0099] In some embodiments, the application selection interface may also display the user's selection results for the first application and the user's selection results for the second application by using the application names. In some embodiments, as shown in FIG7 , the name of the first application is displayed in the first application name display area 73, and the name of the second application is displayed in the second application name display area 74. It should be understood that the location and display content of the application name display area provided in the embodiments of the present disclosure are merely examples and should not be construed as limitations of the present disclosure.

[0100] In some embodiments, after the user selects the first application and the second application, if the first application or the second application needs to be modified, the first application or the second application can be reselected by controlling the position of the focus box, or by editing the application name in the application name display area.

[0101] S503: Display window arrangement selection controls and split-screen controls.

[0102] The window arrangement selection control is used to allow the user to select the arrangement of the first split-screen area and the second split-screen area. The window arrangement selection control can be implemented as a selection control, a drop-down menu control, or other forms of controls, which are not limited in the present embodiment.

[0103] The split-screen control is used for the user to enable the split-screen display function. When the user operates the split-screen control, the first application and the second application can be displayed in split screen on the display device based on the selection result of the first application and the selection result of the second application.

[0104] S504: Receive a second selection input for the window arrangement mode selection control.

[0105] S505: In response to the second selection input, determine an arrangement of the first split screen area and the second split screen area.

[0106] The user can select the arrangement of the split-screen windows. In the embodiment of the present disclosure, in response to the second selection input of the window arrangement selection control, the split-screen display mode of the first application and the second application desired by the user can be determined.

[0107] In some embodiments, the first split-screen area and the second split-screen area can be arranged in a left-right split-screen manner, and the first split-screen area can be displayed on the left side of the screen and the second split-screen area can be displayed on the right side of the screen, or the first split-screen area can be displayed on the right side of the screen and the second split-screen area can be displayed on the left side of the screen; or, the second split-screen area can be arranged in an up-down split-screen manner, and the first split-screen area can be displayed on the upper side of the screen and the second split-screen area can be displayed on the lower side of the screen, or the first split-screen area can be displayed on the upper side of the screen and the second split-screen area can be displayed on the lower side of the screen.

[0108] In some embodiments, as shown in FIG7 , the application selection interface displays a left-right arrangement control 75 and a top-bottom arrangement control 76. When a second selection input is received for the left-right arrangement control 75, it is determined that the user desires to display the first application and the second application in a left-right arrangement manner; when a second selection input is received for the top-bottom arrangement control 76, it is determined that the user desires to display the first application and the second application in a top-bottom arrangement manner. The application selection interface also displays a split-screen control 77 for the user to enable the split-screen display function.

[0109] S506: Receive a first control input for the split-screen control.

[0110] S507: In response to a first control input, display the interface of the first application in the first split-screen area, and display the interface of the second application in the second split-screen area.

[0111] Referring to the above steps, after determining the split-screen mode for the first application and the second application, in response to the first control input to the split-screen control, split-screen display is performed based on the selection result of the first application and the selection result of the second application. Specifically, the interface of the first application is displayed in the first split-screen area, and the interface of the second application is displayed in the second split-screen area.

[0112] Exemplarily, as shown in FIG8 , a third interface diagram of a display device provided in an embodiment of the present disclosure is shown. In conjunction with FIG7 , the first application selected at this time is application 3, and the second application selected is application 5. In response to the user's second selection input for the left-right arrangement control 75 in the interface as shown in FIG7 , the arrangement of the first split-screen area and the second split-screen area is determined to be left-right arrangement; in response to the user's first control input for the split-screen (SPLIT-SCREEM) control 77 in the interface as shown in FIG7 , the application interface of the first application and the application interface of the second application are respectively displayed in the first split-screen area and the second split-screen area arranged left-right, that is, the interface diagram of the display device as shown in FIG8 is displayed.

[0113] In some embodiments, as shown in FIG9 , a fourth schematic diagram of an interface of a display device provided in an embodiment of the present disclosure is shown. In conjunction with FIG7 , the selected first application is application 3, and the selected second application is application 5. In response to a second selection input from a user on the vertical arrangement control 76 in the interface shown in FIG7 , the arrangement of the first split-screen area and the second split-screen area is determined to be vertical; and in response to a first control input from a user on the split-screen control 77 in the interface shown in FIG7 , the application interface of the first application and the application interface of the second application are displayed in the vertically arranged first split-screen area and the second split-screen area, respectively, thereby displaying the interface schematic diagram of the display device shown in FIG9 .

[0114] The technical solution provided by the embodiments of the present disclosure allows users to select the applications to be split-screen and the direction of the application split-screen before the application is split-screen displayed, so that the application interface can be split-screen displayed in the ideal split-screen display area on the screen according to the user's wishes, providing users with a more friendly split-screen display interactive experience.

[0115] In some embodiments, in response to a first selection input for a first application and a second application, the selection results of the first application and the selection results of the second application are displayed, and the window arrangement selection control and the split screen control are displayed. Then, based on the second selection input for the first application and the second application, the selection order of the first application and the second application is displayed.

[0116] Specifically, you can specify the display positions of the first split-screen area and the second split-screen area. For example, when using the left-right split-screen display mode, the first split-screen area is displayed on the left interface of the display device by default, and the second split-screen area is displayed on the right interface of the display device. At this time, when the split-screen function is turned on, the interface of the first application will be displayed on the left interface of the display device, and the interface of the second application will be displayed on the right interface of the display device. When the selection order of the first application and the second application is swapped, the split-screen display positions of the two applications will also be swapped.

[0117] In some embodiments, when the user selects application 3 as the first application and application 5 as the second application, when the left-right split-screen display mode is adopted, when the split-screen function is turned on, the interface of application 3 will be displayed on the left interface of the display device, and the interface of application 5 will be displayed on the right interface of the display device; if the user selects application 5 as the first application and application 3 as the second application, when the left-right split-screen display mode is adopted, when the split-screen function is turned on, the interface of application 5 will be displayed on the left interface of the display device, and the interface of application 3 will be displayed on the right interface of the display device.

[0118] Therefore, based on the second selection input for the first application and the second application, the selection order of the first application and the second application can be displayed, so that the user can understand the positions of the interfaces of the two currently selected applications to be displayed in the split-screen on the display device interface. At the same time, by changing the selection order of the applications to be displayed in the split-screen, the user can also determine the positions of the interfaces of the applications to be displayed in the split-screen on the display device interface.

[0119] In some embodiments, as shown in FIG7 , the name of the first application can be displayed in the first application name display area 73, and the name of the second application can be displayed in the second application name display area 74. That is, the name of the application selected first by the user is displayed in the first application name display area 73, and the name of the application selected later by the user is displayed in the second application name display area 74. If the user selects application 3 and application 5 in sequence and then selects left-right split-screen mode, the interface of application 3 can be displayed on the left interface of the display device, and the interface of application 5 can be displayed on the right interface of the display device; if the user selects application 5 and application 3 in sequence and then selects left-right split-screen mode, the interface of application 5 can be displayed on the left interface of the display device, and the interface of application 3 can be displayed on the right interface of the display device; if the user selects application 3 and application 5 in sequence and then selects top-bottom split-screen mode, the interface of application 3 can be displayed on the top interface of the display device, and the interface of application 5 can be displayed on the bottom interface of the display device; if the user selects application 5 and application 3 in sequence and then selects top-bottom split-screen mode, the interface of application 5 can be displayed on the top interface of the display device, and the interface of application 3 can be displayed on the bottom interface of the display device. It is understandable that the above implementation content is only an example of the embodiment of the present disclosure, and the specific display position of the application program interface should not be used as a limitation of the embodiment of the present disclosure.

[0120] The disclosed embodiment enables users to freely control the display position of the application interface in the split-screen state through the user's selection order of split-screen applications, effectively improves the controllability and interactivity of the split-screen display method, improves the user experience, and is more in line with market and user needs.

[0121] In some embodiments, as shown in FIG10 , a flow chart of a split-screen display method provided by an embodiment of the present disclosure is provided. The method is described below with reference to the flow chart.

[0122] S1001: Determine the selection result of the first application, the selection result of the second application, and the arrangement of the first split screen area and the second split screen area.

[0123] Specifically, S1001 can be implemented through the above steps S501 to S505. The implementation process and principle are consistent and will not be repeated here.

[0124] S1002: Receive a first control input for a split-screen control.

[0125] S1003: Detect startup processes of the first application and the second application.

[0126] S1004: Determine whether the startup processes of the first application and the second application have finished running. If so, execute S1006; if not, execute S1005.

[0127] After the user activates split-screen display, the first and second applications need to execute a startup process, which includes loading the display interface of the first application and the display interface of the second application. When the startup process of the first application is completed, it means that the first application has been successfully launched and loaded, and the first application interface can be displayed. When the startup process of the second application is completed, it means that the second application has been successfully launched and loaded, and the second application interface can be displayed.

[0128] Specifically, the activity manager in the software system can be used to obtain and detect the process task (runningTask) of the first application and the process task (runningTask) of the second application, and detect whether the startup processes of the first application and the second application have been completed, so as to determine whether the first application and the second application have been loaded.

[0129] S1005. Display the preset loading interface in full screen.

[0130] If the startup process of the first and second applications has not yet completed, it means that the first or second application has not yet loaded. At this time, the interface of the first or second application may flicker or other abnormal display issues. To avoid this problem, the preset loading interface is displayed full screen before the startup process of the first and second applications has completed.

[0131] For example, as shown in Figure 11, a schematic diagram of a preset loading interface provided in an embodiment of the present disclosure is provided. In Figure 11, a rotating ring with a notch in the center of the interface indicates to the user that the display interface of the first application and the second application are currently loading. In other possible implementations, the preset loading interface can be any preset loading image or loading animation, which is not limited in the embodiment of the present disclosure.

[0132] S1006. Hide the preset loading interface.

[0133] When it is detected that the startup processes of the first application and the second application have ended, it means that the first application and the second application have been loaded. At this time, the interfaces of the first application and the second application can be displayed normally, and the preset loading interface is hidden.

[0134] S1007. Display the interface of the first application in the first split-screen area, and display the interface of the second application in the second split-screen area.

[0135] The selection results of the first application and the second application obtained in S1001, as well as the arrangement of the first split-screen area and the second split-screen area, are passed as parameters to the Activity Manager and the Window Manager in the Application Framework layer. The Activity Manager and the Window Manager then perform split-screen display, displaying the interface of the first application in the first split-screen area and the interface of the second application in the second split-screen area. For the specific display method, refer to the implementation method in the above embodiment and will not be repeated here.

[0136] The technical solution provided by the embodiment of the present disclosure avoids abnormal display conditions that may occur during the loading process of the split-screen interface by displaying the preset loading interface in full screen during the loading process of the first application and the second application, avoids confusion to the user, and improves the user experience during the split-screen display process.

[0137] In some embodiments, a region control control may be displayed between the first split screen region and the second split screen region; and in response to a second control input to the region control control, the sizes of the first split screen region and the second split screen region are changed.

[0138] The region control control can be displayed as a draggable control or as a dividing line. The user can change the size of the first split-screen area and the second split-screen area by dragging the region control control, or by performing a second control input. In some embodiments, Figure 12 is a fifth diagram of an interface of a display device according to some embodiments. As shown in Figure 12, dragging the region control control 1201 can change the size of the first split-screen area 1202 and the second split-screen area 1203.

[0139] The embodiment of the present disclosure, by setting an area control control, can facilitate users to adjust the size of the two split-screen areas at will, thereby increasing the flexibility of the split-screen display method and further improving the user's interactive experience.

[0140] Figure 13 is a flow chart of a key input processing method provided by an embodiment of the present disclosure. As shown in Figure 13, the method includes the following steps.

[0141] S1301: Receive key input from the user.

[0142] S1302: Check whether the current display is in split-screen mode. If yes, execute S1303; if not, execute S1307.

[0143] S1303: Determine whether the key corresponding to the key input exists in a preset key whitelist. If so, execute S1306; if not, execute S1304.

[0144] Among them, the preset button whitelist is used to store all buttons that will not be responded to in the split-screen display state.

[0145] In the split-screen state, if a key input from the user is received, in order to avoid affecting the split-screen display, it is first determined whether the key corresponding to the received key input is allowed to be responded to.

[0146] Specifically, a preset button white list (white list) is added to the system case manager (Phone Window Manager) of the software system.

[0147] For example, add the following code to the system:

[0148] int[]whitelist={KeyEvent.KEYCODE_MENU,KeyEvent.KEYCODE_INPUT,KeyEvent.KEYCODE_SETTINGS,KeyEvent.KEYCODE_MEDIA,……}

[0149] The buttons recorded in the above preset case whitelist can be set as buttons that are not responded to in the split-screen display state.

[0150] S1304: Determine the current focus application.

[0151] The focus application is the one with a higher priority among the first application and the second application.

[0152] If the key corresponding to the key input is not recorded in the preset key whitelist, it is determined that the key corresponding to the currently received user key input needs to be responded to, and it is further determined that one of the first application and the second application processes the key input.

[0153] Specifically, the priorities of the first application and the second application may be pre-set by the user. Alternatively, the priorities of the first application and the second application may be determined according to a pre-set priority determination rule. For example, based on the user's operation records for the first application and the second application, it is determined that the application that the user has most recently operated has a higher priority. The application with the higher priority between the first application and the second application is selected as the focus application.

[0154] S1305: The focus application processes the key input.

[0155] Send the key input to the focus application, which processes the key input.

[0156] S1306: Abandon processing the key input.

[0157] When the key corresponding to the key input is recorded in the preset key whitelist, no processing is performed on the key input to avoid affecting the current application split-screen display.

[0158] S1307: Directly process the key input.

[0159] If it is detected that the current system is not in split-screen display state, the key input at this time will not affect the application split-screen display, and the key input will be directly processed according to the system native rules.

[0160] The embodiment of the present disclosure sets a preset button whitelist to filter the user's button input in the split-screen state, effectively preventing the user's button input in the split-screen state from affecting the split-screen display state of the application, ensuring the stability of the split-screen display method, and further improving the user experience.

[0161] FIG14 is a flowchart of a method for processing an application startup instruction provided by an embodiment of the present disclosure. As shown in FIG14 , the method includes the following steps.

[0162] S1401: Receive a third application startup instruction from the user.

[0163] The third application refers to any application other than the first application and the second application displayed in the split screen. The third application startup instruction can be any form of operation instruction such as a voice instruction, a gesture instruction, etc. The third application startup instruction is used to instruct to start the third application.

[0164] S1402: Check whether the current display is in split-screen mode. If yes, execute S1403; if not, execute S1404.

[0165] The Activity Manager in the Application Framework layer detects the current system state and blocks the third application startup instruction if it detects that the system is currently in a split-screen state. Otherwise, the third application is run normally according to the third application startup instruction.

[0166] S1403: Shield the third application startup instruction.

[0167] If the app is currently in split-screen mode, i.e., the first and second apps are currently being displayed, launching a third app will overwrite the first and second apps, or interrupt the current split-screen mode. To ensure the normal display of the split-screen interface, the third app launch command needs to be blocked.

[0168] Specifically, the third application startup instruction is shielded or intercepted by an activity manager.

[0169] S1404: Start the third application.

[0170] If the split-screen display state is not currently in progress, the third application will be started normally based on the third application startup instruction according to the system native logic.

[0171] S1405, end.

[0172] The embodiment of the present disclosure detects the current system status after receiving the third application startup instruction. If it is detected that the system is currently in a split-screen state, the third application startup instruction is blocked to avoid abnormalities in the current split-screen display interface caused by the startup of the third application, further ensuring the stability of the split-screen display method and improving the user experience.

[0173] In addition, considering the increase in the display screen of the display device, the performance of memory, central processing unit (CPU) and graphics processing unit (GPU) is constantly improving, and the hardware performance of the current display device can already meet the conditions for running multiple applications at the same time. Therefore, the display device has the hardware conditions for configuring the split-screen function. If the display device is configured with the split-screen function, after turning on the split-screen function, the display device can display at least two windows at the same time, and display different pictures in the at least two windows. However, the user needs to determine which window of the at least two windows occupies the focus through additional operations such as moving the focus, resulting in a poor user experience.

[0174] To solve the above problem, when the display device 200 of the present disclosure displays the first screen of the first application in the first window, it can also perform split-screen display according to the process shown in FIG15 . The specific steps are as follows:

[0175] S51 , in response to a screen splitting instruction input by the user, adding and displaying a second window, and displaying a first screen of a first application in the first window, and displaying a second screen of a second application in the second window.

[0176] The first application is the application currently displayed on the display device 200. The first application is a split-screen application, that is, the first application can be run simultaneously with other split-screen applications in different windows. The screen of the first application is the first screen.

[0177] The first window can be one or more. If there is one first window, it means that the display device 200 is currently displaying only one first application, and the first window can occupy the entire display area of ​​the display. If there are multiple first windows, it means that the display device 200 is currently displaying multiple first applications at the same time, each first application is different and is displayed in a corresponding first window. The multiple first windows respectively occupy part of the display area of ​​the display, and the partial display area occupied by each first window can constitute the entire display area of ​​the display.

[0178] The split-screen instruction is used to instruct the split-screen display of the first application and the second application. The window used to display the second application is the second window, and the second application's screen is the second screen. The second window can be located to the left or right of the first window, that is, the first window and the second window are arranged horizontally, or the second window can be located above or below the first window, that is, the first window and the second window are arranged vertically.

[0179] The first window and the second window can be displayed according to a preset split-screen ratio. For example, if the preset split-screen ratio is 1:1, the first window and the second window are displayed in a manner that evenly distributes the display area. If the preset split-screen ratio is 1:2, the first window occupies one-third of the display area, and the second window occupies two-thirds of the display area.

[0180] In some embodiments, the display device 200 may display the first application and the second application in split screen according to the process shown in FIG16 . The specific steps are as follows:

[0181] S601: In response to a split-screen instruction, display a split-screen application list.

[0182] The user can input a split-screen instruction to the display device 200 based on the split-screen button on the control device 100, and the display device 200 responds to the split-screen instruction by displaying a split-screen application list. The split-screen application list includes items for each split-screen application, and these split-screen applications include the second application. The split-screen applications in the split-screen application list can be split-screen applications installed in the display device 200 and supporting the split-screen function, or they can be split-screen applications installed in the display device 200 and associated with the first application, where the association means that the first application and the split-screen application are displayed in split-screen in a fixed combination.

[0183] S602 : In response to a selection instruction input by the user based on an item of the second application, a second window is added and displayed, and a first screen of the first application is displayed in the first window, and a second screen of the second application is displayed in the second window.

[0184] If the split-screen applications in the split-screen application list are applications installed on the display device 200 and support the split-screen function, the user can select a second application to be displayed in split-screen with the first application, that is, input a selection instruction based on the item of the second application to instruct the display device 200 to display the first application and the second application in split-screen. In this way, the user can flexibly select the second application to be displayed in split-screen with the first application.

[0185] If the split-screen application is installed in the display device 200 and is associated with the first application, typically there is only one split-screen application, namely the second application. The user can directly input a selection instruction based on the item of the split-screen application to instruct the display device 200 to display the fixed combination in split screen. Since the associated first application and the second application are used to achieve specific functions through split-screen display, for example, the first application is a fitness video application and the second application is a camera application. By displaying the first application and the second application in split screen, it is possible to play a fitness video through the fitness video application and capture and display the user's movements through the camera application. Therefore, after selecting the second application, the user can achieve the specific function by displaying the first application and the second application in split screen.

[0186] In some embodiments, the display device 200 may display the first application and the second application in split screen according to the process shown in FIG17 . The specific steps are as follows:

[0187] S701: In response to a screen split instruction, obtain a second application associated with a first application.

[0188] The first application has a specified function, and the first application needs to implement the specified function through split-screen display with the second application. Among them, the split-screen application that is displayed split-screen with the first application to implement the specified function of the first application is the second application associated with the first application.

[0189] For example, the first application is a fitness video application with a designated function of correcting fitness movements, and the second application associated with the first application is a camera application. For another example, the first application is an education application with a designated function of taking study notes, and the second application associated with the first application is a note-taking application.

[0190] The user can input a split screen instruction to the display device 200 based on the split screen button on the control device 100. The display device 200 responds to the split screen instruction and obtains the second application associated with the first application based on the association relationship between the first application and the second application.

[0191] S702 , adding and displaying a second window, and displaying a first screen of a first application in the first window, and displaying a second screen of a second application in the second window.

[0192] A first screen of a first application is displayed in a first window, and a second screen of a second application is displayed in a second window, so that a designated function of the first application is realized by displaying the associated first and second applications in split screens.

[0193] Display device 200 provides a mechanism for accessing the Android split-screen application programming interface (API) in the system user interface (SystemUI), providing a way for upper-level applications to quickly enter split screen. The Android split-screen API for window management (WindowManager) is called through the SystemUI. The reason for providing the interface in the SystemUI is that after the Android system starts, the parent process (init) forks the incubator process (Zygote), which starts the system service (system_server) process, which starts the boot service, core services, and other services.

[0194] The Android boot process shown in Figure 18 includes the following steps:

[0195] Step 181: bootloader starts, kernel idle process starts;

[0196] When Android boots up, it pre-starts the boot loader and starts the kernel idle.

[0197] Step 182: The kernel starts the init process;

[0198] Kernel idle starts process number 0 init (pid0).

[0199] Step 183: The int process starts the Zygote process;

[0200] init starts the first process zygote (pid0) through the fork function.

[0201] Step 184: The Zygote process starts the system_server process;

[0202] Step 185: The system_server process starts the boot service, core service, and other services, and starts the systemUI process when starting other services.

[0203] The system_server process is one of the core processes of the Android system, providing Android's service server capabilities, such as the Window Manager Service (WindowManagerService) and the Activity Manager Service (ActivityManagerService). These core services are started within the main function of the system_server process, and the SystemUI process is started when other services are started. SystemUI is a resident process independent of the system_server process, providing the Android system UI. You can think of SystemUI as a separate app, and SystemUI starts earlier than other system apps, making it a good choice to add a split-screen entry within SystemUI.

[0204] Before Android 12, the Recents class of System UI provided a way to enter split screen, the WindowManagerProxy class provided a way to exit split screen, and the am class provided a way to adjust the window size.

[0205] In Android 12 and above, Android's native split-screen control logic is located in the WindowManager directory, which provides a split-screen control (SplitScreenController) class that provides methods for entering and exiting split screens and adjusting window sizes.

[0206] When the system_server process starts the System UI, it will first pull up the SystemUIService. The initialization (onCreate) method of the SystemUIService will call the startServicesIfNeeded method of the SystemUIApplication. Executing this method will load all System UI service classes that inherit SystemUI (CoreStartable class for Android 13 and above) and execute their start methods.

[0207] In order to provide a quick way to enter split-screen mode in System UI and allow third-party applications to call it directly through System UI, the split-screen function can be injected when System UI starts the System UI service class. To reduce coupling, the split-screen logic can be injected into the vendor service (VendorServices). VendorServices also inherits from System UI. The default implementation of the start method is empty. The split-screen management (SplitScreenManager) class can be instantiated in the VendorService class and registered to listen for broadcasts. The split-screen management class holds a reference to SplitScreenController and can call SplitScreenController methods according to different broadcast types to implement split-screen control operations.

[0208] The split-screen operation process provided by SystemUI can be referred to in Figure 19. SystemUIApplication obtains the SystemUI service class through SystemUIFactory. When calling startServicesIfNeedeed, SystemUIApplication will call the start method of the VendorServices class. The VendorServices class holds a reference to the custom SplitScreenManager and will be instantiated as SplitScreenManagerImpl in the start method. SplitScreenManagerImpl can register for broadcasts of entering split screen, exiting split screen, and adjusting windows through the register receiver (registReceiver) method, and then call the system native split-screen method of the SplitScreenController held by SplitScreenManagerImpl to perform various split-screen operations. By providing services in System UI, the quick entry and exit of split screen are unified. Entering split screen does not require entering from the recent list, but provides a direct call method by sending system notifications to the bottom layer, which does not destroy the system native split-screen logic, facilitates management, makes the split-screen experience more harmonious, and solves the problem that third-party applications cannot directly enter split screen.

[0209] S52: Determine a focus window that occupies focus between the first window and the second window.

[0210] When a first window and a second window are displayed in a split-screen mode in multi-window mode, the window that occupies focus is the focus window. The display device 200 pre-sets a rule for occupying focus after the split screen, for example, the leftmost window occupies focus, the topmost window occupies focus, the first window occupies focus, the second window occupies focus, etc.

[0211] After displaying the first window and the second window in split screen, the display device 200 can determine the focus window by identifying the location of the focus. After displaying the first window and the second window in split screen, the display device 200 can also directly determine the focus window based on a preset rule of occupying focus.

[0212] S53, displaying a focus frame on the upper layer of the focus window.

[0213] The focus frame is a frame-shaped window, that is, the middle area of ​​the focus frame is hollow, and the image in the focus window located below the focus frame can be displayed through this middle area. The size of the focus frame matches the size of the focus window, which can achieve the effect of the focus frame covering the border of the focus frame. Therefore, when the focus frame is displayed on the border of the focus window, the user can see the focus frame and the image in the focus window displayed through the focus frame. At this time, the focus frame can serve as an indicator of the focus window. The user can quickly determine the focus window based on the position of the focus frame, and then quickly determine the location of the focus.

[0214] The display device 200 may display a focus frame according to the process shown in FIG20 , and the specific steps are as follows:

[0215] S2001, obtaining a position of a focus window, a size of a display area of ​​a display, a preset split-screen ratio between a first window and a second window, and an arrangement of the first window and the second window.

[0216] The focus window position is the relative position of the focus window in the display area. For example, if the first window and the second window are arranged side by side, and the first window is to the left of the second window, then if the first window is the focus window, then the focus window position is the left side; if the second window is the focus window, then the focus window position is the right side. For another example, if the first window and the second window are arranged top by top, and the first window is above the second window, then if the first window is the focus window, then the focus window position is the top; if the second window is the focus window, then the focus window position is the bottom.

[0217] The size of the display area can be represented by width and height, wherein the size of the display area in the horizontal direction is width, and the size of the display area in the vertical direction is height.

[0218] The preset split-screen ratio between the first window and the second window is the ratio of the border sizes of the first window and the second window in the arrangement direction. For example, if the first window and the second window are arranged side by side, the bottom border of the first window and the bottom border of the second window, or the top border of the first window and the top border of the second window, constitute the borders in the arrangement direction. The ratio of the size of the bottom border of the first window to the size of the bottom border of the second window is equal to the preset split-screen ratio. Alternatively, the ratio of the size of the top border of the first window to the size of the top border of the second window is equal to the preset split-screen ratio. For another example, if the first window and the second window are arranged top by top, the left border of the first window and the left border of the second window, or the right border of the first window and the right border of the second window, constitute the borders in the arrangement direction. The ratio of the size of the left border of the first window to the size of the left border of the second window is equal to the preset split-screen ratio. Alternatively, the ratio of the size of the right border of the first window to the size of the right border of the second window is equal to the preset split-screen ratio.

[0219] S2002: Determine the size of the focus window according to the size of the display area, the preset split-screen ratio, and the arrangement of the first window and the second window.

[0220] The sizes of the borders of the first and second windows in the non-aligned direction are equal and equal to the size of the display area corresponding to the borders. For example, if the first and second windows are aligned side by side, the left and right borders of the first and second windows are borders in the non-aligned direction, and the sizes of these borders are equal to the height of the display area. For another example, if the first and second windows are aligned top and bottom, the top and bottom borders of the first and second windows are borders in the non-aligned direction, and the sizes of these borders are equal to the width of the display area.

[0221] The sum of the sizes of the borders of the first window and the second window in the arrangement direction is equal to the size of the display area corresponding to the borders, and the ratio of the sizes of the borders of the first window and the second window in the arrangement direction is equal to the preset split-screen ratio. Based on the above relationship, the sizes of the borders of the first window and the second window in the arrangement direction can be calculated. For example, if the first window and the second window are arranged left and right, the sum of the bottom borders of the first window and the second window is equal to the width of the display area, and the ratio of the bottom borders of the first window and the second window is equal to the preset split-screen ratio, such as 2:1. In this case, the size of the bottom border of the first window is equal to two-thirds of the width of the display area, and the size of the bottom border of the second window is equal to one-third of the width of the display area.

[0222] Based on the above process, the size of each frame of the first window and the second window can be determined, and the size of each frame of the focus window can be further determined.

[0223] S2003: Determine the position of the focus frame according to the position of the focus window, and determine the size of the focus frame according to the size of the focus window.

[0224] The position of the focus window is the position of the focus frame in the display area, and the size of the focus window is the size of the focus frame.

[0225] S2004: Display the focus frame according to the position and size of the focus frame.

[0226] In this way, it can be ensured that the focus frame covers the focus window and its size matches the border of the focus window, without blocking the picture displayed in the focus window and the focus frame can be fully displayed.

[0227] In some embodiments, the display device 200 can implement the display of a focus frame based on the system architecture shown in FIG21 . As shown in FIG21 , the split screen application (Split Screen App) can be a YouTube application, an Amazon Prime Video application, a wireless display standard (Miracast) application, or a live TV application. The focus frame (FocusWindow) is a focus frame dynamically added to the split screen focus window. When the split screen application such as YouTube or Prime Video obtains focus in the split screen state, it will be attached above the focus window. The size and position are consistent with the focus window. The task listener (TaskCheckListener) monitors the state changes of the split screen focus window. When the size or position of the focus window changes through the remote control or voice operation, the FocusWindow will also update the size or position accordingly. The TaskStackListener monitors the state changes of the application stack. When the split screen application stack changes, it responds in a timely manner to control the display and hiding of the FocusWindow.

[0228] Based on the above, you can quickly enter the split screen by sending a broadcast to systemUI. After sending the split screen broadcast, SystemUI will call the moveToSideStage method of SplitScreenController to pull the split screen application into the split screen. The Activity Manager (ActivityManager) provides a method to obtain the system's running tasks (task), encapsulates it into the running task information (RunningTaskInfo) and returns it to the caller. RunningTaskInfo inherits from the Task Information (TaskInfo) class, which provides a method to obtain the current task window mode "getWindowingMode". This method can be used to obtain whether the current task is in multi-window mode. By judging whether the task of the current split-screen application is in multi-window mode, when it is determined to enter split screen mode, the focus frame is added to the default focus window. To add a focus window, you can use the addView method of the window manager (WindowManager).

[0229] This method receives two parameters (View view, WindowManager.LaYoutParams params). The first parameter is View, which is the view to be added, that is, the focus frame itself. The second parameter is layout parameters (LayoutParams), which are the configuration of the focus frame, such as size, position, type, window level, etc., which can be obtained through WindowManager.

[0230] In some embodiments, the LayoutParams of the focus frame can be obtained based on the coordinate system. Take the first window and the second window as arranged left and right, wherein the first window is on the left and the second window is on the right, and the preset split screen ratio is 1:1 as an example. A coordinate system is established with the lower left corner vertex of the first window as the origin, the lower border of the display area of ​​the display as the x-axis, and the left border of the display area of ​​the display as the y-axis. After obtaining the width (mScreenWidth) and height (mScreenHeight) of the display area of ​​the display, the width (width) and height (height) of the focus frame can be calculated, as well as the position of the focus frame.

[0231] If the focused window is the first window, the LayoutParams of the focused window can be expressed as follows:

[0232] MLayoutParams.width=mScreenWidth / 2;

[0233] MLayoutParams.height=mScreenHeight;

[0234] MLayoutParams.x = 0;

[0235] MLayoutPartams.y=0.

[0236] If the focused window is the second window, the LayoutParams of the focused window can be expressed as follows:

[0237] MLayoutParams.width=mScreenWidth / 2;

[0238] MLayoutParams.height=mScreenHeight;

[0239] MLayoutParams.x=mScreenWidth / 2;

[0240] MLayoutPartams.y=0.

[0241] In some embodiments, after displaying the first application and the second application in split screen, the display device 200 may switch to displaying one of the applications as follows:

[0242] In response to the movement instruction input by the user, the display device 200 moves the focus to the target window and displays a focus frame on the upper layer of the target window.

[0243] The application displayed in the target window is the target application that needs to be replaced.

[0244] The display device 200 displays the split-screen application list in the target window in response to the split-screen instruction input by the user.

[0245] The split-screen application list includes items of split-screen applications that can be displayed in split-screen with a non-target application.

[0246] The display device 200 exits the target application in response to a selection instruction input by the user based on the item of the third application in the split screen application, and displays the third application in the target window.

[0247] In this way, the user can change any application in the applications currently displayed on the split screen.

[0248] Steps S51 to S53 are exemplarily described with reference to the following examples.

[0249] Example 1

[0250] For example, the display device 200 is a television, the control device 100 is a remote controller, and the first application is a YouTube application.

[0251] As shown in Figure 22 (1), the TV displays the first screen of the YouTube app in first window 1201. This first screen is a screen of Movie A. The user inputs a split-screen command to the TV using the split-screen button on the remote control. In response to this command, as shown in Figure 22 (2), the TV displays a split-screen application list 1202 on the screen of Movie A. This split-screen application list 1202 includes items for split-screen display with the first application, such as App1, App2, and the Prime Video app. For example, if the second application is the Prime Video app, the user inputs a selection command based on the Prime Video app item. In response to this selection command, as shown in Figure 22 (3), the TV adds a second window 1203 to the right of the first window 1201 and displays the second screen of the Prime Video app in second window 1203. This second screen is a screen of Movie B. This achieves the effect of split-screen display of the YouTube app and the Prime Video app. If the TV determines that the first window 1201 is the focused window, it adds a focus frame 1204 above the first window 1201, as shown in Figure 22 (4). Among them, focus frame 1204 is a frame-shaped window, and area 1205 is hollow. The display level of focus frame 1204 is higher than the display level of first window 1201. The size of focus frame 1204 matches the size of first window 1201. When focus frame 1204 is displayed on first window 1201, the border of first window 1201 is covered by focus frame 1204, and the image of movie A displayed in first window 1201 can be presented to the user through area 1205. Based on the split-screen interface shown in Figure 22 (4), the user can quickly determine that the focus window is first window 1201 based on focus frame 1204.

[0252] Example 2

[0253] Take the display device 200 as a television, the control device 100 as a remote controller, and the first application as a fitness application as an example.

[0254] As shown in Figure 23 (1), the TV displays the first screen of a fitness app in first window 1301. This first screen is a picture of a fitness video. The user inputs a split-screen command to the TV using the split-screen button on the remote control. In response to this command, the TV determines that the second app associated with the fitness app is a camera app. As shown in Figure 23 (2), the TV adds a second window 1302 to the right of first window 1301. After turning on the camera, the TV displays the second screen of the camera app in second window 1302. This second screen is the picture captured by the TV's camera. Thus, by displaying the fitness app and camera app in split screen, the user's movements during fitness can be corrected. If the TV determines that second window 1302 is the focus window, it adds a focus frame 1303 above second window 1302, as shown in Figure 23 (3). Focus frame 1303 is a frame-shaped window with a hollow area 1304. Focus frame 1303 is displayed at a higher level than second window 1302. The size of focus frame 1303 matches the size of second window 1302. When focus frame 1303 is displayed on second window 1302, the border of second window 1302 is covered by focus frame 1303, and the image captured by the TV camera displayed in second window 1302 can be presented to the user through area 1304. Based on the split-screen interface shown in ③ of Figure 23, the user can quickly determine that the focus window is second window 1302 based on focus frame 1303.

[0255] Example 3

[0256] For example, the display device 200 is a television, the control device 100 is a remote controller, and the first application is a YouTube application.

[0257] When using the YouTube app, the user can send a split-screen command to the TV through the split-screen button on the remote control. The TV will respond to the split-screen command by displaying a split-screen application list, which includes items of applications that can be displayed split-screen with the YouTube app, such as the YouTube app. If the user selects the item of the Prime Video app, as shown in ① in Figure 24, the TV will display the first screen of the YouTube app, such as movie A, in the first window 1401, and the second screen of the Prime Video app, such as movie B, in the second window 1402. And, if the TV determines that the first window 1401 is the focus window, a focus frame 1403 is displayed on the upper layer of the first window 1401. The above process of displaying the YouTube app and the Prime Video app in split screen is similar to Example 1 and will not be repeated here.

[0258] When the YouTube app and Prime Video app are displayed in split-screen mode, if the user wishes to change the split-screen app, they can use the remote control to move the focus to the window containing the app they wish to change. For example, if the user wishes to change the Prime Video app, they can use the remote control to move the focus to second window 1402. At this point, as shown in FIG24(2), a focus frame 1403 will be updated and displayed on second window 1402, indicating that second window 1402 is the focused window. While the focus is on second window 1402, the user can use the split-screen button on the remote control to send another split-screen command to the TV. In response to this split-screen command, as shown in FIG24(3), the TV displays a split-screen application list 1404 on second window 1402. This list includes items for split-screen apps that can be displayed split-screen with the YouTube app. To prevent invalid user selections, this list does not include the currently displayed Prime Video app but includes items for other split-screen apps, such as App1 and App2. The focus will be displayed on the first item in the split-screen application list 1404 by default, and the user can control the focus to move on each item by using the direction buttons on the remote control. If the user wants to display the App2 application in split screen with the YouTube application, the right button on the remote control can be used, as shown in ③ in Figure 24, to move the focus (shown in a thick box) to the item of the App2 application, and send a confirmation instruction to the TV through the confirmation button on the remote control. In response to the confirmation instruction, the TV will exit the Prime Video application and display the screen of the App2 application in the second window 1402, such as a chat interface, as shown in ④ in Figure 24. At this time, the focus frame 1403 is still displayed on the upper layer of the second window 1402 to indicate to the user that the focus window is the second window 1402.

[0259] Example 4

[0260] Take the display device 200 as a television, the control device 100 as a remote controller, and the first application as a fitness application as an example.

[0261] When a user is using a fitness app, they can send a split-screen command to the TV via the split-screen button on the remote control. The TV will respond to the split-screen command, determine the camera app associated with the fitness app, and display the first screen of the fitness app, such as the screen of a fitness video, in the first window 1501, as shown in ① in Figure 25, and display the second screen of the camera app, such as the captured screen, in the second window 1502. In addition, if the TV determines that the second window 1502 is the focus window, a focus frame 1503 is added to the upper layer of the second window 1502. The above process of displaying the fitness app and the camera app in split screen is similar to Example 2 and will not be repeated here.

[0262] When the Fitness app and the Camera app are displayed in split-screen mode, if the user wishes to change the app displayed in split-screen mode with the Fitness app, they can use the remote control to move the focus to second window 1502. In this example, since the focus is already displayed in second window 1502, the user does not need to perform the additional operation of moving the focus to second window 1502. While the focus is displayed in second window 1502, the user can use the split-screen button on the remote control to send another split-screen instruction to the TV. In response to this split-screen instruction, as shown in FIG. 25 (2), the TV displays a split-screen application list 1504 in second window 1502. This split-screen application list 1504 includes items for split-screen applications that can be displayed in split-screen mode with the Fitness app, as well as items for programs from different signal sources than the Fitness app, such as Live TV and App3. The focus is displayed on the first item in split-screen application list 1504 by default, and the user can use the directional buttons on the remote control to control the focus between items. If the user wishes to display live TV in a split-screen format alongside the fitness app, as shown in step 2 of FIG25 , since the focus (indicated by a bold outline) is displayed by default on the live TV item, the user can send a confirmation command to the TV using the confirmation button on the remote control. In response to this confirmation command, the TV will exit the camera application and display the live TV content, such as Program 1, in second window 1502, as shown in step 3 of FIG25 . At this point, focus frame 1503 remains displayed above second window 1502 to indicate to the user that second window 1502 is the focused window.

[0263] To enhance app and system platform security, untrusted touch events have been added. This prevents touch events from being passed down when the touch window is different from the underlying app. Consequently, when a user uses touch and mouse to operate an application within the focus window under the focus frame, the application becomes unresponsive. This means that the user can operate the application within the focus window under the focus frame using the remote control, but cannot operate it with the mouse, resulting in a peripheral malfunction.

[0264] The display device 200 can respond to the mouse operation according to the process shown in FIG26 , and the specific steps are as follows:

[0265] S1601, setting the focus frame to be trusted by the system.

[0266] In order to prevent the focus frame from affecting mouse operations, it is necessary to set the system to trust the window where the focus frame is located. Because the focus frame and the application in the underlying focus window do not belong to the same application, it is necessary to modify the properties of the focus frame. By analyzing and experimenting with Android 12's restrictions on untrusted touch events, a set of methods that call WindowManager.LaYoutParams through the reflection mechanism is designed to bypass this restriction of the focus window. You can refer to the following code:

[0267] S1602 : In response to an operation instruction input by the user to the application in the focus window based on the focus frame, a touch event corresponding to the operation instruction is transmitted to the application in the focus window.

[0268] With the above settings, when adding a focus frame and initializing LaYoutParams, you can set trustedOverLay by directly setting the private flag (privateFlags), so that the system trusts the added focus frame and allows touch events to be passed down, thereby solving the problem that the application in the focus window cannot respond to mouse operations.

[0269] The user can zoom in or out of the focus window. The display device 200 can zoom in or out of the focus window according to the process shown in FIG. 27 . The specific steps are as follows:

[0270] S1701, in response to a zoom instruction input by a user, determining a target distance between a junction of a first window and a second window and a target border according to a size of a display area, a preset split-screen ratio, and a size that changes in a zooming direction.

[0271] If the first window and the second window are arranged side by side, the zooming direction is horizontal. If the first window and the second window are arranged top by top, the zooming direction is vertical.

[0272] The target border is the first border in the display area that is perpendicular to the scaling direction. For example, if the first and second windows are arranged side by side, the target border is the left border of the display area. If the first and second windows are arranged top by top, the target border is the top border of the display area.

[0273] There is a threshold for the size change in the zoom direction. This threshold is the distance between the boundary between the first and second windows and the border of the display area perpendicular to the zoom direction. For example, if the zoom direction is horizontal to the right, the threshold is the distance between the boundary between the first and second windows and the right border of the display area.

[0274] S1702: Determine the changed positions and sizes of the first window and the second window according to the size of the display area and the target distance, and determine the changed position and size of the focus frame according to the changed position and size of the focus window.

[0275] For example, if the first window and the second window are arranged left and right, with the first window on the left and the second window on the right, the second window is the focus window. The focus window is currently located in the area starting at the target distance from the left border of the display area, and its width is equal to the target distance, and its height is equal to the height of the display area. The width of the second window is equal to the difference between the width of the display area and the target distance, and its height is equal to the height of the display area.

[0276] Based on the changed positions and sizes of the first window and the second window, the changed position and size of the focus window can be determined, and the changed position and size of the focus window can be directly used as the changed position and size of the focus frame.

[0277] S1703 , displaying the changed first window and the second window according to the changed positions and sizes of the first window and the second window, and displaying the changed focus frame according to the changed position and size of the focus frame.

[0278] Display device 200 displays the changed first and second windows based on the changed positions and sizes of the first and second windows, and displays the changed focus frame based on the changed position and size of the focus frame. At this point, the changed focus frame can still be accurately displayed above the changed focus window, and its size matches the changed focus window. This allows the focus window to be zoomed in and out based on the zoom buttons within the focus window, and the focus frame to be dynamically adjusted based on the zoom changes of the focus window.

[0279] As shown in FIG28 ①, the display device 200 displays a first window 1801 and a second window 1802 in a split screen, wherein the second window 1802 is the focus window, and a focus frame 1803 is displayed on the second window 1802. The user can input a menu instruction to the display device 200 via the menu key in the control device 100. In response to the menu instruction, the display device 200 displays a control menu in the focus window (i.e., the second window 1802). As shown in FIG28 ①, the control menu includes a zoom-out button 1804 and an enlargement button 1805. The user can input a zoom-out instruction to the display device 200 using the zoom-out button 1804. In response to the zoom-out instruction, the display device 200 zooms out the second window 1802 and enlarges the first window 1801, as shown in FIG28 ②. The horizontal sizes of the first window 1801 and the second window 1802 change. Correspondingly, based on the changed position and size of the second window 1802, the size of the focus frame 1803 is also reduced accordingly, and the reduction direction is horizontally to the right. The reduced size matches the size of the reduced second window 1802, and the reduced focus frame 1803 is still displayed on the reduced second window 1802.

[0280] The user can input a zoom-in instruction to the display device 200 using the zoom-in button 1805. In response to the zoom-in instruction, the display device 200 enlarges the second window 1802 and reduces the first window 1801. The horizontal sizes of the first window 1801 and the second window 1802 change. Accordingly, based on the changed position and size of the second window 1802, the size of the focus frame 1803 is also enlarged horizontally to the left. The enlarged size matches the size of the enlarged second window 1802, and the enlarged focus frame 1803 remains displayed on the enlarged second window 1802.

[0281] In some embodiments, the updated LaYoutParams of the focus frame can be obtained based on the coordinate system. Take the first window and the second window as an example, where the first window is on the left and the second window is on the right. A coordinate system is established with the lower left corner vertex of the first window as the origin, the lower border of the display area of ​​the display as the x-axis, and the left border of the display area of ​​the display as the y-axis. If the preset split-screen ratio is 1:1, the junction of the first window and the second window is the center line position of the display area. Assuming that the threshold in the zoom direction is x, if the user inputs a zoom-in command, the horizontal coordinate (target distance) of the junction is calculated as: Mx = MscreenWidth / 2+x. If the user inputs a zoom-out command, the horizontal coordinate (target distance) of the junction is calculated as: Mx = MscreenWidth / 2-x. After calculating Mx, the split-screen application will notify SystemUI to call the system SplitScreenController's ChangeBound method to change the horizontal coordinates of the first window and the second window to display the scaled first window and the second window.

[0282] When a split-screen app notifies systemUI to call the changeBound method to resize the first and second windows, it passes the horizontal coordinate parameter Mx of the boundary, allowing the app to use this parameter to update its own focus frame. For example, if the user enters a zoom-out command and the focus window is the second window on the right, the updated focus frame parameters are as follows:

[0283] MLaYoutParams.width=mScreenWidth-Mx;

[0284] MLaYoutParams.height=mScreenHeight;

[0285] MLaYoutParams.x = Mx;

[0286] mlAuTotalParts.y = 0;

[0287] You can use the updateViewLaYout method of WindowManager to update the size of the focus frame, that is, WindowManager.updateViewLaYout(focusWindow,MLaYoutParams).

[0288] The display device 200 can scale the focus window according to the process shown in FIG29 . The specific steps are as follows:

[0289] S1901: Display a separator at the boundary between the first window and the second window.

[0290] After splitting the screen, the Android system can automatically add a divider view (DividerView) at the intersection of the first and second windows. This divider can be in the form of a line, segment, dot, arrow, etc. This divider is used to indicate the intersection of the first and second windows.

[0291] As shown in ① in Figure 30, a separator 2003 is displayed at the junction of the first window 2001 and the second window 2002. The separator 2003 is in a segment shape. If the first window 2001 is the focus window, a focus frame 2004 is also displayed on the first window 2001.

[0292] S1902 : In response to a movement instruction input by the user based on the separator, move the separator, and obtain a position of the separator after the move in the display area.

[0293] The user can move the spacer left or right by operating the mouse. For example, after selecting the spacer by operating the mouse, the user slides the mouse to the right to input a movement instruction to the display device 200, where the movement instruction instructs to move the spacer to the right.

[0294] The display device 200 displays the moving track of the spacer in response to the moving instruction, and obtains the position of the spacer within the display area when the spacer reaches the position after the movement indicated by the user.

[0295] S1903, determining the changed positions and sizes of the first window and the second window according to the position of the separator after it moves in the display area, and determining the changed position and size of the focus frame according to the changed position and size of the focus window.

[0296] The separator is used to indicate the boundary between the first window and the second window. Therefore, the position of the separator after it moves within the display area corresponds to the position of the boundary between the first window and the second window after the change. Therefore, the display area can be re-divided based on the position of the separator after it moves within the display area, with the re-divided display areas corresponding to the changed first window and the second window, respectively. The process of determining the changed positions and sizes of the first window and the second window based on the position of the separator after it moves within the display area is equivalent to the process of determining the changed positions and sizes of the first window and the second window based on the changed boundary between the first window and the second window.

[0297] In some embodiments, a target distance between the boundary between the first and second windows after the change and a target border can be calculated. The target border is the first border in the display area perpendicular to the direction of movement of the separator. The positions and sizes of the first and second windows after the change are then determined based on the target distance and the size of the display area. The above process can be referred to steps S1701-S1702 and will not be further described here.

[0298] Based on the changed positions and sizes of the first window and the second window, the changed position and size of the focus window can be determined, and the changed position and size of the focus window can be directly used as the changed position and size of the focus frame.

[0299] S1904 , displaying the changed first window and the second window according to the changed positions and sizes of the first window and the second window, and displaying the changed focus frame according to the changed position and size of the focus frame.

[0300] Display device 200 displays the changed first and second windows based on the changed positions and sizes of the first and second windows, and displays the changed focus frame based on the changed position and size of the focus frame. At this point, the changed focus frame can still be accurately displayed above the changed focus window, and its size matches the changed focus window. This allows for the effect of scaling the focus window based on the spacer, and dynamically adjusting the focus frame based on the scaling of the focus window.

[0301] Take the example of a user controlling the movement of separator 2003 by mouse control. If the user wants to expand first window 2001, the movement instruction input by the user is to instruct to move separator 2003 to the right. As shown in ② of FIG30 , after separator 2003 moves a distance a, first window 2001 and second window 2002 are adjusted based on the moved separator 2003, wherein first window 2001 is equivalent to expanding a horizontally to the right, while second window 2002 is equivalent to shrinking a horizontally to the right. Focus frame 2004 changes accordingly based on the changed position and size of first window 2001. As shown in ② of FIG30 , the changed focus frame 2004 is still displayed on the changed first window 2001, and its size matches the changed first window 2001.

[0302] In the disclosed embodiment, DividerView can rewrite the touch monitoring callback onTouch method. When the user slides DividerView with the mouse, the callback method will be triggered. When the sliding is detected, DividerView will calculate the sliding distance:

[0303] final int position=mSplitLaYout.getDividePosition()+touchPos–mStartPos

[0304] DividerView will then notify the system to update the sizes of the first and second windows using the window container update (WindowContainerTransaction) method.

[0305] In the onTouch callback method of DividerView, after identifying the touch event as a motion event MotionEvent.ACTION_MOVE, the sliding distance is calculated, and then a notification is sent to the split-screen application, and the calculated sliding distance is passed at the same time.

[0306] After the split-screen application obtains the calculated value, it also uses WindowManager to directly update the size of the focus frame.

[0307] After the first window and the second window are displayed in split screen mode, the user may move the focus from the current focused window to another window by clicking an application button, using voice, or using a peripheral device, i.e., switching the focused window, so as to operate the application in the switched focused window. The display device 200 can periodically detect the application stack and promptly add a focus frame to the window that actually occupies the focus.

[0308] After displaying the first and second windows in split-screen, display device 200 creates a process for periodically checking application stacks corresponding to the first and second applications currently displayed in split-screen. Display device 200 can manage each application stack in the form of an application stack list, where the first application stack is the application stack corresponding to the application that actually occupies focus, that is, the application stack corresponding to the application in the actual focus window.

[0309] The display device 200 may perform a timed detection on the application stack according to the process shown in FIG31 , including:

[0310] Step 311: Start the timing detection application stack;

[0311] Step 312: Obtain an application stack list;

[0312] Step 313: Obtain the first application stack in the application stack list;

[0313] The display device 200 can control the period of timed application stack detection through a timer. After starting the timed application stack detection, the display device 200 detects the application stack according to a preset period. During the detection process, the display device 200 obtains an application stack list and obtains the first application stack in the application stack list. The first application stack indicates the application that currently actually occupies the focus, that is, the application in the focus window of the focus frame that should be displayed.

[0314] Step 314: Determine whether the first application stack is consistent with the application in the window where the focus frame is located;

[0315] Step 315: If not, update the focus frame; otherwise, return to the above step 312;

[0316] The display device 200 determines whether the application indicated by the first application stack is consistent with the application in the window where the focus frame is currently located. If they are consistent, it means that the window where the focus frame is located is the application used to display the application indicated by the first application stack. If they are inconsistent, it means that the window where the focus frame is located is not used to display the application indicated by the first application stack, that is, the focus frame is not displayed on the window where the application indicated by the first application stack is located, and the position of the focus frame display is incorrect. The focus frame needs to be updated and displayed on the window where the application indicated by the first application stack is located. Based on the above process, it can be ensured that the display device 200 promptly updates the focus frame to the window where the application that actually occupies the focus is located, so as to accurately indicate the focus window to the user.

[0317] The process of updating the displayed focus frame can be implemented by TaskCheckListener in the system architecture diagram shown in Figure 21. The ActivityManager provides the getRunningTasks method, which accepts a non-parameter integer value and returns a RunningTaskInfo list representing the running tasks. The first item in this list is the application occupying focus. TaskCheckListener starts a scheduled task, compares the first item with the application currently in focus, and determines whether to update the focus frame position.

[0318] Based on the process shown in FIG31 , if the user switches the focus window, the display device 200 may switch the focus window according to the process shown in FIG32 . The specific steps are as follows:

[0319] S2201 , in response to a focus window switching instruction input by a user, displaying the focus in the switched focus window, and modifying the first application stack in the application stack list to be the application displayed in the switched focus window.

[0320] As shown in ① of FIG. 33 , the display device 200 displays a first window 2301 and a second window 2302 in a split screen. The first window 2301 displays the screen of the first application, and the second window 2302 displays the screen of the second application. The focus window is the first window 2301, and the focus 2303 is displayed in the first window 2301. A focus frame 2304 is displayed above the first window 2301. The user can input a switching instruction using the mouse to instruct the focus to move to the second window 2302. In response to the switching instruction, as shown in ② of FIG. 33 , the display device 200 displays the focus 2303 in the second window 2302, and modifies the first application stack in the application stack list to the application stack corresponding to the second application, indicating that the application actually occupying the focus is the second application.

[0321] S2202: Detect, according to a preset period, whether the first application stack in the application stack list and the focus window where the focus frame is located correspond to the same application.

[0322] After displaying the first window 2301 and the second window 2302 in split-screen mode, the display device 200 initiates a timed detection application stack process. Based on the timed detection application stack process shown in FIG31 , the display device 200 detects whether the focus frame needs to be updated according to a preset period. The detection process is described above with reference to the process shown in FIG31 and will not be further described here.

[0323] S2203: If the applications correspond to different ones, the focus frame is displayed on the window where the application corresponding to the first application stack is located.

[0324] Based on this detection process, the display device 200 can detect that the first application stack (i.e., the second application) and the application in the first window 2301 where the focus frame 2304 is currently located (i.e., the first application) are not the same application, and determines that the display focus frame 2304 needs to be updated, that is, as shown in ③ in Figure 33, the focus frame 2304 is displayed on the second window 2302.

[0325] When the focus moves to the window on the other side, after determining the updated layout parameters of the focus frame, the layout parameters of the focus frame can be updated by calling the updateViewLaYout method of WindowManager, for example: WindowManager.updateViewLaYout(focusWindow, MLaYoutParams), and then the focus frame is updated based on the updated layout parameters.

[0326] When an information prompt box appears in the application in the focus window, the display device 200 can display the message prompt box according to the process shown in FIG34 . The specific steps are as follows:

[0327] S2401: Monitor display messages of the application in the focus window.

[0328] The application's display message is used to indicate the display of an information prompt box. The information prompt box can be a system pop-up window, such as a toast.

[0329] S2402: If a display message is detected, the information prompt box is displayed on the upper layer of the focus frame.

[0330] If the display device 200 monitors the display message, the display device 200 displays the information prompt box in the manner shown in Figure 35. As shown in Figure 35, take the gradual increase of the display level from bottom to top as an example. The display device 200 displays the first window 2501 and the second window 2502 in a split screen at a lower display level, wherein the screen of the first application is displayed in the first window 2501, and the screen of the second application is displayed in the second window 2502. The focus frame 2503 is displayed at a higher display level, and the position and size of the focus frame 2503 match the first window 2501. If the display device 200 monitors the message that the first application displays a pop-up window 2504, the pop-up window 2504 is displayed at the highest display level. As a result, the displayed pop-up window 2504 will not be blocked or affected by the focus frame 2503.

[0331] As explained above, the display device of the present disclosure can be a television. Considering that currently, installing a fitness teaching video APP in a smart TV and following the fitness teaching APP for fitness training has become a mainstream way of home fitness. When following the fitness teaching APP for fitness training, due to problems such as lack of fitness experience, users may not know whether their body movements are correct. To solve this problem, some types of smart TVs have launched a fitness split-screen mode, which plays the user's movement video captured by the camera while displaying the fitness media played by the fitness teaching APP.

[0332] However, the current split-screen fitness mode requires users to first activate the monitor's split-screen mode, then launch the fitness app within split-screen mode. However, in practice, users often trigger the smart TV split-screen display only when following the fitness app to complete warm-up exercises and simple fitness exercises, or when learning complex movements. The current method of launching the fitness app in split-screen mode fails to meet users' needs for instant access to fitness split-screen mode, resulting in a poor user experience.

[0333] In order to solve the above problem, the processor of the present disclosure may further include steps S510-S550 as shown in FIG36 when executing the aforementioned split-screen display method.

[0334] S510: In response to receiving a menu pop-up instruction, determine whether the current video is a fitness teaching video and whether the display plays the current video in full screen mode; if both are true, execute S520.

[0335] In the disclosed embodiment, after the user turns on the display device and issues a video play instruction to the display device, the display device starts playing the current video. The corresponding video file is used to play the current video on the display. In other embodiments, the processor can use a client application such as YouTube to obtain and parse the current video provided by the remote server to play the current video on the display. In other embodiments, the processor can also receive video streams sent by other smart terminals, interpret the video streams, and play the current video on the display.

[0336] When the display device is playing the current video, the user can send a menu pop-up instruction to the processor of the display device through the remote control, smart device or gesture control mentioned above.

[0337] When the processor receives a menu pop-up instruction triggered by the user, it does not determine the type of options included in the pop-up menu according to the pre-configured template in the conventional way, load the corresponding options and control the display to display the pop-up menu, but first determines whether the current video is a fitness teaching video and whether the display is playing the current video in full-screen mode, and determines the type of options included in the pop-up menu based on the judgment result, and displays the pop-up menu.

[0338] In the specific implementation of the embodiment of the present disclosure, the processor can use different methods to determine whether the current video played by the display is a fitness teaching video.

[0339] In some embodiments, when the display device uses a client application to play the current video, the processor can determine whether the current video is a fitness teaching video by obtaining the type identifier of the current video provided by the client application, or based on the network address of the current video stream provided by the client application.

[0340] In some other embodiments, the processor is configured with a pre-trained video type recognition model that can determine the type of a video by processing video frames. The processor can intercept a video frame of the current video and use the video type recognition model to identify the current video to determine whether the current video is a fitness instructional video.

[0341] In the embodiment of the present disclosure, the processor may determine whether the display plays the current video in full-screen mode by calling the playback window configuration parameters.

[0342] S520: Control the display to display a first pop-up menu including a split-screen display option.

[0343] When the processor determines that the current video is a fitness teaching video and the display plays the current video in a full-screen mode, the control device controls the display to display a first pop-up menu including a split-screen display option.

[0344] In a specific implementation, the processor can obtain the default options and the aforementioned split-screen display options included in the first pop-up menu based on the feature that the display is playing the current video, reasonably arrange the default options and the split-screen display options, determine the display style of the first pop-up menu, and control the display to display the first pop-up menu according to the display style.

[0345] In some embodiments of the present disclosure, the processor may call a WindowsManager method provided by the operating system to display a first pop-up menu in the current display window of the display (that is, the window playing the current video).

[0346] FIG37 is a schematic diagram of a user interface of a display according to some embodiments. As shown in FIG37 , when the processor controls the display to display the first pop-up menu 602, the first pop-up menu 602 is displayed in a floating manner within the current display window 601. To enhance the aesthetics of the user interface, the first pop-up menu 602 is displayed below the current display window 601. The first pop-up menu 602 includes a split-screen display option, and the split-screen display option is displayed in a highlighted form, that is, the split-screen display option is in a selected state. In conjunction with FIG37 , in addition to the split-screen display selection, the first pop-up menu 602 may also include a timer setting, a resolution setting option, a parental control option, and a help option.

[0347] It should be noted that in the embodiment of the present disclosure, the "Show Split Screen Display" option is added to the first pop-up menu 602 only when the current video is a fitness instructional video and the display is playing the current video in full-screen mode. If the current video is not a fitness instructional video, or the display is not playing the current video in full-screen mode, the first pop-up menu 602 will not include the "Show Split Screen Display" option.

[0348] S530: In response to receiving a control instruction for triggering the split-screen display option, controlling the front camera to start and receive a local video generated by the front camera.

[0349] After the display displays the first pop-up menu 602, the user can see various selectable options in the first pop-up menu 602. When the user wants to control the display device to start the split-screen display mode, the user can use a remote control, smart device or gesture control to send a control instruction to trigger the split-screen display option to the processor.

[0350] After the processor receives the control instruction for the user to trigger the split-screen display selection, the processor controls the initialization of the front camera, and after successful initialization, captures the environment on the display surface side of the display, thereby obtaining a local video. It should be noted that the front camera referred to in the embodiments of the present disclosure is a camera that captures the environment on the display surface side of the display. In some embodiments, the camera configured with the display device is a rotatable camera. In this case, the camera will be identified as a front camera only when the camera lens is facing the display surface side of the display; when the rotatable camera is facing the non-display surface side of the display, it is no longer regarded as a front camera and will not be activated.

[0351] In a specific embodiment, the processor can start the front camera by calling the camera driving method provided by the bottom layer of the operating system, and receive the local video captured by the front camera through the corresponding interface.

[0352] S540: Control the display to split into two display windows, and play the current video and the local video in the two display windows respectively.

[0353] After receiving the local video from the front camera, the processor determines that the front camera has been successfully started and returns the local video normally. The processor can then use the display window control method provided by the operating system to split the display interface into two display windows, and play the current video and the local video in each display window respectively.

[0354] It should be noted that if the front camera is not started normally, the processor cannot obtain the local video. In order to avoid the situation where one display window has no corresponding display video after splitting the screen, the processor will not control the display screen to split into two display windows.

[0355] Figure 38 is a schematic diagram of a user interface displayed on a display according to some embodiments. As shown in Figure 38, the display interface of the display is divided into two display windows on the left and right: a first display window 701 on the right and a second display window 702 on the left. The first display window 701 plays the local video transmitted by the front camera, and the second display window 702 plays the current video. Because the current video is a fitness instructional video and the local video is a follow-up video of the user's body movements, the user can view both the fitness instructional video and the follow-up video simultaneously while exercising to confirm whether their movements are correct and standard.

[0356] Using the split-screen display method provided in an embodiment of the present disclosure, after receiving a menu pop-up instruction, the processor determines whether the current video being played by the display is a fitness instruction video and determines whether the display is playing the current video in full screen mode. If the current video being played by the display is a fitness instruction video and is being played in full screen mode, the first pop-up menu displayed by the display includes a split-screen display option. If the user triggers the split-screen display option, the processor controls the front-facing camera to start capturing the local video and causes the display to play the current video and the local video in two display windows, respectively, in a split-screen display mode.

[0357] By adopting the split-screen display method provided in the embodiment of the present disclosure, when the display device plays a fitness teaching video and the user wants to trigger the display to realize split-screen display of a local video for follow-up training, the processor provides a first pop-up menu including split-screen display selection, so that the user can quickly switch the display's playback mode to a follow-up split-screen display mode when playing the fitness video, thereby improving the user experience.

[0358] In some embodiments of the present disclosure, after the processor controls the display to play the current video and the local video in two display windows, the user may send a menu pop-up instruction to the processor again for various reasons. In this case, in addition to executing the aforementioned S510-S540, the processor may also execute the following S550-S560 after executing the aforementioned S540.

[0359] S550: In response to receiving the menu pop-up instruction again, controlling the display to display a second pop-up menu including a mirroring option in the first display window playing the local video.

[0360] In some embodiments of the present disclosure, when the processor receives the menu pop-up instruction again, it loads the pre-configured option type and calls the WindowsManager method provided by the operating system to display the second pop-up menu 703 in the first display window 701 .

[0361] Figure 39 is a schematic diagram of a user interface displayed on a display according to some embodiments. As shown in Figure 39 , when the processor controls the display to display a second pop-up menu 703, the second pop-up menu 703 is displayed in a floating manner within the first display window 701. As shown in Figure 39 , the first pop-up menu includes a Mirror option, and the Mirror option is selected. The Mirror option is used to trigger the local video displayed in the first display window 701 to be mirrored left and right.

[0362] In some embodiments, the user's body posture in the local video displayed in the first display window 701 is exactly opposite to the instructor's posture in the fitness instructional video played in the second display window 702. This makes it difficult for the user to quickly identify whether their own training movements are correct. In order to facilitate the user to easily compare their own movements with the instructor's movements to see if they are consistent, and then confirm whether the training movements are correct, the user needs to trigger the mirror flipping of the current video. To meet the user's aforementioned needs, in the disclosed embodiment, a mirror option is displayed in the second pop-up menu 703.

[0363] S560: In response to receiving a control instruction for triggering the mirroring option, the local video is horizontally mirror-flipped, and the display is controlled to play the horizontally mirror-flipped local video in the first display window.

[0364] If the user wants to flip the local video to the left and right mirror images, the user can use a remote control, smart device or gesture control to send a control instruction to trigger the mirroring option to the processor of the display device.

[0365] After receiving the control instruction to trigger the mirroring option, the processor obtains the local video frame in the OnPreviewFrame method provided by the operating system, invokes a pre-set mirror flipping algorithm to mirror-flip the video frame, and thereby obtains the mirror-flipped local video. After obtaining the mirror-flipped local video, the processor controls the display to play the horizontally mirror-flipped local video in the first display window 701.

[0366] Depending on the data storage type and color mode of the local video frame, the processor needs to use the corresponding data processing algorithm to process the video frame to obtain the mirror-flipped video frame. The specific implementation of mirror-flipping for various types of video frames can be found in relevant technical literature and will not be elaborated here.

[0367] In some specific embodiments of the present disclosure, the local video captured by the front camera is in YUV color mode, and the video frames are stored using a yuvbyte array. For this type of local video, the processor uses the following program code to implement mirror flipping of each video frame in the local video.

[0368] As shown in Figures 38 and 39, in some embodiments, when the processor controls the display to split into two display windows, the processor specifically controls the display to split into two display windows in a manner that is evenly divided from left to right. When the display is split into two display windows in a manner that is evenly divided from left to right, in order to ensure that the second pop-up menu 703 displayed in the first display window 701 is positioned reasonably, that is, to achieve the display effect shown in Figure 39, the processor needs to execute the following S551-S553.

[0369] S551: Obtain the window width of the display, the preset margin width, the number of options in the second pop-up menu 703, and the option width.

[0370] The processor can obtain the display window width from the configuration parameters in the WindowManager method of the underlying operating system, and can determine the preset margin width, the number of options in the second pop-up menu 703, and the option width of each option based on the system configuration parameters. The margin width is the width left on the left and right sides of the second pop-up menu 703 when it is displayed, that is, the width from the left and right edges of the second pop-up menu 703 to the edge of the first display window 701.

[0371] S552: Determine the horizontal coordinates of the reference point of the second pop-up menu according to the window width, the margin width, the number of options in the second pop-up menu 703, and the option width. The reference horizontal coordinates are coordinates in the global coordinate system of the display.

[0372] After obtaining the window width, margin width, number of options and option width of the second pop-up menu 703, the processor can then calculate the horizontal coordinates of the reference point of the second pop-up menu using the following formula:

[0373] xray=screenWidth / 2+(screenWidth / 4-(itemWidth*N+itemMargin*(N-1)+padding / 2) / 2)

[0374] In the above formula, screenWidth represents the window width, itemWidth represents the width of a single option, N represents the number of options in the second pop-up menu, itemMargin represents the margin width for each option, and padding represents the margin width. It should be noted that the base horizontal coordinate is in the display's global coordinate system.

[0375] S553 : Control the display to display the second pop-up menu 703 including the mirror option using the horizontal coordinates of the reference point, so as to realize the display of the pop-up menu in the first display window 701 .

[0376] After determining the reference horizontal coordinate, the processor then determines the display position of the second pop-up menu 703 in the display according to the aforementioned reference point horizontal coordinate and the pre-set vertical coordinate, and controls the display to display the second pop-up menu 703 according to the aforementioned display position, so as to realize the display of the second pop-up menu 703 as shown in Figure 39.

[0377] As shown in FIG39 , in some embodiments of the present disclosure, in the second pop-up menu 703 that the processor controls the display to display in a floating manner in the first display window 701, in addition to the aforementioned mirroring option, a focus window switching option is also included. The corresponding processor may also execute the following S570-S580.

[0378] S570: In response to receiving a control instruction to trigger the focus window switching option, the display is controlled to switch the second display window 702 playing the current video to the focus window, and the display is controlled to display a third pop-up menu 704 including a full-screen play option in the second display window 702.

[0379] The focus window is the window in the selected state between the first display window 701 and the second display window 702. When the first display window 701 or the second display window 702 is the focus window, if the processor receives a menu pop-up instruction, it will display a pop-up menu in the focus window.

[0380] If the user wants to control the left-right mirror flipping of the local video displayed in the first display window 701, a control instruction of the focus switching option can be sent to the processor of the display device by using a remote control, a smart device or gesture control.

[0381] When the processor receives the control instruction for the focus switching option, the processor calls the ActivityManager method provided by the operating system to switch the second display window 702 to the focus window, and calls the updateViewLayout method in the WindowsManager provided by the operating system to display the third pop-up menu 704 in the second display window 702 of the display.

[0382] Figure 40 is a schematic diagram of a user interface displayed on a display according to some embodiments. As shown in Figure 40 , the third pop-up menu 704 is located at the center of the lower edge of the second display window 702. In a specific implementation, to display the third pop-up menu 704 at the center of the lower edge of the second display window 702, the processor uses the following method to determine the horizontal coordinates of the reference point of the third pop-up menu 704. First, the display window width, the preset margin width, the number of options in the third pop-up menu 704, and the option width are obtained.

[0383] The horizontal reference coordinate of the third pop-up menu 704 is determined by screenWidth / 4 – (itemWidth*M+(M-1)*itemMargin+padding*2) / 2. As mentioned above, screenWidth represents the window width, itemWidth represents the width of a single option, itemMargin represents the margin width for the option, padding represents the margin width for the margin, and M represents the number of options in the third pop-up menu 704.

[0384] As described above, the third pop-up menu 704 includes a full-screen play option. If the user wants to control the display device to exit the split-screen display and display the current video in full-screen mode, a control instruction to trigger the full-screen play option can be sent to the processor of the display device using a remote control, a smart device, or gesture control. After receiving the control instruction to trigger the full-screen play option, the processor controls the display to resume playing the current video in full-screen mode. In a specific implementation, if the user feels that a certain fitness exercise is particularly difficult to learn and wants to carefully review the content of the fitness video first, the full-screen play option can be used to quickly achieve full-screen playback of the current video, so that the user has a better user experience.

[0385] As shown in Figure 40, in addition to the aforementioned full-screen play option, the third pop-up menu 704 also includes a focus switch option and an exit option. The corner switch option is used to trigger the switch of the currently displayed focus window to the first display window 701, and the exit option is used to control the exit of the third pop-up menu 704.

[0386] Please refer back to Figure 39. In some embodiments of the present disclosure, in addition to the aforementioned mirror option and corner point switching option, the second pop-up menu 703 also includes a full-screen play option. This full-screen play option is used to trigger the display to display the local video in full screen. In a specific implementation, the display device is often equipped with a speaker. When the display plays the video content of the current video, the speaker simultaneously plays the audio content corresponding to the current video, so that the user can receive fitness teaching information through sound and light at the same time. In some cases, the user may have become familiar with the fitness movements through early follow-up training, and at this time there is no need to continue watching the current movement, but only want to complete the fitness movement by themselves under the guidance of the audio corresponding to the current video. At this time, the user also needs to pay attention to whether his or her own movements are correct and standard.

[0387] To meet the aforementioned requirements, a full-screen play option is also provided in the second pop-up menu 703 in the disclosed embodiment. If the user wishes to control full-screen play of the local video, they can use a remote control, smart device, or gesture control to send a control instruction to the processor of the display device to trigger the full-screen play option. Accordingly, the processor executes S580 as follows.

[0388] S580: In response to receiving a control instruction to trigger the full-screen playback option, controlling the display to play the local video in full-screen mode, and controlling the speaker to play the audio corresponding to the current video at the same time.

[0389] In the embodiment of the present disclosure, when the full-screen play option in the second pop-up menu 703 is triggered, the processor controls the display to play the local video in full screen, while continuing to control the speaker to play the audio corresponding to the current video. This allows the user to complete fitness movements on their own under the guidance of the audio, and pay attention to whether their own movements are correct and standard, thereby improving the user experience.

[0390] Please refer to Figures 38 to 40. In some embodiments of the present disclosure, when the display displays the current video and the local video in a left-right split screen, since the screen size of the current video does not match the window size of the first display window 701, and the screen size of the local video does not match the window size of the second display window 702, in order to play the entire current video and local video content, the current video and the local video are both displayed in reduced size, and there will be a large blank background above and below the first display window 701 and the second display window 702. Since the current video and the local video are both displayed in reduced size, the user may need to get closer to the display device to see the fitness content in the current video and the local video more clearly, resulting in a poor user experience. In addition, in some cases, when the user is too close to the display device, certain fitness movements may not be completed because the display device is idle.

[0391] Figure 41 is a flow chart of controlling two display windows to play the current video and the local video according to some embodiments. As shown in Figure 41, in order to solve the problem mentioned in the previous paragraph and make full use of the display interface of the display device, the processor executes the following S541-S542.

[0392] S541: According to the sizes of the two display windows, the fitness image regions of the corresponding current video and the local video are respectively intercepted to obtain the intercepted current video and the intercepted local video.

[0393] In the disclosed embodiment, the processor can obtain the size of the first display window 701 and the size of the second display window 702. After obtaining the size of the first display window 701, the processor can intercept the fitness image area of ​​the local video based on the size of the first display window 701, obtain the fitness image area in the local video, and use the video content of the local fitness image area as the intercepted local video. After obtaining the size of the second display window 702, the processor can intercept the fitness image area of ​​the current video based on the size of the second display window 702, and obtain the fitness image area in the current video.

[0394] In some embodiments, the processor can use a pre-trained body recognition model to identify human body locations in the current video and local video in real time to determine the captured fitness image area. In other embodiments, the processor can use an image difference algorithm to compare pixel content changes between adjacent frames of the current video to determine the corresponding fitness image area, and compare pixel content changes between adjacent frames of the local video to determine the corresponding fitness image area.

[0395] S542: Play the intercepted current video and the intercepted local video in full-window playback mode in two display windows respectively.

[0396] Since the current video and local video fitness images are captured respectively according to the size of the display window, the captured current video and local video are obtained, and the captured current video and local video match the corresponding display window size (specifically, the aspect ratio of the corresponding display window matches), the processor can control the playback of the captured current video and the captured local video in full-screen mode in the two display windows respectively.

[0397] By executing the aforementioned S541-S542, the display device can play the fitness instructional portion of the current video and the user's self-training portion of the local video in full-screen mode, allowing the user to more clearly view the corresponding content and perform movement comparisons. In addition, by playing the intercepted current video and the intercepted local video in full-screen mode, it is possible to avoid or even eliminate the blank background in the display window, thereby improving the user's viewing experience.

[0398] In the aforementioned embodiment, when the processor controls the display to execute S540 to control the display to play the current video and the local video in two display windows respectively, and receives a menu pop-up instruction again, the processor controls the display to display a second display pop-up window in the first display window. In other embodiments, when the processor controls the display to play the current video and the local video in two display windows respectively, and receives a menu pop-up instruction again, the processor controls the display to display a third pop-up menu including a full-screen play option and a focus window switch option in the second display window playing the current video, that is, to display a menu as shown in Figure 40. The third pop-up menu includes a full-screen play option and a focus window switch option. If the processor subsequently receives a control instruction to trigger the full-screen play option, the display controls the display to resume playing the current video in full screen mode; if the processor receives a control instruction to trigger the focus switch option, the display controls the display to switch the first display window playing the local video to the focus window, and controls the display to display a second pop-up menu including a mirroring option and an exit full-screen play option in the first display window. In addition to providing the aforementioned method of achieving split-screen display and exiting split-screen display by evoked pop-up menus, the display device in the embodiment of the present disclosure also provides a solution for pre-setting entry into split-screen mode and selecting applications to be launched in split-screen mode. Specifically, after a user enters a specific interactive interface, the interface displays a split-screen fitness mode and displays available client applications within the split-screen fitness mode. After the user selects the desired client application and confirms the selection, the display device's processor controls the display to enter split-screen display mode, displaying two windows: one window displays the playback interface of the selected client application, and the other window displays the local video captured by the front-facing camera.

Claims

1. A display device, include: a display configured to display an image and / or a user interface; A user interface configured to receive instructions from a user; a communicator configured to communicate with an external device according to a predetermined protocol; a memory configured to store computer instructions and data associated with a display device; At least one processor, connected to the display, user interface, communicator and memory, is configured to execute computer instructions to cause the display device to perform: In response to a first selection input for a first application and a second application received through the user input interface, controlling the display to display a selection result of the first application and a selection result of the second application, and displaying a window arrangement mode selection control and a split-screen control, wherein the first application and the second application support split-screen display; In response to a second selection input of a window arrangement mode selection control received through the user input interface, determining an arrangement mode of the first split screen area and the second split screen area; In response to a first control input for the split-screen control received through the user input interface, the display is controlled to display the interface of the first application in the first split-screen area and to display the interface of the second application in the second split-screen area.

2. The display device according to claim 1, wherein the at least one processor is further configured to execute computer instructions to cause the display device to perform: Control the display to display an application selection interface, a first focus frame and / or a second focus frame, wherein the application selection interface is used to display the application identifiers of one or more applications that support split screen, the first focus frame is used to indicate the application identifier of the first application preselected by the current user, and the second focus frame is used to indicate the application identifier of the second application preselected by the current user.

3. The display device according to claim 2, wherein the at least one processor is further configured to execute computer instructions to cause the display device to perform: In response to a first selection input for a first application and a second application received through the user input interface, move the first focus frame to a location where an application identifier of the first application is located, and move the second focus frame to a location where an application identifier of the second application is located; The color of the first focus frame and / or the second focus frame is changed, the first focus frame after the color change is used to indicate the user's selection result for the first application, and the second focus frame after the color change is used to indicate the user's selection result for the second application.

4. The display device according to claim 1, wherein the at least one processor is further configured to execute computer instructions to cause the display device to perform: Based on the first selection input for the first application and the second application received through the user input interface, the display is controlled to show the selection order of the first application and the second application.

5. The display device according to claim 1, wherein the window arrangement mode selection control comprises at least a left-right arrangement control and a top-bottom arrangement control; The processor is specifically used for: In response to a second selection input for a left-right arrangement control received through the user input interface, determining that the first split screen area and the second split screen area are arranged in a left-right split screen manner; or, In response to a second selection input for the vertical arrangement control received through the user input interface, it is determined that the first split screen area and the second split screen area are arranged in a vertical split screen manner.

6. The display device according to claim 1, wherein the at least one processor is further configured to execute computer instructions to cause the display device to perform: The control device controls the display to display the interface of the first application in the first split-screen area, and before displaying the interface of the second application in the second split-screen area, detects the startup process of the first application and the second application, and controls the display to display a preset loading interface in full screen; After determining that the startup processes of the first application and the second application have finished running, controlling the display to hide the preset loading interface.

7. The display device according to claim 1, wherein the at least one processor is further configured to execute computer instructions to cause the display device to perform: Displaying a region control control between the first split-screen region and the second split-screen region; In response to a second control input to the region control widget received through the user input interface, the sizes of the first split screen region and the second split screen region are changed.

8. The display device according to claim 1, wherein the at least one processor is further configured to execute computer instructions to cause the display device to perform: In the split-screen display state, receiving a key input from a user through the user input interface, and determining whether a key corresponding to the key input exists in a preset key whitelist, wherein the preset key whitelist is used to store all keys that will not be responded to in the split-screen display state; If the key corresponding to the key input does not exist in the preset key whitelist, determining a current focus application, the focus application being the first application or the second application with a higher priority; The key input is processed by the focus application.

9. The display device according to claim 1, wherein the at least one processor is further configured to execute computer instructions to cause the display device to perform: In the split-screen display state, if a third application startup instruction from the user is received through the user input interface, the third application startup instruction is shielded.

10. The display device according to claim 1, wherein the display is further configured to display a first screen of a first application in a first window; and the at least one processor is further configured to execute computer instructions so that the display device executes: In response to a screen splitting instruction input by a user, controlling the display to add and display a second window, and displaying a first screen of the first application in the first window, and displaying a second screen of the second application in the second window; Determine a focus window that occupies focus between the first window and the second window; The display is controlled to display a focus frame on an upper layer of the focus window, wherein the focus frame is a frame-shaped window, and the position and size of the focus frame match the position and size of the focus window.

11. The display device according to claim 10, wherein in response to the screen splitting instruction input by the user, the display is controlled to add and display a second window, and a first screen of the first application is displayed in the first window, and a second screen of the second application is displayed in the second window, and the at least one processor is further configured to execute computer instructions so that the display device executes: In response to the split-screen instruction, controlling the display to display a split-screen application list, the split-screen application list including an item of the second application; In response to a selection instruction input by the user based on the item of the second application, the display is controlled to additionally display the second window, and the first screen of the first application is displayed in the first window, and the second screen of the second application is displayed in the second window.

12. The display device according to claim 10, when, in response to the screen splitting instruction input by the user, the display is controlled to add and display the second window, and the first screen of the first application is displayed in the first window, and the second screen of the second application is displayed in the second window, the at least one processor is further configured to execute computer instructions so that the display device executes: In response to the split screen instruction, acquiring the second application associated with the first application, in, Displaying the first application and the second application in split screen is used to implement a preset function of the first application; The display is controlled to additionally display the second window, and the first screen of the first application is displayed in the first window, and the second screen of the second application is displayed in the second window.

13. The display device according to claim 10, when controlling the display to display a focus frame on the border of the focus window, the at least one processor is further configured to execute computer instructions to cause the display device to execute: Acquire the position of the focus window, the size of the display area of ​​the display, the preset split screen ratio between the first window and the second window, and the arrangement of the first window and the second window; Determining the size of the focus window according to the size of the display area, the preset split-screen ratio, and the arrangement of the first window and the second window; Determining the position of the focus frame according to the position of the focus window, and determining the size of the focus frame according to the size of the focus window; According to the position of the focus frame and the size of the focus frame, the display is controlled to display the focus frame.

14. The display device according to claim 13, wherein the at least one processor is further configured to execute computer instructions to cause the display device to perform: In response to a zoom instruction input by a user, determining a target distance between a junction of the first window and the second window and a target frame according to a size of the display area, the preset split screen ratio, and a size changed in a zooming direction, wherein the target frame is a first frame in the display area perpendicular to the zooming direction; Determining the changed positions and sizes of the first window and the second window according to the size of the display area and the position of the target from the focus window, and determining the changed position and size of the focus frame according to the changed position and size of the focus window; According to the changed positions and sizes of the first window and the second window, the display is controlled to display the changed first window and the second window, and according to the changed position and size of the focus frame, the display is controlled to display the changed focus frame.

15. The display device according to claim 13, wherein the at least one processor is further configured to execute computer instructions to cause the display device to perform: Controlling the display to display a separator at the boundary between the first window and the second window; In response to a movement instruction input by a user based on the spacer, move the spacer, and obtain a position of the spacer after the spacer is moved in the display area; Determine the changed positions and sizes of the first window and the second window according to the position of the spacer after it moves in the display area, and determine the changed position and size of the focus frame according to the changed position and size of the focus window; According to the changed positions and sizes of the first window and the second window, the display is controlled to display the changed first window and the second window, and according to the changed position and size of the focus frame, the display is controlled to display the changed focus frame.

16. The display device according to claim 10, wherein the at least one processor is further configured to execute computer instructions to cause the display device to perform: In response to a switching instruction of the focus window input by a user, the focus is displayed in the switched focus window, and the first application stack in the application stack list is modified to be the application displayed in the switched focus window, in, The application stack list includes the first application and the second application; Detecting, according to a preset period, whether the first application stack in the application stack list and the focus window where the focus frame is located correspond to the same application; If they correspond to different applications, the display is controlled to display the focus frame on the window where the application corresponding to the first application stack is located.

17. The display device according to claim 10, wherein the at least one processor is further configured to execute computer instructions to cause the display device to perform: Setting the focus frame to be trusted by the system; In response to an operation instruction input by a user to an application in the focus window based on the focus frame, the operation instruction is transmitted to the application in the focus window.

18. The display device according to claim 10, wherein the processor is further configured to: Monitoring a display message of the application in the focus window, wherein the display message is used to indicate a display information prompt box; If the display message is monitored, the display is controlled to display the information prompt box on the upper layer of the focus frame.

19. The display device according to claim 1, further comprising: include: A front camera, configured to capture the environment on the side of the display surface of the display and generate a local video in an activated state; The at least one processor is further configured to execute computer instructions to cause the display device to perform: in response to receiving a menu pop-up instruction, determining whether the current video played by the display is a fitness teaching video, and whether the display plays the current video in full screen mode; In a case where the current video is a fitness teaching video and the display plays the current video in full screen mode, controlling the display to display a first pop-up menu including a split-screen display option; In response to receiving a control instruction that triggers the split-screen display option, controlling the front camera to start and receive the local video generated by the front camera; The display is controlled to split into two display windows, and the current video and the local video are played in the two display windows respectively.

20. The display device according to claim 19, wherein the at least one processor, after controlling the display to play the current video and the local video in two display windows respectively, is further configured to execute computer instructions so that the display device executes: In response to receiving the menu pop-up instruction again, controlling the display to display a second pop-up menu including a mirroring option in the first display window playing the local video; In response to receiving a control instruction for triggering the mirror option, the local video is horizontally mirror-flipped, and the display is controlled to play the local video that has been horizontally mirror-flipped in the first display window.

21. The display device according to claim 20, when controlling the display to split into two display windows, the at least one processor is further configured to execute computer instructions so that the display device executes: controlling the display to split into two display windows in a manner of equally dividing the left and right sides; When controlling the display to display a second pop-up menu including a mirroring option in the first display window playing the local video, the at least one processor is further configured to execute computer instructions to cause the display device to execute: Obtaining the window width of the display, the preset margin width, the option margin width, the number of options in the second pop-up menu, and the option width; Determine the horizontal coordinates of the reference point of the second pop-up menu according to the window width, the edge blank width, the option blank width, the number of options in the second pop-up menu, and the option width, where the reference horizontal coordinates are coordinates in the global coordinate system of the display; The display is controlled to display a second pop-up menu including the mirroring option according to the horizontal coordinate of the reference point, so as to display the pop-up menu in the first display window.

22. The display device according to claim 20, wherein while controlling the player to display a second pop-up menu including a mirroring option in the first display window playing the local video, the second pop-up menu also displays a focus window switching option; The at least one processor is further configured to execute computer instructions to cause the display device to perform: In response to receiving a control instruction that triggers the focus window switching option, the at least one processor controls the display to switch the second display window playing the current video to the focus window, and controls the display to display a third pop-up menu including a full-screen playback option in the second display window.

23. The display device according to claim 22, after controlling the display to display a third pop-up menu including a full-screen play option in the second display window, the at least one processor is further configured to execute computer instructions to cause the display device to execute: In response to receiving a control instruction that triggers the full-screen play option, the display is controlled to resume playing the current video in full-screen mode.

24. The display device according to claim 20, further comprising a speaker; while controlling the player to display a second pop-up menu including a mirroring option in the first display window playing the local video, the second pop-up menu also displays a full-screen display option; The at least one processor is further configured to execute computer instructions to cause the display device to perform: in response to receiving a control instruction that triggers the full-screen playback option, control the display to play the local video in full screen mode, and control the speaker to simultaneously play the audio corresponding to the current video.

25. The display device according to claim 19, after controlling the display to play the current video and the local video in the two display windows respectively, the at least one processor is further configured to execute computer instructions to cause the display device to execute: In response to receiving the menu pop-up instruction again, controlling the display to display a third pop-up menu including a full-screen play option and a focus window switching option in the second display window playing the current video; In response to receiving a control instruction for triggering the full-screen play option, controlling the display to resume playing the current video in full-screen mode; In response to receiving a control instruction that triggers the focus switching option, the display is controlled to switch the first display window playing the local video to a focus window, and the display is controlled to display a second pop-up menu including a mirroring option and an option to exit full-screen playback in the first display window.

26. According to any one of claims 19-21, when the display device controls the two display windows to play the current video and the local video respectively, the at least one processor is further configured to execute computer instructions to enable the display device to execute: According to the sizes of the two display windows, respectively intercepting the corresponding current video and the local video to obtain the intercepted current video and the intercepted local video; The intercepted current video and the intercepted local video are played in the two display windows in full-window playback mode respectively.

27. A split-screen display method, the method include: In response to a first selection input for a first application and a second application, displaying a selection result of the first application and a selection result of the second application, and showing a window arrangement mode selection control and a split-screen control, wherein the first application and the second application support split-screen display; In response to a second selection input of the window arrangement mode selection control, determining an arrangement mode of the first split screen area and the second split screen area; In response to a first control input to the split-screen control, an interface of the first application is displayed in the first split-screen area, and an interface of the second application is displayed in the second split-screen area.

Images