Display device and processing method for display device

Abstract

A display device and a processing method for the display device are applied to the technical field of display. The display device comprises a display, a user interface, communication means, a memory configured to save computer instructions and data associated with the display device; the at least one processor is connected with the display and the memory, and the at least one processor is configured to execute a computer instruction so as to enable the display equipment to execute the following operations: when it is detected that a video playing application program plays a target video in the current display equipment, displaying a user prompt interface, the user prompt interface being used for prompting a user whether to enter a split-screen working mode or not; when an instruction of entering a split screen working mode input by a user on the user prompt interface is received, dividing a display interface of the display equipment into a first split screen and a second split screen, starting a search application program, displaying an interface of a video playing application program on the first split screen, and displaying an interface of the search application program on a full screen of the second split screen; manual split screen setting is not needed, and the user experience is improved.

Description

Display device and processing method for display device

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent applications filed on December 30, 2022, with application number 202211721340.4; filed on December 30, 2022, with application number 202211737411.X; filed on February 3, 2023, with application number 202310093843.X; and filed on April 14, 2023, with application number 202310402904.6, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of display technology, and in particular to a display device and a processing method for the display device. Background Art

[0004] At present, with the rapid development of computer technology, users can use search applications on display devices such as smart TVs to browse media information related to the currently playing video while watching videos on the display devices.

[0005] In the related art, when a user needs to search and browse media information related to the currently playing video according to a search application while watching a video, the user needs to manually set it up, that is, the user needs to enter the setting mode and set the display interface of the display device to split the screen, so that on the display interface of the same display device, through different split screens, the user can watch the video and also search through the search application to obtain media information related to the currently playing video.

[0006] However, with the related technology, since the user needs to manually perform split-screen settings, the operation is cumbersome, thereby reducing the user experience.

[0007] Summary of the Invention

[0008] According to a display device of an embodiment of the present application, the display device includes: a display, configured to display an image and / or a user interface; a user interface, configured to receive instructions from a user; a communication device, configured to communicate with an external device according to a predetermined protocol; a memory, configured to store computer instructions and data associated with the display device; at least one processor, connected to the display, the user interface, the communication device and the memory, and configured to execute the computer instructions to enable the display device to execute: when it is detected that a video playback application plays a target video on the current display device, a user prompt interface is displayed, wherein the user prompt interface is used to prompt the user whether to enter a split-screen working mode; when an instruction to enter a split-screen working mode is received from the user on the user prompt interface, the display interface of the display device is divided into a first split screen and a second split screen, and a search application is started; the interface of the video playback application is displayed on the first split screen, and the interface of the search application is displayed in full screen on the second split screen.

[0009] According to an embodiment of the present application, a processing method for a display device includes: when it is detected that a video playback application is playing a target video on the current display device, displaying a user prompt interface, wherein the user prompt interface is used to prompt the user whether to enter a split-screen working mode; when an instruction to enter a split-screen working mode is received from the user input on the user prompt interface, dividing the display interface of the display device into a first split screen and a second split screen, and starting a search application; displaying the interface of the video playback application on the first split screen, and displaying the interface of the search application in full screen on the second split screen.

[0010] According to an embodiment of the present application, a computer-readable non-volatile storage medium stores a computer program, which, when executed by a processor, enables a computer device to perform the steps of the processing method for a display device. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] FIG1A is a schematic diagram of an application scenario among a display device, a control device, and a terminal device according to some embodiments of the present application;

[0012] FIG1B is a schematic diagram of a scenario of split-screen playback during video playback in the related art;

[0013] FIG2 is a block diagram of a hardware configuration of a control device 100 according to some embodiments of the present application;

[0014] FIG3A is a hardware configuration block diagram of a display device 200 according to some embodiments of the present application;

[0015] FIG3B is a schematic diagram of software configuration in the display device 200 according to some embodiments of the present application;

[0016] FIG3C is a schematic diagram showing an icon control interface of an application in a display device 200 according to some embodiments of the present application;

[0017] FIG4A is a framework diagram of a split-screen control system according to some embodiments of the present application;

[0018] FIG4B is a diagram of a split-screen control architecture according to some embodiments of the present application;

[0019] FIG5A is a flowchart diagram 1 of a processing method for a display device according to some embodiments of the present application;

[0020] FIG5B is a schematic diagram of a scenario for displaying a user prompt interface according to some embodiments of the present application;

[0021] FIG5C is a schematic diagram of a scenario in which an application is displayed on a first split screen and a second split screen according to some embodiments of the present application;

[0022] FIG6A is a second flow chart of a processing method for a display device according to some embodiments of the present application;

[0023] FIG6B is a schematic diagram of a scene at a preset display interface size according to some embodiments of the present application;

[0024] FIG7A is a third flow chart of a processing method for a display device according to some embodiments of the present application;

[0025] FIG7B is a schematic diagram of a scenario for quickly searching for media asset information of a target video according to some embodiments of the present application;

[0026] FIG8 is a flowchart 1 of a second processing method for a display device according to some embodiments of the present application;

[0027] FIG9 is a schematic diagram of a user interface of a display according to some embodiments of the present application;

[0028] FIG10 is a schematic diagram of a user interface displayed on a display in another case according to some embodiments of the present application;

[0029] FIG11 is a schematic diagram of a user interface displayed on a display in still further cases according to some embodiments of the present application;

[0030] FIG12 is a schematic diagram of a user interface displayed on a display in still further cases according to some embodiments of the present application;

[0031] FIG13 is a second flowchart of a second processing method for a display device according to some embodiments of the present application;

[0032] FIG14 is a flowchart diagram 1 of a third processing method for a display device according to some embodiments;

[0033] FIG15 is a first schematic diagram of an interface of a display device according to some embodiments;

[0034] FIG16 is a second schematic diagram of an interface of a display device according to some embodiments;

[0035] FIG17 is a third schematic diagram of an interface of a display device according to some embodiments;

[0036] FIG18 is a fourth schematic diagram of an interface of a display device according to some embodiments;

[0037] FIG19 is a second flow chart of a third processing method for a display device according to some embodiments;

[0038] FIG20 is a schematic diagram of a preset loading interface according to some embodiments;

[0039] FIG21 is a fifth schematic diagram of an interface of a display device according to some embodiments;

[0040] FIG22 is a flowchart of a third processing method for a display device according to some embodiments;

[0041] FIG23 is a fourth flow chart of a third processing method for a display device according to some embodiments;

[0042] FIG24 is a flowchart of a fourth processing method for a display device according to some embodiments of the present application;

[0043] FIG25 is a flowchart illustrating a display device displaying a first application and a second application in split-screen mode according to some embodiments of the present application;

[0044] FIG26 is a flowchart illustrating a display device displaying a first application and a second application in split-screen mode according to some embodiments of the present application;

[0045] FIG27 is a schematic diagram of the Android boot process according to some embodiments of the present application;

[0046] FIG28 is a schematic diagram of a split-screen operation process provided by SystemUI according to some embodiments of the present application;

[0047] FIG29 is a flow chart of a display device displaying a focus frame according to some embodiments of the present application;

[0048] FIG30 is a schematic diagram of a system architecture of a display device according to some embodiments of the present application;

[0049] FIG31 is a schematic diagram of a process of displaying a first application and a second application on a television in split-screen mode according to Example 1 of some embodiments of the present application;

[0050] FIG32 is a schematic diagram of a process of displaying a first application and a second application on a television in split-screen mode according to Example 2 of some embodiments of the present application;

[0051] FIG33 is a schematic diagram of a process of displaying a first application and a second application on three televisions in split-screen mode according to some embodiments of the present application;

[0052] FIG34 is a schematic diagram illustrating a process of displaying a first application and a second application on four televisions in split-screen mode according to some embodiments of the present application;

[0053] FIG35 is a flow chart showing a display device responding to a mouse operation according to some embodiments of the present application;

[0054] FIG36 is a flow chart illustrating a display device zooming a focus window using a zoom button according to some embodiments of the present application;

[0055] FIG37 is a flow chart illustrating a display device zooming a focus window using a zoom button according to some embodiments of the present application;

[0056] FIG38 is a flow chart of a display device scaling a focus window using a spacer according to some embodiments of the present application;

[0057] FIG39 is a flow chart illustrating a display device scaling a focus window using a spacer according to some embodiments of the present application;

[0058] FIG40 is a flowchart of a display device performing timing detection on an application stack according to some embodiments of the present application;

[0059] FIG41 is a flowchart of a display device switching a focus window according to some embodiments of the present application;

[0060] FIG42 is a flowchart of a display device switching a focus window according to some embodiments of the present application;

[0061] FIG43 is a flowchart of a display device displaying a message prompt box according to some embodiments of the present application;

[0062] Figure 44 is a schematic diagram of a display device displaying a message prompt box according to some embodiments of the present application. DETAILED DESCRIPTION

[0063] In order to more clearly understand the above-mentioned objectives, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0064] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application can also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present application, not all of the embodiments.

[0065] The terms "first" and "second" in this application are used to distinguish different objects, rather than to describe a specific order of objects. For example, the terms "first processing result" and "second processing result" are used to distinguish different processing results, rather than to describe a specific order of processing results.

[0066] The display device provided in the embodiments of the present application can have various implementation forms, for example, it can be a television, a smart TV, a laser projection device, a monitor, an electronic bulletin board, an electronic table, etc. The embodiments of the present application do not limit the specific type of the display device.

[0067] When a display device detects that the display screen has been stationary for a long time, the system will start the display screen saver to protect the display screen to prevent the display screen from being stationary for a long time, causing screen burn-in and reducing the user's subsequent usage experience.

[0068] Referring to Figures 1A and 1B, Figure 1A is a schematic diagram of an application scenario between a display device, a control device, and a terminal device according to some embodiments of the present application, and Figure 1B is a schematic diagram of a related art scenario of split-screen operation during video playback. When a user needs to further obtain media resource information for the currently playing video while watching a video, the user enters setup mode and configures the display interface of the display device 200 to split the screen. The configured first split screen 201 displays the playing video, while the second split screen 202 displays the application search interface. The user can operate the display device 200 through the control device 100 or the terminal device 300.

[0069] However, with the related technology, when the user needs to further obtain the media resource information of the currently playing video while watching the video, the user needs to manually set the split-screen mode, which is cumbersome and reduces the user experience.

[0070] In order to solve the above problems, an embodiment of the present application proposes a processing method for a display device, which is specifically used for split-screen control. The method includes: displaying a user prompt interface when it is detected that a video playback application is playing a target video on the current display device, wherein the user prompt interface is used to prompt the user whether to enter the split-screen working mode; when an instruction to enter the split-screen working mode is received from the user input on the user prompt interface, the display interface of the display device is divided into a first split screen and a second split screen, and a search application is started, the interface of the video playback application is displayed on the first split screen, and the interface of the search application is displayed in full screen on the second split screen.

[0071] In the above method, when it is detected that the video playback application is playing a video, a user prompt interface can be actively displayed to the user, prompting the user to enter the split-screen working mode. When the user determines that the media information or other information of the target video needs to be obtained, after choosing to enter the split-screen working mode, the display interface will be split-screened, and the search application will be started at the same time. The interface of the video playback application is displayed in the first split screen, and the interface of the search application is displayed in the second split screen. In this way, the user can quickly enter the split-screen working mode, which satisfies the user's need to obtain information through the search application while watching the video, without the user having to manually set the display interface to split screen, thereby improving the user experience.

[0072] In some embodiments, the control device 100 may be a remote control. Communication between the remote control and the display device 200 may include infrared protocol communication, Bluetooth protocol communication, wireless or other wired methods, etc., and the display device 200 can be controlled through the above communication methods. The user can control the display device 200 by inputting user commands through buttons on the remote control, voice input, and control panel input. For example, the user can input corresponding control commands through the volume up and down keys, menu key, power button, etc. on the remote control to realize the function of controlling the display device 200.

[0073] In some embodiments, a tablet computer, a computer, a laptop computer, and other terminal devices may also be used to control the display device 200. For example, the display device 200 may be controlled using an application running on the terminal device.

[0074] In some embodiments, the display device 200 may not use the above-mentioned terminal device or control device to receive instructions, but may receive user control through touch or gestures.

[0075] In some embodiments, the display device 200 can also be controlled in ways other than the control device and the terminal device. For example, the user's voice command control can be directly received through a module for obtaining voice commands configured inside the display device 200, or the user's voice command control can be received through a voice control device set outside the display device 200.

[0076] In some embodiments, the terminal device can install software applications with the display device 200, and achieve connection communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The display device 200 can also communicate data with the server through a variety of communication methods, allowing the display device 200 to communicate and connect through a local area network (LAN), a wireless local area network (WLAN) and other networks. The server can be a cluster or multiple clusters, and can include one or more types of servers. The server can provide various content and interactions to the display device 200. The display device 200 can be a liquid crystal display, an OLED display or a projection display device, etc. In addition to providing broadcast reception television functions, the display device 200 can also provide additional intelligent network TV functions that provide computer support functions.

[0077] Figure 2 is a block diagram of the hardware configuration of the control device 100 according to some embodiments of the present application. As shown in Figure 2, the control device includes at least one processor 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device can receive user input operation instructions and convert the operation instructions into instructions that can be recognized and responded to by the display device 200, acting as an intermediary for interaction between the user and the display device 200. The communication interface 130 is used for external communication and includes at least one of a WIFI chip, a Bluetooth module, NFC, or an alternative module. The user input / output interface 140 includes at least one of a microphone, a touchpad, a sensor, a button, or an alternative module.

[0078] FIG3A is a block diagram of a hardware configuration of a display device 200 according to some embodiments of the present application. As shown in FIG3A , the display device 200 includes at least one of a tuner / demodulator 210, a communicator 220, a detector 230, an external device interface 240, at least one processor 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface (i.e., a user input interface) 280.

[0079] The at least one processor 250 includes at least one of a central processing unit, a video processor, an audio processor, a graphics processor, a RAM, a ROM, a first interface to an nth interface for input / output, a communication bus, and the like.

[0080] Display 260 includes a display screen component for displaying images, a driver component for driving the image display, and components for receiving image signals output by at least one processor to display video content, image content, a menu control interface, and a user interface. Display 260 can be at least one of a liquid crystal display, an organic light emitting diode (OLED) display, a touch display, and a projection display. It can also be a projection device and projection screen.

[0081] The tuner-demodulator 210 receives broadcast television signals via a wired or wireless reception method, and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

[0082] Communicator 220 is a component used to communicate with external devices or servers using various communication protocols. For example, the communicator may include at least one of a Wi-Fi module, a Bluetooth module, a wired Ethernet module, or other network communication protocol chip or a near-field communication protocol chip, as well as an infrared receiver. Display device 200 can use communicator 220 to send and receive control signals and data signals with external control device 100 or server 400.

[0083] Detector 230 is used to collect signals from the external environment or external interactions. For example, detector 230 includes a light receiver, a sensor for collecting ambient light intensity; or detector 230 includes an image collector, such as a camera, for collecting external environmental scenes, user attributes, or user interaction gestures; or detector 230 includes a sound collector, such as a microphone, for receiving external sounds.

[0084] The external device interface 240 may include, but is not limited to, any one or more of the following: a high-definition multimedia interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input / output interface formed by multiple of the above interfaces.

[0085] The user interface 280 may be configured to receive a control signal from the control device 100 (eg, an infrared remote controller, etc.).

[0086] The at least one processor 250 and the tuner / demodulator 210 may be located in different separate devices. That is, the tuner / demodulator 210 may also be located in an external device to the main device where the at least one processor 250 is located, such as an external set-top box. The user interface 280 may be used to receive control signals from a control device (e.g., an infrared remote control).

[0087] In some embodiments, at least one processor 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The at least one processor 250 controls the overall operation of the display device 200. A user can enter user commands through a graphical user interface (GUI) displayed on the display 260, and the user interface receives the user input commands through the graphical user interface (GUI). Alternatively, the user can enter user commands by inputting specific sounds or gestures, and the user interface recognizes the sounds or gestures through sensors to receive the user input commands.

[0088] Figure 3B is a schematic diagram of the software configuration in the display device 200 according to some embodiments of the present application. As shown in Figure 3B, the system is divided into four layers, from top to bottom, namely the application layer (referred to as "application layer"), the application framework layer (referred to as "framework layer"), the Android runtime (Android runtime) and system library layer (referred to as "system runtime library layer"), and the kernel layer.

[0089] In some embodiments, at least one application program runs in the application layer. These applications can be window programs, system settings programs, clock programs, etc. that come with the operating system, or applications developed by third-party developers. In specific implementations, the applications in the application layer include but are not limited to the above examples.

[0090] The framework layer provides applications with an application programming interface (API) and programming framework. The application framework layer includes predefined functions. The application framework layer acts as a processing center, determining the actions taken by applications in the application layer. Through the API, applications can access system resources and services during execution.

[0091] As shown in Figure 3B, the application framework layer in the embodiment of the present application includes managers, content providers, etc., wherein the manager includes at least one of the following modules: an activity manager (Activity Manager) is used to interact with all activities running in the system; a location manager (Location Manager) is used to provide system services or applications with access to system location services; a package manager (Package Manager) is used to retrieve various information related to the application package currently installed on the device; a notification manager (Notification Manager) is used to control the display and clearing of notification messages; a window manager (Window Manager) is used to manage icons, windows, toolbars, wallpapers and desktop components on the user interface.

[0092] In some embodiments, the activity manager is used to manage the lifecycle of each application and common navigation back functions, such as controlling the exit, opening, and backing of an application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, taking screenshots, and controlling display window changes (such as shrinking the display window, shaking the display, distorting the display, etc.).

[0093] In some embodiments, the system runtime layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system will run the C / C++ library contained in the system runtime layer to implement the functions to be implemented by the framework layer.

[0094] In some embodiments, the kernel layer is a layer between hardware and software, and includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0095] Figure 3C is a schematic diagram showing the icon control interface of an application in the display device 200 according to some embodiments of the present application. As shown in Figure 3C, the application layer includes at least one application that can display a corresponding icon control in the display, such as: a live TV application icon control, a video on demand application icon control, a media center application icon control, an application center icon control, a game application icon control, etc. The live TV application can provide live TV through different signal sources. The video on demand application can provide videos from different storage sources. Unlike the live TV application, the video on demand provides video display from certain storage sources. The media center application can provide applications for playing various multimedia content. The application center can provide storage for various applications.

[0096] In some embodiments, the display device is a terminal device with a display function, such as a television, a mobile phone, a computer, a learning machine, etc. In the display device:

[0097] a display 260 configured to display images and / or a user interface;

[0098] The user interface 280 is configured to receive instructions from the user;

[0099] a communication device (communicator 220 ) configured to communicate with an external device according to a predetermined protocol;

[0100] a memory configured to store computer instructions and data associated with the display device;

[0101] At least one processor 250, connected to the display 260, the user interface 280, the communication device and the memory, is configured to execute the computer instructions to cause the display device to perform:

[0102] When it is detected that a video playback application is playing a target video on the current display device, a user prompt interface is displayed, wherein the user prompt interface is used to prompt the user whether to enter the split-screen working mode;

[0103] When receiving an instruction from the user on the user prompt interface to enter a split-screen working mode, dividing the display interface of the display device into a first split screen and a second split screen, and starting a search application;

[0104] The interface of the video playback application is displayed in the first split screen, and the interface of the search application is displayed in full screen in the second split screen.

[0105] In some embodiments, at least one processor 250 is specifically configured to execute the computer instructions to cause the display device to perform:

[0106] Obtaining a preset display interface size of the video playback application and a first display interface size of the first split screen, wherein the preset display interface is an interface for displaying the target video;

[0107] Determining a target display interface size for displaying the video playback application in the first split screen based on the preset display interface size and the first display interface size;

[0108] Based on the target display interface size, the interface of the video playback application is displayed on the first split screen.

[0109] In some embodiments, the at least one processor 250 is further configured to execute the computer instructions to cause the display device to perform:

[0110] Determining a target ratio according to the preset display interface size;

[0111] A target display interface size for displaying the video playback application in the first split screen is determined according to the first display interface size and the target ratio.

[0112] In some embodiments, at least one processor 250 is specifically configured to execute the computer instructions to cause the display device to perform:

[0113] Determining a size of a second display interface of the second split screen;

[0114] Based on the size of the second display interface, the interface of the search application is displayed in full screen within the second split screen.

[0115] In some embodiments, the at least one processor 250 is further configured to execute the computer instructions to cause the display device to perform:

[0116] In response to a search instruction input by a user, displaying a keyword menu;

[0117] When a keyword selection instruction input by the user in the keyword menu is received, the search application is called to obtain the media asset information of the target video.

[0118] In some embodiments, at least one processor 250 is specifically configured to execute the computer instructions to cause the display device to perform:

[0119] In response to a search instruction input by a user, acquiring attribute information of the target video;

[0120] The keyword menu is generated based on the attribute information.

[0121] In some embodiments, the at least one processor 250 is further configured to execute the computer instructions to cause the display device to perform:

[0122] When receiving an instruction input by the user to start a preset application, the split-screen working mode is exited and the preset application is displayed on the display interface.

[0123] In summary, the present application performs the above-mentioned processing method for display devices on a display device, and displays a user prompt interface when it is detected that a video playback application is playing a target video on the current display device, wherein the user prompt interface is used to prompt the user whether to enter the split-screen working mode; when the user receives the instruction to enter the split-screen working mode input by the user on the user prompt interface, the display interface of the display device is divided into a first split screen and a second split screen, and the search application is started, the interface of the video playback application is displayed in the first split screen, and the interface of the search application is displayed in full screen on the second split screen. In the above method, when it is detected that the video playback application is playing a video, the user prompt interface can be actively displayed to the user, prompting the user to enter the split-screen working mode, and when the user determines that the media information or other information of the target video needs to be obtained, after selecting to enter the split-screen working mode, the display interface is split-screen processed, and the search application is started at the same time, the interface of the video playback application is displayed in the first split screen, and the interface of the search application is displayed in the second split screen. In this way, the user can quickly enter the split-screen working mode, satisfying the user's ability to watch the video while also obtaining information through the search application, without the user having to manually set the display interface to split screen, thereby improving the user experience.

[0124] FIG4A is a framework diagram of a split-screen control system according to some embodiments of the present application. As shown in FIG4A , the system may include a user prompt interface display module 401 and a split-screen display module 402. When the system detects that a video playback application is playing a target video on the current display device, the system displays a user prompt interface through the user prompt interface display module 401, wherein the user prompt interface is used to prompt the user whether to enter the split-screen working mode. Then, when the split-screen display module 402 receives an instruction to enter the split-screen working mode input by the user on the user prompt interface, the display interface of the display device is divided into a first split screen and a second split screen, and a search application is started, the interface of the video playback application is displayed on the first split screen, and the interface of the search application is displayed in full screen on the second split screen. In the above method, when it is detected that the video playback application is playing a video, a user prompt interface can be actively displayed to the user, prompting the user to enter the split-screen working mode. When the user determines that the media information or other information of the target video needs to be obtained, after choosing to enter the split-screen working mode, the display interface will be split-screened, and the search application will be started at the same time. The interface of the video playback application is displayed in the first split screen, and the interface of the search application is displayed in the second split screen. In this way, the user can quickly enter the split-screen working mode, which satisfies the user's need to obtain information through the search application while watching the video. There is no need for the user to manually set the display interface to split screen, thereby improving the user experience.

[0125] Figure 4B is a diagram of the split-screen control architecture according to some embodiments of the present application. Based on the above system framework, the present application is implemented as shown in Figure 4B. The Android system mainly includes the application layer, framework layer, system runtime layer and kernel layer. The implementation logic is mainly reflected in the application layer, which includes the user prompt interface display module and the split-screen display module. The functions of each module have been described in detail in the above embodiments. In order to avoid repetition, they will not be repeated here.

[0126] In the processing method for a display device provided in an embodiment of the present application, at least one processor of the display device displays a user prompt interface when detecting that a video playback application is playing a target video on the current display device, wherein the user prompt interface is used to prompt the user whether to enter a split-screen working mode; when receiving an instruction to enter the split-screen working mode input by the user on the user prompt interface, the display interface of the display device is divided into a first split-screen and a second split-screen, and a search application is started, the interface of the video playback application is displayed in the first split-screen, and the interface of the search application is displayed in full screen on the second split-screen. In the above method, when it is detected that the video playback application is playing a video, the user prompt interface can be actively displayed to the user, prompting the user to enter the split-screen working mode, and when the user determines that the media information or other information of the target video needs to be obtained, after selecting to enter the split-screen working mode, the display interface is split-screen processed, and the search application is started at the same time, the interface of the video playback application is displayed in the first split-screen, and the interface of the search application is displayed in the second split-screen. In this way, the user can quickly enter the split-screen working mode, satisfying the user's ability to watch the video while also obtaining information through the search application, without the user having to manually set the display interface to split screen, thereby improving the user experience.

[0127] In order to illustrate the implementation method in more detail, the following is explained in conjunction with Figure 5A. It can be understood that the steps involved in Figure 5A may include more steps or fewer steps in actual implementation, and the order of these steps may also be different, so as to be able to implement the above-mentioned processing method for the display device provided in the embodiment of the present application. The embodiment of the present application does not limit it.

[0128] FIG5A is a flow chart of a processing method for a display device according to some embodiments of the present application. As shown in FIG5A , the method specifically includes the following steps:

[0129] S510: When it is detected that a video playback application is playing a target video on the current display device, a user prompt interface is displayed.

[0130] Among them, the user prompt interface is used to prompt the user whether to enter the split-screen working mode. The target video can be a video saved locally or a video downloaded from the Internet, but is not limited thereto and is not specifically limited in this application.

[0131] Specifically, when at least one processor of the display device detects that a video playback application is playing the target video on the current display device, it displays a user prompt interface on the display interface of the display device 200 to prompt the user whether to enter the split-screen working mode.

[0132] As shown in Figure 5B, when at least one processor of the display device 200 detects that there is a video playback application on the current display device 200 playing the target video, a user prompt interface 203 is displayed on the display interface of the display device 200, and a prompt of "Whether to enter the split-screen working mode" is displayed on the user prompt interface 203, so that the user can determine whether to quickly make the display interface of the display device enter the split-screen working mode according to his or her own wishes.

[0133] It should be noted that the above-mentioned user prompt interface can be a prompt interface that pops up in the video playback application, or a menu, or a floating window that floats up on the video playback application, but is not limited to this. This application does not specifically limit it, and technical personnel in this field can set it according to actual conditions.

[0134] Optionally, the above-mentioned detection of whether there is a video playback application playing the target video on the current display device can be achieved by displaying the playback window corresponding to the video playback application on the display interface, then it is considered that the current video playback application is playing the target video, but it is not limited to this. This application does not specifically limit it, and technical personnel in this field can set it according to actual conditions.

[0135] S520: When receiving an instruction for entering a split-screen working mode input by the user on the user prompt interface, dividing the display interface of the display device into a first split screen and a second split screen, and starting a search application.

[0136] Specifically, when the current user determines that he needs to obtain the media information of the target video played by the video playback application, or needs to know some other information, the user enters an instruction to enter the split-screen working mode on the user prompt interface. When at least one processor of the display device receives the instruction to enter the split-screen working mode input by the user, it divides the display interface of the display device into a first split screen and a second split screen, and starts the search application at the same time.

[0137] It should be noted that at this time, the display interface of the display device is split into a split screen to obtain a first split screen and a second split screen. In order to enable the first split screen and the second split screen to have the largest display interface, there is no gap between the first split screen and the second split screen, and the first split screen and the second split screen are both half of the display interface of the display device, but not limited to this. This application does not specifically limit the layout of the first split screen and the second split screen on the display interface, and technical personnel in this field can set it according to actual conditions.

[0138] S530: Display the interface of the video playback application on the first split screen, and display the interface of the search application in full screen on the second split screen.

[0139] Specifically, after at least one processor of the display device divides the display interface of the display device into a first split screen and a second split screen, the interface of the video playback application is displayed in the first split screen, that is, the target video currently being played by the video playback application is displayed in the first split screen, and the interface of the search application is displayed in full screen in the second split screen.

[0140] 5B-5C , on the user prompt interface, the user can determine whether to make the display interface of the display device 200 enter the split-screen working mode according to his or her own wishes. When the user clicks the virtual button "Agree", the display interface of the display device 200 is divided into a first split screen 204 and a second split screen 205. At the same time, the search application is started, and the interface of the video playback application is displayed on the first split screen 204, that is, the target video currently being played by the video playback application is displayed on the first split screen 204, and the interface of the search application is displayed in full screen on the second split screen 205.

[0141] In an embodiment of the present application, at least one processor of a display device, when detecting that a video playback application is playing a target video on the current display device, displays a user prompt interface, wherein the user prompt interface is used to prompt the user whether to enter a split-screen working mode; when receiving an instruction to enter the split-screen working mode input by the user on the user prompt interface, the display interface of the display device is divided into a first split-screen and a second split-screen, and a search application is started, the interface of the video playback application is displayed in the first split-screen, and the interface of the search application is displayed in full screen on the second split-screen. In the above method, when it is detected that the video playback application is playing a video, the user prompt interface can be actively displayed to the user, prompting the user to enter the split-screen working mode, and when the user determines that the media information or other information of the target video needs to be obtained, after selecting to enter the split-screen working mode, the display interface is split-screen processed, and the search application is started at the same time, the interface of the video playback application is displayed in the first split-screen, and the interface of the search application is displayed in the second split-screen. In this way, the user can quickly enter the split-screen working mode, satisfying the user's ability to obtain information through the search application while watching the video, without the user having to manually set the display interface to split screen, thereby improving the user experience.

[0142] FIG6A is a second flow chart of a processing method for a display device according to some embodiments of the present application. FIG6A is based on the embodiment shown in FIG5A , and further, as shown in FIG6A , is a possible implementation manner of displaying the interface of the video playback application on the first split screen:

[0143] S610: Obtain a preset display interface size of the video playback application and a first display interface size of the first split screen.

[0144] The preset display interface is the interface for displaying the target video. It should be noted that the preset interface here is the interface for playing the corresponding screen of the target video through the video playback application before the display device is split. Referring to Figure 6B, for the entire window interface 601 corresponding to the video playback application, the interface 602 of the target video played in the window interface 601 is the preset display interface, but is not limited to this and is not specifically limited in this application.

[0145] S620: Determine a target display interface size for displaying the video playback application on the first split screen based on the preset display interface size and the first display interface size.

[0146] Specifically, at least one processor of the display device obtains the preset display interface size of the video playback application and the first interface size of the first split screen. After obtaining the preset display interface size of the video playback application and the first interface size of the first split screen, it determines the target display interface size for displaying the video playback application in the first split screen based on the preset display interface size of the video playback application and the first interface size of the first split screen.

[0147] Optionally, based on the above embodiment, in some embodiments of the present application, an implementation of S620 may be:

[0148] S6201: Determine a target ratio based on a preset display interface size.

[0149] Among them, the preset display interface size can be the length and height of the interface displaying the target video. Referring to Figure 6B, for the preset display interface 602, its corresponding size is length l and height h. According to the preset display interface size, i.e., length and height, the corresponding target ratio is determined, such as l:h. The target ratio can be, for example, 16:9, or 4:3, but is not limited to this. This application does not specifically limit it, and technical personnel in this field can set it according to actual conditions.

[0150] Optionally, the size of the first display interface of the first split screen can be determined based on the pixel points of the first split screen, and the preset display interface size can be determined based on the pixel points of the picture corresponding to the target video currently being played, but is not limited to this. This application does not specifically limit it, and technical personnel in this field can set it according to actual conditions.

[0151] S6202: Determine a target display interface size for displaying the video playback application on the first split screen according to the first display interface size and the target ratio.

[0152] The target display interface size refers to the interface size that can maximize the display of the target video image on the first split screen after the split screen is performed.

[0153] Specifically, after obtaining the preset display interface size, at least one processor of the display device can calculate the target ratio of the interface for displaying the target video based on the preset display interface size. After determining the target ratio of the interface for displaying the target video, it can determine the target display interface size for displaying the video playback application in the first split screen based on the first display interface size and the target ratio.

[0154] S630: Displaying an interface of the video playback application program on the first split screen based on the target display interface size.

[0155] Specifically, after obtaining the target display interface size according to the first display interface size and the target ratio, at least one processor of the display device uses the target display interface size to display the interface of the video playback application on the first split screen.

[0156] In an embodiment of the present application, in the above process, the display interface size of the first split screen and the target ratio determined according to the preset display interface size of the video playback application are used to determine the display interface size of the interface of the video playback application displayed on the first split screen, so as to ensure that the target video played by the current video playback application can be displayed to the user to the maximum extent on the first split screen, thereby improving the user experience.

[0157] Optionally, based on the above embodiment, in some embodiments of the present application, with continued reference to FIG6A , one implementation of displaying the interface of the search application in full screen on the second split screen may be:

[0158] S640: Determine the size of the second display interface of the second split screen.

[0159] S650: Based on the size of the second display interface, display the search application interface in full screen in the second split screen.

[0160] Specifically, at least one processor of the display device obtains the second interface size of the second split screen, and after obtaining the second interface size of the second split screen, displays the interface corresponding to the search application in full screen on the second split screen.

[0161] Continuing with the above embodiment, referring to FIG5C , for the second split screen 205, after obtaining the second interface size of the second split screen 205, the interface corresponding to the search application can be displayed in full screen on the second split screen 205, but is not limited to this. This application does not impose specific restrictions, and those skilled in the art can set it according to actual conditions.

[0162] In an embodiment of the present application, in the above process, the interface size of the second split screen is used to display the interface corresponding to the search application in full screen on the second split screen, so that the interface corresponding to the search application can be displayed to the user in the maximum extent on the second split screen, thereby improving the user experience.

[0163] FIG7A is a third flow chart of a processing method for a display device according to some embodiments of the present application. FIG7A is based on the embodiment shown in FIG6A . As shown in FIG7A , further, in order to enable a user to quickly obtain media asset information of a target video currently being played by a video playback application on a display device, one implementation method may be:

[0164] S710 , in response to a search instruction input by a user, displaying a keyword menu.

[0165] Among them, the search instruction is used to indicate the instruction input when the user currently wants to obtain media information related to the target video. The keyword menu includes attribute information corresponding to the target video, such as the name of the target video, the ranking of the target video, the type of the target video, the director of the target video, and the starring actor of the target video, but is not limited to this. This application does not specifically limit it, and technical personnel in this field can set it according to actual conditions.

[0166] Specifically, at least one processor of the display device receives a search instruction input by a user, and in response to the user's search instruction, displays a keyword menu related to the target video to the user.

[0167] As shown in reference Figure 7B, the search command input by the user can be clicking a virtual button 702 such as "Search" on the submenu 701 on the first split-screen interface. When the user clicks the virtual button 702 "Search", a keyword menu 703 is displayed on the interface corresponding to the video playback application, but is not limited to this. This application does not specifically limit it, and technical personnel in this field can set it according to actual conditions.

[0168] Optionally, based on the above embodiment, in some embodiments of the present application, a method for obtaining the keyword menu may be:

[0169] In response to a search instruction input by a user, attribute information of a target video is acquired.

[0170] Among them, the attribute information of the target video includes but is not limited to the name of the target video, the ranking of the target video such as the top 10, the type of the target video such as comedy, horror, adventure, the score of the target video such as 8.6 points, the director of the target video and the starring actor of the target video, etc. This application does not specifically limit it, and technical personnel in this field can set it according to actual conditions.

[0171] Optionally, the attribute information of the target video may be obtained through an electronic program guide, but is not limited thereto. This application does not impose any specific restrictions, and those skilled in the art may configure the information based on actual conditions.

[0172] Based on the attribute information, a keyword menu is generated.

[0173] Specifically, at least one processor of the display device receives a search instruction input by a user and obtains attribute information of the target video currently being played in response to the user's search instruction. After obtaining the attribute information of the target video currently being played, it can extract multiple keywords based on the attribute information, thereby generating a keyword menu, and displaying the keyword menu related to the target video to the user.

[0174] S720: When a keyword selection instruction input by the user in the keyword menu is received, a search application is called to obtain media resource information of the target video.

[0175] Specifically, at least one processor of the display device displays a keyword menu related to the target video, and the user selects from multiple keywords according to his or her own wishes to determine one or more keywords. After determining one or more keywords, the search application is called to search for the one or more keywords to obtain media information of the target video.

[0176] Continuing with the above embodiment, referring to FIG. 7B , after selecting the keyword “name: xx” in the keyword menu 703 , a search application is called to search for the keyword “name: xx” to obtain the media asset information of the target video.

[0177] In an embodiment of the present application, in the above process, when the user wants to know more about the media information of the target video currently being played, the user can directly obtain a keyword menu related to the target video by entering a search command, so that the user can search for the one or more keywords based on the keyword menu by selecting one or more keywords on the keyword menu. In this way, the user can quickly obtain the media information of the target video currently being played without the user manually entering a search cluster, thereby improving the user experience.

[0178] Optionally, based on the above embodiments, some embodiments of the present application further include:

[0179] When receiving the instruction input by the user to start the preset program, the split-screen working mode is exited and the preset program is displayed on the display interface.

[0180] Among them, the preset program refers to other programs that are not running, such as an audio playback application, and the current display device can also pre-set shortcut keys, but is not limited to this. This application does not specifically limit it, and technical personnel in this field can set it according to actual conditions.

[0181] Specifically, when at least one processor of the display device receives an instruction input by the user to start a preset program, it responds to the instruction, exits the split-screen working mode, and displays the preset program to the user on the display interface of the display device.

[0182] Optionally, based on the above embodiments, in some embodiments of the present application, one way to exit the split-screen working mode may also be to pre-set an exit shortcut key so that the user can quickly exit the split-screen working mode based on the exit shortcut key and return to the interface of the focus application.

[0183] It should be noted that the focus application refers to the application that the current user can control. The focus application can be a video playback application or a search application. The determination of the focus application can be implemented based on the selection instruction input by the user. For example, the user can click the virtual button "R" on the preset menu to determine that the focus application is the search application, or click the virtual button "L" to determine that the focus application is the video playback application, but is not limited to this. This application does not specifically limit it, and technical personnel in this field can set it according to actual conditions.

[0184] In an embodiment of the present application, in the above process, when the display device is working in the split-screen working mode, when it receives an instruction from the user to start other preset programs, or an instruction to exit the split-screen working mode, it can enable the user to quickly exit the split-screen working mode, thereby improving the user experience.

[0185] Currently, when users use display devices for fitness training, they need to first activate the split-screen mode of the display device, and then activate the fitness teaching app and open the fitness teaching video in the split-screen mode. It is impossible to immediately activate the split-screen mode after the fitness teaching app is activated or during the playback of the fitness teaching video, resulting in poor user experience.

[0186] In order to improve the user experience, the embodiment of the present application provides a second processing method for a display device, which is specifically used for display control, so that when the display device plays a fitness teaching video, the split-screen mode can be turned on immediately, thereby improving the user experience.

[0187] FIG8 is a flowchart of a second processing method for a display device according to some embodiments of the present application. The method shown in FIG8 is executed by at least one processor in the display device. As shown in the figure, the method executed by at least one processor includes S810-S850.

[0188] S810: In response to receiving a menu pop-up instruction, determine whether the current video is a fitness teaching video and whether the display plays the current video in full screen mode; if both are true, execute S820.

[0189] In an embodiment of the present application, after a user turns on a display device and issues a video play instruction to the display device, the display device starts playing the current video. The corresponding video file is used to play the current video on the display. In other embodiments, at least one processor can use a client application such as YouTube to obtain and parse the current video provided by a remote server to play the current video on the display. In other embodiments, at least one processor can also receive a video stream sent by another smart terminal, interpret the video stream, and play the current video on the display.

[0190] When the display device is playing the current video, the user can send a menu pop-up instruction to at least one processor of the display device through the remote control, terminal device or gesture control mentioned above.

[0191] When at least one processor receives a menu pop-up instruction triggered by the user, it does not determine the type of options included in the pop-up menu according to the pre-configured template in the conventional manner, load the corresponding options and control the display to display the pop-up menu, but first determines whether the current video is a fitness teaching video and whether the display is playing the current video in full-screen mode, and determines the type of options included in the pop-up menu based on the judgment result, and displays the pop-up menu.

[0192] In the specific implementation of the embodiment of the present application, at least one processor can use different methods to determine whether the current video played by the display is a fitness teaching video.

[0193] In some embodiments, when the display device uses a client application to play the current video, at least one processor can determine whether the current video is a fitness teaching video by obtaining the type identifier of the current video provided by the client application, or based on the network address of the current video stream provided by the client application.

[0194] In some other embodiments, at least one processor is configured with a pre-trained video type recognition model that can determine the type of a video by processing video frames. The at least one processor can intercept a video frame of a current video and use the video type recognition model to identify the current video and determine whether the current video is a fitness instructional video.

[0195] In an embodiment of the present application, at least one processor may determine whether the display plays the current video in full-screen mode by calling a playback window configuration parameter.

[0196] S820: Control the display to display a first pop-up menu including a split-screen display option.

[0197] When the at least one processor determines that the current video is a fitness teaching video and the display plays the current video in a full-screen mode, the control device controls the display to display a first pop-up menu including a split-screen display option.

[0198] In a specific implementation, at least one processor can obtain the default options and the aforementioned split-screen display options included in the first pop-up menu based on the feature that the display is playing the current video, reasonably arrange the default options and the split-screen display options, determine the display style of the first pop-up menu, and control the display to display the first pop-up menu according to the display style.

[0199] In some embodiments of the present application, at least one processor may call a WindowsManager method provided by the operating system to display a first pop-up menu in the current display window of the display (that is, the window playing the current video).

[0200] FIG9 is a schematic diagram of a user interface of a display according to one embodiment of the present application. As shown in FIG9 , when at least one processor controls the display to display a first pop-up menu 902, the first pop-up menu 902 is displayed in a floating manner in the current display window 901. In order to make the user interface more aesthetically pleasing, the first pop-up menu 902 is displayed on the lower side of the current display window 901. The first pop-up menu 902 includes a split-screen display option, and the split-screen display option is displayed in a highlighted form, that is, the split-screen display option is in a selected state. In conjunction with FIG9 , in addition to the split-screen display selection, the first pop-up menu 902 may also include a timer setting, a resolution setting option, a parental control option, and a help option.

[0201] It should be noted that in the embodiment of the present application, the display split-screen display option is added to the first pop-up menu 902 only when the current video is a fitness instructional video and the display is playing the current video in full-screen mode. If the current video is not a fitness instructional video, or the display is not playing the current video in full-screen mode, the first pop-up menu 902 will not include the display split-screen display option.

[0202] S830: In response to receiving a control instruction for triggering the split-screen display option, controlling the front camera to start and receive a local video generated by the front camera.

[0203] After the display displays the first pop-up menu 902, the user can see various selectable options in the first pop-up menu 902. When the user wants to control the display device to start the split-screen display mode, the user can use a remote control, a terminal device, or gesture control to send a control instruction to trigger the split-screen display option to at least one processor.

[0204] After at least one processor receives a control instruction from the user to trigger the split-screen display selection, at least one processor controls the initialization of the front camera, and after successful initialization, captures the environment on the display surface side of the display, thereby obtaining a local video. It should be noted that the front camera referred to in the embodiments of the present application is a camera that captures the environment on the display surface side of the display. In some embodiments, the camera configured with the display device is a rotatable camera. In this case, the camera will be identified as a front camera only when the camera lens is facing the display surface side of the display; and when the rotatable camera is facing the non-display surface side of the display, it is no longer regarded as a front camera and will not be activated.

[0205] In a specific embodiment, at least one processor can start the front camera by calling a camera driving method provided by the bottom layer of the operating system, and receive the local video captured by the front camera through a corresponding interface.

[0206] S840: Control the display to split into two display windows, and play the current video and local video in the two display windows respectively.

[0207] After receiving the local video transmitted from the front camera, the at least one processor determines that the front camera is successfully activated and transmits the local video normally. The at least one processor can then use the display window control method provided by the operating system to split the display interface of the display into two display windows, and play the current video and the local video in each of the two display windows.

[0208] It should be noted that if the front camera is not started normally, at least one processor cannot obtain the local video. In order to avoid the situation where one display window has no corresponding display video after splitting the screen, at least one processor will not control the display screen to split into two display windows.

[0209] FIG10 is a schematic diagram of a user interface displayed on a display in another embodiment of the present application. As shown in FIG10 , the display interface of the display is divided into two display windows on the left and right, namely a first display window 1001 on the right and a second display window 1002 on the left. The first display window 1001 plays the local video transmitted by the front camera, and the second display window 1002 is used to play the current video. Because the current video is a fitness teaching video and the local video is a follow-up training video that captures the user's body movements, when the user follows the fitness teaching video for fitness, he can view the fitness teaching video and the follow-up training video at the same time to determine whether his movements are correct and standardized.

[0210] As mentioned above, using the second processing method for a display device provided in an embodiment of the present application, at least one processor, after receiving a menu pop-up instruction, determines whether the current video played by the display is a fitness instruction video and determines whether the display plays the current video in full screen mode. If the current video played by the display is a fitness instruction video and the fitness instruction video is played in full screen mode, then the first pop-up menu displayed by the display includes a split-screen display option. If the user triggers the split-screen display option, the at least one processor controls the front camera to start shooting the local video, and causes the display to play the current video and the local video in two display windows respectively in a split-screen display mode.

[0211] By adopting the second processing method for a display device provided in an embodiment of the present application, when the display device plays a fitness teaching video and the user wants to trigger the display to realize split-screen display of a local video for follow-up training, at least one processor provides a first pop-up menu including a split-screen display selection, so as to facilitate the user to quickly switch the display's playback mode to a follow-up split-screen display mode when playing the fitness video, thereby improving the user experience.

[0212] In some embodiments of the present application, after at least one processor controls the display to play the current video and the local video in two display windows, respectively, the user may send a menu pop-up instruction to the at least one processor again for various reasons. In this case, in addition to executing the aforementioned S810-S840, the at least one processor may also execute the following S850-S860 after executing the aforementioned S840.

[0213] S850: In response to receiving the menu pop-up instruction again, controlling the display to display a second pop-up menu including a mirroring option in the first display window playing the local video.

[0214] In some embodiments of the present application, when receiving a menu pop-up instruction again, at least one processor will load the pre-configured option type and call the WindowsManager method provided by the operating system to display the second pop-up menu 1003 in the first display window 1001.

[0215] Figure 11 is a schematic diagram of a user interface displayed on a display in further embodiments according to some embodiments of the present application. As shown in Figure 11 , when at least one processor controls the display to display a second pop-up menu 1003, the second pop-up menu 1003 is displayed in a floating manner within the first display window 1001. As shown in Figure 11 , the first pop-up menu includes a Mirror option, and the Mirror option is selected. The Mirror option is used to trigger the local video displayed in the first display window 1001 to be mirrored left to right.

[0216] In some embodiments, the user's body posture in the local video displayed in the first display window 1001 is exactly opposite to the coach's posture in the fitness teaching video played in the second display window 1002. At this time, it is not convenient for the user to quickly identify whether their own training movements are correct. In order to facilitate the user to easily compare their own movements with the coach's movements to see if they are consistent, and then confirm whether the training movements are correct, the user has the need to trigger the mirror flipping of the current video. To meet the user's aforementioned needs, in the embodiment of the present application, a mirror option is displayed in the second pop-up menu 1003.

[0217] S860: In response to receiving a control instruction for triggering the mirroring option, the local video is horizontally mirror-flipped, and the display is controlled to play the horizontally mirror-flipped local video in the first display window.

[0218] If the user wants to flip the left and right mirror images of the local video, a control instruction for triggering the mirroring option can be sent to at least one processor of the display device by using a remote control, a terminal device or gesture control.

[0219] After receiving the control instruction to trigger the mirroring option, at least one processor obtains a video frame of the local video using the OnPreviewFrame method provided by the operating system, invokes a pre-set mirror flipping algorithm to mirror-flip the video frame, and thereby obtains a mirror-flipped local video. After obtaining the mirror-flipped local video, at least one processor controls the display to play the horizontally flipped local video in first display window 1001.

[0220] Based on the different data storage types and color modes of the local video frames, at least one processor must use the corresponding data processing algorithm to process the video frames to obtain mirror-flipped video frames. Specific methods for implementing mirror-flipping for various video frame types are available in relevant technical literature and will not be elaborated here.

[0221] In some specific embodiments of the present application, the local video captured by the front camera is in YUV color mode, and the video frames are stored in a yuvbyte array. For this type of local video, at least one processor implements the following program code to implement mirror flipping of each video frame in the local video.

[0222] As shown in Figures 10 and 11, in some embodiments, when at least one processor controls the display to split into two display windows, the processor specifically controls the display to split into two display windows in an evenly spaced manner. When the display is split into two display windows in an evenly spaced manner, in order to ensure that the second pop-up menu 1003 displayed in the first display window 1001 is positioned appropriately, that is, to achieve the display effect shown in Figure 11, the at least one processor needs to execute the following steps S851-S853.

[0223] S851: Obtain the window width of the display, the preset margin width, the number of options in the second pop-up menu 1003, and the option width.

[0224] At least one processor can obtain the display window width from configuration parameters in the WindowManager method of the underlying operating system, and can determine the preset margin width, the number of options in the second pop-up menu 1003, and the option width of each option based on the system configuration parameters. The margin width is the width left on the left and right sides of the second pop-up menu 1003 when the second pop-up menu 703 is displayed, that is, the width from the left and right edges of the second pop-up menu 1003 to the edge of the first display window 1001.

[0225] S852: Determine the horizontal coordinates of the reference point of the second pop-up menu according to the window width, the margin width, the number of options in the second pop-up menu 1003, and the option width. The reference horizontal coordinates are coordinates in the global coordinate system of the display.

[0226] After obtaining the window width, margin width, number of options in the second pop-up menu 1003, and option width, at least one processor can then calculate the horizontal coordinates of the reference point of the second pop-up menu using xray = screenWidth / 2 + (screenWidth / 4 - (itemWidth*N + itemMargin*(N-1) + padding / 2) / 2). In the above formula, screenWidth represents the window width, itemWidth represents the width of a single option, N represents the number of options in the second pop-up menu, itemMargin represents the option margin width, and padding represents the margin width. It should be noted that the reference horizontal coordinates are coordinates in the display's global coordinate system.

[0227] S853 : Controlling the display to display the second pop-up menu 1003 including the mirror option using the horizontal coordinates of the reference point, so as to realize the display of the pop-up menu in the first display window 1001 .

[0228] After determining the reference horizontal coordinate, at least one processor then determines the display position of the second pop-up menu 1003 on the display according to the aforementioned reference point horizontal coordinate and the pre-set vertical coordinate, and controls the display to display the second pop-up menu 1003 according to the aforementioned display position, so as to realize the display of the second pop-up menu 1003 as shown in Figure 11.

[0229] As shown in FIG11 , in some embodiments of the present application, in addition to the aforementioned mirroring option, a focus window switching option is also included in the second pop-up menu 1003 displayed in a floating manner within the first display window 1001 by controlling the at least one processor. The corresponding at least one processor may further execute the following steps S870-S880.

[0230] S870: In response to receiving a control instruction to trigger the focus window switching option, the display is controlled to switch the second display window 1002 playing the current video to the focus window, and the display is controlled to display a third pop-up menu 1004 including a full-screen play option in the second display window 1002.

[0231] The focus window is the window in the selected state between the first display window 1001 and the second display window 1002. When the first display window 1001 or the second display window 1002 is the focus window, if at least one processor receives a menu pop-up instruction, a pop-up menu will be displayed in the focus window.

[0232] If the user wants to control the left-right mirror flipping of the local video displayed in the first display window 1001, a control instruction of the focus switching option can be sent to at least one processor of the display device by using a remote control, a terminal device or gesture control.

[0233] When at least one processor receives a control instruction for the focus switching option, at least one processor calls the ActivityManager method provided by the operating system to switch the second display window 1002 to the focus window, and calls the updateViewLayout method in the WindowsManager provided by the operating system to display the third pop-up menu 1004 in the second display window 1002 of the display.

[0234] FIG12 is a schematic diagram of a user interface displayed on a display in some further scenarios according to some embodiments of the present application. As shown in FIG12 , third pop-up menu 1004 is located at the center of the bottom edge of second display window 1002. In a specific implementation, to display third pop-up menu 1004 at the center of the bottom edge of second display window 1002, at least one processor determines the horizontal coordinates of the reference point of third pop-up menu 1004 using the following method. First, the display window width, the preset margin width, the number of options in third pop-up menu 1004, and the option width are obtained. Then, the horizontal reference coordinates of third pop-up menu 1004 are determined using the formula xray = screenWidth / 4 - (itemWidth*M+(M-1)*itemMargin+padding*2) / 2. As previously mentioned, screenWidth represents the window width, itemWidth represents the width of a single option, itemMargin represents the option margin width, padding represents the margin width, and M represents the number of options in third pop-up menu 1004.

[0235] As previously described, the third pop-up menu 1004 includes a full-screen play option. If the user wishes to control the display device to exit split-screen display and display the current video in full-screen mode, a control instruction to trigger the full-screen play option can be sent to at least one processor of the display device using a remote control, a terminal device, or gesture control. Upon receiving the control instruction to trigger the full-screen play option, the at least one processor controls the display to resume playing the current video in full-screen mode. In a specific implementation, if a user finds a particular exercise particularly difficult to learn and wants to carefully review the content of the exercise video first, the full-screen play option can be used to quickly enable full-screen playback of the current video, providing the user with a better user experience.

[0236] As shown in FIG12 , in addition to the aforementioned full-screen play option, the third pop-up menu 1004 also includes a focus switch option and an exit option. The corner switch option is used to trigger switching the currently displayed focus window of the display back to the first display window 1001, and the exit option is used to control the exit of the third pop-up menu 1004.

[0237] Please refer back to Figure 11. In some embodiments of the present application, in addition to the aforementioned mirror option and corner point switching option, the second pop-up menu 1003 also includes a full-screen play option. This full-screen play option is used to trigger the display to display the local video in full screen mode. In a specific implementation, the display device is often equipped with a speaker. When the display plays the video content of the current video, the speaker simultaneously plays the audio content corresponding to the current video, so that the user can receive fitness teaching information through sound and light at the same time. In some cases, the user may have been familiar with the fitness movements through early follow-up training, and at this time there is no need to continue watching the current movement, but only want to complete the fitness movement by themselves under the guidance of the audio corresponding to the current video. At this time, the user also has the need to always pay attention to whether his or her own movements are correct and standard.

[0238] To meet the aforementioned requirements, in this embodiment of the present application, a full-screen play option is also provided in the second pop-up menu 1003. If the user wishes to control full-screen play of the local video, they can use a remote control, a terminal device, or gesture control to send a control instruction to trigger the full-screen play option to at least one processor of the display device. In response, the at least one processor will execute the following S880.

[0239] S880: In response to receiving a control instruction to trigger the full-screen playback option, controlling the display to play the local video in full-screen mode, and controlling the speaker to play the audio corresponding to the current video at the same time.

[0240] In an embodiment of the present application, when the full-screen play option in the second pop-up menu 1003 is triggered, at least one processor controls the display to play the local video in full screen, while continuing to control the speaker to play the audio corresponding to the current video. This allows the user to complete fitness movements on their own under the guidance of the audio, and pay attention to whether their own movements are correct and standard, thereby improving the user experience.

[0241] Please refer to Figures 10 to 12. In some embodiments of the present application, when the display displays the current video and the local video in a left-right split screen, since the screen size of the current video does not match the window size of the first display window 1001, and the screen size of the local video does not match the window size of the second display window 1002, in order to play the entire current video and local video content, the current video and the local video are both reduced in size, and there will be a large blank background above and below the first display window 1001 and the second display window 1002. Since the current video and the local video are both reduced in size, the user may need to get closer to the display device to see the fitness content in the current video and the local video more clearly, resulting in a poor user experience. In addition, in some cases, when the user is too close to the display device, certain fitness movements may not be completed because the display device is idle.

[0242] Figure 13 is a flowchart of a second processing method for a display device according to some embodiments of the present application, for controlling two display windows to play a current video and a local video. As shown in Figure 13 , to resolve the aforementioned issues and fully utilize the display interface of the display device, at least one processor executes the following steps S841-S842.

[0243] S841: According to the sizes of the two display windows, the fitness image regions of the corresponding current video and the local video are respectively intercepted to obtain the intercepted current video and the intercepted local video.

[0244] In an embodiment of the present application, at least one processor may obtain the size of the first display window 1001 and the size of the second display window 1002. After obtaining the size of the first display window 1001, the at least one processor may intercept the fitness image area of ​​the local video based on the size of the first display window 1001, obtain the fitness image area in the local video, and use the video content of the local fitness image area as the intercepted local video. After obtaining the size of the second display window 1002, the at least one processor may intercept the fitness image area of ​​the current video based on the size of the second display window 1002, and obtain the fitness image area in the current video.

[0245] In some embodiments, at least one processor may utilize a pre-trained body recognition model to identify human body locations in the current video and local video in real time, thereby determining the region of the captured fitness image. In other embodiments, at least one processor may utilize an image difference algorithm to compare pixel content changes between adjacent frames of the current video to determine the corresponding fitness image region, and to compare pixel content changes between adjacent frames of the local video to determine the corresponding fitness image region.

[0246] S842: Play the intercepted current video and the intercepted local video in full-window playback mode in two display windows respectively.

[0247] Since the current video and the local video fitness images are respectively captured according to the size of the display window to obtain the captured current video and the local video, and the captured current video and the local video match the corresponding display window size (specifically, the aspect ratio of the corresponding display window matches), at least one processor can control the playback of the captured current video and the captured local video in full-screen mode in the two display windows respectively.

[0248] By executing steps S841-S842, the display device can play the fitness instructional portion of the current video and the user's self-training portion of the local video in full-screen mode, allowing the user to more clearly view the corresponding content and compare movements. Furthermore, by playing the captured current video and the captured local video in full-screen mode, blank backgrounds in the display window can be avoided or even removed, thereby improving the user's viewing experience.

[0249] In the aforementioned embodiment, when at least one processor controls the display to execute S840 to control the display to play the current video and the local video in two display windows respectively, and receives a menu pop-up instruction again, at least one processor controls the display to display a second display pop-up window in the first display window. In other embodiments, when at least one processor controls the display to play the current video and the local video in two display windows respectively, and receives a menu pop-up instruction again, at least one processor controls the display to display a third pop-up menu including a full-screen play option and a focus window switching option in the second display window playing the current video, that is, displaying a menu as shown in Figure 12. The third pop-up menu includes a full-screen play option and a focus window switching option. Subsequently, if at least one processor receives a control instruction to trigger the full-screen play option, the display is controlled to resume playing the current video in full screen mode; if at least one processor receives a control instruction to trigger the focus switching option, the display is controlled to switch the first display window playing the local video to the focus window, and the display is controlled to display a second pop-up menu including a mirroring option and an exit full-screen play option in the first display window. In addition to providing the aforementioned method of realizing split-screen display and exiting split-screen display by evoked pop-up menu, the display device in the embodiment of the present application also provides a solution for pre-setting entry into split-screen mode and selecting an application to be launched in split-screen mode. Specifically, after a user enters a specific interactive interface, the interface displays a split-screen fitness mode and displays client applications available in split-screen fitness mode. After the user selects a desired client application and confirms the selection, at least one processor of the display device controls the display to enter split-screen display mode, displaying two windows: one window displays the playback interface of the selected client application, and the other window displays the local video captured by the front camera.

[0250] Currently, the software system of smart devices supports split-screen display of applications, but the applications for split-screen display can only be obtained from the recently accessed application list, and the order in which the applications are loaded in the recently accessed application list is fixed, which greatly limits the scope of applications for split-screen display. Users are also unable to split-screen display applications as they wish, which is not friendly to the user's interactive experience.

[0251] In view of this, as shown in FIG14 , there is a flowchart diagram 1 of a third processing method for a display device according to some embodiments of the present application, specifically for split-screen display, the method comprising:

[0252] S1401: Receive a first selection input for a first application and a second application.

[0253] The various inputs described in the embodiments of the present application (such as the first selection input, the second selection input, the first control input, the second control input, etc.) can all be inputs by the user for the corresponding application identifier, or inputs by the user for the corresponding operation control. The input method can be click input, voice input, etc.

[0254] Among them, the click input can include single-click input or double-click input. The click input can be a finger click input, or a click input through an input device such as a mouse, or a click input issued by a remote control or other control device. The embodiment of the present application does not limit this.

[0255] The application identifier can be understood as an application icon, or any other control that can represent an application, such as the application identifier of the first application, the application identifier of the second application, etc. in the embodiment of the present application.

[0256] S1402: In response to a first selection input, display a selection result of the first application and a selection result of the second application.

[0257] FIG15 shows a first schematic diagram of an interface for a display device according to some embodiments of the present application. FIG15 illustrates an application selection interface, which includes application identifiers for multiple applications supported by the current platform and available for user selection, representing Application 1, Application 2, and so on. The application selection interface also includes a focus frame 151, which is used to represent the application currently preselected by the user. For example, in the interface shown in FIG15 , the application currently preselected by the user is Application 2.

[0258] FIG16 is a second schematic diagram of an interface of a display device according to an embodiment of the present application. FIG16 shows, based on the application selection interface shown in FIG15 , the selection results of the first application and the second application based on the user's first selection input. As shown in FIG16 , the first focus frame 161 and the second focus frame 162 represent the user's selection results for the first application and the second application, respectively. The current user's selection results for the first application and the second application are application 3 and application 5, respectively.

[0259] In some embodiments, in response to a first selection input for a first application and a second application received through the user interface, the first focus frame is moved to the location of the application identifier of the first application, and the second focus frame is moved to the location of the application identifier of the second application; the color of the first focus frame and / or the second focus frame is changed, and the first focus frame after the color change is used to indicate the user's selection result for the first application, and the second focus frame after the color change is used to indicate the user's selection result for the second application.

[0260] Split-screen display requires the user to select at least two applications. The user first selects the first application through the first focus frame, and then selects the second application through the second focus frame. When the user needs to open the split-screen application, the position of the first focus frame in the display device is controlled; when the first focus frame moves to the identification position of the first application that the user needs to select, the user's selection result for the first application is identified by changing the color of the first focus frame. Similarly, the position of the second focus frame in the display device is controlled; when the second focus frame moves to the identification position of the second application that the user needs to select, the user's selection result for the second application is identified by changing the color of the second focus frame.

[0261] In some embodiments, the shape of the focus frame may be changed to indicate the user's selection result for the first application or the second application, which is not limited in this embodiment of the present application.

[0262] In some embodiments, the application selection interface may also display the user's selection results for the first application and the user's selection results for the second application by using the application names. As shown in Figure 16, the name of the first application is displayed in the first application name display area 163, and the name of the second application is displayed in the second application name display area 164. It should be understood that the location and display content of the application name display area provided in the embodiments of the present application are merely examples and should not be used as limitations on the implementation of the present application.

[0263] In some embodiments, after the user selects the first application and the second application, if the first application or the second application needs to be modified, the first application or the second application can be reselected by controlling the position of the focus box, or by editing the application name in the application name display area.

[0264] S1403: Display window arrangement selection controls and split-screen controls.

[0265] The window arrangement selection control is used to allow the user to select the arrangement of the first split-screen area and the second split-screen area. The window arrangement selection control can be implemented as a selection control, or as a drop-down menu control, or as other forms of controls, which are not limited in the embodiments of the present application.

[0266] The split-screen control is used for the user to enable the split-screen display function. When the user operates the split-screen control, the first application and the second application can be displayed in split screen on the display device based on the selection result of the first application and the selection result of the second application.

[0267] S1404: Receive a second selection input for the window arrangement mode selection control.

[0268] S1405 . In response to the second selection input, determine an arrangement of the first split screen area and the second split screen area.

[0269] The user can select the arrangement of the split-screen windows. In the embodiment of the present application, in response to the second selection input of the window arrangement selection control, the split-screen display mode of the first application and the second application desired by the user can be determined.

[0270] The first split-screen area and the second split-screen area can be arranged in a left-right split-screen manner, the first split-screen area can be displayed on the left side of the screen, and the second split-screen area can be displayed on the right side of the screen, or the first split-screen area can be displayed on the right side of the screen, and the second split-screen area can be displayed on the left side of the screen; or, the second split-screen area can be arranged in an up-down split-screen manner, the first split-screen area can be displayed on the upper side of the screen, and the second split-screen area can be displayed on the lower side of the screen, or the first split-screen area can be displayed on the upper side of the screen, and the second split-screen area can be displayed on the lower side of the screen.

[0271] In some embodiments, as shown in FIG16 , the application selection interface displays a left-right arrangement control 165 and a top-bottom arrangement control 166. When a second selection input is received for the left-right arrangement control 165, it is determined that the user desires to display the first application and the second application in a left-right arrangement manner; when a second selection input is received for the top-bottom arrangement control 166, it is determined that the user desires to display the first application and the second application in a top-bottom arrangement manner. The application selection interface also displays a split-screen control 167 for the user to enable the split-screen display function.

[0272] S1406: Receive a first control input for the split-screen control.

[0273] S1407. In response to a first control input, display the interface of the first application in the first split-screen area, and display the interface of the second application in the second split-screen area.

[0274] Referring to the above steps, after determining the split-screen mode for the first application and the second application, in response to the first control input to the split-screen control, split-screen display is performed based on the selection result of the first application and the selection result of the second application. Specifically, the interface of the first application is displayed in the first split-screen area, and the interface of the second application is displayed in the second split-screen area.

[0275] As shown in Figure 17, this is a third interface diagram of a display device according to some embodiments of the present application. In conjunction with Figure 16, the first application selected at this time is application 3, and the second application selected is application 5. In response to the user's second selection input for the left-right arrangement control 165 in the interface shown in Figure 16, the arrangement mode of the first split-screen area and the second split-screen area is determined to be left-right arrangement; in response to the user's first control input for the split-screen (SPLIT-SCREEM) control 167 in the interface shown in Figure 16, the application interface of the first application and the application interface of the second application are respectively displayed in the first split-screen area and the second split-screen area arranged left-right, that is, the interface diagram of the display device shown in Figure 17 is displayed.

[0276] FIG18 is a fourth schematic diagram of an interface of a display device according to some embodiments of the present application. In conjunction with FIG16 , the selected first application is Application 3, and the selected second application is Application 5. In response to a user's second selection input to the vertical arrangement control 166 in the interface shown in FIG16 , the arrangement of the first split-screen area and the second split-screen area is determined to be vertical; and in response to a user's first control input to the split-screen control 167 in the interface shown in FIG16 , the application interface of the first application and the application interface of the second application are displayed in the vertically arranged first split-screen area and the second split-screen area, respectively, thereby presenting the interface schematic diagram of the display device shown in FIG18 .

[0277] In an embodiment of the present application, by allowing the user to select the application to be split and the direction of the application split before the application is split-screen displayed, the application interface can be split-screen displayed in the ideal split-screen display area on the screen according to the user's wishes, providing the user with a more friendly split-screen display interactive experience.

[0278] In some embodiments, in response to a first selection input for a first application and a second application, the selection results of the first application and the selection results of the second application are displayed, and the window arrangement selection control and the split screen control are displayed. Then, based on the second selection input for the first application and the second application, the selection order of the first application and the second application is displayed.

[0279] Specifically, you can specify the display positions of the first split-screen area and the second split-screen area. For example, when using the left-right split-screen display mode, the first split-screen area is displayed on the left interface of the display device by default, and the second split-screen area is displayed on the right interface of the display device. At this time, when the split-screen function is turned on, the interface of the first application will be displayed on the left interface of the display device, and the interface of the second application will be displayed on the right interface of the display device. When the selection order of the first application and the second application is swapped, the split-screen display positions of the two applications will also be swapped.

[0280] When the user selects application 3 as the first application and application 5 as the second application, and when the left-right split-screen display mode is used, when the split-screen function is turned on, the interface of application 3 will be displayed on the left interface of the display device, and the interface of application 5 will be displayed on the right interface of the display device; if the user selects application 5 as the first application and application 3 as the second application, when the left-right split-screen display mode is used, when the split-screen function is turned on, the interface of application 5 will be displayed on the left interface of the display device, and the interface of application 3 will be displayed on the right interface of the display device.

[0281] Therefore, based on the second selection input for the first application and the second application, the selection order of the first application and the second application can be displayed, so that the user can understand the positions of the interfaces of the two currently selected applications to be displayed in the split-screen on the display device interface. At the same time, by changing the selection order of the applications to be displayed in the split-screen, the user can also determine the positions of the interfaces of the applications to be displayed in the split-screen on the display device interface.

[0282] As shown in FIG16 , the name of the first application can be displayed in the first application name display area 163, and the name of the second application can be displayed in the second application name display area 164. That is, the name of the application selected first by the user is displayed in the first application name display area 163, and the name of the application selected later by the user is displayed in the second application name display area 164. If the user selects application 3 and application 5 in sequence and then selects the left-right split-screen mode, the interface of application 3 can be displayed on the left interface of the display device, and the interface of application 5 can be displayed on the right interface of the display device; if the user selects application 5 and application 3 in sequence and then selects the left-right split-screen mode, the interface of application 5 can be displayed on the left interface of the display device, and the interface of application 3 can be displayed on the right interface of the display device; if the user selects application 3 and application 5 in sequence and then selects the top-bottom split-screen mode, the interface of application 3 can be displayed on the top interface of the display device, and the interface of application 5 can be displayed on the bottom interface of the display device; if the user selects application 5 and application 3 in sequence and then selects the top-bottom split-screen mode, the interface of application 5 can be displayed on the top interface of the display device, and the interface of application 3 can be displayed on the bottom interface of the display device. It is understandable that the above implementation content is only an example of the embodiment of the present application, and the specific display position of the application interface should not be used as a limitation of the embodiment of the present application.

[0283] The embodiment of the present application enables the user to freely control the display position of the application interface in the split-screen state through the user's selection order of split-screen applications, effectively improving the controllability and interactivity of the above-mentioned third processing method for the display device, improving the user experience, and better meeting the needs of the market and users.

[0284] In some embodiments, as shown in FIG19 , there is a second flowchart of a third processing method for a display device according to some embodiments of the present application. The method is described below with reference to the flowchart.

[0285] S1901: Determine the selection result of the first application, the selection result of the second application, and the arrangement of the first split screen area and the second split screen area.

[0286] Specifically, S1901 can be implemented through the above steps S1401 to S1405. The implementation process and principle are consistent and will not be repeated here.

[0287] S1902: Receive a first control input for a split-screen control.

[0288] S1903: Detect startup processes of the first application and the second application.

[0289] S1904: Determine whether the startup processes of the first application and the second application have finished running. If so, execute S1906; if not, execute S1905.

[0290] After the user activates split-screen display, the first and second applications need to execute a startup process, which includes loading the display interface of the first application and the display interface of the second application. When the startup process of the first application is completed, it means that the first application has been successfully launched and loaded, and the first application interface can be displayed. When the startup process of the second application is completed, it means that the second application has been successfully launched and loaded, and the second application interface can be displayed.

[0291] Specifically, the activity manager in the software system can be used to obtain and detect the process task (runningTask) of the first application and the process task (runningTask) of the second application, and detect whether the startup processes of the first application and the second application have been completed, so as to determine whether the first application and the second application have been loaded.

[0292] S1905. Display the preset loading interface in full screen.

[0293] If the startup process of the first and second applications has not yet completed, it means that the first or second application has not yet loaded. At this time, the interface of the first or second application may flicker or other abnormal display issues. To avoid this problem, the preset loading interface is displayed full screen before the startup process of the first and second applications has completed.

[0294] Figure 20 shows a schematic diagram of a preset loading interface according to some embodiments of the present application. In Figure 20, a rotating ring with a notch in the center of the interface indicates to the user that the first application and the second application are currently loading. In other possible implementations, the preset loading interface can be any preset loading image or loading animation, which is not limited in this embodiment of the present application.

[0295] S1906. Hide the preset loading interface.

[0296] When it is detected that the startup processes of the first application and the second application have ended, it means that the first application and the second application have been loaded. At this time, the interfaces of the first application and the second application can be displayed normally, and the preset loading interface is hidden.

[0297] S1907. Display the interface of the first application in the first split-screen area, and display the interface of the second application in the second split-screen area.

[0298] The selection results of the first application and the second application obtained in S1901, as well as the arrangement of the first split-screen area and the second split-screen area, are passed as parameters to the Activity Manager and the Window Manager in the Application Framework layer. The Activity Manager and the Window Manager then perform split-screen display, displaying the interface of the first application in the first split-screen area and the interface of the second application in the second split-screen area. For the specific display method, refer to the implementation method in the above embodiment and will not be repeated here.

[0299] In an embodiment of the present application, by displaying the preset loading interface in full screen during the loading process of the first application and the second application, abnormal display situations that may occur during the loading process of the split-screen interface are avoided, confusion for the user is avoided, and the user experience during the split-screen display process is improved.

[0300] In some embodiments, a region control control may be displayed between the first split screen region and the second split screen region; and in response to a second control input to the region control control, the sizes of the first split screen region and the second split screen region are changed.

[0301] The region control control can be displayed as a draggable control or as a dividing line. The user can change the size of the first split-screen area and the second split-screen area by dragging the region control control, or by performing a second control input. Figure 21 is a fifth diagram of an interface of a display device according to some embodiments. As shown in Figure 21, dragging the region control control 2101 can change the size of the first split-screen area 2102 and the second split-screen area 2103.

[0302] The embodiment of the present application sets an area control control, which allows users to adjust the size of the two split-screen areas at will, thereby increasing the flexibility of the third processing method for the display device and further improving the user's interactive experience.

[0303] FIG22 is a flowchart of a third processing method for a display device according to some embodiments of the present application, specifically for key input processing. As shown in FIG22 , the method includes the following steps.

[0304] S2201: Receive key input from the user.

[0305] S2202: Check whether the current display is in split-screen mode. If yes, execute S2203; if not, execute S2207.

[0306] S2203: Determine whether the key corresponding to the key input exists in a preset key whitelist. If so, execute S2206; if not, execute S2204.

[0307] Among them, the preset button whitelist is used to store all buttons that will not be responded to in the split-screen display state.

[0308] In the split-screen state, if a key input from the user is received, in order to avoid affecting the split-screen display, it is first determined whether the key corresponding to the received key input is allowed to be responded to.

[0309] Specifically, a preset button white list (white list) is added to the system case manager (Phone Window Manager) of the software system.

[0310] For example, add the following code to the system:

[0311] int[]whitelist={KeyEvent.KEYCODE_MENU,KeyEvent.KEYCODE_INPUT,KeyEvent.KEYCODE_SETTINGS,KeyEvent.KEYCODE_MEDIA,……}

[0312] The buttons recorded in the above preset case whitelist can be set as buttons that are not responded to in the split-screen display state.

[0313] S2204: Determine the current focus application.

[0314] The focus application is the one with a higher priority among the first application and the second application.

[0315] If the key corresponding to the key input is not recorded in the preset key whitelist, it is determined that the key corresponding to the currently received user key input needs to be responded to, and it is further determined that one of the first application and the second application processes the key input.

[0316] Specifically, the priorities of the first application and the second application may be pre-set by the user. Alternatively, the priorities of the first application and the second application may be determined according to a pre-set priority determination rule. For example, based on the user's operation records for the first application and the second application, it is determined that the application that the user has most recently operated has a higher priority. The application with the higher priority between the first application and the second application is selected as the focus application.

[0317] S2205: The focus application processes the key input.

[0318] Send the key input to the focus application, which processes the key input.

[0319] S2206: Abandon processing the key input.

[0320] When the key corresponding to the key input is recorded in the preset key whitelist, no processing is performed on the key input to avoid affecting the current application split-screen display.

[0321] S2207: Directly process the key input.

[0322] If it is detected that the current system is not in split-screen display state, the key input at this time will not affect the application split-screen display, and the key input will be directly processed according to the system native rules.

[0323] The embodiment of the present application sets a preset button whitelist to filter the user's button input in the split-screen state, effectively preventing the user's button input in the split-screen state from affecting the split-screen display state of the application, ensuring the stability of the split-screen display method, and further improving the user experience.

[0324] Figure 23 is a fourth flowchart of the third processing method for a display device according to an embodiment of the present application, which is specifically used for application startup instruction processing. As shown in Figure 23, the method includes the following steps.

[0325] S2301: Receive a third application startup instruction from the user.

[0326] The third application refers to any application other than the first application and the second application displayed in the split screen. The third application startup instruction can be any form of operation instruction such as a voice instruction, a gesture instruction, etc. The third application startup instruction is used to instruct to start the third application.

[0327] S2302: Check whether the current display is in split-screen mode. If yes, execute S2303; if not, execute S2304.

[0328] The Activity Manager in the Application Framework layer detects the current system state and blocks the third application startup instruction if it detects that the system is currently in a split-screen state. Otherwise, the third application is run normally according to the third application startup instruction.

[0329] S2303: Shield the third application startup instruction.

[0330] If the app is currently in split-screen mode, i.e., the first and second apps are currently being displayed, launching a third app will overwrite the first and second apps, or interrupt the current split-screen mode. To ensure the normal display of the split-screen interface, the third app launch command needs to be blocked.

[0331] Specifically, the third application startup instruction is shielded or intercepted by an activity manager.

[0332] S2304: Start the third application.

[0333] If the split-screen display state is not currently in progress, the third application will be started normally based on the third application startup instruction according to the system native logic.

[0334] S2305, end.

[0335] The embodiment of the present application detects the current system status after receiving the third application startup instruction. If it is detected that the system is currently in a split-screen state, the third application startup instruction is blocked to avoid abnormalities in the current split-screen display interface caused by the startup of the third application, further ensuring the stability of the above-mentioned third processing method for the display device and improving the user experience.

[0336] With the increase in the display screen of the display device 200 and the improvement in memory, CPU and GPU performance, the hardware performance of the display device 200 can now meet the conditions for running multiple applications simultaneously. Therefore, the display device 200 has the hardware conditions for configuring the split-screen function. If the display device 200 is configured with the split-screen function, after the split-screen function is turned on, the display device 200 can display at least two windows at the same time, and display the screens of different applications in the at least two windows. However, the user needs to perform additional operations such as moving the focus to determine which window of the at least two windows occupies the focus, resulting in a poor user experience.

[0337] To address the above issues, embodiments of the present application provide a fourth processing method for a display device, specifically for multi-window display, which can be applied to the display device 200 shown in Figures 1A-4B, or other display devices. Embodiments of the present application use the display device 200 shown in Figures 1A-4B as an example to illustrate this fourth processing method for a display device.

[0338] When the display device 200 displays the first screen of the first application in the first window, multi-window display can be performed according to the process shown in FIG24 . The specific steps are as follows:

[0339] S2401 , in response to a screen splitting instruction input by the user, adding and displaying a second window, and displaying a first screen of a first application in the first window, and displaying a second screen of a second application in the second window.

[0340] The first application is the application currently displayed on the display device 200. The first application is a split-screen application, that is, the first application can be run simultaneously with other split-screen applications in different windows. The screen of the first application is the first screen.

[0341] The first window can be one or more. If there is one first window, it means that the display device 200 is currently displaying only one first application, and the first window can occupy the entire display area of ​​the display. If there are multiple first windows, it means that the display device 200 is currently displaying multiple first applications at the same time, each first application is different and is displayed in a corresponding first window. The multiple first windows respectively occupy part of the display area of ​​the display, and the partial display area occupied by each first window can constitute the entire display area of ​​the display.

[0342] The split-screen instruction is used to instruct the split-screen display of the first application and the second application. The window used to display the second application is the second window, and the second application's screen is the second screen. The second window can be located to the left or right of the first window, that is, the first window and the second window are arranged horizontally, or the second window can be located above or below the first window, that is, the first window and the second window are arranged vertically.

[0343] The first window and the second window can be displayed according to a preset split-screen ratio. For example, if the preset split-screen ratio is 1:1, the first window and the second window are displayed in a manner that evenly distributes the display area. If the preset split-screen ratio is 1:2, the first window occupies one-third of the display area, and the second window occupies two-thirds of the display area.

[0344] In some embodiments, the display device 200 may display the first application and the second application in split screen according to the process shown in FIG25 . The specific steps are as follows:

[0345] S2501: In response to a split-screen instruction, display a split-screen application list.

[0346] The user can input a split-screen instruction to the display device 200 based on the split-screen button on the control device 100, and the display device 200 responds to the split-screen instruction by displaying a split-screen application list. The split-screen application list includes items for each split-screen application, and these split-screen applications include the second application. The split-screen applications in the split-screen application list can be split-screen applications installed in the display device 200 and supporting the split-screen function, or they can be split-screen applications installed in the display device 200 and associated with the first application, where the association means that the first application and the split-screen application are displayed in split-screen in a fixed combination.

[0347] S2502 : In response to a selection instruction input by the user based on an item of the second application, a second window is added and displayed, and a first screen of the first application is displayed in the first window, and a second screen of the second application is displayed in the second window.

[0348] If the split-screen applications in the split-screen application list are applications installed on the display device 200 and support the split-screen function, the user can select a second application to be displayed in split-screen with the first application, that is, input a selection instruction based on the item of the second application to instruct the display device 200 to display the first application and the second application in split-screen. In this way, the user can flexibly select the second application to be displayed in split-screen with the first application.

[0349] If the split-screen application is installed in the display device 200 and is associated with the first application, typically there is only one split-screen application, namely the second application. The user can directly input a selection instruction based on the item of the split-screen application to instruct the display device 200 to display the fixed combination in split screen. Since the associated first application and the second application are used to achieve specific functions through split-screen display, for example, the first application is a fitness video application and the second application is a camera application. By displaying the first application and the second application in split screen, it is possible to play a fitness video through the fitness video application and capture and display the user's movements through the camera application. Therefore, after selecting the second application, the user can achieve the specific function by displaying the first application and the second application in split screen.

[0350] In some embodiments, the display device 200 may display the first application and the second application in split screen according to the process shown in FIG26 . The specific steps are as follows:

[0351] S2601: In response to a screen split instruction, obtain a second application associated with the first application.

[0352] The first application has a specified function, and the first application needs to implement the specified function through split-screen display with the second application. Among them, the split-screen application that is displayed split-screen with the first application to implement the specified function of the first application is the second application associated with the first application.

[0353] For example, the first application is a fitness video application with a designated function of correcting fitness movements, and the second application associated with the first application is a camera application. For another example, the first application is an education application with a designated function of taking study notes, and the second application associated with the first application is a note-taking application.

[0354] The user can input a split screen instruction to the display device 200 based on the split screen button on the control device 100, and the display device 200 responds to the split screen instruction and obtains the second application associated with the first application according to the association relationship between the first application and the second application.

[0355] S2602 , adding and displaying a second window, and displaying a first screen of the first application in the first window, and displaying a second screen of the second application in the second window.

[0356] A first screen of a first application is displayed in a first window, and a second screen of a second application is displayed in a second window, so that a designated function of the first application is realized by displaying the associated first and second applications in split screens.

[0357] Display device 200 provides a mechanism for accessing the Android split-screen application programming interface (API) in the system user interface (SystemUI), providing a way for upper-level applications to quickly enter split screen. The Android split-screen API for window management (WindowManager) is called through the SystemUI. The reason for providing the interface in the SystemUI is that after the Android system starts, the parent process (init) forks the incubator process (Zygote), which starts the system service (system_server) process, which starts the boot service, core services, and other services.

[0358] The Android boot process shown in Figure 27 includes the following steps:

[0359] S2701: Bootloader starts and kernel idle process starts.

[0360] S2702, start the init process (pid0).

[0361] S2703, fork starts the zygote process (pid1).

[0362] S2704: The system_server process starts.

[0363] S2705, start the boot service.

[0364] S2706, start core services.

[0365] S2707, start other services.

[0366] S2708, start systemui.

[0367] Specifically, in the Android boot process, the kernel starts process No. 0 init, init starts process No. 1 zygote, and the zygote process starts the system_server process. The system_server process is one of the core processes of the Android system and provides the service server capabilities within Android, such as the window management service (WindowManagerService) and the activity management service (ActivityManagerService). These core services will be started in the main function of the system_server process, and when other services are started, the SystemUI process will be started. SystemUI is a resident process independent of the sytem_server process and provides the UI of the Android system. SystemUI can be regarded as an independent app. SystemUI starts earlier than other system apps, so adding a split-screen entry in SystemUI is a good choice.

[0368] Before Android 12, the Recents class of System UI provided a way to enter split screen, the WindowManagerProxy class provided a way to exit split screen, and the am class provided a way to adjust the window size.

[0369] In Android 12 and above, Android's native split-screen control logic is located in the WindowManager directory, which provides a split-screen control (SplitScreenController) class that provides methods for entering and exiting split screens and adjusting window sizes.

[0370] When the system_server process starts the System UI, it will first pull up the SystemUIService. The initialization (onCreate) method of the SystemUIService will call the startServicesIfNeeded method of the SystemUIApplication. Executing this method will load all System UI service classes that inherit SystemUI (CoreStartable class for Android 13 and above) and execute their start methods.

[0371] In order to provide a quick way to enter split-screen mode in System UI and allow third-party applications to call it directly through System UI, the split-screen function can be injected when System UI starts the System UI service class. To reduce coupling, the split-screen logic can be injected into the vendor service (VendorServices). VendorServices also inherits from System UI. The default implementation of the start method is empty. The split-screen management (SplitScreenManager) class can be instantiated in the VendorService class and registered to listen for broadcasts. The split-screen management class holds a reference to SplitScreenController and can call SplitScreenController methods according to different broadcast types to implement split-screen control operations.

[0372] The split-screen operation process provided by SystemUI can be referred to in Figure 28. SystemUIApplication obtains the SystemUI service class through SystemUIFactory. When calling startServicesIfNeedeed, SystemUIApplication will call the start method of the VendorServices class. The VendorServices class holds a reference to the custom SplitScreenManager and will be instantiated as SplitScreenManagerImpl in the start method. SplitScreenManagerImpl can register for broadcasts of entering split screen, exiting split screen, and adjusting windows through the register receiver (registReceiver) method, and then call the system native split-screen method of the SplitScreenController held by SplitScreenManagerImpl to perform various split-screen operations. By providing services in System UI, the quick entry and exit of split screen are unified. Entering split screen does not require entering from the recent list, but provides a direct call method by sending system notifications to the underlying layer. It does not destroy the system native split-screen logic, facilitates management, makes the split-screen experience more harmonious, and solves the problem that third-party applications cannot directly enter split screen.

[0373] S2402: Determine a focus window that occupies focus between the first window and the second window.

[0374] When a first window and a second window are displayed in a split-screen mode in multi-window mode, the window that occupies focus is the focus window. The display device 200 pre-sets a rule for occupying focus after the split screen, for example, the leftmost window occupies focus, the topmost window occupies focus, the first window occupies focus, the second window occupies focus, etc.

[0375] After displaying the first window and the second window in split screen, the display device 200 can determine the focus window by identifying the location of the focus. After displaying the first window and the second window in split screen, the display device 200 can also directly determine the focus window based on a preset rule of occupying focus.

[0376] S2403, displaying a focus frame on the upper layer of the focus window.

[0377] The focus frame is a frame-shaped window, that is, the middle area of ​​the focus frame is hollow, and the image in the focus window located below the focus frame can be displayed through this middle area. The size of the focus frame matches the size of the focus window, which can achieve the effect of the focus frame covering the border of the focus frame. Therefore, when the focus frame is displayed on the border of the focus window, the user can see the focus frame and the image in the focus window displayed through the focus frame. At this time, the focus frame can serve as an indicator of the focus window. The user can quickly determine the focus window based on the position of the focus frame, and then quickly determine the location of the focus.

[0378] The display device 200 may display the focus frame according to the process shown in FIG29 , and the specific steps are as follows:

[0379] S2901, obtaining the position of the focus window, the size of the display area of ​​the display, the preset split screen ratio between the first window and the second window, and the arrangement of the first window and the second window.

[0380] The focus window position is the relative position of the focus window in the display area. For example, if the first window and the second window are arranged side by side, and the first window is to the left of the second window, then if the first window is the focus window, then the focus window position is the left side; if the second window is the focus window, then the focus window position is the right side. For another example, if the first window and the second window are arranged top by top, and the first window is above the second window, then if the first window is the focus window, then the focus window position is the top; if the second window is the focus window, then the focus window position is the bottom.

[0381] The size of the display area can be represented by width and height, wherein the size of the display area in the horizontal direction is width, and the size of the display area in the vertical direction is height.

[0382] The preset split-screen ratio between the first window and the second window is the ratio of the border sizes of the first window and the second window in the arrangement direction. For example, if the first window and the second window are arranged side by side, the bottom border of the first window and the bottom border of the second window, or the top border of the first window and the top border of the second window, constitute the borders in the arrangement direction. The ratio of the size of the bottom border of the first window to the size of the bottom border of the second window is equal to the preset split-screen ratio. Alternatively, the ratio of the size of the top border of the first window to the size of the top border of the second window is equal to the preset split-screen ratio. For another example, if the first window and the second window are arranged top by top, the left border of the first window and the left border of the second window, or the right border of the first window and the right border of the second window, constitute the borders in the arrangement direction. The ratio of the size of the left border of the first window to the size of the left border of the second window is equal to the preset split-screen ratio. Alternatively, the ratio of the size of the right border of the first window to the size of the right border of the second window is equal to the preset split-screen ratio.

[0383] S2902: Determine the size of the focus window according to the size of the display area, the preset split-screen ratio, and the arrangement of the first window and the second window.

[0384] The sizes of the borders of the first and second windows in the non-aligned direction are equal and equal to the size of the display area corresponding to the borders. For example, if the first and second windows are aligned side by side, the left and right borders of the first and second windows are borders in the non-aligned direction, and the sizes of these borders are equal to the height of the display area. For another example, if the first and second windows are aligned top and bottom, the top and bottom borders of the first and second windows are borders in the non-aligned direction, and the sizes of these borders are equal to the width of the display area.

[0385] The sum of the sizes of the borders of the first window and the second window in the arrangement direction is equal to the size of the display area corresponding to the borders, and the ratio of the sizes of the borders of the first window and the second window in the arrangement direction is equal to the preset split-screen ratio. Based on the above relationship, the sizes of the borders of the first window and the second window in the arrangement direction can be calculated. For example, if the first window and the second window are arranged left and right, the sum of the bottom borders of the first window and the second window is equal to the width of the display area, and the ratio of the bottom borders of the first window and the second window is equal to the preset split-screen ratio, such as 2:1. In this case, the size of the bottom border of the first window is equal to two-thirds of the width of the display area, and the size of the bottom border of the second window is equal to one-third of the width of the display area.

[0386] Based on the above process, the size of each frame of the first window and the second window can be determined, and the size of each frame of the focus window can be further determined.

[0387] S2903: Determine the position of the focus frame according to the position of the focus window, and determine the size of the focus frame according to the size of the focus window.

[0388] The position of the focus window is the position of the focus frame in the display area, and the size of the focus window is the size of the focus frame.

[0389] S2904: Display the focus frame according to the position and size of the focus frame.

[0390] In this way, it can be ensured that the focus frame covers the focus window and its size matches the border of the focus window, without blocking the picture displayed in the focus window and the focus frame can be fully displayed.

[0391] In some embodiments, the display device 200 can realize the display of the focus frame based on the system architecture shown in Figure 30. As shown in Figure 30, the split screen application (Split Screen App) can be a YouTube application, a Prime Video application, a Miracast application, and a LiveTV application. FocusWindow is a focus frame dynamically added to the split screen focus window. In the split screen state, when the split screen application such as YouTube and Prime Video obtains the focus, it will be attached above the focus window. The size and position are consistent with the focus window. TaskCheckListener monitors the state changes of the split screen focus window. When the user changes the size or position of the focus window through the remote control or voice operation, FocusWindow will also update the size or position with the focus window. TaskStackListener monitors the state changes of the application stack. When the split screen application stack changes, it responds in time to control the display and hiding of FocusWindow.

[0392] Based on the above, you can quickly enter the split-screen mode by sending a broadcast to systemUI. After sending the split-screen broadcast, SystemUI will call the moveToSideStage method of SplitScreenController to pull the split-screen application into the split-screen mode. ActivityManager provides a method to obtain the system's currently running task (task), encapsulates it into the running task information (RunningTaskInfo) and returns it to the caller. RunningTaskInfo inherits from the task information (TaskInfo) class, which provides a method to obtain the current task window mode "getWindowingMode". This method can be used to obtain whether the current task is in multi-window mode. By judging whether the task of the current split-screen application is in multi-window mode, when it is determined to enter split-screen mode, the focus frame is added to the default focus window.

[0393] To add a focus window, you can use the addView method of WindowManager, which accepts two parameters (View view, WindowManager.LayoutParams params). The first parameter is View, which is the view to be added, that is, the focus frame itself, and the second parameter is layout parameters (LayoutParams), which are the configuration of the focus frame, such as size, position, type, window hierarchy, etc., which can be obtained through WindowManager.

[0394] In some embodiments, the LayoutParams of the focus frame can be obtained based on the coordinate system. Take the first window and the second window as arranged left and right, wherein the first window is on the left and the second window is on the right, and the preset split screen ratio is 1:1 as an example. A coordinate system is established with the lower left corner vertex of the first window as the origin, the lower border of the display area of ​​the display as the x-axis, and the left border of the display area of ​​the display as the y-axis. After obtaining the width (mScreenWidth) and height (mScreenHeight) of the display area of ​​the display, the width (width) and height (height) of the focus frame can be calculated, as well as the position of the focus frame.

[0395] If the focus window is the first window, the LayoutParams of the focus window may be expressed as follows: MLayoutParams.width=mScreenWidth / 2; MLayoutParams.height=mScreenHeight; MLayoutParams.x=0; MLayoutParams.y=0.

[0396] If the focus window is the second window, the LayoutParams of the focus window may be expressed as follows: MLayoutParams.width=mScreenWidth / 2; MLayoutParams.height=mScreenHeight; MLayoutParams.x=mScreenWidth / 2; MLayoutParams.y=0.

[0397] In some embodiments, after displaying the first application and the second application in split screen, the display device 200 may switch to displaying one of the applications as follows:

[0398] In response to the movement instruction input by the user, the display device 200 moves the focus to the target window and displays a focus frame on the upper layer of the target window.

[0399] The application displayed in the target window is the target application that needs to be replaced.

[0400] The display device 200 displays the split-screen application list in the target window in response to the split-screen instruction input by the user.

[0401] The split-screen application list includes items of split-screen applications that can be displayed in split-screen with a non-target application.

[0402] The display device 200 exits the target application in response to a selection instruction input by the user based on the item of the third application in the split screen application, and displays the third application in the target window.

[0403] In this way, the user can change any application in the applications currently displayed on the split screen.

[0404] Steps S2401-S2403 are explained with reference to the following examples.

[0405] Example 1

[0406] For example, the display device 200 is a television, the control device 100 is a remote controller, and the first application is a YouTube application.

[0407] As shown in Figure 31 (1), the TV displays the first screen of the YouTube app in first window 3101. This first screen is a screen of Movie A. The user inputs a split-screen command to the TV using the split-screen button on the remote control. In response to this command, as shown in Figure 31 (2), the TV displays a split-screen application list 3102 on the screen of Movie A. This split-screen application list 3102 includes items for split-screen display with the first application, such as App1, App2, and the Prime Video app. For example, if the second application is the Prime Video app, the user inputs a selection command based on the Prime Video app item. In response to this selection command, as shown in Figure 31 (3), the TV adds a second window 3103 to the right of the first window 3101 and displays the second screen of the Prime Video app in second window 3103. This second screen is a screen of Movie B. This achieves the effect of split-screen display of the YouTube app and the Prime Video app. If the TV determines that the first window 3101 is the focused window, it adds a focus frame 3104 above the first window 3101, as shown in Figure 31 (4). Among them, focus frame 3104 is a frame-shaped window, and area 3105 is hollow. The display level of focus frame 3104 is higher than the display level of first window 3101. The size of focus frame 3104 matches the size of first window 3101. When focus frame 3104 is displayed on first window 3101, the border of first window 3101 is covered by focus frame 3104, and the image of movie A displayed in first window 3101 can be presented to the user through area 3105. Based on the split-screen interface shown in Figure 31 (4), the user can quickly determine that the focus window is first window 3101 based on focus frame 3104.

[0408] Example 2

[0409] Take the display device 200 as a television, the control device 100 as a remote controller, and the first application as a fitness application as an example.

[0410] As shown in Figure 32 (1), the TV displays the first screen of a fitness app in first window 3201. This first screen is a picture of a fitness video. The user inputs a split-screen command to the TV using the split-screen button on the remote control. In response to this command, the TV determines that the second app associated with the fitness app is a camera app. As shown in Figure 32 (2), the TV adds a second window 3202 to the right of first window 3201. After turning on the camera, the TV displays the second screen of the camera app in second window 3202. This second screen is the picture captured by the TV's camera. Thus, by displaying the fitness app and camera app in split screen, the user's movements can be corrected while exercising. If the TV determines that second window 3202 is the focus window, it adds a focus frame 3203 above second window 3202, as shown in Figure 32 (3). Focus frame 3203 is a frame-shaped window with a hollow area 3204. The display level of focus frame 3203 is higher than that of second window 3202. The size of focus frame 3203 matches the size of second window 3202. When focus frame 3203 is displayed on second window 3202, the border of second window 3202 is covered by focus frame 3203, and the image captured by the TV camera displayed in second window 3202 can be presented to the user through area 3204. Based on the split-screen interface shown in ③ of Figure 32, the user can quickly determine that the focus window is second window 3202 based on focus frame 3203.

[0411] Example 3

[0412] For example, the display device 200 is a television, the control device 100 is a remote controller, and the first application is a YouTube application.

[0413] When using the YouTube app, the user can send a split-screen command to the TV through the split-screen button on the remote control. The TV will respond to the split-screen command and display a split-screen application list. The split-screen application list includes items of applications that can be displayed split-screen with the YouTube app, such as the YouTube app. If the user selects the item of the Prime Video app, as shown in ① in Figure 33, the TV will display the first screen of the YouTube app, such as movie A, in the first window 3301, and the second screen of the Prime Video app, such as movie B, in the second window 3302. And, if the TV determines that the first window 3301 is the focus window, a focus frame 3303 is displayed on the upper layer of the first window 3301. The above process of displaying the YouTube app and the Prime Video app in split screen is similar to Example 1 and will not be repeated here.

[0414] When the YouTube app and Prime Video app are displayed in split-screen mode, if the user wishes to change the split-screen app, they can use the remote control to move the focus to the window containing the app they wish to change. For example, if the user wishes to change the Prime Video app, they can use the remote control to move the focus to second window 3302. At this point, as shown in FIG33 (2), a focus frame 3304 will be updated and displayed on second window 3302, indicating that second window 3302 is the focused window. While the focus is on second window 3302, the user can use the split-screen button on the remote control to send a split-screen command to the TV again. In response to this split-screen command, as shown in FIG33 (3), the TV displays a split-screen application list 3304 on second window 3302. This list includes items for split-screen apps that can be displayed split-screen with the YouTube app. To prevent invalid user selections, this list does not include the currently displayed Prime Video app but includes items for other split-screen apps, such as App1 and App2. The focus will be displayed on the first item in the split-screen application list 3304 by default, and the user can control the focus to move on each item by using the direction buttons on the remote control. If the user wants to display the App2 application in split screen with the YouTube application, the right button on the remote control can be used, as shown in ③ in Figure 33, to move the focus (shown in a thick box) to the item of the App2 application, and send a confirmation instruction to the TV through the confirmation button on the remote control. In response to the confirmation instruction, the TV will exit the Prime Video application and display the screen of the App2 application in the second window 3302, such as a chat interface, as shown in ④ in Figure 33. At this time, the focus frame 3303 is still displayed on the upper layer of the second window 3302 to indicate to the user that the focus window is the second window 3302.

[0415] Example 4

[0416] Take the display device 200 as a television, the control device 100 as a remote controller, and the first application as a fitness application as an example.

[0417] When a user is using a fitness app, they can send a split-screen command to the TV via the split-screen button on the remote control. The TV will respond to the split-screen command, determine the camera app associated with the fitness app, and display the first screen of the fitness app, such as the screen of a fitness video, in a first window 3401, as shown in Example 1 in FIG34, and display the second screen of the camera app, such as the captured screen, in a second window 3402. Furthermore, if the TV determines that the second window 3402 is the focus window, a focus frame 3403 is added to the upper layer of the second window 3402. The above process of displaying the fitness app and the camera app in split screen is similar to that in Example 2 and will not be repeated here.

[0418] When the Fitness app and the Camera app are displayed in split-screen mode, if the user wishes to change the app displayed split-screen with the Fitness app, they can use the remote control to move the focus to second window 3402. In this example, since the focus is already displayed in second window 3402, the user does not need to perform the additional operation of moving the focus to second window 3402. While the focus is displayed in second window 3402, the user can use the split-screen button on the remote control to send a split-screen instruction to the TV again. In response to the split-screen instruction, as shown in FIG. 34 (2), the TV displays a split-screen application list 3404 in second window 3402. This split-screen application list 3404 includes items for split-screen applications that can be displayed split-screen with the Fitness app, as well as items for programs from different signal sources corresponding to the Fitness app, such as Live TV and App3. The focus is displayed on the first item in split-screen application list 3404 by default, and the user can use the directional buttons on the remote control to control the focus between items. If the user wishes to display live TV in a split-screen format alongside the fitness app, as shown in step 2 of FIG34 , since the focus (indicated by a bold outline) is by default on the live TV item, the user can send a confirmation command to the TV using the confirmation button on the remote control. In response to this confirmation command, the TV will exit the camera app and display the live TV content, such as Program 1, in second window 3402, as shown in step 3 of FIG34 . At this point, focus frame 3403 remains displayed above second window 3402, indicating to the user that second window 3402 is the focused window.

[0419] To enhance app and system platform security, untrusted touch events have been added. This prevents touch events from being passed down when the touch window is different from the underlying app. Consequently, when a user uses touch and mouse to operate an application within the focus window under the focus frame, the application becomes unresponsive. This means that the user can operate the application within the focus window under the focus frame using the remote control, but cannot operate it with the mouse, resulting in a peripheral malfunction.

[0420] The display device 200 can respond to the mouse operation according to the process shown in FIG35 , and the specific steps are as follows:

[0421] S3501, setting the focus frame to be trusted by the system.

[0422] In order to prevent the focus frame from affecting mouse operations, it is necessary to set the system to trust the window where the focus frame is located. Because the focus frame and the application in the underlying focus window do not belong to the same application, it is necessary to modify the properties of the focus frame. By analyzing and experimenting with Android 12's restrictions on untrusted touch events, a set of methods that call the setTrustedOverlay method of WindowManager.LayoutParams through the reflection mechanism is designed to bypass this restriction of the focus window. You can refer to the following code:

[0423] S3502 : In response to an operation instruction input by the user to the application in the focus window based on the focus frame, a touch event corresponding to the operation instruction is transmitted to the application in the focus window.

[0424] With the above settings, when adding a focus frame and initializing LayoutParams, you can set trustedOverLay by directly setting the private flag (privateFlags) so that the system trusts the added focus frame and allows touch events to be passed down, thus solving the problem of the application in the focus window not being able to respond to mouse operations.

[0425] The user can zoom in or out of the focus window. The display device 200 can zoom in or out of the focus window according to the process shown in FIG. 36 . The specific steps are as follows:

[0426] S3601, in response to a zoom instruction input by the user, determining a target distance between a junction of a first window and a second window and a target border according to a size of a display area, a preset split-screen ratio, and a size that changes in a zooming direction.

[0427] If the first window and the second window are arranged side by side, the zooming direction is horizontal. If the first window and the second window are arranged top by top, the zooming direction is vertical.

[0428] The target border is the first border in the display area that is perpendicular to the scaling direction. For example, if the first and second windows are arranged side by side, the target border is the left border of the display area. If the first and second windows are arranged top by top, the target border is the top border of the display area.

[0429] There is a threshold for the size change in the zoom direction. This threshold is the distance between the boundary between the first and second windows and the border of the display area perpendicular to the zoom direction. For example, if the zoom direction is horizontal to the right, the threshold is the distance between the boundary between the first and second windows and the right border of the display area.

[0430] S3602: Determine the changed positions and sizes of the first window and the second window according to the size of the display area and the target distance, and determine the changed position and size of the focus frame according to the changed position and size of the focus window.

[0431] For example, if the first window and the second window are arranged left and right, with the first window on the left and the second window on the right, the second window is the focus window. The focus window is currently located in the area starting at the target distance from the left border of the display area, and its width is equal to the target distance, and its height is equal to the height of the display area. The width of the second window is equal to the difference between the width of the display area and the target distance, and its height is equal to the height of the display area.

[0432] Based on the changed positions and sizes of the first window and the second window, the changed position and size of the focus window can be determined, and the changed position and size of the focus window can be directly used as the changed position and size of the focus frame.

[0433] S3603: Display the changed first window and the second window according to the changed positions and sizes of the first window and the second window, and display the changed focus frame according to the changed position and size of the focus frame.

[0434] Display device 200 displays the changed first and second windows based on the changed positions and sizes of the first and second windows, and displays the changed focus frame based on the changed position and size of the focus frame. At this point, the changed focus frame can still be accurately displayed above the changed focus window, and its size matches the changed focus window. This allows the focus window to be zoomed in and out based on the zoom buttons within the focus window, and the focus frame to be dynamically adjusted based on the zoom changes of the focus window.

[0435] As shown in FIG37 ①, the display device 200 displays a first window 3701 and a second window 3702 in a split screen, wherein the second window 3702 is the focus window, and a focus frame 3703 is displayed on the second window 3702. The user can input a menu instruction to the display device 200 via the menu key in the control device 100. In response to the menu instruction, the display device 200 displays a control menu in the focus window (i.e., the second window 3702). As shown in FIG37 ①, the control menu includes a zoom-out button 3704 and an enlargement button 3705. The user can input a zoom-out instruction to the display device 200 using the zoom-out button 3704. In response to the zoom-out instruction, the display device 200 zooms out the second window 3702 and enlarges the first window 3701, as shown in FIG37 ②. The horizontal sizes of the first window 3701 and the second window 3702 change. Correspondingly, based on the changed position and size of the second window 3702, the size of the focus frame 3703 is also reduced accordingly, and the reduction direction is horizontally to the right. The reduced size matches the size of the reduced second window 3702, and the reduced focus frame 3703 is still displayed on the reduced second window 3702.

[0436] The user can input a zoom-in instruction to the display device 200 using the zoom-in button 3705. In response to the zoom-in instruction, the display device 200 enlarges the second window 3702 and reduces the first window 3701. The horizontal sizes of the first window 3701 and the second window 3702 change. Accordingly, based on the changed position and size of the second window 3702, the size of the focus frame 3703 is also enlarged horizontally to the left. The enlarged size matches the size of the enlarged second window 3702, and the enlarged focus frame 3703 remains displayed on the enlarged second window 3702.

[0437] In some embodiments, the updated LayoutParams of the focus frame can be obtained based on the coordinate system. Take the first window and the second window as an example, where the first window is on the left and the second window is on the right. A coordinate system is established with the lower left corner vertex of the first window as the origin, the lower border of the display area of ​​the display as the x-axis, and the left border of the display area of ​​the display as the y-axis. If the preset split-screen ratio is 1:1, the junction of the first window and the second window is the center line position of the display area. Assuming that the threshold in the zoom direction is x, if the user inputs a zoom-in command, the horizontal coordinate (target distance) of the junction is calculated as: Mx = MscreenWidth / 2+x. If the user inputs a zoom-out command, the horizontal coordinate (target distance) of the junction is calculated as: Mx = MscreenWidth / 2-x. After calculating Mx, the split-screen application will notify SystemUI to call the system SplitScreenController's ChangeBound method to change the horizontal coordinates of the first window and the second window to display the scaled first window and the second window.

[0438] When a split-screen app notifies systemUI to call the changeBound method to change the size of the first and second windows, it passes the horizontal coordinate parameter Mx of the intersection. The app can use this parameter to update its own focus frame. For example, if the user enters a zoom-out command and the focus window is the second window on the right, the updated focus frame parameters are as follows: MLayoutParams.width = mScreenWidth - Mx; MLayoutParams.height = mScreenHeight; MLayoutParams.x = Mx; MLayoutParams.y = 0;

[0439] You can use the updateViewLayout method of WindowManager to update the size of the focus frame, that is, WindowManager.updateViewLayout(focusWindow,MLayoutParams).

[0440] The display device 200 can scale the focus window according to the process shown in FIG38 . The specific steps are as follows:

[0441] S3801: Display a separator at the boundary between the first window and the second window.

[0442] After splitting the screen, the Android system can automatically add a divider view (DividerView) at the intersection of the first and second windows. This divider can be in the form of a line, segment, dot, arrow, etc. This divider is used to indicate the intersection of the first and second windows.

[0443] As shown in ① in FIG39 , a segment-shaped separator 3903 is displayed at the junction of the first window 3901 and the second window 3902. If the first window 3901 is the focus window, a focus frame 3904 is also displayed on the first window 3901.

[0444] S3802 : In response to a movement instruction input by the user based on the spacer, move the spacer and obtain a position of the spacer after the move within the display area.

[0445] The user can move the spacer left or right by operating the mouse. For example, after selecting the spacer by operating the mouse, the user slides the mouse to the right to input a movement instruction to the display device 200, where the movement instruction instructs to move the spacer to the right.

[0446] The display device 200 displays the moving track of the spacer in response to the moving instruction, and obtains the position of the spacer within the display area when the spacer reaches the position after the movement indicated by the user.

[0447] S3803, determining the changed positions and sizes of the first window and the second window according to the position of the separator after it moves in the display area, and determining the changed position and size of the focus frame according to the changed position and size of the focus window.

[0448] The separator is used to indicate the boundary between the first window and the second window. Therefore, the position of the separator after it moves within the display area corresponds to the position of the boundary between the first window and the second window after the change. Therefore, the display area can be re-divided based on the position of the separator after it moves within the display area, with the re-divided display areas corresponding to the changed first window and the second window, respectively. The process of determining the changed positions and sizes of the first window and the second window based on the position of the separator after it moves within the display area is equivalent to the process of determining the changed positions and sizes of the first window and the second window based on the changed boundary between the first window and the second window.

[0449] In some embodiments, a target distance between the boundary between the first and second windows after the change and a target border can be calculated. The target border is the first border in the display area perpendicular to the direction of movement of the separator. The positions and sizes of the first and second windows after the change are then determined based on the target distance and the size of the display area. The above process can be referred to steps S3601-S3602 and will not be further described here.

[0450] Based on the changed positions and sizes of the first window and the second window, the changed position and size of the focus window can be determined, and the changed position and size of the focus window can be directly used as the changed position and size of the focus frame.

[0451] S3804: Display the changed first window and the second window according to the changed positions and sizes of the first window and the second window, and display the changed focus frame according to the changed position and size of the focus frame.

[0452] Display device 200 displays the changed first and second windows based on the changed positions and sizes of the first and second windows, and displays the changed focus frame based on the changed position and size of the focus frame. At this point, the changed focus frame can still be accurately displayed above the changed focus window, and its size matches the changed focus window. This allows for the effect of scaling the focus window based on the spacer, and dynamically adjusting the focus frame based on the scaling of the focus window.

[0453] Take the example of a user controlling the movement of the separator 3903 by the mouse. If the user wants to expand the first window 3901, the movement instruction input by the user is to instruct the separator 3903 to move to the right. As shown in ② in Figure 39, after the separator 3903 moves a distance a, the first window 3901 and the second window 3902 are adjusted based on the moved separator 3903, wherein the first window 3901 is equivalent to expanding a to the right in the horizontal direction, while the second window 3902 is equivalent to shrinking a to the right in the horizontal direction. The focus frame 3904 changes accordingly according to the changed position and size of the first window 3901. As shown in ② in Figure 39, the changed focus frame 3904 is still displayed on the changed first window 3901, and its size matches the changed first window 3901.

[0454] In this embodiment of the present application, the touch monitoring callback onTouch method can be rewritten by DividerView. When the user slides the DividerView with the mouse, the callback method will be triggered. When the sliding is detected, DividerView will calculate the sliding distance:

[0455] final int position=mSplitLayout.getDividePosition()+touchPos-mStartPos

[0456] DividerView will then notify the system to update the sizes of the first and second windows using the window container update (WindowContainerTransaction) method.

[0457] In the onTouch callback method of DividerView, after identifying the touch event as a motion event MotionEvent.ACTION_MOVE, the sliding distance is calculated, and then a notification is sent to the split-screen application, and the calculated sliding distance is passed at the same time.

[0458] After the split-screen application obtains the calculated value, it also uses WindowManager to directly update the size of the focus frame.

[0459] After the first window and the second window are displayed in split screen mode, the user may move the focus from the current focused window to another window by clicking an application button, using voice, or using a peripheral device, i.e., switching the focused window, so as to operate the application in the switched focused window. The display device 200 can periodically detect the application stack and promptly add a focus frame to the window that actually occupies the focus.

[0460] After displaying the first and second windows in split-screen mode, display device 200 creates a process for periodically checking application stacks corresponding to the first and second applications currently displayed in split-screen mode. Display device 200 can manage each application stack in the form of an application stack list, where the first application stack is the application stack corresponding to the application that actually occupies focus, i.e., the application stack corresponding to the application in the actual focus window.

[0461] The display device 200 can perform a timed detection on the application stack according to the process shown in Figure 40. The display device 200 can control the period of the timed detection of the application stack through a timer. As shown in Figure 40, the display device 200 specifically performs the following steps:

[0462] S4001: Start the timing detection application stack.

[0463] After starting the timed detection of the application stack, the application stack is detected according to a preset period. During the detection process, the display device 200 executes step S4002.

[0464] S4002: Obtain an application stack list.

[0465] S4003 , obtaining the first application stack in the application stack list, where the first application stack indicates the application that currently actually occupies the focus, that is, the application in the focus window of which the focus frame should be displayed.

[0466] S4004, determine whether the application indicated by the first application stack is consistent with the application in the window where the focus frame is currently located. If they are consistent, it means that the window where the focus frame is located is the application used to display the application indicated by the first application stack, and return to execute step S4002; if they are inconsistent, it means that the window where the focus frame is located is not used to display the application indicated by the first application stack, that is, the focus frame is not displayed on the window where the application indicated by the first application stack is located, and there is an error in the position where the focus frame is displayed, and execute step S4005.

[0467] S4005, update the focus frame.

[0468] Specifically, the focus frame is updated and displayed on the window where the application indicated by the first application stack is located.

[0469] Based on the above process, it can be ensured that the display device 200 updates the focus frame in a timely manner and displays it on the window where the application actually occupies the focus, so as to accurately indicate the focus window to the user.

[0470] The process of updating the displayed focus frame can be implemented by TaskCheckListener in the system architecture diagram shown in Figure 30. The ActivityManager provides the getRunningTasks method, which accepts a non-parameter integer value and returns a RunningTaskInfo list representing the running tasks. The first item in this list is the application occupying focus. TaskCheckListener starts a scheduled task, compares the first item with the application currently in focus, and determines whether to update the focus frame position.

[0471] Based on the process shown in FIG40 , if the user switches the focus window, the display device 200 may switch the focus window according to the process shown in FIG41 . The specific steps are as follows:

[0472] S4101 , in response to a focus window switching instruction input by a user, displaying the focus in the switched focus window, and modifying the first application stack in the application stack list to be the application displayed in the switched focus window.

[0473] As shown in FIG42 ①, the display device 200 displays a first window 4201 and a second window 4202 in a split screen. The first window 4201 displays the screen of the first application, and the second window 4202 displays the screen of the second application. The focus window is the first window 4201, and the focus 4203 is displayed in the first window 4201. A focus frame 4204 is displayed above the first window 4201. The user can input a switching instruction using the mouse to instruct the focus to move to the second window 4202. In response to the switching instruction, as shown in FIG42 ②, the display device 200 displays the focus 4203 in the second window 4202 and modifies the first application stack in the application stack list to the application stack corresponding to the second application, indicating that the application actually occupying the focus is the second application.

[0474] S4102: Detect, according to a preset period, whether the first application stack in the application stack list and the focus window where the focus frame is located correspond to the same application.

[0475] After displaying the first window 4201 and the second window 4202 in split-screen mode, the display device 200 initiates a timed detection application stack process. Based on the timed detection application stack process shown in FIG40 , the display device 200 detects whether the focus frame needs to be updated according to a preset period. The detection process can be found in the description of the process shown in FIG40 above and will not be further described here.

[0476] S4103: If the applications correspond to different ones, the focus frame is displayed on the window where the application corresponding to the first application stack is located.

[0477] Based on this detection process, the display device 200 can detect that the first application stack (i.e., the second application) and the application in the first window 4201 where the focus frame 4204 is currently located (i.e., the first application) are not the same application, and determines that the display focus frame 4204 needs to be updated, that is, as shown in ③ in Figure 42, the focus frame 4204 is displayed on the second window 4202.

[0478] When the focus moves to the other window, after determining the updated layout parameters of the focus frame, the layout parameters of the focus frame can be updated by calling the updateViewLayout method of WindowManager, for example: WindowManager.updateViewLayout(focusWindow, MLayoutParams), and then the focus frame is updated based on the updated layout parameters.

[0479] When an information prompt box appears in the application in the focus window, the display device 200 can display the message prompt box according to the process shown in FIG43 . The specific steps are as follows:

[0480] S4301: Monitor display messages of the application in the focus window.

[0481] The application's display message is used to indicate the display of an information prompt box. The information prompt box can be a system pop-up window, such as a toast.

[0482] S4302: If a display message is detected, the information prompt box is displayed on the upper layer of the focus frame.

[0483] If the display device 200 monitors the display message, the display device 200 displays the information prompt box in the manner shown in Figure 44. As shown in Figure 44, take the example of gradually increasing the display level from bottom to top. The display device 200 displays the first window 4401 and the second window 4402 in a split screen at a lower display level, wherein the screen of the first application is displayed in the first window 4401, and the screen of the second application is displayed in the second window 4402. The focus frame 4403 is displayed at a higher display level, and the position and size of the focus frame 4403 match the first window 4401. If the display device 200 monitors the message that the first application has a display pop-up window 4404, the pop-up window 4404 is displayed at the highest display level. As a result, the displayed pop-up window 4404 will not be blocked or affected by the focus frame 4403.

[0484] According to a computer-readable non-volatile storage medium according to an embodiment of the present application, a computer program is stored thereon. When the computer program is executed by a processor, the various processes of any one of the above-mentioned processing methods for a display device are implemented, and the same technical effects can be achieved. To avoid repetition, they are not described here.

[0485] The computer-readable non-volatile storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0486] A computer program product according to an embodiment of the present application includes: a computer program, which, when executed on a computer, enables the computer to implement any one of the aforementioned processing methods for a display device.

[0487] For ease of explanation, the above description has been made in conjunction with specific embodiments. However, the above discussion of some embodiments is not intended to be exhaustive or to limit the embodiments to the specific forms of the above applications. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are intended to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A display device, comprising: a display configured to display an image and / or a user interface; A user interface configured to receive instructions from a user; a communication device configured to communicate with an external device according to a predetermined protocol; a memory configured to store computer instructions and data associated with the display device; At least one processor, connected to the display, the user interface, the communication device and the memory, is configured to execute the computer instructions to cause the display device to perform: When it is detected that a video playback application is playing a target video on the current display device, a user prompt interface is displayed, wherein the user prompt interface is used to prompt the user whether to enter a split-screen working mode; When receiving an instruction to enter a split-screen working mode input by the user on the user prompt interface, dividing the display interface of the display device into a first split screen and a second split screen, and starting a search application; The interface of the video playback application is displayed in the first split screen, and the interface of the search application is displayed in full screen in the second split screen.

2. The display device according to claim 1, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: Obtaining a preset display interface size of the video playback application and a first display interface size of the first split screen, wherein: The preset display interface is an interface for displaying the target video; Based on the preset display interface size and the first display interface size, determining a target display interface size for displaying the video playback application in the first split screen; Based on the target display interface size, the interface of the video playback application is displayed in the first split screen.

3. The display device according to claim 2, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: Determining a target ratio according to the preset display interface size; A target display interface size for displaying the video playback application in the first split screen is determined according to the first display interface size and the target ratio.

4. The display device according to claim 1, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: Determining a size of a second display interface of the second split screen; Based on the size of the second display interface, the interface of the search application is displayed in full screen in the second split screen.

5. The display device according to claim 1, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: In response to a search instruction input by a user, displaying a keyword menu; When a keyword selection instruction input by the user in the keyword menu is received, the search application is called to obtain the media asset information of the target video.

6. The display device according to claim 5, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: In response to a search instruction input by a user, acquiring attribute information of the target video; Based on the attribute information, the keyword menu is generated.

7. The display device according to claim 1, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: When an instruction input by the user to start a preset application is received, the split-screen working mode is exited and the preset application is displayed on the display interface.

8. The display device according to claim 1, further comprising: A front camera, configured to capture the environment on the side of the display surface of the display and generate a local video in an activated state; The at least one processor is further configured to execute the computer instructions to cause the display device to perform: In response to receiving a menu pop-up instruction, determining whether the current video played by the display is a fitness teaching video, and whether the display plays the current video in full screen mode; In a case where the current video is a fitness teaching video and the display plays the current video in full screen mode, controlling the display to display a first pop-up menu including a split-screen display option; In response to receiving a control instruction that triggers the split-screen display option, controlling the front camera to start and receive the local video generated by the front camera; The display is controlled to split into two display windows, and the current video and the local video are played in the two display windows respectively.

9. The display device according to claim 8, after the at least one processor controls the display to play the current video and the local video in two display windows respectively, the at least one processor is further configured to execute the computer instructions to enable the display device to execute: In response to receiving the menu pop-up instruction again, controlling the display to display a second pop-up menu including a mirroring option in the first display window playing the local video; In response to receiving a control instruction for triggering the mirror option, the local video is horizontally mirror-flipped, and the display is controlled to play the local video that has been horizontally mirror-flipped in the first display window.

10. The display device according to claim 9, when the at least one processor controls the display to split into two display windows, the at least one processor is further configured to execute the computer instructions to enable the display device to execute: Controlling the display to split the screen into two display windows in a manner of equally dividing the left and right parts; When the at least one processor controls the display to display a second pop-up menu including a mirroring option in the first display window playing the local video, the at least one processor is specifically configured to execute the computer instructions to enable the display device to: Obtaining the window width of the display, the preset margin width, the option margin width, the number of options in the second pop-up menu, and the option width; Determine the horizontal coordinates of the reference point of the second pop-up menu according to the window width, the edge blank width, the option blank width, the number of options in the second pop-up menu, and the option width, where the reference horizontal coordinates are coordinates in the global coordinate system of the display; The display is controlled to display a second pop-up menu including the mirroring option according to the horizontal coordinate of the reference point, so as to display the pop-up menu in the first display window.

11. The display device according to claim 9, wherein when the at least one processor controls the player to display a second pop-up menu including a mirroring option in the first display window playing the local video, the second pop-up menu also displays a focus window switching option; The at least one processor is further configured to execute the computer instructions to cause the display device to perform: In response to receiving a control instruction that triggers the focus window switching option, the display is controlled to switch the second display window playing the current video to the focus window, and the display is controlled to display a third pop-up menu including a full-screen playback option in the second display window.

12. The display device according to claim 11, after controlling the display to display a third pop-up menu including a full-screen play option in the second display window, the at least one processor is further configured to execute the computer instructions to enable the display device to execute: In response to receiving a control instruction that triggers the full-screen play option, the display is controlled to resume playing the current video in full-screen mode.

13. The display device according to claim 9, further comprising a speaker; When the at least one processor controls the player to display a second pop-up menu including a mirroring option in the first display window playing the local video, the second pop-up menu also displays a full-screen display option; The at least one processor is further configured to execute the computer instructions to cause the display device to perform: In response to receiving a control instruction to trigger the full-screen playback option, controlling the display to play the video in full-screen mode. The local video is played, and the speaker is controlled to play the audio corresponding to the current video at the same time.

14. The display device according to claim 8, after the at least one processor controls the display to play the current video and the local video in the two display windows respectively, the at least one processor is further configured to execute the computer instructions to enable the display device to execute: In response to receiving the menu pop-up instruction again, controlling the display to display a third pop-up menu including a full-screen play option and a focus window switching option in the second display window playing the current video; In response to receiving a control instruction for triggering the full-screen play option, controlling the display to resume playing the current video in full-screen mode; In response to receiving a control instruction that triggers the focus switching option, the display is controlled to switch the first display window playing the local video to a focus window, and the display is controlled to display a second pop-up menu including a mirroring option and an option to exit full-screen playback in the first display window.

15. The display device according to any one of claims 8 to 10, wherein in the step of the at least one processor controlling the two display windows to respectively play the current video and the local video, the at least one processor is further configured to execute the computer instruction to enable the display device to execute: According to the sizes of the two display windows, respectively intercepting the corresponding current video and the local video to obtain the intercepted current video and the intercepted local video; The intercepted current video and the intercepted local video are played in the two display windows in full-window playback mode respectively.

16. The display device according to claim 1, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: In response to a first selection input for a first application and a second application received through the user interface, controlling the display to display a selection result of the first application and a selection result of the second application, and displaying a window arrangement mode selection control and a split-screen control, wherein the first application and the second application support split-screen display; In response to a second selection input for a window arrangement mode selection control received through the user interface, determining an arrangement mode of the first split screen area and the second split screen area; In response to a first control input for the split-screen control received through the user interface, the display is controlled to display the interface of the first application in the first split-screen area and to display the interface of the second application in the second split-screen area.

17. The display device of claim 16, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: Control the display to display an application selection interface, a first focus frame and / or a second focus frame, wherein the application selection interface is used to display the application identifiers of one or more applications that support split screen, the first focus frame is used to indicate the application identifier of the first application preselected by the current user, and the second focus frame is used to indicate the application identifier of the second application preselected by the current user.

18. The display device of claim 17, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: In response to a first selection input for a first application and a second application received through the user interface, move the first focus frame to a location where an application identifier of the first application is located, and move the second focus frame to a location where an application identifier of the second application is located; The color of the first focus frame and / or the second focus frame is changed, the first focus frame after the color change is used to indicate the user's selection result for the first application, and the second focus frame after the color change is used to indicate the user's selection result for the second application.

19. The display device of claim 16, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: Based on the first selection input for the first application and the second application received through the user interface, the display is controlled to show the selection order of the first application and the second application.

20. The display device according to claim 16, wherein the window arrangement mode selection control comprises at least a left-right arrangement control and a top-bottom arrangement control; The at least one processor is further configured to execute the computer instructions to cause the display device to perform: In response to a second selection input for a left-right arrangement control received through the user interface, determining that the first split screen area and the second split screen area are arranged in a left-right split screen manner; or, In response to a second selection input for the top-bottom arrangement control received through the user interface, it is determined that the first split screen area and the second split screen area are arranged in a top-bottom split screen manner.

21. The display device of claim 16, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: The control device controls the display to display the interface of the first application in the first split-screen area, and before displaying the interface of the second application in the second split-screen area, detects the startup process of the first application and the second application, and controls the display to display a preset loading interface in full screen; After determining that the startup processes of the first application and the second application have finished running, controlling the display to hide the preset loading interface.

22. The display device of claim 16, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: Displaying a region control control between the first split-screen region and the second split-screen region; In response to a second control input to the region control widget received through the user interface, the sizes of the first split screen region and the second split screen region are changed.

23. The display device of claim 16, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: In the split-screen display state, receiving a key input from a user through the user interface, and determining whether a key corresponding to the key input exists in a preset key whitelist, wherein the preset key whitelist is used to store all keys that will not be responded to in the split-screen display state; If the key corresponding to the key input does not exist in the preset key whitelist, determining a current focus application, the focus application being the first application or the second application with a higher priority; The key input is processed by the focus application.

24. The display device of claim 16, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: In the split-screen display state, if a third application startup instruction from the user is received through the user interface, the third application startup instruction is shielded.

25. The display device according to claim 1, wherein the display is further configured to display a first screen of the first application in the first window; The at least one processor is further configured to execute the computer instructions to cause the display device to perform: In response to a screen splitting instruction input by a user, controlling the display to add and display a second window, and displaying a first screen of the first application in the first window, and displaying a second screen of the second application in the second window; Determine a focus window that occupies focus between the first window and the second window; The display is controlled to display a focus frame on an upper layer of the focus window, wherein the focus frame is a frame-shaped window, and the position and size of the focus frame match the position and size of the focus window.

26. The display device according to claim 25, wherein the at least one processor controls the display to add and display a second window in response to a split screen instruction input by a user, and displays a first screen of the first application in the first window and a second screen of the second application in the second window, and is configured to execute the computer instructions to enable the display device to execute: In response to the split-screen instruction, controlling the display to display a split-screen application list, the split-screen application list including an item of the second application; In response to a selection instruction input by a user based on an item of the second application, the display is controlled to add the display of the A second window is provided, and a first screen of the first application is displayed in the first window, and a second screen of the second application is displayed in the second window.

27. The display device according to claim 25, wherein the at least one processor controls the display to add and display a second window in response to a split screen instruction input by a user, and displays a first screen of the first application in the first window and a second screen of the second application in the second window, and is configured to execute the computer instructions to enable the display device to execute: In response to the split screen instruction, the second application associated with the first application is acquired, wherein: Displaying the first application and the second application in split screen is used to implement a preset function of the first application; The display is controlled to additionally display the second window, and the first screen of the first application is displayed in the first window, and the second screen of the second application is displayed in the second window.

28. The display device according to claim 1, wherein the at least one processor controls the display to display a focus frame on the border of the focus window, and is configured to execute the computer instructions to cause the display device to perform: Acquire the position of the focus window, the size of the display area of ​​the display, the preset split screen ratio between the first window and the second window, and the arrangement of the first window and the second window; Determining the size of the focus window according to the size of the display area, the preset split-screen ratio, and the arrangement of the first window and the second window; Determining the position of the focus frame according to the position of the focus window, and determining the size of the focus frame according to the size of the focus window; According to the position of the focus frame and the size of the focus frame, the display is controlled to display the focus frame.

29. The display device according to claim 28, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: In response to a zoom instruction input by a user, determining a target distance between a junction of the first window and the second window and a target frame according to a size of the display area, the preset split screen ratio, and a size changed in a zooming direction, wherein the target frame is a first frame in the display area perpendicular to the zooming direction; Determining the changed positions and sizes of the first window and the second window according to the size of the display area and the position of the target from the focus window, and determining the changed position and size of the focus frame according to the changed position and size of the focus window; According to the changed positions and sizes of the first window and the second window, the display is controlled to display the changed first window and the second window, and according to the changed position and size of the focus frame, the display is controlled to display the changed focus frame.

30. The display device according to claim 28, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: Controlling the display to display a separator at the boundary between the first window and the second window; In response to a movement instruction input by a user based on the spacer, move the spacer, and obtain a position of the spacer after the spacer is moved in the display area; Determine the changed positions and sizes of the first window and the second window according to the position of the spacer after it moves in the display area, and determine the changed position and size of the focus frame according to the changed position and size of the focus window; According to the changed positions and sizes of the first window and the second window, the display is controlled to display the changed first window and the second window, and according to the changed position and size of the focus frame, the display is controlled to display the changed focus frame.

31. The display device according to claim 25, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: In response to a switching instruction of the focus window input by a user, the focus is displayed in the switched focus window, and the first application stack in the application stack list is modified to be the application displayed in the switched focus window, wherein: The application stack list includes the first application and the second application; Detecting, according to a preset period, whether the first application stack in the application stack list and the focus window where the focus frame is located correspond to the same application; If they correspond to different applications, the display is controlled to display the focus frame on the window where the application corresponding to the first application stack is located.

32. The display device according to claim 25, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: Setting the focus frame to be trusted by the system; In response to an operation instruction input by a user to an application in the focus window based on the focus frame, the operation instruction is transmitted to the application in the focus window.

33. The display device according to claim 25, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: Monitoring a display message of the application in the focus window, wherein the display message is used to indicate a display information prompt box; If the display message is monitored, the display is controlled to display the information prompt box on the upper layer of the focus frame.

34. A processing method for a display device, comprising: When it is detected that a video playback application is playing a target video on the current display device, a user prompt interface is displayed, wherein the user prompt interface is used to prompt the user whether to enter a split-screen working mode; When receiving an instruction to enter a split-screen working mode input by the user on the user prompt interface, dividing the display interface of the display device into a first split screen and a second split screen, and starting a search application; The interface of the video playback application is displayed in the first split screen, and the interface of the search application is displayed in full screen in the second split screen.

35. A computer-readable non-volatile storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer device executes the steps of the processing method for a display device as claimed in claim 34.

Images