A display device and a mode switching method for the display device.

Abstract

This application discloses a display device and a mode switching method for the display device. The display device may include at least one processor connected to a display, a communication device, and a memory, configured to execute a computer program stored in the memory to cause the display device to perform the following: when a mode switching application detects that a first projection device has been connected, the mode switching application can be controlled to create a first window to display projection data sent by the first projection device. The mode switching application is started after the display device is powered on. In response to a mode switching operation, the mode switching application can be controlled to switch to a target mode indicated by a mode switching command, and the content to be displayed can be displayed according to a first layout data matching the target mode. The content to be displayed includes the projection data sent by the first projection device, and the first layout data is used to indicate the display parameters of each item of content to be displayed, including at least the display size and display position of the first window after reset; thereby improving the mode switching efficiency.

Description

Display device and mode switching method for display device

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent applications filed on March 27, 2024, with application number 202410355943.X; and filed on March 27, 2024, with application number 202410355935.5, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the technical field of display devices, and in particular to a display device and a mode switching method for the display device. Background Art

[0004] If the display device serves as a screen projection receiver, it can display the projection content in different modes. For example, in a multi-channel screen projection scenario, the user can choose a multi-channel screen projection viewing mode, which may include a two-channel screen projection viewing mode, a three-channel screen projection viewing mode or a more multi-channel screen projection viewing mode; if the user is interested in a certain channel of screen projection, the user can switch to the exclusive mode to display the projection content in full screen, thereby monopolizing the screen; if the user selects the floating mode, the display device displays the main screen content in full screen, and the projection content is displayed as a secondary screen floating on the upper layer of the main screen content; if the user selects the conference mode, the display device can display a list of signal sources (presenting physical signal sources and projection signal sources, etc.), and at the same time display the content of the signal source selected by focus.

[0005] When using a display device, users can switch the projection display mode as needed, for example, switching from dual-screen projection viewing mode to exclusive mode, or from exclusive mode to meeting mode. When switching projection display modes, the display device needs to exit the original mode, cancel the interface elements in the original mode, start the new mode, and create interface elements and layouts adapted to the new mode. This mode switching is cumbersome and inefficient, and can easily lead to jerky interface changes when switching modes. Summary of the Invention

[0006] According to some embodiments of the present application, a display device is provided, which may include: a display, which may be configured to display a user interface in different modes; a communication device, which may be configured to communicate with a projection device; a memory, which may be configured to store a computer program; at least one processor, which may run a mode switching application, and the at least one processor is connected to the display, the communication device and the memory, and may be configured to execute the computer program to enable the display device to execute: after the display device is turned on, the mode switching application may be controlled to start; when the mode switching application detects that a first projection device is connected, the mode switching application may be controlled to create a first window, and the display may be controlled to display the first window in the first window. The projection data is sent by the first projection device; in response to the mode switching operation input by the user, the mode switching application can be controlled to switch the currently recorded projection display mode to the target mode indicated by the mode switching instruction; the mode switching application can be controlled to obtain the content to be displayed in the target mode, and the content to be displayed may include the projection data sent by the first projection device; the mode switching application can be controlled to generate first layout data matching the target mode, and the first layout data can be used to indicate the display parameters of each content to be displayed, and the first layout data may at least include the display size and display position of the first window after reset; the display can be controlled to display the content to be displayed according to the first layout data.

[0007] According to some embodiments of the present application, a display device is provided, which may include: a display, which may be configured to display a user interface in different modes; a communication device, which may be configured to communicate with a projection device; a memory, which may be configured to store a computer program; at least one processor, which may run a mode switching application, and the at least one processor is connected to the display, the communication device and the memory, and the at least one processor may be configured to execute the computer program to enable the display device to perform: after the display device is turned on, the mode switching application may be controlled to start; in response to a mode switching instruction indicating switching from a first multi-channel projection mode to a second multi-channel projection mode, the mode switching application may be controlled to switch the currently recorded projection display mode to the second multi-channel projection mode; the mode switching application may be controlled to obtain first projection information and second projection information; the first projection information The information may be the projection information of at least one screen shared by the first multi-channel projection mode and the second multi-channel projection mode, the second projection information may be the projection information of at least one screen deleted or newly added by the second multi-channel projection mode compared with the first multi-channel projection mode, the projection information may include projection device information and projection window information; the mode switching application may be controlled to calculate interface layout data adapted to the second multi-channel projection mode according to the first projection information and the second projection information, the interface layout data may include display parameters after the reset of the previously created shared window indicated by the first projection information, the shared window may be the projection window retained from the first multi-channel projection mode to the second multi-channel projection mode; the display may be controlled to display the projection content of at least one screen contained in the second multi-channel projection mode according to the interface layout data.

[0008] According to some embodiments of the present application, a mode switching method for a display device is also provided, which may include: after the display device is turned on, a mode switching application can be controlled to start; when the mode switching application monitors the access of the first projection device, the mode switching application can be controlled to create a first window, and the projection data sent by the first projection device can be displayed in the first window; in response to the mode switching operation input by the user, the mode switching application can be controlled to switch the currently recorded projection display mode to the target mode indicated by the mode switching instruction; the mode switching application can be controlled to obtain the content to be displayed in the target mode, and the content to be displayed may include the projection data sent by the first projection device; the mode switching application can be controlled to generate first layout data matching the target mode, and the first layout data can be used to indicate the display parameters of each content to be displayed, and the first layout data may at least include the display size and display position of the first window after reset; the content to be displayed can be displayed according to the first layout data.

[0009] According to some embodiments of the present application, a mode switching method for a display device is also provided, which may include: in response to a mode switching instruction indicating switching from a first multi-way screen projection mode to a second multi-way screen projection mode, the mode switching application may be controlled to switch the currently recorded screen projection display mode to the second multi-way screen projection mode; the mode switching application may be controlled to obtain first screen projection information and second screen projection information; the first screen projection information may be screen projection information of at least one screen projection shared by the first multi-way screen projection mode and the second multi-way screen projection mode, and the second screen projection information may be screen projection information of at least one screen projection shared by the first multi-way screen projection mode and the second multi-way screen projection mode, and the second screen projection information may be screen projection information of at least one screen projection deleted by the second multi-way screen projection mode compared to the first multi-way screen projection mode. The projection information of at least one screen projection that is reduced or added may include projection device information and projection window information; the mode switching application may be controlled to calculate interface layout data adapted to the second multi-screen projection mode according to the first projection information and the second projection information, and the interface layout data may include display parameters after the reset of the previously created shared window indicated by the first projection information, and the shared window may be a projection window retained from the first multi-screen projection mode to the second multi-screen projection mode; the projection content of at least one screen projection contained in the second multi-screen projection mode may be displayed according to the interface layout data. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technology descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0011] FIG1 illustrates an operation scenario between a display device and a control apparatus according to some embodiments of the present application;

[0012] FIG2 is a hardware configuration block diagram of a control device according to some embodiments of the present application;

[0013] FIG3 is a hardware configuration block diagram of a display device according to some embodiments of the present application;

[0014] FIG4A is a diagram illustrating software configuration in a display device according to some embodiments of the present application;

[0015] FIG4B is a software architecture diagram of a mode switching application according to some embodiments of the present application;

[0016] FIG5A is a first schematic diagram of trigger mode switching according to some embodiments of the present application;

[0017] FIG5B is a second schematic diagram of trigger mode switching according to some embodiments of the present application;

[0018] FIG6A is a schematic diagram of an interface in a dual-screen projection viewing mode according to some embodiments of the present application;

[0019] FIG6B is a schematic diagram of an interface in a three-way projection viewing mode according to some embodiments of the present application;

[0020] FIG7A is a first schematic diagram of displaying a first prompt pop-up window on a user interface according to some embodiments of the present application;

[0021] FIG7B is a second schematic diagram of displaying a first prompt pop-up window on a user interface according to some embodiments of the present application;

[0022] FIG8A is a first schematic diagram of displaying a second prompt pop-up window on a user interface according to some embodiments of the present application;

[0023] FIG8B is a second schematic diagram of displaying a second prompt pop-up window on the user interface according to some embodiments of the present application;

[0024] FIG9 is a schematic diagram of an interface of a conference mode provided according to some embodiments of the present application;

[0025] FIG10 is a schematic diagram showing a third prompt pop-up window displayed when switching from a three-way projection viewing mode to a dual-way projection viewing mode according to some embodiments of the present application;

[0026] FIG11 is a display interface 1 in a dual-screen projection viewing mode according to some embodiments of the present application;

[0027] FIG12 is a second display interface in a dual-screen projection viewing mode according to some embodiments of the present application;

[0028] FIG13 is a display interface in a three-way projection viewing mode according to some embodiments of the present application;

[0029] FIG14 is a first diagram illustrating interface changes when switching from a three-way projection viewing mode to a dual-way projection viewing mode according to some embodiments of the present application;

[0030] FIG15 is a second schematic diagram of interface changes when switching from a three-way projection viewing mode to a dual-way projection viewing mode according to some embodiments of the present application;

[0031] FIG16 is a display interface in exclusive mode according to some embodiments of the present application;

[0032] FIG17 is a schematic diagram of interface changes when switching from exclusive mode to conference mode according to some embodiments of the present application;

[0033] FIG18 is a first diagram illustrating an interface change from a dual-screen viewing mode to a conference mode according to some embodiments of the present application;

[0034] FIG19 is a second schematic diagram of the interface change from the dual-screen projection viewing mode to the conference mode according to some embodiments of the present application;

[0035] FIG20 is a schematic diagram of an interface change from exclusive mode to floating mode according to some embodiments of the present application;

[0036] FIG21 is a flowchart of a mode switching method for a display device according to some embodiments of the present application;

[0037] Figure 22 is a flowchart of another mode switching method for a display device provided according to some embodiments of the present application. DETAILED DESCRIPTION

[0038] In order to make the purpose and implementation methods of the present application clearer, some implementation methods of the present application will be clearly and completely described below in combination with the drawings in some embodiments of the present application. Obviously, one or more embodiments described are only part of the embodiments of the present application, rather than all the embodiments.

[0039] It should be noted that the brief descriptions of terms in this application are only for the purpose of facilitating the understanding of the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their ordinary and usual meanings.

[0040] In the specification and claims of this application and the accompanying drawings, the terms "first," "second," "third," etc. are used to distinguish similar or similar objects or entities, and are not necessarily intended to limit a particular order or sequence, unless otherwise noted. It should be understood that the terms used in this manner are interchangeable under appropriate circumstances.

[0041] The terms "comprise," "include," and "have," and any variations thereof, are intended to cover but not exclude inclusion; for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0042] The display device provided in the embodiments of the present application can have various implementation forms, such as a television, smart TV, laser projection device, monitor, electronic bulletin board, electronic table, and other terminal devices. Figures 1 and 2 illustrate a specific embodiment of the display device of the present application.

[0043] FIG1 is a schematic diagram of an operation scenario between a display device and a control device according to some embodiments of the present application. As shown in FIG1 , a user can operate a display device 200 through a smart device 300 or a control device 100 .

[0044] In some embodiments, the control device 100 may be a remote controller, and communication between the remote controller and the display device may include infrared protocol communication, Bluetooth protocol communication, or other short-range communication methods, to control the display device 200 wirelessly or wiredly. The user may control the display device 200 by inputting user commands through buttons on the remote controller, voice input, control panel input, etc.

[0045] In some embodiments, a smart device 300 (such as a mobile terminal, tablet computer, computer, laptop computer, etc.) can also be used to control the display device 200. For example, the display device 200 can be controlled using an application running on the smart device.

[0046] In some embodiments, the display device may not use the aforementioned smart device or control device to receive instructions, but may receive user control through touch or gestures.

[0047] In some embodiments, the display device 200 can also be controlled in a manner other than the control device 100 and the smart device 300. For example, the user's voice command control can be received through a module for obtaining voice commands configured inside the display device 200, or the user's voice command control can be received through a voice control device set outside the display device 200.

[0048] In some embodiments, the display device 200 can also communicate data with the server 400. The display device 200 can be connected to a local area network (LAN), a wireless local area network (WLAN), or other networks. The server 400 can provide various content and interactions to the display device 200. The server 400 can be a single cluster or multiple clusters, and can include one or more types of servers.

[0049] Figure 2 is a block diagram of a control device according to some embodiments of the present application. Referring to Figure 2 , the control device 100 may include at least one processor 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 may receive user input commands and convert them into commands that the display device 200 can recognize and respond to, acting as an intermediary for interaction between the user and the display device 200.

[0050] FIG3 is a block diagram of a hardware configuration of a display device according to some embodiments of the present application. Referring to FIG3 , a display device 200 may include at least one of a tuner / demodulator 210, a communication device 220, a detector 230, an external device interface 240, at least one processor 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0051] In some embodiments, the processor may include at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), a RAM (Random Access Memory), a read-only memory (ROM), a first interface to an nth interface for input / output, a communication bus, etc.

[0052] In some embodiments, the display 260 may include a display screen component for displaying images and a driver component for driving the image display. The display 260 may be configured to receive image signals output from at least one processor and display video content, image content, a menu control interface, and a user control UI interface. The display 260 may be a liquid crystal display, an organic light-emitting diode (OLED) display, or a projection display. It may also be a projection device and a projection screen.

[0053] In some embodiments, the communication device 220 is a component that can be used to communicate with external devices or servers using various communication protocols. For example, the communication device can include at least one of a Wi-Fi module, a Bluetooth module, a wired Ethernet module, or other network communication protocol chip or a near-field communication protocol chip, as well as an infrared receiver. The display device 200 can use the communication device 220 to send and receive control signals and data signals with the external control device 100 or the server 400.

[0054] In some embodiments, the user interface may be configured to receive a control signal from the control device 100 (eg, an infrared remote controller, etc.).

[0055] In some embodiments, the external device interface 240 may include, but is not limited to, any one or more of the following: a High Definition Multimedia Interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (Composite Video Broadcast Signal or Composite Video Blanking and Sync, CVBS), a Universal Serial Bus (USB) input interface (USB for short), an RGB port, or the like. It may also be a composite input / output interface formed by multiple of the above interfaces. The external device interface 240 may be used to connect a signal source device (e.g., a set-top box, a TV box, etc.) so that the display device 200 displays the signal source connected through the external device interface 240.

[0056] In some embodiments, the external device interface 240 may include at least one HDMI, which can be used to connect to external signal source devices such as set-top boxes, TV boxes, etc. When the HDMI is connected to the signal source, the at least one processor 250 can control the display 260 to display the video signal received by the HDMI.

[0057] In some embodiments, the tuner-demodulator 210 receives broadcast television signals via wired or wireless reception, and demodulates audio and video signals, such as an Electronic Program Guide (EPG) data signal, from multiple wireless or wired broadcast television signals.

[0058] In some embodiments, at least one processor 250 and the tuner / demodulator 210 may be located in different separate devices, that is, the tuner / demodulator 210 may also be located in an external device of the main device where the at least one processor 250 is located, such as an external set-top box.

[0059] In some embodiments, at least one processor 250 can control the operation of the display device and respond to user operations using various software control programs or computer programs stored in the memory. The at least one processor 250 can control the overall operation of the display device 200. For example, in response to receiving a user command to select a UI object to be displayed on the display 260, the at least one processor 250 can perform operations related to the object selected by the user command.

[0060] The user can input user commands through the Graphic User Interface (GUI) displayed on the display 260, and the user input interface receives the user input commands through the Graphic User Interface (GUI). Alternatively, the user can input user commands by inputting specific sounds or gestures, and the user input interface recognizes the sounds or gestures through sensors to receive the user input commands. "User interface" can be a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The commonly used form of user interface is the Graphic User Interface (GUI), which refers to a user interface related to computer operations that is displayed in a graphical manner. It can be an interface element such as an icon, window, control, etc. displayed on the display screen of an electronic device, where the control can include visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc.

[0061] Figure 4A is a diagram of the software configuration of a display device according to some embodiments of the present application. Referring to Figure 4A , the system can be divided into four layers: from top to bottom, the Applications layer (referred to as the "Application layer"), the Application Framework layer (referred to as the "Framework layer"), the Android Runtime and System Library layer (referred to as the "System Runtime Library layer"), and the Kernel layer.

[0062] In some embodiments, at least one application program runs in the application layer. These applications can be window programs, system settings programs, clock programs, etc. that come with the operating system, or applications developed by third-party developers. The application packages in the application layer are not limited to the above examples.

[0063] In some embodiments, in order to solve the technical problems raised in the background technology of this application, this application provides an application. In the following embodiments, this application is named "mode switching application", but it should be noted that the application name of this application is not limited to the embodiments of this application. Referring to Figure 4A, the mode switching application can run in the application layer. When the display device 200 serves as the screen projection receiving end, in this screen projection scenario, the user can select or switch the screen projection display mode. The screen projection display mode may include but is not limited to: multi-channel screen projection viewing mode, exclusive mode, conference mode, picture-in-picture mode (PictureInPicture, also known as floating mode), split-screen game mode, etc., follow-up mode, etc. According to the number of screen projection channels supported by the display device, the multi-channel screen projection viewing mode can be subdivided into screen projection viewing modes under different channels. For example, if the display device supports up to three-channel screen projection at the same time, the multi-channel screen projection viewing mode can include a two-channel screen projection viewing mode and a three-channel screen projection viewing mode.

[0064] In some embodiments, the user can select or switch the projection display mode as needed, for example, switching from a two-way projection viewing mode to an exclusive mode, or from an exclusive mode to a conference mode, etc. The display device responds to the mode switching operation input by the user and can start the mode switching application and its modules. Through the interaction and information flow between the modules within the mode switching application, the projection display mode can be quickly and smoothly switched to the target mode selected by the user.

[0065] In some embodiments, the user can input a preset operation to call up a mode list, where the preset operation is not limited, for example: the preset operation is to long press the signal source key or other preset button on the control device 100, or the preset operation is a voice operation input by the user, for example, the user can input similar voice information such as "I want to switch to the projection display mode" through the near field / far field or the voice key on the control device 100, so that the display device knows that the user has entered the preset operation by analyzing the intention.

[0066] Figure 5A is a schematic diagram of a mode list provided in accordance with some embodiments of the present application, and Figure 5B is a schematic diagram of a mode list provided in accordance with some embodiments of the present application. Referring to Figures 5A and 5B , the display device, in response to a preset operation input by a user, can control the display to display a mode list 51. Mode list 51 can be displayed in a floating layer at a designated location above the current interface (e.g., at the bottom of the screen).

[0067] In some embodiments, referring to Figures 5A and 5B , the mode list 51 may include at least one mode option 511. The mode option 511 may be set to be in the form of a control that can be selected by focus and triggered. The mode option 511 may display content such as mode category, mode description information, and mode icon (not shown in the figure). Among them, the mode category can be used to enable users to identify different screen projection display modes. The mode category can be presented as a mode name, for example, the mode name is "two-way screen projection viewing"; the mode description information can be used to describe the display form of the mode, and the mode description information corresponding to the exclusive mode is an example of "one-way screen projection full screen exclusive screen"; the mode icon can be used to display the interaction and operation form of the mode, for example, the mode icon in the two-way screen projection viewing mode can be presented as two smartphones projecting to the large screen.

[0068] In some embodiments, the user can select any mode from the mode list 51 according to preferences and needs to trigger the input of the mode switching operation. Continuing to refer to Figures 5A and 5B, the user can adjust the focus position through the direction keys on the control device 100, and when the mode option 511 corresponding to the target mode is selected in focus, click the confirmation key / OK key on the control device 100 to input the mode switching operation. Alternatively, the user can input a voice command indicating the switch to the target mode through near-field voice / far-field voice or the voice key on the control device 100, for example, the user says "switch to two-way projection viewing mode".

[0069] In some embodiments, the display device, in response to a mode switching instruction input by the user, launches a mode switching application and controls the mode switching application to complete the mode switch, i.e., switch to the target mode selected by the user. The mode switching application can be pre-configured with a variety of different mode templates to dynamically update View and UI elements in the interface that change with the mode switch based on the mode categories before and after the switch. This improves mode switching efficiency by enabling fast and smooth switching between different modes within the same application.

[0070] Still as shown in Figure 4A, the framework layer can provide application programming interfaces (APIs) and programming frameworks for applications. The application framework layer may include some predefined functions. The application framework layer is equivalent to a processing center that determines the actions to be taken by applications in the application layer. Applications (which may include mode switching applications) can access resources in the system and obtain system services and required information through API interfaces during execution. For example, mode switching applications can obtain required information (such as the number of projection paths, projection device information, device connection status, etc.) from lower layers (such as the kernel layer) through API interfaces (such as Flow interfaces, etc.).

[0071] As shown in Figure 4A, in some embodiments of the present application, the application framework layer may include managers, content providers, etc., where the manager may include at least one of the following modules: an activity manager can be used to interact with all activities running in the system; a location manager can be used to provide system services or applications with access to system location services; a package manager can be used to retrieve various information related to the application packages currently installed on the device; a notification manager can be used to control the display and clearing of notification messages; and a window manager can be used to manage icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0072] In some embodiments, the activity manager can be used to manage the lifecycle of each application and common navigation back functions, such as controlling the exit, opening, and backing of an application. The window manager can be used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, taking screenshots, and controlling display window changes (for example, shrinking the display window, shaking the display, distorting the display, etc.).

[0073] In some embodiments, the system runtime layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system will run the C / C++ libraries contained in the system runtime layer to implement the functions required by the framework layer. The system runtime layer may include native libraries, which may include but are not limited to WEBKIT (a web browser engine) and OpenGL ES (Open Graphics Library for Embedded Systems).

[0074] In some embodiments, a hardware abstraction layer (HAL) may be included between the system runtime layer and the kernel layer. The HAL is an interface layer between the kernel layer and the hardware circuit, and its purpose is to abstract the hardware.

[0075] In some embodiments, the kernel layer can be a layer between hardware and software. As shown in FIG4A , the kernel layer can include a power management module and at least one of the following drivers: an audio driver, a display driver, a Bluetooth driver, a camera driver, a WiFi driver, a USB driver, an HDMI driver, a sensor driver (such as a fingerprint sensor, a temperature sensor, a pressure sensor, etc.), and a power driver.

[0076] Figure 4B is a diagram of the software architecture of a mode switching application provided according to some embodiments of the present application. Referring to Figure 4B , the software architecture of the mode switching application may include, but is not limited to, a screen projection management module 201, a data management module 202, a message processing module 203, a view management module 204, a mode management module 205, a sound management module 206, a location management module 207, an algorithm processing module 208, an application interaction processing module 209, a service management module 210, a replacement management module 211, a broadcast control management module 212, a time management module 213, and a focus management module 214.

[0077] In some embodiments, the screen projection management module 201 can be used to interact with the screen projection device through the screen projection service, manage the connection and disconnection of the screen projection, manage the status and list of the screen projection device, etc. The screen projection protocol is not limited, for example, Miracast, AirPlay, DLNA, etc. can be used.

[0078] In some embodiments, the data management module 202 may be used to manage information and data involved in different modes of a mode switching application, and to synchronously update related information and data.

[0079] In some embodiments, the message processing module 203 may be used to implement message transmission and flow between modules in a mode switching application.

[0080] In some embodiments, the View management module 204 may be used to implement UI display and update of different mode templates included in the mode switching application.

[0081] In some embodiments, the mode management module 205 can be used to implement switching between different modes based on a mode switching operation input by the user. For example, the mode switching operation input by the user may indicate that the projection display mode should be switched from a dual-screen projection viewing mode to an exclusive mode. In this case, the mode management module 205 can switch the current mode to the exclusive mode. In this way, other modules in the mode switching application can interact, ultimately causing the view management module 204 to adapt and update the UI to the display effect in the exclusive mode.

[0082] In some embodiments, the mode management module 205 can also be used to configure templates in different modes (hereinafter referred to as: mode templates), and the mode templates can be used to specify the operating mechanism and typesetting characteristics in different modes.

[0083] In some embodiments, for the exclusive mode, the mode management module 205 may set a mode template such as: the projection screen is displayed in full screen size.

[0084] In some embodiments, for the dual-screen projection viewing mode, the mode template set by the mode management module 205 is, for example, two-way projection, one left and one right, or one up and one down, display, or it can match the typesetting and layout according to the screen direction of each projection (landscape or portrait).

[0085] In some embodiments, for the three-way projection viewing mode, the mode management module 205 can set a mode template such as three-way projection arranged horizontally, one up and two down, two up and one down, etc., or can match the typesetting and layout according to the screen direction of each projection (landscape or portrait).

[0086] In some embodiments, for the conference mode, the mode template set by the mode management module 205 is, for example, an interface including a playback window and a signal source list. The signal source list can be used to display relevant information of at least one signal source supported and accessed by the display device (such as the category, name, and connection status of the signal source, etc.). Among them, the categories of signal sources may include but are not limited to: HDMI signal source, screen projection signal source, USB signal source, Bluetooth signal source, infrared signal source, and other signal sources. In this way, the user can quickly switch the signal source displayed in the playback window by adjusting the position of the focus in the signal source list based on the signal source list.

[0087] In some embodiments, for the watch-while-training mode, the mode management module 205 may configure a mode template such as displaying the media content in full screen and displaying a floating window for fitness training in a floating layer on top of the media content, so that the user can watch the media while following the training through the floating window. It should be noted that the mode management module 205 can adaptively configure the mode template corresponding to each mode based on the characteristics and scenario requirements of different modes, and is not limited to the examples in the embodiments of the present application.

[0088] In some embodiments, for the floating mode, the mode management module 205 may set a mode template such as displaying the main screen content in full screen, displaying a floating window on top of the main screen content, and displaying the secondary screen content in the floating window, so that the user can view the main screen content while viewing the secondary screen content through the floating window. It should be noted that the mode management module 205 can adaptively configure the mode template corresponding to each mode based on the characteristics and scenario requirements of different modes, and is not limited to the examples in the embodiments of the present application.

[0089] In some embodiments, the sound management module 206 can be used to process the sound of the projection screen in different modes, such as muting and unmuting. For example, assuming that in a dual-screen viewing mode, including projection screen A and projection screen B, if the user chooses to play sound for projection screen A, the sound management module 206 can set setMute(false) for projection screen A and set setMute(true) for projection screen B, so that projection screen A plays sound and projection screen B is muted.

[0090] In some embodiments, the position management module 207 can be used to set the display position of each projection window in the multi-projection viewing mode according to the connection status of the projection device and the interface focus. Taking the dual-projection viewing mode as an example, including projection A and projection B, the position management module 207 can set the relative positions of projection A and projection B in the interface. For example, if projection A is earlier than projection B, projection A can be set on the left side of the interface, and projection B can be set on the right side of the interface. Taking the three-projection viewing mode as an example, including projection A, projection B and projection C, the position management module 207 can set the relative positions of projection A, projection B and projection C in the interface. For example, if projection A is earlier than projection B and projection C is the latest connected, the three-projection screens can be arranged in the order of access time, for example, projection A is set on the left side of the upper area of ​​the interface, projection B is set on the right side of the upper area of ​​the interface, and projection C is set in the center of the lower area of ​​the interface.

[0091] In some embodiments, still taking the dual-screen projection viewing mode as an example, if projection screen A is disconnected and projection screen C is newly connected, the position management module 207 can keep projection screen B unchanged on the right side of the interface, and set projection screen C on the left side of the interface to avoid the problem of poor projection viewing effect caused by the displacement and shaking of the projection content. For another example, projection screen A is set on the left side of the interface, and projection screen B is set on the right side of the interface. The focus is on the window of projection screen B. At this time, if projection screen C is newly connected, projection screen C is used to replace the focus position of projection screen B, so that projection screen A is set on the left side of the interface and projection screen C is on the left side of the interface. It should be noted that the position management module 207 can be configured with positioning rules that match different modes, and is not limited to the examples of this application. Still taking the three-way projection viewing mode as an example, if projection screen A is disconnected and projection screen D is newly connected, the position management module 207 can keep the positions of projection screens B and C in the interface unchanged, and set projection screen D at the position of the original projection screen A to avoid the problem of poor projection viewing effect caused by the displacement and shaking of the projection content. For example, when D projection is connected, if the focus falls on the window of B projection, D projection can be used to replace B projection at the focus position, and the positions of the original A projection and C projection remain unchanged.

[0092] It should be noted that the location management module 207 can be configured with positioning rules that match different modes, and is not limited to the examples in this application. For example, the location management module 207 can also re-match the relative position arrangement of each projection window in the interface according to the updated projection path when the projection device is connected or disconnected.

[0093] In some embodiments, the algorithm processing module 208 can be used to adaptively and dynamically match the UI typesetting and layout in the target mode when switching to the target mode, which may include the size (including width and height) and display position (such as position coordinate setting) of different Views and windows in the interface, thereby providing parameter basis for the View management module to update the interface.

[0094] In some embodiments, the application interaction processing module 209 can be used to implement interaction with other external applications during mode switching. For example, mode switching can set multiple startup entries, including but not limited to the mode option 511 presented after calling up the mode list, voice entry, etc. These startup entries can interact with the application interaction processing module 209 and perform different customized management according to the differences and categories of the startup entries.

[0095] In some embodiments, the service management module 210 can be used to start the mode switching application, manage the opening and closing of related services involved in the mode switching application, and can also be used to customize specific services in specific scenarios and modes.

[0096] In some embodiments, the replacement management module 211 can be used to replace a previous screen projection according to the replacement rules when the number of screen projection channels exceeds the upper limit of the display device's bearing capacity, so that the most recently connected screen projection can be displayed. For example, the display device supports up to three screen projection channels. At this time, the display device has connected screen projections A, B, and C. If screen projection D is received, the replacement management module 211 can make screen projection D replace the earliest connected screen projection A, thereby updating to screen projection B, C, and D. For another example, when the display device receives screen projection D, the interface focus falls on the window of screen projection B, then the replacement management module 211 can make screen projection D replace the focused screen projection B, thereby ensuring that the latest screen projection can be connected and displayed. It should be noted that the replacement rules configured by the replacement management module 211 are not limited to the embodiments shown in this application, and the replacement rules can be set according to different scenarios and different modes.

[0097] In some embodiments, the playback control management module 212 can be used to control the playback of media content in different modes of projection. For example, when the focus is on projection A, operations such as fast forward, rewind, speed control, pause, and resume playback can be performed on projection A. It should be noted that the playback control management module 212 can perform different playback control operations based on the type of media being projected. For example, video media can be fast forwarded, rewinded, speed control, paused, and volume controlled, while text media can be zoomed, font adjusted, annotated, and marked.

[0098] In some embodiments, the time management module 213 can be used to process time events involved in the screen projection process. For example, the screen projection window may be configured with some additional content such as controls, icons, or prompts. These additional contents will be automatically hidden after being displayed for a preset period of time. Therefore, the time management module 213 can start a timer when these additional contents begin to be displayed. After the timer reaches the preset period of time, the message processing module 203 notifies the view management module 204 to hide these additional contents.

[0099] In some embodiments, for example, the first episode of a TV series is being played in projection screen A. When the first episode is finished, it can automatically jump to the second episode. However, there is a certain transition time between episode 1 and episode 2. If the transition time does not exceed the time threshold, the display device can maintain projection screen A and wait for the loading and playback of episode 2. If the transition time exceeds the time threshold, it indicates that the second episode has not been broadcast after a long time, and the display device can disconnect projection screen A. Therefore, the time management module 213 can start the timing at the end of the first episode. When the timing reaches the time threshold, the View management module 204 can be notified through the message processing module 203. The View management module 204 detects whether the second episode has started. If the second episode has not started, the View management module 204 can notify the projection management module 201 through the message processing module 203 to disconnect the projection screen A; if the second episode has started, the projection screen A can be kept connected and the content of the second episode can be played normally.

[0100] In some embodiments, the focus management module 214 can be used to process the focus involved in the operation of the mode switching application, such as responding to user key operations, controlling the display, hiding and movement of the focus, recording focus information, etc., where the focus information may include but is not limited to the focus position, focus display status, etc., and the focus information can be synchronized to other related modules through the message processing module 203. For example, the focus management module 214 can synchronize the focus information to the position management module 207 through the message processing module 203, so that the position management module 207 decides the position of the projection window according to the focus information; for another example, the focus management module 214 can synchronize the focus information to the broadcast control management 212 through the message processing module 203, so that the broadcast control management 212 can control the playback of the projection with focus according to the focus information, and so on.

[0101] The above describes the functions of each module configuration in the mode switching application. Next, we will use specific mode switching scenarios to explain the specific execution procedures of each module in the mode switching application, as well as the interaction and flow of information and data between modules, to achieve fast, easy, and smooth mode switching within the mode switching application. It should also be noted that the modules in the mode switching application can be increased or decreased based on other functional requirements of the mode switching.

[0102] In some embodiments, the display device can send a mode switching instruction to the mode switching application in response to a mode switching operation input by the user. The mode switching instruction can be used to instruct switching to the target mode. In response to the mode switching instruction, the mode switching application can control the mode management module 205 to switch to the target mode and perform initialization tasks based on the current mode switching process. The initialization tasks may include but are not limited to: controlling the service management module 210 to start a service for obtaining a signal source list, controlling the service management module 210 to start a screen projection service to listen for screen projection requests and / or exit screen projection requests initiated by other terminal devices, controlling the View management module 204 to initialize the configuration of the View displayed by the application layer, and controlling the data management module 202 to create some preparation data, etc.

[0103] In some embodiments, after the initialization task is completed, the mode switching application can control the screen projection management module 201 to register the listening logic of UsessionConnectionListener, UsessionPrepareListener, USessionDisplayListener, and UsessionEventListener. Among them, Session represents a conversation between the client and the server, and can record client information (also known as device information). The client information can be configured as {id=, type=, protocol=,...}, where id represents the client's device id for identifying the client, type represents the type of the client, and protocol represents the screen projection protocol used by the client.

[0104] In some embodiments, UsessionConnectionListener can be used to monitor changes in the connection status of the client in the Session. The connection status may include a connected state and an unconnected state (i.e., a disconnected state). UsessionPrepareListener can be used to prepare for screen projection interaction with the client. UsessionDisplayListener can be used to monitor changes in interface playback information. Interface playback information may include but is not limited to playback content information and playback status. UsessionEventListener can be used to monitor client control events, etc. In this way, the screen projection management module 201 can monitor the information of the screen projection signal source (hereinafter referred to as: screen projection signal source information). The screen projection signal source information may include but is not limited to screen projection device information, connection status, etc.

[0105] In some embodiments, the screen projection management module 201 can send the screen projection signal source information to the message processing module 203, and the message processing module 203 forwards the screen projection signal source information to the data management module 202. The data management module 202 can create and store a screen projection device list based on the screen projection signal source information. The screen projection device list may include relevant information of terminal devices (referred to as: screen projection devices) that have established a screen projection connection with the display device 200. Some embodiments of the present application provide an underlying structure of a screen projection device list as follows. It should be noted that the structure in some embodiments of the present application is only an example and is not specifically limited in this application: {mInputName=screen projection device 1, mIsConnected=true, mInputSourceId=, mVisible=true, mSourceType=, mSession=SessionBase{id=Client 1,type=,protocol=}…}, {mInputName=screen projection guide, mIsConnected=, mInputSourceId=, mVisible=false, mSourceType=…}]

[0106] Among them, mInputName is the device name set by the data management module 202 for the screen projection device, such as screen projection device 1. The numerical suffix of the screen projection device does not exceed the real-time screen projection number of the display device. mIsConnected indicates the connection status of the screen projection device. mIsConnected = true means that the screen projection device is connected. mIsConnected = false means that the screen projection device is disconnected. If the screen projection device list only maintains connected screen projection devices, if the screen projection device A is disconnected, the information of the screen projection device A can be deleted from the screen projection device list, or the mIsConnected corresponding to the screen projection device A can be changed to false. mInputSourceId is used to indicate the signal source ID of the screen projection device. mSourceType is used to indicate the signal source type; mVisible = true is used to indicate the display status of the signal source. If mVisible = true, it means that the screen projection is in a visible display state. If mVisible = false, it means that the screen projection is in an invisible or hidden state. mSession = SessionBase {id = Client 1, type =, protocol =} ...} is used to record the device information of the screen projection device. For example, the ID of screen projection device 1 is Client 1. A display device can have an upper limit on its screen projection capabilities. For example, if it supports up to K screen projection channels, the screen projection device list will include information on up to K screen projection devices with mIsConnected = true.

[0107] In some embodiments, the mode switching application can set a screen projection guidance mode and its mode template. The mode template can include tags such as the local area network name and screen projection operation items, thereby guiding the user to initiate execution from the client terminal to the display device. Because the screen projection guidance mode is highly related to screen projection, the data management module 202 can add information corresponding to the screen projection guidance mode to the screen projection device list.

[0108] In some embodiments, if the mode management module 205 switches to the target mode, it can send a notification message indicating that the mode has been switched to notify other modules that they are about to enter the target mode, so that the switching and updating of the mode on the user interface can be achieved through program processing and interactive flow between modules. For example, after the mode management module 205 switches the mode to the single-channel projection mode / exclusive mode, it can notify the algorithm processing module 208 through the message processing module 203 to calculate the typesetting data according to the mode template of the exclusive mode, and the algorithm processing module 208 sends the typesetting data to the View management module 204 through the message processing module 203. In addition, if the data management module 202 finds that the projection device list does not contain information about the projection device, that is, the display device is not connected to the projection device, it can notify the View management module 204 through the message processing module 203 to display the projection guide page / projection help page.

[0109] In some embodiments, for example, after the mode management module 205 switches to the dual-screen projection viewing mode, it can notify the algorithm processing module 208 through the message processing module 203 to calculate the layout data according to the mode template of the dual-screen projection viewing mode, and the algorithm processing module 208 sends the layout data to the View management module 204 through the message processing module 203. In addition, if the data management module 202 queries that the projection device list only contains information about one projection device 1 (corresponding to projection A), it can notify the position management module 207 through the message processing module 203 to set the position distribution information. The position distribution information is used to indicate the position distribution of projection A and the projection guide in the interface. For example, projection A can be displayed on the left side of the interface, and the projection guide can be displayed on the right side of the interface. The position management module 207 can pass the position distribution information to the View management module 204 through the message processing module 203. Figure 6A is a schematic diagram of the interface in the dual-screen projection viewing mode provided according to some embodiments of the present application. Referring to Figure 6A, when the user switches to the dual-screen projection viewing mode, the View management module 204 can display the first projection window 61 corresponding to the A projection and the second projection window 62 corresponding to the projection guide on the interface according to the layout data and position distribution information, such as the first projection window 61 is displayed on the left side of the interface, and the second projection window 62 is displayed on the right side of the interface.

[0110] In some embodiments, for example, after the mode management module 205 switches to the three-way projection viewing mode, it can notify the algorithm processing module 208 through the message processing module 203 to calculate the layout data in the three-way projection viewing mode. The algorithm processing module 208 sends the layout data to the View management module 204 through the message processing module 203. In addition, if the data management module 202 finds that the projection device list only contains information about projection device 1 (corresponding to projection A) and projection device 2 (corresponding to projection B), the position management module 207 is notified through the message processing module 203 to set the position distribution information. The position distribution information is used to indicate the position distribution of projection A, projection B and projection guide in the interface, for example, displaying projection A on the left side of the upper area of ​​the interface, displaying projection B on the right side of the upper area of ​​the interface, and displaying the projection guide in the middle of the lower area of ​​the interface. In this scenario, the projection guide is used to make up for and fill the gap of the third projection. The position management module 207 transmits the position distribution information to the View management module 204 through the message processing module 203. FIG6B is a schematic diagram of an interface in a three-way projection viewing mode according to some embodiments of the present application. Referring to FIG6B , when a user switches to the three-way projection viewing mode, the View management module 204 displays a first projection window 61 corresponding to projection A, a second projection window 63 corresponding to projection B, and a third projection window 62 corresponding to the projection guide on the interface based on the layout data and position distribution information. For example, the first projection window 61 is displayed on the left side of the upper area of ​​the interface, the second projection window 62 is displayed on the right side of the upper area of ​​the interface, and the third projection window 63 is displayed in the center of the lower area of ​​the interface. The screen orientation of the three-way projection windows is not limited.

[0111] In some embodiments, for example, after the mode management module 205 switches to the M-way projection viewing mode, the algorithm processing module 208 can be notified through the message processing module 203 to calculate the layout data according to the mode template of the M-way projection viewing mode, and the algorithm processing module 208 sends the layout data to the View management module 204 through the message processing module 203, where M≤K, that is, the multi-way projection viewing mode cannot exceed the upper limit of the projection capability of the display device, and if the data management module 202 queries that the projection device list includes information of N projection devices, and N is greater than M, then M-way target projections can be filtered out from the N-way projection according to the first preset rule. For example, if the user switches to the two-way projection viewing mode (that is, M=2), the screen projection device list includes screen projection device 1, screen projection device 2 and screen projection device 3 (that is, N=3), then the data management module 202 filters two screen projection devices according to the first preset rule, for example, screen projection device 2 and screen projection device 3 are filtered out, and the message processing module 203 notifies the position management module 207 to set the position distribution information, and the position distribution information can be used to indicate the position distribution of the projection content of screen projection device 2 and screen projection device 3 in the interface, so that the View management module 204 can display the projection window 1 corresponding to the projection device 2 and the projection window 2 corresponding to the projection device 3 on the interface according to the layout data and the position distribution information, for example, displaying the projection window 1 on the left side of the interface and the projection window 2 on the right side of the interface.

[0112] In some embodiments, the first preset rule is not limited, for example: according to the screen projection connection order / access time, select the M target screen projection devices that have been connected recently, or select the M target screen projection devices that rank in the top M in terms of the number of historical connections. Figure 7A is a schematic diagram of displaying a first prompt pop-up window in the user interface according to some embodiments of the present application. Figure 7B is a schematic diagram of displaying a first prompt pop-up window in the user interface according to some embodiments of the present application. Referring to Figures 7A and 7B, when M<N≤K, the data management module 202 can notify the View management module 204 through the message processing module 203, so that the View management module 204 displays the first prompt pop-up window 71. The first prompt pop-up window 71 may include a first prompt message 71a, a screen projection device list 71b, a confirmation button 71c, and a cancel button 71d. Among them, the first prompt message 71a can be used to prompt the selection of M target screen projection devices from the N screen projection devices included in the screen projection device list to join the M-way screen projection viewing mode. In Figure 7A, the user switches to the dual-screen projection viewing mode, but the display device is connected to the three-way projection as an example, that is, M = 2, N = 3. In Figure 7B, the user switches to the three-way projection viewing mode, but the display device is connected to the four-way projection as an example, that is, M = 3, N = 4.

[0113] In some embodiments, after the user selects M target projection devices, he can click the confirmation button 71c, so that the View management module 204 obtains the projection device set. The projection device set is a true subset of the projection device list, which contains information of M target projection devices. The View management module 204 can send the projection device set to the position management module 207 through the message processing module 203. The position management module 207 determines the position distribution information of the M target projections and sends the position distribution information to the View management module 204 through the message processing module 203. In this way, the View management module 204 can display M projection windows according to the layout data adapted to the M-way projection viewing mode and the position distribution information.

[0114] In some embodiments, if the user clicks the cancel button 71d, indicating that the user does not specify M target screen projection devices, the M target screen projection devices can be matched from the screen projection device list according to the first preset rule.

[0115] In some embodiments, after the management module 205 switches to the M-way projection viewing mode, it can notify the algorithm processing module 208 through the message processing module 203 to calculate the layout data according to the mode template of the M-way projection viewing mode, and the algorithm processing module 208 sends the layout data to the View management module 204 through the message processing module 203. In addition, if the data management module 202 queries that the projection device list includes information of N projection devices, and N is less than M, for example, the user switches to the four-way split-screen game mode, but the display device is only connected to two-way projection, so the number of projection channels is not enough to support the display of the four-way split-screen game mode. For example, the user switches to the three-way projection viewing mode, but the display device is only connected to one-way projection, so even if the projection guide is filled in, it is not enough to support the display of the three-way projection viewing mode.

[0116] Figure 8A is a schematic diagram of displaying a second prompt pop-up window in the user interface according to some embodiments of the present application. Figure 8B is a schematic diagram of displaying a second prompt pop-up window in the user interface according to some embodiments of the present application. Referring to Figures 8A and 8B, the second prompt pop-up window 81 may include a second prompt message 81a, a reselect button 81b, and a wait button 81c. Among them, the second prompt message 81a can be used to prompt the user to switch to the M-way projection viewing mode, but the display device is only connected to N-way projection (N<M), and ask the user whether to reselect the mode to switch. In Figure 8A, the user switches to the dual-way projection viewing mode, but the display device is only connected to a single-way projection, that is, M=2, N=1. In Figure 8B, the user switches to the three-way projection viewing mode, but the display device is only connected to a single-way projection, that is, M=3, N=1.

[0117] In some embodiments, if the user clicks the reselect button 81 b , the View management module 204 may pop up a mode list 51 on the upper layer of the interface, and the user may reselect a switchable mode category from the mode list 51 .

[0118] In some embodiments, if the user clicks the wait button 81c, the View management module 204 may no longer display the mode list 51, and may notify the screen projection management module 201 through the message processing module 203 to continue monitoring the connection status of the screen projection device. If a screen projection device is connected and / or disconnected, the data management module 202 synchronously updates the screen projection device list until the screen projection device list can include information of at least M-1 screen projection devices, and then switches to the M-way screen projection viewing mode according to a process similar to the above.

[0119] In some embodiments, if the updated list of projection devices includes information on M-1 projection devices, as shown in Figures 6A and 6B, the final View management module 204 can display projection windows corresponding to M-1 projection devices on the interface, as well as a window corresponding to the projection guide. If the updated list of projection devices includes information on more than or equal to M projection devices, you can switch to the M-way projection viewing mode in a similar process to the above. The second prompt pop-up window 81 can avoid user misoperation when switching modes, avoid mode switching failure, and support users to reselect modes or wait for mode switching, thereby improving user experience.

[0120] The above solution provides mode switching for screen projection scenarios in non-conference mode. In some embodiments, users can switch to conference mode. The mode switching application supports switching between multiple different modes and, when switching to the target mode, adapts to changes and updates in the interface layout, layout, and content of the target mode.

[0121] FIG9 is a schematic diagram of the interface of the conference mode provided according to some embodiments of the present application. Referring to FIG9 , the interface in the conference mode may include a first area 91 and a second area 92. The first area 91 and the second area 92 may have relatively fixed sizes and position distributions. Among them, the second area 92 can be used to display the signal source list View. The signal source list View may include information about interface type signal sources supported by the display device, connected projection signal sources or other signal sources. The information may include signal source icons, signal source categories, signal source names (such as HDMI1 to 3, projection device 1, projection device 2, projection help, etc.), signal source connection status, display status (including display status or non-display status), focus acquisition status (with focus and without focus), etc.

[0122] Among them, interface-type signal sources may include but are not limited to at least one HDMI interface, and other signal sources may include but are not limited to Bluetooth signal sources, infrared signal sources, etc. In conference mode, the focus management module 214 can control the position of the focus in the signal source list based on user operations to indicate which signal source is selected. In this way, when the user confirms the selection (for example, clicking the confirmation key / OK key on the control device 100), the signal source selected by the focus becomes the target signal source, and the View management module 204 can display the content of the target signal source in the playback window of the first area 91. In this way, the user can learn the category and connection status of the signal source through the signal source list, and adjust the position of the focus in the signal source list according to preference, so as to quickly switch the target signal source displayed in the playback window.

[0123] In some embodiments, the screen projection management module 201 can also query whether the service for obtaining the first signal source information is available through the onServiceAvailable interface. If the service is available and the SourceManager has been created, the screen projection management module 201 can call SourceManager (signal source management) through the Flow interface provided by the framework layer, and can obtain the first signal source information from the SourceManager. The first signal source information may include information about the interface type signal source supported by the display device, where the interface type may include an HDMI interface. Among them, one way to call SourceManager may be TVManager.getInstance().getSourceManager(mContext), and one way to obtain the first signal source information may be SourceManager.getInputSources().

[0124] In some embodiments, the first signal source information may also include the signal source being played by the user interface recorded by the SourceManager before the mode switch is initiated or before the screen projection is received (hereinafter referred to as: initialization signal source).

[0125] In some embodiments, after obtaining the first signal source information, the screen projection management module 201 can define the signal source name of each HDMI based on the first signal source information. The signal source name may include the category of the signal source and the arrangement number in the signal source list (hereinafter referred to as: index number).

[0126] Assume that an example of the first signal source information is {HDMI-ID1, HDMI-ID2, HDMI-ID3}, that is, the display device has three HDMI interfaces, and the screen projection management module 201 can name the signal source mapped by HDMI-ID1 as HDMI1, the signal source mapped by HDMI-ID2 as HDMI2, and the signal source mapped by HDMI-ID3 as HDMI3. Among them, HDMI is the type of signal source, and the digital suffix of HDMI is the index number of the signal source, which can be the sorting position of HDMI in the signal source list in conference mode set by the screen projection management module 201. The screen projection management module 201 can construct a mapping relationship between the index number and the identification information of the signal source, so that each HDMI interface can be identified by the index number.

[0127] In some embodiments, after setting the HDMI signal source name configured for the display device according to the first signal source information, the screen projection management module 201 can obtain the HDMI connection status included in the first signal source information through the isSourceConnected interface of SourceManager and mark the connection status to obtain the second signal source information. For example, an example of the second signal source information can be as follows:

[0128] {[mInputName=HDMI1,mIsConnected=true,mInputSourceId=,mSourceType=…],

[0129] [mInputName=HDMI2,mIsConnected=false,mInputSourceId=,mSourceType=…],

[0130] [mInputName=HDMI3,mIsConnected=false,mInputSourceId=,mSourceType=…]}

[0131] In the second signal source information, mInputName is used to indicate the signal source name, mInputSourceId is used to indicate the signal source ID, and mSourceType is used to indicate the signal source type. mIsConnected is used to indicate the connection status of the signal source. If mIsConnected = false, it indicates that the mapped HDMI is not connected; if mIsConnected = true, it indicates that the mapped HDMI is connected. The above example shows that in the current scenario, none of the three HDMI interfaces are connected to a signal source.

[0132] In some embodiments, the screen projection management module 201 can send the second signal source information to the message processing module 203, and the message processing module 203 can forward the second signal source information to the data management module 202. The data management module 202 can create and store an interface type signal source list based on the second signal source information. The interface type signal source list may include but is not limited to: the signal source name, signal source ID, signal source type, connection status and display status of the HDMI mapping configured by the display device. Taking the display device with 3 HDMI as an example, the underlying structure of an interface type signal source list provided in some embodiments of the present application can be as follows: [{mInputName=HDMI1, mIsConnected=false, mInputSourceId=, mVisible=false, mSourceType=…}, {mInputName=HDMI2, mIsConnected=false, mInputSourceId=, mVisible=false, mSourceType=…}, {mInputName=HDMI3, mIsConnected=true, mInputSourceId=, mVisible=true, mSourceType=…}]

[0133] The above-mentioned interface signal source list structure may also include mVisible set by the data management module 202 for each HDMI. mVisible is a status value that can be used to indicate the display status of the HDMI. mVisible = true indicates that the data management module 202 can set the corresponding HDMI as the current signal source, thereby instructing the View management module 204 to display the video signal received by the HDMI with mVisible = true in the user interface in conference mode, so that the user can see the signal source data. The HDMI with mVisible = false is not set as the current signal source by the data management module 202. In this way, the View management module 204 does not display the signal source data associated with mVisible = true in the user interface in conference mode, that is, the signal source data of the HDMI is not visible to the user.

[0134] When the projection management module 201 monitors the connection status change of any HDMI through the isSourceConnected interface of SourceManager, the data management module 202 can synchronously rewrite the mIsConnected status value corresponding to the HDMI, so that the data management module 202 can dynamically refresh the interface type signal source list.

[0135] In some embodiments, the data management module 202 may set the mVisible of the HDMI mapping included in the interface type signal source list according to a second preset rule.

[0136] In some embodiments, one possible configuration of the second preset rule is: the mVisible status value of the initialization signal source indicated by the first signal source information can be set to true, and the mVisible status value of the signal source other than the initialization signal source in the interface type signal source list can be set to false.

[0137] For example, assuming that the display device has HDMI1, HDMI2 and HDMI3, and the user interface is currently displaying the signal source content of HDMI1, the SourceManager records that the currently displayed signal source is HDMI1. At this time, if the display device receives a mode switching operation instructed by the user to switch to conference mode, it starts the mode switching process. The mode switching process can be executed in accordance with the aforementioned module processing and interaction method. After the data management module 202 obtains the second signal source information, it can obtain the initialization signal source as HDMI1. Then, the mVisible of HDMI1 can be set to true, and the mVisible of HDMI2 and HDMI3 can be set to false, so that the signal source displayed in the user interface before and after the mode switching remains consistent. If mIsConnected = false of HDMI1, it indicates that HDMI1 is not connected. Since HDMI1 cannot receive the video signal, a prompt message indicating that HDMI1 has no signal will be displayed in the playback window of the interface after switching to conference mode; if mIsConnected = true of HDMI1, the playback window of the conference mode can normally display the video signal received by HDMI1.

[0138] In some embodiments, another possible configuration of the second preset rule is: the connection status of the initialization signal source can be queried. If the initialization signal source is in a connected state (i.e., mIsConnected = true), the status value of the mVisible of the initialization signal source can be set to true, and the status value of the mVisible of the signal sources other than the initialization signal source in the interface type signal source list can be set to false; if the initialization signal source is in an unconnected state (i.e., mIsConnected = false), any signal source in a connected state can be selected from the interface type signal source list. For example, if the signal source is HDMI2, the status value of the mVisible mapped by HDMI2 can be set to true, and the status value of the mVisible mapped by HDMI1 and HDMI3 can be set to false. This embodiment realizes that after switching to conference mode, the application interface can automatically locate and switch to the signal source with signal access for display, avoiding the interface displaying no signal.

[0139] In some embodiments, in non-conference mode, the data management module 202 can independently maintain a list of projection devices and a list of interface signal sources. For example, when the display device is in a dual-channel projection viewing mode, the two-channel projection content displayed in the dual-channel projection viewing mode can be decided based on the projection device list.

[0140] In some embodiments, after the mode management module 205 switches to conference mode, the data management module 202 can integrate the projection device list and the interface type signal source list to obtain a complete signal source list. The signal source list may include information of different categories of signal sources. The signal source categories may include but are not limited to interface type signal sources and projection signal sources.

[0141] In some embodiments, the data management module 202 can send the signal source list to the view management module 204 via the message processing module 203. When the signal source list is updated, the updated signal source list can be synchronized with the view management module 204. In this way, the view management module 204 can control the display and refresh of the signal sources and the signal source list View displayed in the playback window in conference mode based on the signal source list.

[0142] In some embodiments, the conference mode can be subdivided into mSourceMode and mAppMode, where mSourceMode can be an interface type signal source conference mode, and mAppMode can be a screen projection signal source conference mode. If mSourceMode=true, mAppMode=false, it indicates that the video signal transmitted by the interface type signal source needs to be played in the playback window in conference mode, that is, the current signal source is an interface type signal source such as HDMI; conversely, if mSourceMode=false, mAppMode=true, it indicates that the screen projection content needs to be played in the playback window in conference mode, that is, the current signal source is a screen projection signal source. In this way, after entering the conference mode, the display device switches between mSourceMode and mAppMode.

[0143] In some embodiments, in conference mode, the data management module 202 may query the status value of mVisible mapped to each signal source in the signal source list, and may obtain the current signal source mapped with mVisible=true.

[0144] In some embodiments, if the data management module 202 detects that the current signal source is an interface type signal source, it can notify the mode management module 205 to switch to mSourceMode through the message processing module 203; after the mode management module 205 switches to mSourceMode, it can send a switched mode message to the data management module 202 through the message processing module 203; the data management module 202 receives the switched mode message, and can set mSourceMode = true for each interface type signal source mapping in the signal source list, and can set mAppMode = false for each projection signal source mapping; the mode management module 205 switches to mSourceMode ode, a first signal source switching instruction can also be sent to the screen projection management module 201 through the message processing module 203. The first signal source switching instruction can include the identification information of the current signal source (for example, signal source ID or signal source name, etc.); the screen projection management module 201 responds to the first signal source switching instruction, and can call the Flow interface of the framework layer to control the setCurrentSource of SourceManager to switch Source to the current signal source corresponding to mVisible=true, so that the transmission channel of the current signal source is turned on, and the View management module 204 can display the video signal received by the current signal source in the playback window of the conference mode.

[0145] In some embodiments, if the data management module 202 detects that the current signal source is a projection signal source, it can notify the mode management module 205 to switch to mAppMode through the message processing module 203; after the mode management module 205 switches to mAppMode, it can send a switched mode message to the data management module 202 through the message processing module 203; the data management module 202 receives the switched mode message, and can set mAppMode = true for each projection signal source mapping in the signal source list, and can set mSourceMode = false for each interface class signal source mapping, so that the View management module 204 can display the projection content pushed by the current signal source in the playback window of the conference mode.

[0146] In some embodiments, when switching from mode A to mode B, the display device needs to exit mode A. This may include ending the activity cycle corresponding to mode A to cancel some or all of the UI content created in mode A, such as windows, views, and controls, and then starting the activity corresponding to mode B to create the UI content required for mode B, thereby appearing to refresh the user interface as the mode switches. However, this mode switching method is cumbersome and inefficient, resulting in a less than smooth user interface as the mode switches, potentially causing issues such as freezes and screen flickering, and even preventing the same source content from being effectively played back in different modes. This problem is particularly prominent in scenarios where users switch modes continuously, such as switching from multi-screen mode to exclusive mode, then from exclusive mode to conference mode, and then from conference mode to watch-and-practice mode, or switching from multi-screen mode A (e.g., three-screen viewing mode) to multi-screen mode B (e.g., two-screen viewing mode), and then from multi-screen mode B to conference mode. This affects the user experience.

[0147] This application provides a mode switching application, in which multiple different modes and their mode templates can be configured. During the continuous switching of multiple modes, the same application is always used to run and control, without the need to switch applications and activities. The same UI elements between different modes can be shared without having to undo and rebuild. By resetting the same UI elements (for example, adjusting the size and display position, etc.), the layout in different modes can be adapted. For example, in the dual-screen projection viewing mode (including projection A and projection B), if the user switches to the exclusive mode for projection A, the projection management module 201 can disconnect projection B, or the View management module 204 can hide the display window of projection B. The algorithm processing module 208 calculates the layout data in the exclusive mode, and can send the layout data to the View management module 204 through the message processing module 203. The View management module 204 can adjust the size and position of the display window corresponding to projection A (abbreviated as: projection window A) according to the layout data, that is, projection window A is a UI element shared before and after the mode switch. By synchronously adjusting the shared element with the target mode after the switch, the mode switching speed and efficiency are also improved, and the content of projection window A can be played continuously before and after the switch, thereby achieving smoothness and continuity of the mode switch.

[0148] In some embodiments, in the scenario of switching from M-way projection viewing mode to Q-way projection viewing mode, if Q is greater than M, the mode switching application can follow the processing logic of the newly connected projection, and ultimately make the View management module 204 add QM windows to the interface and update the interface layout following the mode switch.

[0149] In some embodiments, if Q is less than M, for example, when switching from a three-way projection viewing mode to a dual-way projection viewing mode, the mode switching application can follow the processing logic of screen disconnection or screen hiding, and ultimately enable the View management module 204 to delete MQ windows in the interface and update the interface layout following the mode switching, that is, to filter out M target projections from the Q-way projections according to predetermined filtering rules for display, and in addition, the MQ-way projections can be disconnected or hidden.

[0150] In some embodiments, the screening rules are not limited. One possible screening rule can be configured as follows: the most recently connected Q-channel target screen can be selected in the order of screen projection access time, and the other MQ-channel screen projections that were connected earlier can be disconnected or hidden. For example, in a three-way screen projection viewing mode, screen A, screen B, and screen C can be included, where screen A is earlier than screen C, and screen C is earlier than screen B. If the user switches to a dual-way screen projection viewing mode, screen C and screen B can be displayed in the dual-way screen projection viewing mode, and screen A, which was connected the earliest, can be disconnected or hidden.

[0151] In some embodiments, another possible screening rule may be configured as: Q target screens may be randomly selected from M screens.

[0152] In some embodiments, another possible filtering rule may be configured as follows: when switching from M-way screen projection viewing mode to Q-way screen projection viewing mode, and Q is less than M, the View management module 204 may prompt the user through the UI (e.g., a pop-up window) to select the Q-way target screen projection from the M-way screen projection that is desired to be retained after switching to the Q-way screen projection viewing mode.

[0153] Figure 10 is a schematic diagram of displaying a third prompt pop-up window when switching from a three-way projection viewing mode to a dual-way projection viewing mode according to some embodiments of the present application. Referring to Figure 10, the View management module 204 can display a third prompt pop-up window 101, and the third prompt pop-up window 101 may include a third prompt information 101a, an optional device list 101b, a confirmation control 101c, and a cancel control 101d. Among them, the third prompt information 101a can be used to prompt the user to select Q target projection devices from the M projection devices included in the optional device list 101b to remain in the switched M-way projection viewing mode. Figure 10 takes the user switching from the three-way projection viewing mode to the dual-way projection viewing mode as an example, that is, M=3, N=2.

[0154] In some embodiments, after the user selects Q target projection devices from the optional device list 101b, he can click the confirmation control 101c, so that the View management module 204 obtains the first projection device set. The first projection device set can be a true subset of the optional device list and can include information of Q target projection devices. For example, the target projection devices in Figure 10 can be selected as projection device 1 and projection device 2. The View management module 204 can send the first projection device set to the position management module 207 through the message processing module 203. The position management module 207 can determine the position distribution information of the Q target projections, and can send the position distribution information to the algorithm processing module 208 and the View management module 204 respectively through the message processing module 203. In this way, the algorithm processing module 208 calculates the layout data of the Q-way projection, and can send the layout data to the View management module 204 through the message processing module 203. The View management module 204 can display Q projection windows according to the layout data and position distribution information adapted to the Q-way projection viewing mode, thereby updating to the interface effect under the Q-way projection viewing mode.

[0155] In some embodiments, if the user clicks the cancel control 101d, it may indicate that the user does not specify Q target screen projection devices, and then the user can select Q target screen projections retained after switching modes from the M screen projections according to predetermined filtering rules.

[0156] In some embodiments, if the viewing mode is switched from the M-way projection viewing mode to the Q-way projection viewing mode, for the MQ-way projection that is not retained, the projection management module 201 can disconnect the MQ-way projection so that the View management module 204 can close / destroy the display window of the MQ-way projection. Alternatively, the projection management module 201 can still maintain the connection of the MQ-way projection, and the View management module 204 can hide the display window of the MQ-way projection. Both methods can achieve that the MQ-way projection is invisible to the user at the UI level.

[0157] In some embodiments, the algorithm processing module 208 can calculate the layout data adapted to the Q-channel projection viewing mode, and can send the layout data to the View management module 204 through the message processing module 203. The View management module 204 can reset the size and display position of the retained Q-channel projection display window according to the layout data, that is, the display window of the Q-channel projection can be a UI element shared before and after the mode switch. By synchronously resetting the shared element following the target mode after the switch, the mode switching speed and efficiency are improved, and the content of the Q-channel projection can be played continuously before and after the switch, thereby achieving smoothness and continuity of the mode switch.

[0158] Next, the specific mode switching scenario will be explained. In some embodiments, when the display device receives the mode switching operation input by the user, it starts the modules in the mode switching application. After the mode switching application performs the initialization task, it can control the screen projection management module 201 to register the monitoring logic such as UsesionConnectionListener, UsesionPrepareListener, USessionDisplayListener and UsesionEventListener, so as to monitor the connection and disconnection of the screen projection signal source, and manage each Session (dialogue). Assuming that the mode switching operation indicates switching to the target mode, the mode switching application can control the mode management module 205 to switch the display mode to the target mode.

[0159] In some embodiments, when the screen projection service receives a screen projection request sent by the first screen projection device (corresponding ID is Client 1), the screen projection management module 201 can obtain the device information of the first screen projection device contained in the screen projection request (hereinafter referred to as: first screen projection device information). The structure of the first screen projection device information can be, for example, {id=, type=, protocol=,…..}, and the first screen projection device information can be synchronized to the data management module 202 through the message processing module 203, so that the data management module 202 manages the first screen projection device information. Among them, one structure of the screen projection request can be {"cmd":"REQUEST","data":"{\"info\":{\"id\":\"\",\"name\":\"\"},\"version\":,\"type\":}"}.

[0160] In some embodiments, for example, the user agrees to the screen projection request of the first screen projection device, or the number of screen projection channels does not reach the upper limit of the screen projection capability of the display device (such as supporting up to K-channel screen projection, K is a positive integer greater than 1), indicating that the display device can receive the screen projection, then the screen projection service can send a screen projection consent message to the first screen projection device through the screen projection protocol layer. A structure of the screen projection consent message can be configured as {"cmd":"ACK","data":"{\"version\":,\"ack\":,\"status\":\"\",\"reason\":\"\",\"type\":0}"}. In this way, when the first screen projection device receives the screen projection consent message, it can perform tasks related to preparing for screen projection, such as establishing a screen projection connection with the display device, and then starting screen recording. After the screen recording is ready, it can send a screen projection ready message to the screen projection service through the screen projection protocol layer. Among them, a structure of the screen projection ready message can be configured as {"cmd":"REQUEST_DATA_OK","data":"{\"info\":{\"id\":\"\",\"name\":\"\"},\"version\":,\"type\":}"}.

[0161] In some embodiments, when the screen projection service receives the screen projection ready message, it can feedback an ACK message to the first screen projection device, so that the first screen projection device pushes the screen projection data to the display device when it receives the ACK message. Among them, one structure of the ACK message can be configured as {"cmd":"DATA_ACK","data":"{\"version\":,\"ack\":,\"status\":\"\",\"type\":0}"}.

[0162] In some embodiments, when the screen projection service receives a screen projection ready message, the screen projection management module 201 may send the first screen projection device information to the data management module 202 through the message processing module 203; the data management module 202 may insert the first screen projection device information in the screen projection device list, and may set the device name of the first screen projection device (e.g., screen projection device 1), connection status value (mIsConnected) and visibility status value (mVisible), etc. After that, the mode switching application may prepare for View layout and other processing on the display device side according to the target mode, number of projection paths, screen projection device information, etc., so as to display the received screen projection data through the target mode. The information of the first screen projection device configured in the screen projection device list may be: {mInputName=screen projection device 1, mIsConnected=true, mInputSourceId=, mVisible=true, mSourceType=, mSession=SessionBase{id=Client 1,type=,protocol=}…}

[0163] According to the above processing flow, the mode switching application can monitor the screen projection requests of more terminals and complete the connection and reception of the screen projection until the number of screen projection channels reaches the upper limit K of the display device's screen projection capability. If the K+1th screen projection is received, the replacement management module 211 can select the previous screen projection (hereinafter referred to as A projection) as the replacement object, and notify the screen projection management module 201 through the message processing module 203 to disconnect the connection of A projection. In this way, the number of screen projection channels is reduced by one and is less than the K value, and the screen projection management module 201 can access the latest received K+1th screen projection.

[0164] In some embodiments, for example, if the target mode to be switched to by the mode switching operation is a dual-channel projection viewing mode, after the mode management module 205 switches to the dual-channel projection viewing mode, the data management module 202 queries the projection device list for the number of projection devices with mIsConnected = true (i.e., the number of projection channels). If the projection device list only includes projection device 1 (i.e., the number of projection channels is 1), that is, the number of projection channels is less than the number of projection displays specified by the target mode, then the "screen projection guide" can be used with reference to FIG. 6A to compensate for and fill in the second projection channel.

[0165] In some embodiments, taking the first projection device as projection device 1 as an example, when projection device 1 sends projection data to the display device, it can send Session configuration information to the display device at the same time. The Session configuration information may include but is not limited to Ratio1 of the projection data. Ratio1 represents the aspect ratio of the projection data, that is, Ratio1 = width1 / height1, width1 is the width of the projection data, and height1 is the height of the projection data. The aspect ratio can be used to indicate the screen display direction of the projection data, and the screen display direction is horizontal or vertical. If the width and height of the projection data are less than the height, it corresponds to vertical screen. If the width and height of the projection data are not less than the height, it corresponds to horizontal screen.

[0166] In some embodiments, the screen projection service can receive the session configuration information sent by the screen projection device 1 based on the screen projection connection established by the screen projection protocol, and the screen projection management module 201 can send the session configuration information to the data management module 202 through the message processing module 203; the data management module 202 queries that the screen projection device list only includes the screen projection device 1, then sends a typesetting calculation instruction to the algorithm processing module 208 through the message processing module 203. The typesetting calculation instruction can be used to instruct the algorithm processing module 208 to calculate the typesetting data in the dual-channel screen projection viewing mode, including one-channel screen projection and screen projection guide sent by the screen projection device 1, and the typesetting calculation instruction may include the session configuration information of the screen projection device.

[0167] In some embodiments, the algorithm processing module 208 can obtain the Session configuration information of the projection device 1 from the instruction in response to the typesetting calculation instruction, and obtain the template data corresponding to the projection guide. The template data corresponding to the projection guide may include the width ratio Ratio_guide of the projection guide page, Ratio_guide = width_guide / height_guide, wherein width_guide represents the width of the projection guide page, and height_guide represents the height of the projection guide page. In this way, the algorithm processing module 208 can calculate the width width1 and height height1 of the projection data sent by the projection device 1 in the local dual-channel projection viewing mode according to parameters such as Ratio1, Ratio_guide, width_screen (display device screen width) and height_screen (display device screen height), and calculate the width width_guide and height height_guide of the projection guide page in the local dual-channel projection viewing mode, thereby obtaining size information.

[0168] In some embodiments, the data management module 202 finds that the list of projection devices only includes projection device 1, and can also send a first positioning instruction to the position management module 207 through the message processing module 203. The first positioning instruction can be used to instruct the position management module 207 to determine the display position of the projection data and projection guide sent by projection device 1 in the dual-channel projection viewing mode. In the dual-channel projection viewing mode, a "one up and one down" or "one left and one right" display template can be used. Taking the use of several "one left and one right" display templates as an example, the position management module 207 can be configured to: display the window for displaying projection data (referred to as "projection window") on the left side of the interface, and display the projection guide page on the right side of the interface. It should be noted that this application does not limit the positioning mechanism of the position management module 207.

[0169] In some embodiments, the position management module 207 can send the first position information to the algorithm processing module 208 through the message processing module 203. The first position information can be used to indicate the relative position distribution of the projection window and the projection guide page in the interface. In this way, the algorithm processing module 208 can determine the display position of the projection window and the projection guide page based on width1, height1, width_guide and height_guide, thereby obtaining the second position information. The second position information includes the position coordinates of the key points on the projection window and the projection guide page, where the key point can be any vertex (such as the upper left corner vertex) or the center point on the window / page.

[0170] In some embodiments, the layout data calculated by the algorithm processing module 208 may include size information and the above-mentioned second position information. The size information may include: the width and height of the projection window, and the width and height of the projection guide page. The algorithm processing module 208 may send the layout data to the View management module 204 through the message processing module 203. The View management module 204 displays the interface layout adapted to the dual-screen projection viewing mode based on the layout data, displays the projection data sent by the projection device 1 in the first projection window, and displays the projection guide page in the second projection window.

[0171] Figure 11 is a display interface 1 in the dual-screen projection viewing mode provided according to some embodiments of the present application. Referring to Figure 11, the interface may include a projection display window 11a and a projection guide window 11b. The second position information may indicate that the projection display window 11a is displayed on the left side of the interface, and the projection display window 11a may be displayed on the right side of the interface. Assuming that the upper left corner vertex of the screen is the coordinate origin o, the x-axis is horizontally to the right along the upper edge of the screen, and the y-axis is vertically downward along the left edge of the screen. Assuming that the distance between the projection display window 11a and the x-axis is top, in order to improve the interface display effect, the projection display window 11a and the projection guide window 11b can be made to have the same height, that is, height1 = height_guide, and the height of the same height can be uniformly expressed as height, then the algorithm processing module 208 can calculate the layout data according to the following equation group: width1+width_guide=width_screen; height=height1=height_guide; top*2+height=height_screen; width1 / height=Ratio1; width_guide / height=Ratio_guide; x1=0; y1=y2=top; x2=z+width1;

[0172] In the above equations, width_screen, height_screen, Ratio1, and Ratio_guide are all known parameters. By analyzing the set of equations, width1, height1, width_guide, and height_guide can be calculated to obtain the size information. Among them, z represents the horizontal separation distance between the projection display window 11a and the projection guide window 11b, z ≥ 0. When z = 0, the projection display window 11a and the projection guide window 11b are seamlessly connected in the horizontal direction. In this way, the second position information can be calculated based on the size information, top value, and z value. The second position information may include: the position coordinates (x1, y1) of the key point on the projection display window 11a, and the position coordinates (x2, y2) of the key point on the projection guide window 11b. The key point here can be the upper left corner vertex of the window.

[0173] In some embodiments, the algorithm processing module 208 can generate layout data based on the size information and the second position information, and can send the layout data to the data processing module 202 and the View management module 204 respectively through the message processing module 203, so that the data processing module 202 can sort and store the layout data, and the View management module 204 can display the interface in the dual-screen viewing mode according to the layout data. A data structure of the layout data generated by the algorithm processing module 208 can be as follows: {Session:SessionClient1, View:SurfaceView / TextureView, MViewPosition:1, Position:(0,310), Width:1092, Height:460, …} {Session:ClientGuide, View:layout, MViewPosition:2, Position:(1104,310), Width:816, Height:460, …}

[0174] Among them, View:SurfaceView / TextureView corresponds to the projection display window 11a of SessionClient1, View:layout corresponds to the projection guide window 11b of ClientGuide, MviewPosition represents the window display position, MViewPosition:1 represents display on the left side of the interface, and MViewPosition:2 represents display on the right side of the interface. For example, the size of the projection display window 11a is 1092*460, and the key point position coordinates of the projection display window 11a are (0,310); the size of the projection guide window 11b is 816*460, and the key point position coordinates of the projection guide window 11b are (1104,310). The window interval distance z in this scene is 12.

[0175] In some embodiments, after receiving the layout data, the View management module 204 can set interface display parameter information through the setLayoutParams interface. The interface display parameter information may include parameters that can be recognized and set by the display and generated based on the layout data. A data structure of the interface display parameter information may be as follows: FrameLayout.LayoutParams lp1 = new FrameLayout.LayoutParams(width, height, 0); lp1.width = 1092; lp1.height = 460; lp1.leftMargin = 0; lp1.topMargin = 310; mPlayerView.setLayoutParams(lp1); FrameLayout.LayoutParams lp1 = new FrameLayout.LayoutParams(width, height, 0); lp1.width = 816; lp1.height = 460; lp1.leftMargin = 1104; lp1.topMargin = 310; mPlayerView.setLayoutParams(lp1);

[0176] In this way, the View management module 204 can set the size and display position of the SurfaceView / TextureView used to display the projection data, and set the size and position of the layout used to display the projection guide according to the interface display parameter information, thereby obtaining the display interface shown in Figure 11.

[0177] Figure 12 is a second display interface in the dual-channel projection viewing mode provided according to some embodiments of the present application. If the display device is connected to the projection device 2, that is, the two-way projection of the corresponding projection device 1 and the projection device 2 is connected, and the requirements of the dual-channel projection viewing mode are met, then the use of the projection guide to make up for and fill the second-way projection can be revoked, and when the mode management module 205 does not switch the mode (that is, maintain the dual-channel projection viewing mode), based on the logic similar to the aforementioned access to the projection device 1, the View management module 204 can eventually update the interface in the dual-channel projection viewing mode: Referring to Figure 12, a first projection window 12a with a size of width1*height1 can be displayed at the position of (x1, y1), and the first projection window 12a can display the first projection, and a second projection window 12b with a size of width2*height2 can be displayed at the position of (x2, y2), and the second projection window 12b can display the second projection. The difference from the example in Figure 11 is that width_guide in the aforementioned set of equations can be replaced with width2, height_guide can be replaced with height2, and Ratio_guide can be replaced with Ratio2, Ratio2 = width2 / height2, so that according to the aspect ratio and screen direction of the multi-channel projection configuration, the layout data can be dynamically adaptively configured and updated, and the interface can be refreshed based on the layout data, including refreshing the layout, canceling the projection guide, and displaying the newly connected second projection.

[0178] In some embodiments, for example, the mode switching application responds to the user's operation of switching to the three-way projection viewing mode, that is, the mode switching operation indicates that the target mode to be switched to is the three-way projection viewing mode. After the mode management module 205 switches to the three-way projection viewing mode, the data management module 202 can be controlled to query the number of projection devices with mIsConnected = true in the projection device list (that is, the number of projection paths). If the projection device list only includes information on projection device 1 and projection device 2 (that is, the number of projection paths is 2), that is, the number of projection paths is less than the number of projection displays specified by the target mode, then with reference to Figure 6B, the "screen projection guide" can be used to compensate and fill the third projection path.

[0179] In some embodiments, after the mode management module 205 switches to the three-way projection viewing mode, assuming that the projection management module 201 has connected projection device 1 (corresponding to Client 1), projection device 2 (corresponding to Client 2) and projection device 3 (corresponding to Client 3), that is, the number of projection channels is exactly equal to the number of projection displays specified by the target mode, then the data management module 202 can set mVisible=true corresponding to projection device 1, projection device 2 (corresponding to Client 2) and projection device 3. In this way, a structure of the projection device list can be: [{mInputName=projection device 1, mIsConnected=true, mInputSourceId=, mVisible=true, mSourceType=, mSession=SessionBase{id=Client 1,type=,protocol=}…}, {mInputName=projection device 2, mIsConnected=true, mInputSourceId=, mVisible=true, mSourceType=, mSession=SessionBase{id=Client 2, type=, protocol=}…}, {mInputName=Screen projection device 3, mIsConnected=true, mInputSourceId=, mVisible=true, mSourceType=, mSession=SessionBase{id=Client 3, type=, protocol=}…}, {mInputName=Screen projection guide, mIsConnected=, mInputSourceId=, mVisible=false, mSourceType=…}]

[0180] Based on the above-mentioned projection device list, the position distribution and layout of the three-way projection can be further set so that the View management module 204 can complete the interface display and update in the three-way projection viewing mode.

[0181] In some embodiments, taking the screen projection device 1 as an example, when the screen projection device 1 sends the projection data to the display device, it can also send the session configuration information to the display device. The session configuration information may include but is not limited to the Ratio1 of the projection data. The screen projection service can receive the session configuration information sent by the screen projection device 1 based on the screen projection connection established by the screen projection protocol, and the screen projection management module 201 can send the session configuration information to the data management module 202 through the message processing module 203. If the data management module 202 finds that the screen projection device list includes the screen projection device 1, the screen projection device 2 and the screen projection device 3, the message processing module 203 will send a typesetting calculation instruction to the algorithm processing module 208 so that the algorithm processing module 208 completes the typesetting operation according to the session configuration information; the typesetting calculation instruction can be used to instruct the algorithm processing module 208 to calculate the typesetting data of the three-way projection sent by the screen projection device 1, the screen projection device 2 and the screen projection device 3 in the three-way projection viewing mode.

[0182] In some embodiments, the algorithm processing module 208 can obtain Session configuration information 1 of projection device 1, Session configuration information 2 of projection device 2, and Session configuration information 3 of projection device 3 in response to the typesetting calculation instruction. Among them, Session configuration information 1 may include the aspect ratio Ratio1 of the projected content transmitted by projection device 1, Ratio1 = width1 / height1, width1 is the width of the projected content transmitted by projection device 1, and height1 is the height of the projected content transmitted by projection device 1; Session configuration information 2 may include the aspect ratio Ratio2 of the projected content transmitted by projection device 2, Ratio2 = width2 / height2, width2 is the width of the projected content transmitted by projection device 2, and height2 is the height of the projected content transmitted by projection device 2; Session configuration information 3 may include the aspect ratio Ratio3 of the projected content transmitted by projection device 3, Ratio13 = width3 / height3, width3 is the width of the projected content transmitted by projection device 3, and height3 is the height of the projected content transmitted by projection device 3.

[0183] In some embodiments, after the mode management module 205 switches to the three-way projection viewing mode, if the data management module 202 finds that the projection device list includes projection device 1, projection device 2, and projection device 3, it can also send a second positioning instruction to the position management module 207 through the message processing module 203. The second positioning instruction can be used to instruct the position management module 207 to determine the display position of the three-way projection content sent by projection device 1, projection device 2, and projection device 3 in the three-way projection viewing mode. The position display template of the three-way projection viewing mode is not limited. For example, the position display template can be configured as "two up and one down", "one up and two down", "two left and one right", "one left and two right", "three arranged in rows", "three sorted in columns", etc. The position management module 207 can flexibly decide the relative position distribution of the three-way projection according to the screen direction of the three-way projection indicated by Ratio1, Ratio2, and Ratio3.

[0184] In some embodiments, referring to FIG10 , for example, the projection content sent by projection device 1 and projection device 2 are both horizontal (i.e., Ratio1 and Ratio2 are not less than 1), and the projection content sent by projection device 3 is vertical (i.e., Ratio3 is less than 1). Then, the position management module 207 can match the position display template of “two left and one right”, that is, the projection content sent by projection device 1 is displayed in projection window 1 at the upper left position of the interface, the projection content sent by projection device 2 is displayed in projection window 2 at the lower left position of the interface, and the projection content sent by projection device 3 is displayed in projection window 3 on the right side of the interface. It should be noted that this application does not limit the positioning mechanism of the position management module 207.

[0185] In some embodiments, the position management module 207 can send third position information to the algorithm processing module 208 through the message processing module 203. The third position information can be used to indicate the relative position distribution of projection window 1, projection window 2, and projection window 3 in the interface. In this way, the algorithm processing module 208 can calculate the width1*height1, width2*height2, and width3*height3 presented in the three-way projection viewing mode based on parameters such as Ratio1, Ratio2, Ratio3, width_screen (display device screen width), height_screen (display device screen height), and the third position information, and then obtain the size information in the layout data.

[0186] In some embodiments, the algorithm processing module 208 can also determine the display positions of projection window 1, projection window 2 and projection window 3 in the interface based on width1*height1, width2*height2, width3*height3 and the third position information, thereby obtaining fourth position information. The fourth position information may include the position coordinates of key points on projection window 1, projection window 2 and projection window 3, wherein the key point can be any vertex of the window (for example, the upper left corner vertex) or the center point, etc., and the key point can be selected according to the calculation requirements.

[0187] In some embodiments, the layout data calculated by the algorithm processing module 208 may include the above-mentioned size information and the fourth position information. The size information may include: the width and height of projection window 1, projection window 2, and projection window 3. The algorithm processing module 208 may send the layout data to the View management module 204 through the message processing module 203. The View management module 204 may display the interface layout and layout adapted to the three-way projection viewing mode based on the layout data, and may display the projection data sent by projection device 1 in projection window 1, the projection data sent by projection device 2 in projection window 2, and the projection data sent by projection device 3 in projection window 3.

[0188] Figure 13 illustrates a display interface in a three-way projection viewing mode according to some embodiments of the present application. Referring to Figure 13 , the interface may include a first projection window 12a (horizontal), a second projection window 12b (horizontal), and a third projection window 12c (portrait). The fourth position information may indicate that the first projection window 12a is displayed in the upper left corner of the interface, the second projection window 12b is displayed in the lower left corner, and the third projection window 12c is displayed on the right side of the interface. Assuming that the upper left corner vertex of the screen is the coordinate origin o, the x-axis is the positive direction along the upper edge of the screen horizontally to the right, and the y-axis is the positive direction along the left edge of the screen vertically downward, in order to improve the interface display effect, the upper edges of the first projection window 12a and the third projection window 12c can be set to the same height, and the distance between the upper edge of the first projection window 12a and the x-axis is top (the top setting in Figure 13 can be 0), and the first projection window 12a and the second projection window 12b can be set to have the same width (for example, the width can be uniformly width), and there is a gap d between the first projection window 12a and the third projection window 12c. Then, under the layout template of the example in Figure 13, the algorithm processing module 208 can calculate the layout data according to the following equations: width=width1=width2; width+d+width3=width_screen; top=0; z=0; top*2+height1+height2+z=height_screen; height3=height_screen; width1 / height1=Ratio1; width2 / height2=Ratio2; width3 / height3=Ratio3; (x1,y1)=(0,top); (x2,y2)=(0,top+height1+z); (x3,y3)=(width+d,0);

[0189] In the above equations, width_screen, height_screen, Ratio1, Ratio2, and Ratio3 are all known parameters. By analyzing the equations, width1*height1, width2*height2, and width3*height3 can be calculated to obtain the size information. Here, z represents the vertical distance between the first and second projection windows 12a, 12b. When z ≥ 0, the first and second projection windows 12a, 12b are seamlessly connected vertically. When top ≥ 0, the first and second projection windows 12a, 12b, and 12c fill the entire screen height. When d ≥ 0, the first, second, and third projection windows 12a, 12b, 12c fill the entire screen width. In this way, the fourth position information can be calculated through the size information, top value, d value and z value. The fourth position information may include: the position coordinates (x1, y1) of the key point on the first projection window 12a, the position coordinates (x2, y2) of the key point on the second projection window 12b, and the position coordinates (x3, y3) of the key point on the third projection window 12c. The key point in Figure 13 can be selected as the upper left corner vertex of each window.

[0190] In some embodiments, the algorithm processing module 208 can generate layout data based on the size information and the fourth position information, and can send the layout data to the data processing module 202 and the View management module 204 respectively through the message processing module 203, so that the data processing module 202 can sort and store the layout data, and the View management module 204 can display the interface in the three-way projection viewing mode based on the layout data.

[0191] It should be noted that in the multi-screen projection viewing mode, the mode switching application can adaptively match the display position template, typesetting and layout of each projection screen according to information such as mode category, number of projection displays, screen orientation and aspect ratio of each projection screen, thereby calculating the size and display position coordinates of the multi-screen projection window.

[0192] In some embodiments, a data structure of the layout data generated by the algorithm processing module 208 may be as follows: {Session:SessionClient1, View:SurfaceView / TextureView, MViewPosition:11, Position:(x1,y1), Width:width1, Height:height1, …} {Session:SessionClient2, View:layout1, MViewPosition:21, Position:(x2,y2), Width:width2, Height:height2, …} {Session:SessionClient3, View:layout2, MViewPosition:12, Position:(x3,y3), Width:width3, Height:height3, …}

[0193] Among them, View:SurfaceView / TextureView corresponds to the first projection window 12a of SessionClient1, and the other two View:layout correspond to the second projection window 12b of SessionClient2 and the third projection window 12c of SessionClient3 respectively; MViewPosition represents the window display position. As an example, MViewPosition:11 means display in the upper left part of the interface, MViewPosition:21 means display in the lower left part of the interface, and MViewPosition:12 means display on the right side of the interface.

[0194] In some embodiments, after receiving the layout data, the View management module 204 can set interface display parameter information through the LayoutParams interface. The interface display parameter information may include parameters generated based on the layout data that can be recognized and set by the display. A data structure for displaying parameter information on the interface can be as follows: FrameLayout.LayoutParams lp1=new FrameLayout.LayoutParams(width,height,0); lp1.width=width1; lp1.height=height1; lp1.leftMargin=x1; lp1.topMargin=y1; mPlayerView.setLayoutParams(lp1); FrameLayout.LayoutParams lp1=new FrameLayout.LayoutParams(width,height,0); lp1.width=width2; lp1.height=height2; lp1.leftMargin=x2; lp1.topMargin=y2; mPlayerView.setLayoutParams(lp1); FrameLayout.LayoutParams lp1=new FrameLayout.LayoutParams(width,height,0); lp1.width=width3; lp1.height=height3; lp1.leftMargin=x3; lp1.topMargin=y3; mPlayerView.setLayoutParams(lp1);

[0195] In this way, similar to the aforementioned switch to dual-way projection mode, the View management module 204 can set the size and display position of SurfaceView / TextureView, layout1 and layout2 used to display three-way projection according to the interface display parameter information, thereby obtaining the display interface shown in Figure 13.

[0196] In some embodiments, when the display device is in a three-way projection viewing mode, the user can input an operation to switch to a dual-way projection viewing mode. For example, the user can call out the mode list 51 and select the option of the dual-way projection viewing mode from the mode list 51. The mode switching application responds to the mode switching operation and controls the mode management module 205 to switch to the dual-way projection viewing mode. In some embodiments, after the mode management module 205 switches to the dual-way projection viewing mode, since the number of projection displays changes from 3 to 2, the two target projections in the original three-way projection viewing mode can be retained, and the discarded one-way projection can be removed (Gone) or hidden (Hidden).

[0197] In some embodiments, the screen projection management module 201 can automatically filter out the two target screen projections that are retained after switching to the dual-screen projection viewing mode based on the screening conditions. The screening conditions may include but are not limited to: any two target screen projections can be randomly selected, or the two most recently connected target screen projections can be selected according to the screen projection connection order, or the target screen projections sent by the two screen projection devices most frequently used by the user can be selected based on the historical number of screen projection connections / usage frequency, etc.

[0198] In some embodiments, the screen projection management module 201 can retain the connection with the 2-way target screen projection, and can disconnect the other screen projection that is considered to be abandoned through the Session.finish() interface, and can send a screen projection disconnection message to the data management module 202 through the message processing module 203. Upon receiving the screen projection disconnection message, the data management module 202 can delete the information of the screen projection device indicated by the screen projection disconnection message from the screen projection device list. In this way, the View management module 204 can destroy the window corresponding to the disconnected screen projection, so that the interface only retains the window of the 2-way target screen projection. The window of the 2-way target screen projection can be the screen projection window that continues from the original three-way screen projection viewing mode to the dual-way screen projection viewing mode, and can refresh the interface according to the layout data adapted to the dual-way screen projection viewing mode provided by the algorithm processing module 208.

[0199] In some embodiments, the screen projection management module 201 can still retain the three-way projection in the original three-way projection viewing mode (without cutting off any one of the ways), and can send a screen projection hide message to the data management module 202 through the message processing module 203. The screen projection hide message can be used to indicate the hiding of the one-way projection that was abandoned in the two-way projection viewing mode. When the data management module 202 receives the screen projection hide message, it can change the mVisible state value of the projection device indicated by the screen projection hide message from true to false. In this way, the View management module 204 will not destroy the window of the abandoned one-way projection, but will hide the window. For example, the window size can be set to be extremely small and the position can be outside the screen display range. In this way, the interface has only two visual windows corresponding to the target projections, and can refresh the interface according to the layout data updated by the algorithm processing module 208 to achieve the interface display effect of the dual-way projection viewing mode. In this embodiment, since the three-way projection in the original three-way projection viewing mode is still in a connected state, if the user switches from the dual-way projection viewing mode to the three-way projection viewing mode, the data management module 202 can reset the mVisible of the projection device corresponding to the hidden window to true, so that the View management module 204 can redisplay the three-way projection window, and can synchronously update the interface layout adapted to the three-way projection viewing mode, thereby realizing automatic recovery of the interface layout and projection content when switching from three-way projection viewing mode → dual-way projection viewing mode → three-way projection viewing mode, and realizing fast mode switching and continuous display of projection content after switching.

[0200] In some embodiments, referring to Figure 10, the View management module 204 can display a third prompt pop-up window 101, so that the user can select the two-way projection to be retained in the dual-way projection viewing mode based on the third prompt pop-up window 101, so that the remaining 1-way projection is considered to be discarded. For example, after the user selects projection device 1 and projection device 2, and clicks the confirmation control 101c, the interface can display the content sent by projection device 1 and projection device 2 in the dual-way projection viewing mode, and not display the content sent by projection device 3, thereby realizing the 2-way target projection retained after the user customizes the switching mode based on viewing needs.

[0201] In some embodiments, after obtaining the 2-way target screen projection selected by the user through the third prompt pop-up window 101, the View management module 204 can send a screen projection disconnection instruction to the screen projection management module 201 through the message processing module 203. The screen projection disconnection instruction can be used to instruct the screen projection management module 201 to disconnect the 1-way screen projection abandoned by the user (for example, the screen projection device 3). In response to the screen projection disconnection instruction, the screen projection management module 201 can disconnect from the screen projection device 3 through the Session.finish() interface, and can send a screen projection disconnection message to the data management module 202 through the message processing module 203, so that the data management module 202 can delete the information of the screen projection device indicated by the screen projection disconnection message from the screen projection device list.

[0202] In some embodiments, after obtaining the two-way target screen projection selected by the user through the third prompt pop-up window 101, the View management module 204 can send a screen projection hiding instruction to the data management module 202 through the message processing module 203. In response to the screen projection hiding instruction, the data management module 202 can change the mVisible status value of the screen projection device indicated by the screen projection hiding instruction from true to false.

[0203] In some embodiments, for the mode switching scenario from M-way projection viewing mode → Q-way projection viewing mode → M-way projection viewing mode, and Q is less than M, when switching from M-way projection viewing mode to Q-way projection viewing mode, the projection management module 201 can maintain access to the Q-way target projection and disconnect the other MQ-way discarded projection, so that after switching from the Q-way projection viewing mode back to the M-way projection viewing mode, the mode switching application can query the projection content type previously transmitted by the disconnected MQ-way projection. If the projection content type is a resource link (such as a URL, etc.), the View management module 204 can access the resource link to obtain the resource content after updating the interface to the original M-way projection viewing mode layout, and can display the resource content in the corresponding projection window, and can also realize automatic recovery of the layout and projection content.

[0204] In some embodiments, when the screen projection management module 201 disconnects the abandoned MQ-channel screen projection, the View management module 204 can obtain the browsed progress of the resource link pushed by the MQ-channel screen projection, so that when switching from the Q-channel screen projection viewing mode back to the M-channel screen projection viewing mode, the View management module 204 can access the resource link to obtain the resource content, and can automatically locate the resource content and jump to the position indicated by the browsed progress based on the previously stored browsed progress, thereby achieving consistency in the browsing progress before and after the mode switch, allowing users to continue watching the resource content and improving the user experience.

[0205] In some embodiments, in response to the user's operation of switching from the Q-way projection viewing mode to the M-way projection viewing mode, the mode switching application can query the type of projection content previously transmitted by the disconnected MQ-way projection. If the projection content type is the screen recording content of the projection device or the projection data stream transmitted in real time, then after the MQ-way projection is disconnected, the data stream transmission of the projection device is terminated and cannot be restored or continued to play. In this way, if the user wants to restore the M-way projection content played in the original M-way projection viewing mode, he can control the MQ projection devices that were disconnected earlier to re-initiate a projection request to the display device.

[0206] In some embodiments, the mode switching application responds to the user's operation of switching from the Q-way screen projection viewing mode back to the M-way screen projection viewing mode, and can query the type of screen projection content previously transmitted by the disconnected MQ-way screen projection. If the screen projection content type is the screen recording content of the screen projection device or the real-time transmission of the screen projection data stream, the data management module 202 can query whether the screen projection device list includes information of no less than MQ-1 first target screen projection devices. The first target screen projection device here is other connected screen projection devices except the Q-way screen projection in the Q-way screen projection viewing mode.

[0207] In some embodiments, if the data management module 202 queries the information that the projection device list includes MQ-1 first target projection devices, it can construct the M-way projection content displayed in the M-way projection viewing mode based on the Q-way projection that continues from the Q-way projection viewing mode, MQ-1 first target projection devices and 1 projection guide, and the projection guide can be used to make up for and fill the 1-way vacancy.

[0208] In some embodiments, if the data management module 202 queries the information that the projection device list includes MQ first target projection devices, then based on the Q-way projection that continues from the Q-way projection viewing mode and these MQ first target projection devices, the M-way projection content displayed in the M-way projection viewing mode can be constructed.

[0209] In some embodiments, if the data management module 202 queries that the screen projection device list includes information of more than MQ first target screen projection devices, it can filter out MQ second target screen projection devices from the first target screen projection devices according to certain rules, so as to construct M-way screen projection content displayed in the M-way screen projection viewing mode based on the Q-way screen projection that continues from the Q-way screen projection viewing mode and these MQ second target screen projection devices.

[0210] In some embodiments, if the data management module 202 finds that the projection device list includes information of less than MQ-1 first target projection devices, the gap cannot be filled even if the projection guide is filled. The View management module 204 can prompt the user to switch to other available modes, or wait for the number of projection paths to increase.

[0211] Figure 14 is a schematic diagram of the interface change when switching from the three-way projection viewing mode to the dual-way projection viewing mode according to some embodiments of the present application. For example, through automatic screening rules or through a user-selected mechanism, it can be determined that the two-way projections retained after switching to the dual-way projection viewing mode correspond to projection device 1 and projection device 2, respectively. Referring to Figure 14, assuming that the position management module 207 sets the first projection window 12a corresponding to projection device 1 to be displayed on the left side of the interface, and sets the second projection window 12b corresponding to projection device 2 to be displayed on the right side of the interface, the algorithm processing module 208 can calculate the size of the first projection window 12a width1*height1, the size of the second projection window 12b width2*height2, the display position (x1, y1) of the first projection window 12a, and the display position (x2, y2) of the second projection window 12b based on parameters such as Ratio1, Ratio2, width_screen, and height_screen. The algorithm processing module 208 can calculate the layout data when switching to the dual-screen viewing mode according to the following equations: width1+width2=width_screen; height=height1=height2; top*2+height=height_screen; width1 / height=Ratio1; width2 / height2=Ratio2; (x1, y1)=(0, top); (x2, y2)=(d+width1, top);

[0212] In the above equations, width_screen, height_screen, Ratio1 and Ratio2 are all known parameters. By analyzing the equations, width1, height1, width2 and height2 can be calculated to obtain the size information in the typesetting data. Among them, d represents the horizontal spacing between the first projection window 12a and the second projection window 12b, z≥0, and when z=0, the first projection window 12a and the second projection window 12b are seamlessly connected in the horizontal direction. In this way, the fourth position information can be calculated through the size information, top value and d value. The fourth position information may include: the position coordinates (x1, y1) of the key point on the first projection window 12a, and the position coordinates (x2, y2) of the key point on the second projection window 12b. The key point in Figure 14 can be the upper left corner vertex of the window.

[0213] In some embodiments, the algorithm processing module 208 generates layout data based on the size information and the fourth position information, and sends the layout data to the data processing module 202 and the View management module 204 respectively through the message processing module 203, so that the data processing module 202 can sort and store the layout data, and the View management module 204 can switch to the interface in the dual-screen projection viewing mode according to the layout data display. A data structure of the layout data generated by the algorithm processing module 208 can be as follows: {Session:SessionClient1, View:SurfaceView / TextureView, MViewPosition:11, Position:(x1,y1), Width:width1, Height:height1, …} {Session:SessionClient2, View:layout, MViewPosition:12, Position:(x2,y2), Width:width2, Height:height2, …}

[0214] Among them, View:SurfaceView / TextureView corresponds to the first projection window 12a of SessionClient1, View:layout corresponds to the second projection window 12b of SessionClient2, and MviewPosition represents the relative position distribution of the projection windows on the interface. For example, MViewPosition:11 represents the left side of the interface, and MViewPosition:12 represents the right side of the interface. After receiving the layout data, the View management module 204 can set the interface display parameter information through the setLayoutParams interface. The interface display parameter information may include parameters generated based on the layout data that can be recognized and set by the display.

[0215] In some embodiments, the View management module 204 can set the size and display position of the SurfaceView / TextureView used to display the projection content sent by the projection device 1, and set the size and position of the layout used to display the projection content sent by the projection device 2 according to the interface display parameter information, thereby obtaining a display interface as shown in Figure 14, and realizing synchronous and rapid refresh of the interface following the mode switching.

[0216] In some embodiments, if the sound rule of the dual-channel projection viewing mode is to support only one channel of projection, the sound management module 206 can make one of the projections sound and mute the other projection according to the sound processing mechanism. The sound processing mechanism is not limited. For example, the sound processing mechanism can be configured to make the focus window in the dual-channel projection viewing mode sound, or make the most recently connected projection sound according to the order of projection connection time, or make the projection corresponding to the projection device with the most connections sound according to the number of historical projection connections, etc.

[0217] In some embodiments, referring to FIG14 , for example, the sound management module 206 makes the projection window 1 corresponding to the projection device 1 sound and mutes the projection window 2 corresponding to the projection device 2 according to the sound processing mechanism. In this way, the sound management module 206 can call the setMute(false) interface of SessionClient1 to set the projection window 1 to sound / unmute, and call the setMute(true) interface of SessionClient2 to mute the projection window.

[0218] FIG15 is a second schematic diagram of the interface change when switching from the three-way projection viewing mode to the dual-way projection viewing mode according to some embodiments of the present application. For example, through automatic screening rules or through a user-selected mechanism, it is determined that the two-way projections retained after switching to the dual-way projection viewing mode correspond to projection device 2 and projection device 3, respectively. Referring to FIG15 , assuming that the position management module 207 sets the projection window 13a corresponding to projection device 2 to be displayed on the left side of the interface, and sets the projection window 13b corresponding to projection device 3 to be displayed on the right side of the interface, the algorithm processing module 208 can calculate the size of the projection window 13a width2*height3, the size of the projection window 13b width3*height3, the display position (x2, y2) of the projection window 13a, and the display position (x3, y3) of the projection window 13b based on parameters such as Ratio2, Ratio3, width_screen, and height_screen. The algorithm processing module 208 can calculate the layout data when switching to the dual-screen viewing mode according to the following equations: width2+width3=width_screen; height3=height_screen; top*2+height2=height_screen; width2 / height2=Ratio2; width3 / height3=Ratio3; (x2, y2)=(0, top); (x3, y3)=(d+width2, 0);

[0219] In the above equations, width_screen, height_screen, Ratio2 and Ratio3 are all known parameters. By analyzing the equations, width2, height2, width3 and height3 can be calculated to obtain the size information in the typesetting data. Among them, d represents the horizontal spacing between the projection window 13a and the projection window 13b, z≥0, and when z=0, the projection window 13a and the projection window 13b are seamlessly connected in the horizontal direction. In this way, the second position information can be calculated through the size information, top value and d value. The second position information includes: the position coordinates (x2, y2) of the key point on the projection window 13a, and the position coordinates (x3, y3) of the key point on the projection window 13b. The key point shown in Figure 15 can still be the upper left corner vertex of the window.

[0220] In some embodiments, the algorithm processing module 208 can generate layout data based on the size information and the fourth position information, and can send the layout data to the data processing module 202 and the View management module 204 respectively through the message processing module 203, so that the data processing module 202 can sort and store the layout data, and the View management module 204 can switch to the interface in the dual-screen viewing mode according to the display of the layout data. A data structure of the layout data generated by the algorithm processing module 208 can be as follows: {Session:SessionClient2, View:layout1, MViewPosition:11, Position:(x2,y2), Width:width2, Height:height2, …} {Session:SessionClient3, View:layout2, MViewPosition:12, Position:(x3,y3), Width:width3, Height:height3, …}

[0221] Among them, View:layout1 corresponds to SessionClient2's projection window 13a, View:layout2 corresponds to SessionClient3's projection window 13b, and MviewPosition represents the relative position distribution of the projection windows on the interface. For example, MViewPosition:11 represents the left side of the interface, and MViewPosition:12 represents the right side of the interface. After receiving the layout data, the View management module 204 can set the interface display parameter information through the setLayoutParams interface. The interface display parameter information includes parameters generated based on the layout data that can be recognized and set by the display.

[0222] In some embodiments, the View management module 204 can set the size and display position of layout1 for displaying the projection content sent by projection device 2, and set the size and position of layout2 for displaying the projection content sent by projection device 3 according to the interface display parameter information, thereby obtaining a display interface as shown in Figure 15, and realizing synchronous and rapid refresh of the interface following mode switching.

[0223] It should be noted that when switching from the M-way projection viewing mode to the Q-way projection viewing mode, and M is greater than Q, the screening rules for screening the retained and discarded projections are not limited, and the position display template in the Q-way projection viewing mode can be flexibly set. In addition, according to the aspect ratio and screen direction of the Q-way projection, the algorithm processing module 208 can adaptively match the typesetting operation, thereby generating a typesetting and layout suitable for the Q-way projection viewing mode, and is not limited to the examples of Figures 14 and 15. When switching from M-way projection viewing mode to Q-way projection viewing mode, shared UI elements (such as SurfaceView / TextureView, layout, etc.) can be retained and used, and non-shared elements can be adaptively deleted or added according to the target mode after switching. By switching the interactive flow between modules within the application in the same mode, a mode switching mechanism is implemented without exiting and restarting the mode or switching activities. This improves the mode switching efficiency, especially for multi-mode continuous switching scenarios. The above solution implements interface display control and sound processing when switching from one multi-way projection mode to another. For other mode switching scenarios, a similar processing flow can be used to complete the mode switching.

[0224] In some embodiments, based on the two-way projection viewing mode shown in Figures 11 and 12, or based on the three or more multi-way projection viewing mode shown in Figures 13 and 14, the user can switch to exclusive mode, that is, when switching from M-way projection viewing mode to Q-way projection viewing mode, if Q=1, it is equivalent to switching to exclusive mode, so that a single projection occupies the screen exclusively. Referring to Figure 11 (or Figure 12), the projection display window 11a / first projection window 12a and the projection guide window 11b / second projection window 12b may include a switch full screen button 10c, and the user can click the switch full screen button 10c in any window. Referring to Figure 13, each projection window includes a switch full screen button 10c, and the user can click the switch full screen button 10c in any projection window to input the operation of switching to exclusive mode. Alternatively, referring to Figure 5A (or Figure 5B), the user can call out the mode list 51 and select the exclusive mode option in the mode list 51 to input the mode switching operation of switching to exclusive mode.

[0225] In some embodiments, the mode switching application can control the mode management module 205 to switch to the exclusive mode in response to the operation of switching to the exclusive mode to execute the processing flow of switching to the exclusive mode. The processing flow can adaptively refer to the above-mentioned implementation method of switching from the M-way projection viewing mode to the Q-way projection viewing mode. The main difference is that only 1-way projection is retained in the exclusive mode, and the other M-1-way projections will be disconnected or hidden.

[0226] Taking Figure 11 as an example, when the user clicks the switch full screen button 10c in the projection display window 11a, the View management module 204 will not destroy the SurfaceView / TextureView of the corresponding projection device 1. The View management module 204 can remove (Gone) or hide (Hidden) the projection guide window 11b. The algorithm processing module 208 can recalculate the layout data in exclusive mode based on Ratio1, width_screen and height_screen, so that the View management module 204 can reset (Reset) the SurfaceView / TextureView that displays the projection data according to the updated layout data to adjust the size and key point position coordinates of the projection display window 11a.

[0227] Alternatively, referring to Figure 13, for example, if the user clicks the switch full screen button 10c in the first projection window 12a, then referring to the above-mentioned implementation method of switching from M-way projection viewing mode to Q-way projection viewing mode, only 1-way projection will be retained, and the other M-1-way projections will be disconnected or hidden. Then, the algorithm processing module 208 can recalculate the layout data in exclusive mode based on Ratio1, width_screen and height_screen, so that the View management module 204 resets the SurfaceView / TextureView according to the updated layout data to adjust the size and display position of the first projection window 12a, so that the first projection window 12a fills and monopolizes the screen, so that the user can pay attention to the projection data pushed by the projection device 1 alone.

[0228] In some embodiments, for example, the width and height of the projection data generated by the screen recording on projection device 1 are 1920*808, that is, Ratio1=width1 / height1==1920 / 808≈2.38, and the screen width and height of the display device are width_screen*height_screen=1920*1080. Then, the algorithm processing module 208 calculates width1*height1 as 1920*808, top=136, (x1,y1)=(0,136), and the structure of the updated layout data can be: { Session:SessionClient1, View:SurfaceView / TextureView, MViewPosition:1, Position:(0,136), Width:1092, Height:808, …}

[0229] In this way, the View management module 204 receives the updated layout data, resets the interface display parameter information through the setLayoutParams interface, and switches the interface to exclusive mode based on the updated interface display parameter information. Figure 16 shows the display interface in exclusive mode provided according to some embodiments of the present application. Referring to Figure 16, after the user clicks the switch full screen button 10c in the first projection window 12a to switch to exclusive mode, the original first projection window 12a occupies the screen exclusively, allowing the user to focus only on the projection data pushed by the projection device 1.

[0230] In some embodiments, referring to FIG. 16 , the exclusive mode interface may include an exit full screen button 16 a . When the user clicks the exit full screen button 16 a , the exclusive mode is exited by default.

[0231] In some embodiments, when switching from M (M≥2)-way projection viewing mode to exclusive mode, the data management module 202 can set mVisible = true of projection device 1, and set the mVisible status value of the other M-1 projection devices to false, so that the View management module 204 can make the projection window A corresponding to projection device 1 exclusive to the screen, and hide the projection window B corresponding to the other M-1 projection devices. The mode switching application responds to the user triggering the exit full-screen button 16a in the exclusive mode, and can control the mode management module 205 to switch back to the original M-way projection viewing mode. The data management module 202 sets the mVisible of the projection device corresponding to the previously hidden M-1-way projection to true. The View management module 204 can refresh the interface layout according to the interface display parameter information in the original M-way projection viewing mode before switching to the exclusive mode, and display the corresponding projection data in the M projection windows respectively, thereby realizing automatic recovery of the interface layout and projection content when switching from M-way projection viewing mode → exclusive mode → M-way projection viewing mode.

[0232] In some embodiments, when switching from M (M≥2)-way projection viewing mode to exclusive mode, the projection management module 201 can maintain the projection connection with projection device 1 and disconnect the projection connection with the other M-1 projection devices, so that the data management module 202 can set mIsConnected=false of these M-1 projection devices, or delete the information of these M-1 projection devices from the projection device list, so that the View management module 204 can make the projection window A corresponding to projection device 1 exclusive to the screen, and at the same time destroy the projection window B corresponding to the other M-1 projection devices. The mode switching application responds to the user triggering the exit full-screen button 16a in exclusive mode, and can query the type of projection content previously transmitted by the M-1 projection devices that have been disconnected. If the projection content type is a resource link (such as a URL, etc.), the View management module 204 can access the resource link to obtain the resource content after updating the interface to the original M-way projection viewing mode layout, and display the resource content in the corresponding projection window, and can also realize automatic recovery of the layout and projection content.

[0233] In some embodiments, when the screen projection management module 201 disconnects the screen projection connection with another M-1 screen projection devices, the View management module 204 can obtain the browsed progress of the resource link pushed by these M-1 screen projection devices. In this way, when switching from the exclusive mode to the M-way screen projection viewing mode, the View management module 204 can access the resource link to obtain the resource content, and automatically locate the resource content and jump to the position indicated by the browsed progress based on the previously stored browsed progress, thereby achieving consistency in the browsing progress before and after the mode switch, allowing users to continue viewing the resource content and improving the user experience.

[0234] In some embodiments, in response to the user triggering the exit full-screen button 16a in exclusive mode, the mode switching application can query the type of screen projection content previously transmitted by the M-1 screen projection devices that have been disconnected. If the screen projection content type is the screen recording content of the screen projection device or the real-time transmission of the screen projection data stream, then after the connection is disconnected, the data stream transmission of the screen projection device is terminated and cannot be restored or continued. In this way, if the user wants to restore the M-way screen projection content played in the original M-way screen projection viewing mode, he can control the M-1 screen projection devices that were previously disconnected to re-initiate a screen projection request to the display device.

[0235] In some embodiments, the mode switching application responds to the user triggering the exit full-screen button 16a in exclusive mode, and can query the type of projection content previously transmitted by the M-1 projection devices that have been disconnected. If the projection content type is the recorded content of the projection device or the real-time transmission of the projection data stream, the data management module 202 can query whether the projection device list includes information of no less than M-2 first target projection devices, where the first target projection device is other projection devices in a connected state except projection device 1.

[0236] In some embodiments, if the data management module 202 queries the information of M-2 first target projection devices in the projection device list, it can construct M-way projection content displayed in the M-way projection viewing mode based on projection device 1, M-2 first target projection devices and projection guide, that is, use the projection guide to make up for and fill in one vacancy.

[0237] In some embodiments, if the data management module 202 queries the information that the projection device list includes M-1 first target projection devices, M-way projection content displayed in the M-way projection viewing mode can be constructed based on the projection device 1 and these M-1 first target projection devices.

[0238] In some embodiments, if the data management module 202 queries the screen projection device list and finds that the list includes information of more than M-1 first target screen projection devices, M-1 second target screen projection devices can be filtered out from the first target screen projection devices according to certain rules, so as to construct M-way screen projection content displayed in the M-way screen projection viewing mode based on the screen projection device 1 and these M-1 second target screen projection devices.

[0239] In some embodiments, if the data management module 202 finds that the projection device list includes information of less than M-2 first target projection devices, the gap cannot be filled even if the projection guide is filled, and the View management module 204 can prompt the user to switch to other available modes.

[0240] In some embodiments, users can also switch from M-way projection viewing mode to exclusive mode to other modes, including but not limited to conference mode, picture-in-picture mode (also known as floating mode), split-screen gaming mode, etc. This mode switching path uses exclusive mode as a transition, enabling rapid and continuous switching between multiple modes.

[0241] In some embodiments, when the display device is in exclusive mode, see FIG5A , the user can call up mode list 51 and select the conference mode option, thereby inputting a mode switching operation to switch to conference mode. In response to the user's mode switching operation to switch to conference mode, the mode switching application controls mode management module 205 to switch to conference mode. Based on the description of conference mode in the aforementioned related embodiments, see FIG9 , the interface in conference mode may include a first area 91 and a second area 92 . The second area 92 may be used to display a signal source list View , and the first area 91 may be used to display the signal source content transmitted by the signal source item with focus in the signal source list View .

[0242] Figure 17 is a schematic diagram of the interface changes when switching from exclusive mode to conference mode according to some embodiments of the present application. Referring to Figure 17, the SurfaceView / TextureView in exclusive mode is retained, and other View contents newly added in conference mode need to be preloaded. Among them, the View contents newly added in conference mode may include but are not limited to the signal source list View, which not only provides projection signal source items, but also provides interface signal source items (such as HDMI).

[0243] In some embodiments, after the mode management module 205 switches to conference mode, the data management module 202 can generate a complete signal source list based on the interface signal source list and the projection device list, and synchronize the signal source list to the View management module 204 through the message processing module 203. This signal source list can be the content basis of the signal source list View created by the View management module 204 in conference mode.

[0244] In some embodiments, since the View management module 204 displays the projection data sent by the projection device 1 in exclusive mode, after the mode management module 205 switches to conference mode, the data management module 202 can keep the mVisible of the projection device 1 = true, and can set the mVisible of other signal sources in the signal source list to false. The focus management module 214 can set the focus on the projection device 1 and send the focus information to the projection management module 201 through the message processing module 203.

[0245] In some embodiments, the screen projection management module 201 receives focus information and can set the current signal source to screen projection device 1 through setCurrentSource of SourceManager. In this way, after switching the signal source in conference mode, the algorithm processing module 208 can calculate the width width1 and height height1 of the screen projection data pushed by screen projection device 1 displayed in the playback window, and calculate the display position coordinates (x1, y1) of the screen projection data based on the aspect ratio Ratio1 of the screen projection data pushed by screen projection device 1, the preset width width0 of the playback window of the current signal source in conference mode, and the preset height height0 of the playback window of the current signal source in conference mode. Referring to Figure 17, it can be calculated as follows: width1 = width0; height1 = width1 / Ratio1; x1 = 0; y1 = (height0-height1) / 2;

[0246] In some embodiments, in conference mode, the conference mode template is relatively fixed. For example, the position and size of the signal source playback window, the position and size of the signal source list View, etc. are preset in the conference mode template. In this way, the algorithm processing module 208 only needs to dynamically and adaptively match the size and position of the projection data pushed by the projection device 1 in the playback window of the signal source according to Ratio1, thereby generating layout data, and can send the layout data to the View management module 204 through the message processing module 203. For example, the playback window size of the signal source preset in the conference mode template is 1600*900, and the aspect ratio of the data stream pushed by the projection device 1 is Ratio1=1920 / 808. Then the algorithm processing module 208 can calculate the size of SurfaceView / TextureView as width1*height1=1600*746, (x1,y1)=(0,77). The structure of the layout data can be: { Session:SessionClient1, View:SurfaceView / TextureView, MViewPosition:1, Position:(0,77), Width:1600, Height:746, …}

[0247] In some embodiments, when switching to conference mode, the View management module 204 can load a conference mode template, which may specifically include but is not limited to: initializing a playback window View and a signal source list View for displaying the signal source, and using the Recyclerview component to display the signal source list in the signal source list View according to the signal source list sent by the data management module 202. The Recyclerview component may include multiple item layouts, which may include but are not limited to: signal source name items, interface icon items, connection status items, etc. mapped to each signal source option, and may also include the focus item of the current signal source (mVisible=true). Among them, items involving text information (for example: signal source name items) can be displayed using TextView (text control); items involving image content (for example: interface icons, connection status, focus, etc.) can be displayed using ImageView (image control), thereby creating a complete signal source list View. When the current signal source is switched or the user moves the focus in the signal source list View, the View management module 204 can reset the focus state, and / or, when the connection status of the signal source changes, the View management module 204 can synchronize the connection status item of the signal source, thereby completing the refresh of the signal source list View.

[0248] In some embodiments, after the View management module 204 receives the layout data in the conference mode, it can set the width width1 and height height1 of the projection data through the setLayoutParams interface, and can set the display position of the projection data according to (x1, y1), thereby displaying the projection data sent by the current signal source (for example, projection device 1) in the playback window, and can set the focus item for the signal source item corresponding to projection device 1 in the signal source list View, thereby achieving the interface effect as shown in Figure 17.

[0249] In this application, in the continuous mode switching scenario of M-way projection viewing mode → exclusive mode → conference mode, the SurfaceView / TextureView that can be shared by multiple modes can always be retained. In this way, when switching between different modes, it is only necessary to update the layout data, reset the size and position of the SurfaceView / TextureView, and hide, delete or add certain UI elements (such as the signal source list View) to adapt to different modes. Through the unified scheduling of the mode switching application and the flow of information between modules, continuous and rapid switching of multiple modes is achieved without the need to tediously exit and restart the mode or switch activities (Activity), thereby improving the speed and efficiency of mode switching. When switching from exclusive mode to conference mode, the same signal source can be displayed continuously before and after the switch. The preset conference mode template is loaded by the mode switching application, and the size and position of the current signal source displayed in the playback window are refreshed, thereby achieving a smooth and natural mode switching effect.

[0250] In other embodiments, users can also switch from M-way projection viewing mode → Q-way projection viewing mode → other modes, where other modes include but are not limited to conference mode, picture-in-picture mode (also known as: floating mode), etc., to achieve fast and continuous switching between multiple different modes.

[0251] In some embodiments, when the display device is in Q-channel projection viewing mode (Q≥1), where Q=1, i.e., the aforementioned exclusive mode (see FIG5B ), the user can still call up the mode list 51 and select the conference mode option, thereby inputting a mode switching operation to switch to conference mode. The mode switching application responds to the user's operation to switch to conference mode by controlling the mode management module 205 to switch to conference mode. The conference mode is still shown in FIG9 in detail and will not be further described here.

[0252] Figure 18 is a schematic diagram of the interface change from the dual-screen projection viewing mode to the conference mode according to some embodiments of the present application. Referring to Figure 18, the dual-screen projection viewing mode may include two projections, corresponding to projection device 2 (Client2) and projection device 3 (Client3), respectively. The mode switching application responds to the user's operation of switching to the conference mode, and the control mode management module 205 switches to the conference mode. The SurfaceView / TextureView (corresponding to SessionClient2) and layout (corresponding to SessionClient3) in the original dual-screen projection viewing mode continue to the conference mode, and other newly added View content in the conference mode needs to be preloaded. Among them, the newly added View content in the conference mode is detailed in the above description and will not be repeated here.

[0253] In some embodiments, after the mode management module 205 switches to conference mode, the data management module 202 can rewrite and update the signal source list, then generate a complete signal source list based on the interface signal source list and the projection device list, and synchronize the signal source list to the View management module 204 through the message processing module 203. This signal source list is the content basis of the signal source list View created by the View management module 204 in conference mode.

[0254] In some embodiments, the data management module 202 can rewrite and update the signal source list before synchronizing the signal source list to the View management module 204. See Figure 18. Before switching to the conference mode, the device names corresponding to the two projections in the dual-screen viewing mode are "projection device 2" (ID is Client2) and "projection device 3" (ID is Client3). After the mode management module 205 switches to the conference mode, the data management module 202 can rewrite the index numbers of the device names of the two existing projections.

[0255] In some embodiments, the name rewriting rules of the data management module 202 are not limited. For example, the index number of the projection device that is connected later can be set to a smaller one according to the order of projection access. For example, if Client2 is connected earlier than Client3, the data management module 202 can rewrite the name of the original projection device 3 (Client3) to "projection device 1", and the name of the original projection device 2 (Client2) remains unchanged (that is, it is still "projection device 2"), so that the most recently connected projection device is displayed in the front position of the signal source list View. For another example, before switching to conference mode, the focus is on the projection window corresponding to Client2, then the data management module 202 can rewrite the name of the original projection device 2 (Client2) to "projection device 1", and rewrite the name of the original projection device 3 (ID is Client3) to "projection device 2".

[0256] In some embodiments, after rewriting the index number of the device name, the data management module 202 can preferentially set the mVisible status value of "screen projection device 1" to true, and set the mVisible of other signal sources in the signal source list to false. The focus management module 214 can set the focus on screen projection device 1 and send the focus information to the screen projection management module 201 through the message processing module 203.

[0257] In some embodiments, the screen projection management module 201 receives focus information and can set the current signal source to screen projection device 1 through setCurrentSource of SourceManager. In this way, after switching the current signal source in conference mode, the algorithm processing module 208 can calculate the width width1 and height height1 of the screen projection data pushed by the screen projection device 1 displayed in the first area 91, and calculate the display position coordinates (x1, y1) of the screen projection data according to the aspect ratio Ratio1 of the screen projection data pushed by the screen projection device 1, the preset width width0 of the playback window of the current signal source in conference mode, and the preset height height0 of the playback window of the current signal source in conference mode.

[0258] In Figure 18, the data management module 202 has rewritten the name of the original projection device 2 (Client2) to "projection device 1", and the name of the original projection device 3 (ID is Client3) has been rewritten to "projection device 2". Referring to (b) in Figure 18, when switching to conference mode, the focus is on "projection device 1" in the signal source list View, and the first area 91 displays the projection content pushed by projection device 1 (Client2). The algorithm processing module 208 can calculate it in the same way as (x1, y1) when switching from exclusive mode to conference mode, which will not be repeated here.

[0259] In some embodiments, in conference mode, the conference mode template is relatively fixed. For example, the position and size of the signal source playback window, the position and size of the signal source list View, etc. are preset in the conference mode template. In this way, the algorithm processing module 208 only needs to dynamically and adaptively match the size and position of the projection data pushed by the projection device 1 in the playback window of the signal source according to Ratio1, thereby generating layout data and sending the layout data to the View management module 204 through the message processing module 203. For example, the playback window size of the signal source preset in the conference mode template is 1600*900, and the aspect ratio of the data stream pushed by projection device 1 is Ratio1=1920 / 808. Then the algorithm processing module 208 calculates width1*height1=1600*746, (x1, y1)=(0,77). The structure of the layout data can be: { Session:SessionClient2, View:SurfaceView / TextureView, MViewPosition:1, Position:(0,77), Width:1600, Height:746, …}

[0260] In some embodiments, in conference mode, the user can switch the signal source, see (c) in Figure 18, the user can switch the signal source to the projection device 2, the focus management module 214 can set the focus on the projection device 2, and send the focus information to the projection management module 201 through the message processing module 203.

[0261] In some embodiments, the screen projection management module 201 receives focus information and can set the current signal source to screen projection device 2 through setCurrentSource of SourceManager, and can send a Source switching message to the data management module 202 through the message processing module 203. When the data management module 202 receives the Source switching message, it can change the mVisible status value of screen projection device 1 from true to false, and can change the mVisible status value of screen projection device 2 from false to true. At this time, the mVisible of other signal sources except screen projection device 2 are all false. In this way, after switching the current signal source in conference mode, the algorithm processing module 208 can calculate the width width2 and height height2 of the screen projection data pushed by screen projection device 2 in the first area 91, and calculate the display position coordinates (x2, y2) of the screen projection data based on the aspect ratio Ratio2 of the screen projection data pushed by screen projection device 2, the preset width width0 of the playback window of the current signal source in conference mode, and the preset height height0 of the playback window of the current signal source in conference mode.

[0262] In some embodiments, referring to (c) in FIG18 , after the signal source is switched in conference mode, the focus moves to “Projection Device 2” in the signal source list View, and the first area 91 displays the projection content pushed by projection device 2 (Client 3). The algorithm processing module 208 can calculate as follows: height2 = height0; width2 = height2 × Ratio2; x2 = (width0 - width2) / 2; y2 = 0;

[0263] In some embodiments, when switching to conference mode, the View management module 204 can load the conference mode template in the same manner as when switching from exclusive mode to conference mode, and when the current signal source is switched or the user moves the focus in the signal source list View, the View management module 204 can reset the focus state, and / or, when the connection status of the signal source changes, the View management module 204 can synchronize the connection status item of the signal source, thereby completing the refresh of the signal source list View.

[0264] In some embodiments, taking the current signal source as projection device 1 as an example, after the View management module 204 receives the layout data in the conference mode, it can set the width width1 and height height1 of the projection data through the setLayoutParams interface, and set the display position of the projection data according to (x1, y1), so as to display the projection data sent by the current signal source (for example, projection device 1) in the playback window, and set the focus item for the signal source item corresponding to projection device 1 in the signal source list View, thereby achieving the interface display effect as shown in (b) or (c) in Figure 18.

[0265] In some embodiments, see (b) in 18, when the current signal source is projection device 1 (Client2), the window corresponding to projection device 2 (Client3) can be hidden, and the algorithm processing module 208 can also set the size and display position of the window corresponding to projection device 2 (Client3) to achieve a hidden / invisible effect. The calculated layout data can also include: { Session:SessionClient3, View:layout, MViewPosition:, Position:(-1,-1), Width:1, Height:1, …}

[0266] In the layout data of the signal source of the above-mentioned projection device 2 (Client3), the algorithm processing module 208 can set width2=1, height2=1, (x2, y2)=(-1, -1), thereby reducing the window corresponding to the projection device 2 and setting its display position beyond the screen display range, thereby achieving the effect of hiding the window.

[0267] Figure 19 is a second schematic diagram of the interface changes when switching from dual-channel projection viewing mode to conference mode according to some embodiments of the present application. In Figure 19, the data management module 202 rewrites the name of the original projection device 3 (Client3) to "projection device 1", and the original projection device 2 (Client2) remains unchanged. In this way, after switching to conference mode, the first area 91 can give priority to displaying the projection content sent by projection device 1 (Client3). The user can move the focus to projection device 2 in the signal source list View and trigger the confirmation key / OK key to switch the Source to projection device 2 in conference mode. In this way, in the scenario of switching from Q-channel projection viewing mode to conference mode, the dual-channel projection can be automatically converted to a single-channel signal source display in conference mode, and the current signal source after switching to conference mode can be automatically decided. There is no need to exit the dual-channel projection viewing mode and restart the conference mode, such as starting a specific conference template application, thereby reducing user manual operations.

[0268] In the continuous mode switching scenario of M-way projection viewing mode → Q-way projection viewing mode → conference mode, the SurfaceView / TextureView and layout that can be shared between multiple modes can always be retained. In this way, when switching between different modes, you only need to update the layout data, reset the size and position of SurfaceView / TextureView and layout, and hide, delete or add certain UI elements (such as signal source list View) to adapt to different modes. Through the unified scheduling of the mode switching application and the flow of information between modules, continuous and fast switching of multiple modes can be achieved without the tedious exit and restart of the mode, and without switching activities (Activity), thereby improving the speed and efficiency of mode switching. Among them, when switching from exclusive mode to conference mode, the continuous display of the same signal source is maintained before and after the switch. The preset conference mode template is loaded through the mode switching application, and the size and position of the current signal source displayed in the playback window are refreshed, achieving a smooth and natural mode switching effect.

[0269] In some embodiments, when the display device is in exclusive mode, as shown in FIG. 5A or FIG. 5B , the user may call up mode list 51 and select the suspension mode option, thereby inputting a mode switching operation to switch to suspension mode. The mode switching application controls mode management module 205 to switch to suspension mode in response to the user's mode switching operation to switch to suspension mode. It should be noted that the user's mode switching entry is not limited to the examples in some embodiments of the present application.

[0270] Figure 20 is a schematic diagram of the interface change from exclusive mode to floating mode according to some embodiments of the present application. Referring to Figure 20, floating mode can refer to the floating display of another relatively small window in the form of a floating layer or floating window on the full-screen interface. For example, when a TV series is playing in full screen on the interface, a small window is floating to display the projection data pushed by the projection device 1.

[0271] For ease of description, in floating mode, in some embodiments of the present application, the large screen window that displays content in full screen can be called the "main screen", and the small screen window that is displayed floating on the main screen can be called the "secondary screen".

[0272] In some embodiments, when the display device is playing the main screen content, for example, when playing the film source a provided by a video application, if a mode switching operation input by the user is received (for example, switching to the M-way projection viewing mode), the SourceManager can record the main screen content information. The main screen content information may include but is not limited to part or all of the content name, content ID, resource link, content playback progress, etc.

[0273] In some embodiments, after the mode management module 205 switches to the floating mode, the projection management module 201 can query the main screen content information through the SourceManager, and can send the main screen content information to the View management module 204 through the message processing module 203, so that the View management module 204 can display the content corresponding to the main screen content information in full screen in the floating mode.

[0274] In some embodiments, when the user switches to the floating mode, the projection management module 201 can query the main screen content information through the interface provided by the Android system (such as PackageManager, etc.). The main screen content information may include the target package name corresponding to the application displayed on the main screen, where the package name is the unique identifier of the application, and the target package name can be sent to the view management module 204 through the message processing module 203, so that the view management module 204 can obtain the main screen content according to the target package name and can display the main screen content in full screen. It should be noted that the method of obtaining the main screen content information is not limited to the examples in some embodiments of the present application.

[0275] In some embodiments, when switching to floating mode, the SurfaceView / TextureView in exclusive mode can be retained, and the size and position of the SurfaceView / TextureView can be reset to obtain the secondary screen. Other View content added in floating mode (such as the main screen) needs to be preloaded. The floating mode can be implemented through the PictureInPicture template or other methods (such as Window).

[0276] In some embodiments, taking the switching and display of the floating mode implemented by the PictureInPicture template as an example, the PictureInPicture template can preset the maximum side length Length(max) of the secondary screen (i.e., the floating window). The maximum side length Length(max) can be the maximum width or maximum height of the secondary screen. For example, if the secondary screen is displayed in landscape mode, the maximum side length Length(max) defines the maximum width of the secondary screen; if the secondary screen is displayed in portrait mode, the maximum side length Length(max) defines the maximum height of the secondary screen.

[0277] In some embodiments, when switching from exclusive mode to floating mode, the projection data pushed by the projection device 1 in exclusive mode can be displayed on the secondary screen, and the main screen content recorded by the SourceManager can be displayed on the main screen. Since the main screen content fills the full screen display, that is, the size of the main screen is width_screen*height_screen, the display position of the main screen can be (0,0), which can be the main screen display parameter preset by the PictureInPicture template. The algorithm processing module 208 can calculate the layout data of the corresponding secondary screen based on the display position of the main screen. The layout data may include the size and position coordinates of the SurfaceView / TextureView.

[0278] In some embodiments, the algorithm processing module 208 can calculate the size of the secondary screen based on the aspect ratio Ratio1 and the maximum side length Length(max) of the projection data pushed by the projection device 1. For example, if the projection data is a horizontal screen, the algorithm processing module 208 can set the secondary screen width width1 = Length(max) and the secondary screen height height1 = Length(max) / Ratio1. If the projection data is a vertical screen, the algorithm processing module 208 can set the secondary screen height height1 = Length(max) and the secondary screen width width1 = Length(max)×Ratio1.

[0279] In some embodiments, referring to FIG20 , when the secondary screen is suspended at a specified position on the main screen, a certain margin may be provided between the secondary screen and the main screen. For example, in FIG20 , the secondary screen is suspended at the lower right position of the main screen. The lower edge of the secondary screen and the lower edge of the main screen may have a margin d1, and the right edge of the secondary screen and the right edge of the main screen may have a margin d2. The margins d1 and d2 may be used to constrain the floating position of the secondary screen relative to the main screen. The floating position may be preset by the PictureInPicture template. The algorithm processing module 208 may set the margin parameters related to the floating position through the setPictureInPictureParams interface. It should be noted that the margin parameters are not limited to d1 and d2. For example, the floating position may be constrained by setting d3 and d4, where d3 may be the distance between the upper edge of the secondary screen and the upper edge of the main screen, and d4 may be the distance between the left edge of the secondary screen and the left edge of the main screen. In this way, the layout data calculated by the algorithm processing module 208 may include the size of the secondary screen width1*height1, and the coordinates (x1, y1) of the floating position of the secondary screen. Referring to the example of FIG. 20 , x1 = width_screen - height(max) - d2, y1 = height_screen - height1 - d1.

[0280] In some embodiments, the View management module 204 can display the content indicated by the main screen content information in full screen on the main screen based on the main screen content information sent by the projection management module 201, and set the size and floating position of the SurfaceView / TextureView according to the layout data through the setLayoutParams interface, and then reset the SurfaceView / TextureView to the secondary screen, finally presenting the effect of switching from exclusive mode to floating mode as shown in the example of Figure 20.

[0281] In the continuous switching scenario of M-way projection viewing mode → exclusive mode → floating mode, the SurfaceView / TextureView that can be shared by multiple modes can always be retained. In this way, when switching between different modes, you only need to update the layout data, reset the size and display / floating position of the SurfaceView / TextureView, and hide, delete or add certain UI elements (such as the main screen, etc.) to adapt to different modes. In this way, through the unified scheduling of the mode switching application and the flow of information between modules, when multiple modes are switched continuously and quickly, there is no need to tediously exit and restart the mode, and no switching of activities (Activity), thereby improving the speed and efficiency of mode switching. Among them, when the exclusive mode is switched to the floating mode, the same projection signal source before and after the switch only presents changes in display form (exclusive or floating), window size (main screen to secondary screen) and display position, and the projection signal source can continue to play as the mode switches, and it is possible to watch the main content on the main screen and the projection content on the secondary screen at the same time.

[0282] In some embodiments, in the mode switching scenario, not only the display control of the interface is involved, but also the sound control is involved.

[0283] In some embodiments, the data management module 202 can also record the sound status of each projection session in the signal source list. The sound status can include the sound state setMute(false) and the mute state setMute(true). Referring to Figure 13, if the three-way projection viewing mode only supports audio output of a single projection, the sound processing module 206 can control the sound status of each projection according to the sound processing mechanism preset in the three-way projection viewing mode. A sound processing mechanism can make sound for the focus window. Assuming that the focus is on the first projection window 12a, the sound processing module 206 can call the setMute(false) interface of SessionClient1 to make the first projection window 12a make sound, and can call the setMute(true) interface of SessionClient2 and SessionClient3 to mute the second projection window 11b and the third projection window 11c. In this way, the sound processing module 206 can send the sound status information to the data management module 202, so that the data management module 202 sets the mute=false of projection device 1, and sets the mute=true of projection device 2 and projection device 3.

[0284] In some embodiments, the sound processing module 206 can also control the sound of the most recently connected screen projection.

[0285] In some embodiments, when switching from a three-way projection viewing mode to a dual-way projection viewing mode, if projection device 1 (mute=false) is abandoned, and projection device 2 (mute=true) and projection device 3 (mute=true) are retained in the two-way projection of the dual-way projection viewing mode, the sound processing module 206 can re-match the projection sound object according to the preset sound processing mechanism. For example, Client3 is connected later than Client2, and since Client2 and Client3 are in a muted state in the original three-way projection viewing mode, the sound processing module 206 can call the setMute(false) interface of SessionClient3 to unmute the window corresponding to Client3 so that it makes a sound, keeping the muted state of Client2 unchanged.

[0286] In some embodiments, when switching from a three-way projection viewing mode to a dual-way projection viewing mode, if projection device 2 (mute=true) is abandoned, projection device 1 (mute=false) and projection device 3 (mute=true) are retained in the two-way projection of the dual-way projection viewing mode, the sound processing module 206 may not perform any processing, so that Client 1 remains in a sounding state and Client 3 remains in a silent state.

[0287] In some embodiments, for the scenario of switching from dual-screen projection viewing mode to conference mode, see (a) in Figure 18, if Client2's mute=false and Client3's mute=true in dual-screen projection viewing mode, then when switching to conference mode, see (b) in Figure 18, the sound processing module 206 may not perform any processing, so that Client2 remains in a sounding state and Client3 remains in a silent state. When switching the signal source in conference mode, see (c) in Figure 18, the sound processing module 206 may call the setMute(true) interface of SessionClient2 to mute the signal source of projection device 1, and call the setMute(false) interface of SessionClient3 to unmute / sound the signal source of projection device 2.

[0288] In some embodiments, for the scenario of switching from dual-channel projection viewing mode to conference mode, see (a) in Figure 18, if in the dual-channel projection viewing mode, mute = true of Client2 and mute = false of Client3, then when switching to conference mode, see (b) in Figure 18, the sound processing module 206 can call the setMute(false) interface of SessionClient2 to unmute / sound the signal source of projection device 1, and call the setMute(true) interface of SessionClient3 to mute the signal source of projection device 2.

[0289] In some embodiments, the mode switching application can configure the mode template and can also configure the sound processing rules for each mode. For example, when entering the M-way projection viewing mode, a sound processing rule can be: allowing at most only one projection to make sound, then the sound processing module 206 can query the focus management module 214 for the projection window that currently has the focus (hereinafter referred to as: focus window), and can set the focus window setMute(false) through the setMute (set mute) interface of each Session to make the focus window make sound, and set the other M-1 projection windows setMute(true) to make the M-1 non-focus windows mute. In some embodiments of the present application, setMute(false) can be used to indicate a non-muted or audible state, and setMute(true) can be used to indicate a muted or silent state.

[0290] In some embodiments, for example, when entering exclusive mode, a sound processing rule may be: the sound processing module 206 setsMute(false) the sound signal transmitted by the signal source that exclusively occupies the screen (such as the projection device 1), and setsMute(true) the sound signal of other connected or hidden signal sources.

[0291] In some embodiments, for example, when entering conference mode, a sound processing rule may be: the sound processing module 206 sets the current signal source (mVisible=true) to setMute(false) and sets other connected signal sources (mVisible=false) to setMute(true).

[0292] In some embodiments, for example, when entering the floating mode, a sound processing rule may be: only the main screen is allowed to make sound, and the secondary screen is not allowed to make sound. The sound processing module 206 may set the sound signal of the main screen to setMute(false) and the sound signal of the secondary screen to setMute(true).

[0293] In some embodiments, for example, when entering a three-way split-screen game mode, a sound processing rule may be to allow multiple split screens, such as a three-way split screen, to emit sound simultaneously to facilitate interaction between game players. In this case, the sound processing module 206 may set the sound signals of the multiple split screens to setMute(false). It should be noted that sound processing rules can be defined according to different modes, and the sound processing rules in each mode are not limited to the examples in some embodiments of the present application.

[0294] The UI drawings provided in this application are only examples. According to the configuration of the mode category, projection capability, number and category of signal sources, connection status of each signal source, changes in the current signal source, user interaction, mode templates and typesetting layout of the mode switching application, the underlying software architecture, module function configuration and interactive flow between modules of the mode switching application can be designed to realize the display control and refresh of the mode interface in the scenario of continuous mode switching, as well as the sound processing adapted to different modes, and are not limited to the examples in the drawings of this application. The typesetting algorithms of the algorithm processing module 208 in different modes and the mode switching scenarios are not limited to the examples of the embodiments of this application. There are many mode switching scenarios, and this application does not list them one by one. When switching modes, any mode switching and continuous switching of multiple modes can be realized through the mode switching application provided by this application.

[0295] In some embodiments, the mode switching application runs at the application layer and is controlled by at least one processor 250 in the display device. Therefore, the at least one processor 250 can execute a computer program stored in a memory to cause the display device to control the mode switching application to perform a mode switching method for the display device. The software architecture of the mode switching application is not limited to the example provided in FIG. 4B .

[0296] FIG21 is a flowchart of a mode switching method for a display device according to some embodiments of the present application. Referring to FIG21 , the method may include but is not limited to the following steps:

[0297] Step S151: After the display device is powered on, a control mode switching application is started.

[0298] In some embodiments, the display device may start the mode switching application when the display device is turned on, so that when a mode switching operation is received, the mode switching application can respond more quickly and control the mode switching application to switch to the target mode.

[0299] Step S152: When the mode switching application monitors the connection of the first screen projection device, it creates a first window and controls the display to display the screen projection data sent by the first screen projection device in the first window.

[0300] In some embodiments, the first window corresponds to a SurfaceView / TextureView.

[0301] Step S153, in response to the mode switching instruction input by the user, the control mode switching application switches the currently recorded projection display mode to the target mode indicated by the mode switching instruction.

[0302] Step S154: Control the mode switching application to obtain the content to be displayed in the target mode, where the content to be displayed includes the projection data sent by the first projection device.

[0303] In some embodiments, the content to be displayed in the target mode can be adapted based on factors such as different mode scenarios, the number of projection channels, the current signal source, etc., and the details can be referred to the description of the relevant embodiments above. For example, if the display device is only connected to one projection channel, but the user switches to dual-screen viewing mode, the projection guide can be used to compensate and fill the second projection channel. In this way, the content to be displayed can include the projection data sent by the first projection device and the projection guide page.

[0304] Step S155: Control the mode switching application to generate interface layout data matching the target mode. The interface layout data is used to indicate display parameters of each content to be displayed. The interface layout data at least includes the size and display position information of the first window after reset.

[0305] The method for the algorithm processing module 208 to generate typesetting data adapted to different modes has been described in detail in the previous embodiments. Please refer to the description of the aforementioned related embodiments and will not be repeated here.

[0306] Step S156 , controlling the display to display the content to be displayed according to the interface layout data, so that the user interface is updated following the mode switching.

[0307] Some embodiments of the present application provide a mode switching application that can be controlled by at least one processor of a display device, and switching between different modes is achieved through the mode switching application. The mode switching application can monitor the connection and disconnection of the projection device, and when monitoring the connection of the first projection device, create a first window to display the projection data transmitted by the first projection device. At this time, if a mode switching operation is received, the mode switching application first needs to change the recorded mode to the target mode so that the mode switching application can execute the layout operation and interface update procedures for the target mode. In this way, the mode switching application can match the content to be displayed in the target mode with the first layout data. The first layout data may include parameters such as the size and display position after the first window created previously is reset. In this way, the content to be displayed can be displayed according to the first layout data, so that the interface can be automatically refreshed following the mode switch.

[0308] In this application, the first window can be a UI element that can be shared before and after the mode switch. It only needs to reset the shared UI elements, such as resetting the size and display position of the first window in different modes, so that the UI elements shared between different modes can be retained for continuous use in the same mode switching application, and other non-shared UI elements can be added or deleted to adapt to the mode. In this way, there is no need to switch modes by exiting the old mode and restarting the new mode, nor is there a need to switch activities, which improves the speed and efficiency of mode switching, especially for multi-mode continuous switching scenarios, which can achieve smooth and easy mode switching and improve the refresh efficiency of the interface following the mode switch.

[0309] FIG22 is a flowchart of another mode switching method for a display device according to some embodiments of the present application. Referring to FIG22 , the method may include but is not limited to the following steps:

[0310] Step S161: After the display device is powered on, a control mode switching application is started.

[0311] In some embodiments, the display device may start the mode switching application when the display device is turned on, so that when a mode switching operation is received, the mode switching application can respond more quickly and control the mode switching application to switch to the target mode.

[0312] Step S162, in response to the mode switching instruction indicating switching from the first multi-way screen projection mode to the second multi-way screen projection mode, control the mode switching application to obtain the first screen projection information and the second screen projection information.

[0313] In some embodiments, the first multi-channel projection mode can be equivalent to the aforementioned M-channel projection viewing mode, and the second multi-channel projection mode can be equivalent to the aforementioned Q-channel projection viewing mode. The first projection information is the projection information of at least one projection channel shared by the first and second multi-channel projection modes, and the second projection information is the projection information of at least one projection channel deleted or added in the second multi-channel projection mode compared to the first multi-channel projection mode. The projection information includes, but is not limited to, projection device information and projection window information.

[0314] Step S163, the control mode switching application calculates the interface layout data adapted to the second multi-way screen projection mode according to the first screen projection information and the second screen projection information. The interface layout data includes the display parameters of the previously created shared window after reset indicated by the first screen projection information.

[0315] The method for the algorithm processing module 208 to generate typesetting data adapted to different modes has been described in the previous embodiment. Please refer to the description of the aforementioned related embodiments and will not be repeated here.

[0316] Step S164, controlling the display to display the projection content of at least one projection included in the second multi-screen projection mode according to the interface layout data.

[0317] According to a mode switching application provided in some embodiments of the present application, interface layout and content updates are implemented when switching between different modes. For the switching scenarios between multi-way projection modes, such as switching from the first multi-way projection mode to the second multi-way projection mode, the mode switching application can capture the commonalities and differences between the two modes. Among them, the commonalities are reflected by the first projection information, and the first projection information may include the projection information that is the same and shareable between the two modes before and after the switch, such as projection device information, projection window UI information, etc., so that the association between the two modes before and after the switch can be established based on the first projection information, so that the shared content in the first multi-way projection mode (such as the same UI window, other elements, etc.) can be used in the second multi-way projection mode. The differences are reflected by the second projection information, and the second projection information is used to indicate the projections that need to be deleted or added after the mode is switched. For example, when switching from a three-way projection mode to a two-way projection mode, one projection needs to be discarded.

[0318] In this way, the mode switching application uses the first projection information and the second projection information as the basis to calculate the interface layout data of the second multi-way projection mode after the switching. The interface layout data includes the display parameters after the shared window is reset. For example, when switching from three-way projection mode to dual-way projection mode, it is only necessary to reset the shared window corresponding to the two-way projection retained in the dual-way projection mode and change it to the size and display position required by the dual-way projection mode template. There is no need to exit the three-way projection mode and restart the dual-way projection mode, nor is there any need to recreate the window in the dual-way projection mode. The interface can be refreshed quickly according to the updated layout data following the mode switch.

[0319] According to the mode switching method shown in some embodiments of the present application, it is possible to retain UI elements shared between different modes in the same mode switching application for continuous use, and to add or delete other non-shared UI elements to adapt to the mode. In this way, there is no need to switch modes by exiting the old mode and restarting the new mode, nor is there a need to switch activities, which improves the speed and efficiency of mode switching. In particular, for multi-mode continuous switching scenarios, smooth and easy mode switching can be achieved, and the refresh efficiency of the interface following the mode switching is improved.

[0320] In the process of any of the above methods, the implementation scheme of mode switching is described from the perspective of the overall control mode switching application of the display device. When the mode switching application is controlled by the display device to execute relevant program steps, you can refer to the previous relevant embodiments and realize the automatic refresh of the user interface following the mode switch through the interaction and flow of the underlying modules of the mode switching application. I will not go into details here.

[0321] According to some embodiments of the present application, a computer-readable non-volatile storage medium is also provided, which may store a program. When the storage medium is configured in the display device 200, the program may include the program steps involved in the mode switching application or at least one process 250 configured in the above embodiment. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0322] Finally, it should be noted that the above embodiments are only used to illustrate the solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding solutions from the scope of the embodiments of the present application.

[0323] For ease of explanation, the above description has been provided with reference to specific embodiments. However, the above discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations are possible. The above embodiments have been selected and described to better explain the content of this disclosure, thereby enabling those skilled in the art to better utilize the embodiments.

Claims

1. A display device, comprising: a display configured to display a user interface in different modes; A communication device configured to communicate with the screen projection device; a memory configured to store a computer program; At least one processor running a mode switching application, the at least one processor connected to the display, the communication device and the memory, and configured to execute the computer program to cause the display device to perform: After the display device is powered on, the control mode switching application is started; When the mode switching application monitors that the first screen projection device is connected, the mode switching application is controlled to create a first window, and the display is controlled to display the screen projection data sent by the first screen projection device in the first window; In response to a mode switching operation input by a user, controlling the mode switching application to switch the currently recorded projection display mode to a target mode indicated by the mode switching instruction; Controlling the mode switching application to obtain content to be displayed in the target mode, where the content to be displayed includes the projection data sent by the first projection device; Controlling the mode switching application to generate first layout data matching the target mode, the first layout data being used to indicate display parameters of each item of content to be displayed, the first layout data at least including a display size and a display position of the first window after being reset; Control the display to display the content to be displayed according to the first layout data.

2. The display device according to claim 1 , wherein the controlling the mode switching application acquires the content to be displayed in the target mode, and the at least one processor is further configured to execute the computer program to cause the display device to perform: If the target mode is an M-way projection mode, control the mode switching application to query the number of projection channels, where the number of projection channels is the number N of projection devices connected to the display device, where M≥2; If N is equal to M-1, the mode switching application determines that the content to be displayed includes the projection content and the projection guide page sent by N projection devices; If N is equal to M, the mode switching application determines that the content to be displayed includes the projection content sent by N projection devices; If N is greater than M, control the mode switching application to filter out M second projection devices from N projection devices according to preset rules, and determine that the content to be displayed includes the projection content sent by the M second projection devices.

3. The display device according to claim 1 , wherein the controlling the mode switching application generates first layout data matching the target mode, and the at least one processor is further configured to execute the computer program to cause the display device to perform: If the target mode is an M-way projection mode, control the mode switching application to calculate the display size and display position of the first window after reset, and calculate the display size and display position of the newly created M-1 second windows according to the projection configuration information and screen resolution sent by the M projection devices; wherein, The screen projection configuration information includes the aspect ratio of the screen projection data recorded by the screen projection device.

4. The display device according to claim 2, wherein the at least one processor is further configured to execute the computer program to cause the display device to perform: When the display device is in the M-way screen projection mode, in response to the user switching the focus window to the exclusive mode, the mode switching application is controlled to switch the currently recorded screen projection display mode to the exclusive mode; wherein, The focus window is any one of the M windows in M-way projection mode; Controlling the mode switching application to calculate second layout data matching the exclusive mode according to the aspect ratio and screen resolution of the target projection content displayed by the focus window, where the second layout data includes the display size and display position of the focus window after it is reset; Control the display to display the target projection content according to the second layout data.

5. The display device according to claim 4, wherein the at least one processor is further configured to execute the computer program to cause the display device to perform: When switching to exclusive mode, controlling the display to hide non-focus windows, where the non-focus windows are M-1 windows other than the focus window in the M-way screen projection mode; In response to the user's operation of returning to the M-way screen projection mode, the display is controlled to display the non-focus window that was previously hidden, and M windows are displayed according to the first layout data.

6. The display device according to claim 4, wherein the at least one processor is further configured to execute the computer program to cause the display device to: When switching to exclusive mode, controlling the display to close non-focus windows, wherein the non-focus windows are other M-1 windows except the focus window in the M-way screen projection mode; Query the third projection device from which the projection content displayed in the non-focus window comes, and control the mode switching application to disconnect from the third projection device through the communication device.

7. The display device according to claim 6, wherein the at least one processor is further configured to execute the computer program to cause the display device to: In response to the user's operation of returning to the M-way screen projection mode, controlling the mode switching application to query the types of screen projection content previously sent by M-1 third screen projection devices; If the M-1 third projection devices first send a resource link, control the mode switching application to create M-1 third windows corresponding to the M-1 third projection devices, control the display to display 1 of the focus windows and M-1 third windows according to the first layout data, and load the content indicated by the corresponding resource link in the M-1 third window; If the M-1 third screen projection devices first sent screen recording data, the display is controlled to display a first prompt message, and the first prompt message is used to prompt reconnection with the M-1 third screen projection devices to restore to the M-way screen projection mode before switching to the exclusive mode.

8. The display device according to claim 4, wherein the at least one processor is further configured to execute the computer program to cause the display device to: When the display device is in the exclusive mode, in response to the user's operation of switching to the conference mode, the mode switching application is controlled to switch the currently recorded projection display mode to the conference mode; wherein, The conference mode is configured to: display the content transmitted by the current signal source in the first area of ​​the conference interface, and display a signal source list in the second area of ​​the conference interface, wherein the signal source list includes information items of interface signal sources and projection signal sources; Controlling the mode switching application to calculate third layout data matching the conference mode according to the aspect ratio of the target projection content displayed by the focus window and the preset width and preset height of the first area, wherein the third layout data includes a display position and a display size of the reset focus window within the first area; Control the display to display the target projection content in the first area according to the third layout data, display the signal source list in the second area, and control the focus in the signal source list to select the information item of the target projection device from which the target projection content comes, so as to prompt the target projection signal source as the current signal source through the focus.

9. The display device according to claim 4, wherein the at least one processor is further configured to execute the computer program to cause the display device to: When the display device is in the exclusive mode, in response to the user's operation of switching to the floating mode, the mode switching application is controlled to switch the currently recorded projection display mode to the floating mode; wherein, The floating mode is configured to: display the secondary screen in a floating manner on the main screen, the main screen is full screen size, and the secondary screen is smaller than the main screen; Controlling the mode switching application to calculate fourth layout data matching the suspension mode based on the aspect ratio of the target projection content displayed by the focus window and the maximum side length preset for the suspension mode; wherein the maximum side length is used to constrain the maximum width or maximum height of the secondary screen, and the fourth layout data includes the display size and display position when the focus window is reset to become the secondary screen; Control the display to display the first content in full screen on the main screen, and display the target projection content on the secondary screen according to the fourth layout data; wherein, the first content is the content played by the display device before receiving the mode switching operation, or the first content is the main screen content specified by the user.

10. A display device comprising: a display configured to display a user interface in different modes; A communication device configured to communicate with the screen projection device; a memory configured to store a computer program; At least one processor running a mode switching application, the at least one processor being connected to the display, the communication device, and the memory, and configured to execute the computer program to cause the display device to perform: After the display device is powered on, the control mode switching application is started; In response to a mode switching instruction indicating switching from a first multi-channel screen projection mode to a second multi-channel screen projection mode, controlling the mode switching application to switch a currently recorded screen projection display mode to the second multi-channel screen projection mode; Control the mode switching application to obtain first screen projection information and second screen projection information; wherein, the first screen projection information is the projection information of at least one screen projection shared by the first multi-channel screen projection mode and the second multi-channel screen projection mode, and the second screen projection information is the projection information of at least one screen projection deleted or added by the second multi-channel screen projection mode compared to the first multi-channel screen projection mode, and the screen projection information includes screen projection device information and screen projection window information; Controlling the mode switching application to calculate interface layout data adapted to the second multi-screen projection mode based on the first screen projection information and the second screen projection information, the interface layout data including display parameters of a previously created shared window indicated by the first screen projection information after being reset, the shared window being a projection window retained from the first multi-screen projection mode to the second multi-screen projection mode; Control the display to display the projection content of at least one projection included in the second multi-screen projection mode according to the interface layout data.

11. The display device according to claim 10, wherein the controlling the mode switching application to obtain the first projection information and the second projection information, the at least one processor is further configured to execute the computer program to cause the display device to perform: Control the mode switching application to query the first projection path M included in the first multi-channel projection mode, and obtain the second projection path Q included in the second multi-channel projection mode; If the first projection path M is less than the second projection path Q, the mode switching application determines that the first projection information includes the projection information of the M-path projection displayed in the first multi-path projection mode, and determines that the second projection information includes the projection information of the QM-path newly added projection.

12. The display device according to claim 11, wherein the controlling the mode switching application to obtain the first projection information and the second projection information, the at least one processor is further configured to execute the computer program to cause the display device to perform: If the first projection path M is greater than the second projection path Q, the mode switching application is controlled to automatically filter out the Q-path first target projection from the M-path projection of the first multi-path projection mode according to the first preset filtering rule, and determine that the first projection information includes the projection information of the Q-path first target projection, and determine that the second projection information includes the projection information of the MQ-path projection that is not selected by the preset first filtering rule and is discarded.

13. The display device according to claim 11, wherein the controlling the mode switching application to obtain the first projection information and the second projection information, the at least one processor is further configured to execute the computer program to cause the display device to perform: If the first projection path number M is greater than the second projection path number Q, the display is controlled to display a first prompt pop-up window, where the first prompt pop-up window is used to prompt the user to select Q-path first target projection from the optional device list, where the optional device list includes the projection information of the M-path projection included in the first multi-path projection mode; Control the mode switching application to obtain the first projection screen set submitted by the user through the first prompt pop-up window, determine that the first projection screen information includes the projection screen information in the first projection screen set, and determine that the second projection screen information includes the projection screen information in the second projection screen set; wherein, The first projection set includes the projection information of the Q-channel first target projection selected by the user, and the second projection set includes the projection information that is not included in the first projection set in the optional device list.

14. The display device according to claim 11, wherein the control mode switching application calculates interface layout data adapted to the second multi-screen projection mode based on the first screen projection information and the second screen projection information, and the at least one processor is further configured to execute the computer program to cause the display device to perform: Control the mode switching application to create the first window corresponding to the newly added projection screen on the QM path; Control the mode switching application to set the size and display position of QM first windows according to the mode template of the second multi-channel projection mode, the aspect ratio of the Q-channel projection and the screen resolution, and reset the size and display position of the shared window corresponding to the M-channel projection displayed in the first multi-channel projection mode.

15. The display device according to claim 12 or 13, wherein the control mode switching application calculates interface layout data adapted to the second multi-screen projection mode based on the first screen projection information and the second screen projection information, and the at least one processor is further configured to execute the computer program to cause the display device to perform: Controlling the mode switching application to disconnect the abandoned MQ path screen projection through the communication device; Control the mode switching application to destroy the second window corresponding to the abandoned MQ path screen projection; Control the mode switching application to reset the size and display position of the shared window corresponding to the Q-way first target projection according to the mode template of the second multi-way projection mode, the aspect ratio and screen resolution of the Q-way first target projection.

16. The display device according to claim 12 or 13, wherein the control mode switching application calculates interface layout data adapted to the second multi-screen projection mode based on the first screen projection information and the second screen projection information, and the at least one processor is further configured to execute the computer program to cause the display device to perform: Control the mode switching application to hide the second window corresponding to the abandoned MQ screen projection, wherein the abandoned MQ screen projection remains connected; Control the mode switching application to reset the size and display position of the shared window corresponding to the Q-way first target projection according to the mode template of the second multi-way projection mode, the aspect ratio and screen resolution of the Q-way first target projection.

17. The display device according to claim 11, wherein the at least one processor is further configured to execute the computer program to cause the display device to: When the display device is in the second multi-channel screen projection mode, in response to the user's operation of switching to the conference mode, the mode switching application is controlled to switch the currently recorded screen projection display mode to the conference mode; wherein, The conference mode is configured to: display the content transmitted by the current signal source in the first area of ​​the conference interface, and display a signal source list in the second area of ​​the conference interface, wherein the signal source list includes information items of interface signal sources and projection signal sources; Control the mode switching application to select a second target projection screen as the current signal source from the Q projection screens included in the second multi-projection mode according to a second preset screening rule; The control mode switching application switches the Source to the current signal source; Control the display to display the projection content sent by the current signal source in the first area of ​​the conference interface, and display the signal source list in the second area of ​​the conference interface, and move the focus to the information item corresponding to the current signal source in the signal source list.

18. The display device according to claim 17, wherein after the control mode switching application switches Source to the current signal source, the at least one processor is further configured to execute the computer instructions to cause the display device to execute: Control the mode switching application to hide the third window corresponding to the other Q-1 screen projections except the second target screen projection; Control the mode switching application to reset the size and display position of the shared window corresponding to the second target screen projection according to the mode template of the conference mode and the aspect ratio of the screen projection content sent by the current signal source.

19. A mode switching method for a display device, comprising: After the display device is powered on, the control mode switching application is started; When the mode switching application monitors access to the first screen projection device, controlling the mode switching application to create a first window and displaying the projection data sent by the first screen projection device in the first window; In response to a mode switching operation input by a user, controlling the mode switching application to switch the currently recorded projection display mode to a target mode indicated by the mode switching instruction; Controlling the mode switching application to obtain content to be displayed in the target mode, where the content to be displayed includes the projection data sent by the first projection device; Controlling the mode switching application to generate first layout data matching the target mode, the first layout data being used to indicate display parameters of each item of content to be displayed, the first layout data at least including a display size and a display position of the first window after being reset; The content to be displayed is displayed according to the first layout data.

20. A mode switching method for a display device, comprising: In response to a mode switching instruction indicating switching from a first multi-channel screen projection mode to a second multi-channel screen projection mode, controlling the mode switching application to switch a currently recorded screen projection display mode to the second multi-channel screen projection mode; The control mode switching application obtains the first projection information and the second projection information; wherein, the first projection information is the projection information of at least one projection shared by the first multi-channel projection mode and the second multi-channel projection mode, and the second projection information is the projection information of at least one projection deleted or added by the second multi-channel projection mode compared with the first multi-channel projection mode, and the projection information includes projection device information and projection window information; Controlling the mode switching application to calculate interface layout data adapted to the second multi-screen projection mode based on the first screen projection information and the second screen projection information, the interface layout data including display parameters of a previously created shared window indicated by the first screen projection information after being reset, the shared window being a projection window retained from the first multi-screen projection mode to the second multi-screen projection mode; The screen content of at least one screen projection included in the second multi-screen projection mode is displayed according to the interface layout data.

Images