Electronic equipment and method for adjusting projection screen position

Abstract

The invention provides electronic equipment and a method for adjusting the position of a screen projection picture, and the method comprises the steps: starting a camera in response to a screen projection connection request sent by a mobile terminal, and obtaining a user image shot by the camera; identifying direction information of a user relative to a screen center position through the user image; the center position of the screen is the center line position of the electronic equipment in the vertical direction; calculating a target position according to the direction information; and receiving a screen projection picture sent by the mobile terminal to the electronic equipment, and controlling the display to display the screen projection picture at the target position. The electronic equipment can identify the direction information of the user relative to the center of the screen through the user image shot by the camera, so that the target position of the screen projection picture sent to the electronic equipment by the mobile terminal is automatically adjusted, the direction of the target position of the screen projection picture is consistent with the direction information of the user, the user can conveniently check or control the screen projection picture, and the user experience is improved. Therefore, the problem that the user experience is affected when the direction information of the user is inconsistent with the projection screen direction is solved.

Description

Electronic device and method for adjusting projection screen position

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent applications filed with the China Patent Office on April 12, 2023, with application number 202310391352.3, filed with the China Patent Office on June 7, 2023, with application number 202310673427.7, and filed with the China Patent Office on June 30, 2023, with application number 202310798851.4, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the technical field of electronic devices, and in particular to an electronic device and a method for adjusting the position of a projection screen. Background Art

[0004] Electronic devices can establish a screen projection connection relationship with communication terminals through communication methods such as wireless local area networks, for example, mirroring. Mirroring refers to connecting a mobile terminal to an electronic device via a wired or wireless method, so that the screen displayed on the mobile terminal is projected onto the electronic device. When users mirror the screen of a mobile terminal and an electronic device, they can pre-connect the mobile terminal and the electronic device to a local area network formed by the same router. The mobile terminal will record the content displayed on the mobile terminal in real time, and forward the recorded data to the electronic device through the router, which will decode and display it in real time.

[0005] In electronic devices with a multi-window system, multiple playback windows can be displayed simultaneously. For example, a smart TV can include two playback windows: the left playback window can display the TV homepage or settings page, and the right playback window can display the projected screen of the terminal device. Alternatively, the TV homepage can be displayed full screen on the TV, while the projected screen can be displayed as a floating window on the left or right side of the TV.

[0006] During the screen projection process, the direction of the projected screen will not adjust with the direction of the user's movement. For example, after the projection user establishes the projection connection, the projection screen is in the left window of the electronic device, and the user is also in the left window of the electronic device, that is, the user and the projection screen are in the same direction, which makes it easy for the user to view the projection screen. However, when the user's direction changes, such as moving from the left side of the electronic device to the right side of the electronic device, the projection screen will not move, and the direction of the user and the projection screen will be inconsistent, which is inconvenient for the user to view the projection screen and affects the user experience.

[0007] Summary of the Invention

[0008] In a first aspect, an embodiment of the present application provides an electronic device, including:

[0009] a display configured to display an image and / or a user interface;

[0010] A user interface configured to receive instructions from a user;

[0011] a communication device configured to communicate with an external device according to a predetermined protocol;

[0012] a memory configured to store computer instructions and data associated with a display device;

[0013] An image acquisition interface is configured to connect to a camera to capture a user image, wherein the camera is arranged at the center of the upper or lower frame of the display;

[0014] At least one processor, connected to the display, user interface, communicator and memory, is configured to execute computer instructions to cause the display device to perform:

[0015] In response to a screen projection connection request issued by a mobile terminal, starting the camera, and obtaining a user image captured by the camera;

[0016] Identifying the user's direction information relative to the center of the screen through the user image; the center of the screen is the center line position of the electronic device in the vertical direction;

[0017] Calculating a target position based on the direction information;

[0018] Receive the projection screen sent by the mobile terminal to the electronic device, and control the display to display the projection screen at the target location.

[0019] In a second aspect, an embodiment of the present application provides an electronic device, including:

[0020] a display configured to display an image and / or a user interface;

[0021] A user interface configured to receive instructions from a user;

[0022] a communication device configured to communicate with an external device according to a predetermined protocol;

[0023] a memory configured to store computer instructions and data associated with a display device;

[0024] An image acquisition interface is configured to connect to a camera to acquire local video data, wherein the camera is arranged at the center of the upper or lower frame of the display;

[0025] At least one processor is connected to the display, user interface, communicator and memory, and is configured to execute computer instructions to cause the display device to perform:

[0026] In response to a screen projection request for fitness media resources issued by a mobile terminal, starting the camera and acquiring local video data captured by the camera;

[0027] Controlling the display to display the local video data in full screen in the user interface, and controlling the display to display a projection screen of the fitness media of the mobile terminal in a floating manner in the user interface;

[0028] Identifying, based on the local video data, the direction of the user's head relative to the center of the screen; the center of the screen is the centerline position of the electronic device in the vertical direction;

[0029] Calculating a target position based on the direction information;

[0030] Receive a projection screen of fitness media resources sent by the mobile terminal to the electronic device, and control the display to display the projection screen at the target location.

[0031] In a third aspect, an embodiment of the present application provides a method for adjusting the position of a projection screen, which is applied to the electronic device of the first aspect, wherein the electronic device includes a display, a memory, a communication device, a user interface, an image acquisition interface, and a processor, wherein the communication device is configured to communicate with an external device according to a predetermined protocol, and the image acquisition interface is configured to connect to a camera to capture a user image, and the camera is set at the center of the upper or lower frame of the display; the method for adjusting the position of the projection screen includes:

[0032] In response to a screen projection connection request issued by a mobile terminal, starting the camera, and obtaining a user image captured by the camera;

[0033] Identifying the user's direction information relative to the center of the screen through the user image; the center of the screen is the center line position of the electronic device in the vertical direction;

[0034] Calculating a target position based on the direction information;

[0035] Receive the projection screen sent by the mobile terminal to the electronic device, and control the display to display the projection screen at the target location.

[0036] In a fourth aspect, an embodiment of the present application provides a method for adjusting the position of a projection screen, which is applied to the electronic device of the second aspect, wherein the electronic device includes a display, a memory, a communication device, a user interface, an image acquisition interface, and a processor, wherein the communication device is configured to communicate with an external device according to a predetermined protocol, and the image acquisition interface is configured to connect to a camera to capture a user image, and the camera is set at the center of the upper or lower frame of the display; the method for adjusting the position of the projection screen includes:

[0037] In response to a screen projection request for fitness media resources issued by a mobile terminal, starting the camera and acquiring local video data captured by the camera;

[0038] Controlling the display to display the local video data in full screen in the user interface, and controlling the display to display a projection screen of the fitness media of the mobile terminal in a floating manner in the user interface;

[0039] Identifying, based on the local video data, the direction of the user's head relative to the center of the screen; the center of the screen is the centerline position of the electronic device in the vertical direction;

[0040] Calculating a target position based on the direction information;

[0041] Receive a projection screen of fitness media resources sent by the mobile terminal to the electronic device, and control the display to display the projection screen at the target location. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] FIG1 is a schematic diagram of an operation scenario between an electronic device and a control device according to an embodiment of the present application;

[0043] FIG2 is a block diagram of a hardware configuration of an electronic device according to an embodiment of the present application;

[0044] FIG3 is a block diagram of a hardware configuration of a control device according to an embodiment of the present application;

[0045] FIG4 is a schematic diagram of software configuration in an electronic device according to an embodiment of the present application;

[0046] FIG5 is a schematic diagram of a method for adjusting the position of a projection screen by an electronic device according to an embodiment of the present application;

[0047] FIG6 is a schematic diagram of a process for an electronic device to identify the direction information of a user relative to the center of a screen through a user image according to an embodiment of the present application;

[0048] FIG7 is a schematic diagram showing the relationship between user direction information and the screen center direction according to an embodiment of the present application;

[0049] FIG8 is a schematic diagram showing another relationship between user direction information and the screen center direction according to an embodiment of the present application;

[0050] FIG9 is a schematic diagram of a process of displaying a projection image at a target location by an electronic device according to an embodiment of the present application;

[0051] FIG10 is a schematic diagram of a process for an electronic device to determine a third direction according to an embodiment of the present application;

[0052] FIG11 is a schematic diagram of a process of an electronic device identifying a key user target from multiple users according to an embodiment of the present application;

[0053] FIG12 is a schematic diagram of a method for adjusting the position of a projection screen by an electronic device according to an embodiment of the present application;

[0054] FIG13 is a schematic diagram of a process for an electronic device to identify the direction information of a user's head relative to the center of a screen based on local video data according to an embodiment of the present application;

[0055] FIG14 is a schematic diagram of a user training scenario according to an embodiment of the present application;

[0056] FIG15 is a schematic diagram of a training scenario when the user's head is oriented in a first direction according to an embodiment of the present application;

[0057] FIG16 is a schematic diagram of a training scenario when the user's head direction is in the second direction according to an embodiment of the present application;

[0058] FIG17 is a schematic diagram of a process of adjusting the position of a projection screen according to a user's posture by an electronic device according to an embodiment of the present application;

[0059] FIG18 is a schematic diagram showing an effect of an electronic device adjusting a projection image according to a user's posture according to an embodiment of the present application;

[0060] FIG19 is a schematic diagram of a flow chart of a method for executing multi-channel screen projection by an electronic device according to an embodiment of the present application;

[0061] FIG20 is a schematic diagram of a process of an electronic device interacting with a mobile terminal to obtain an HTTP interaction port number according to an embodiment of the present application;

[0062] FIG21 is a schematic diagram of a process of another electronic device interacting with a mobile terminal to obtain an HTTP interaction port number according to an embodiment of the present application;

[0063] FIG22 is a schematic diagram of a process for obtaining a screen projection device name according to an embodiment of the present application;

[0064] FIG23 is a schematic diagram showing the effect of binding a playback window to a screen projection device name according to an embodiment of the present application;

[0065] FIG24 is a schematic diagram showing the effect of performing multi-channel screen projection based on IP as the dimension according to an embodiment of the present application;

[0066] FIG25 is a schematic diagram showing the effect of performing multi-channel screen projection based on HTTP interaction port numbers according to an embodiment of the present application;

[0067] FIG26 is a schematic diagram of a process for controlling the playback progress and playback status of the projected content in the playback window according to an embodiment of the present application;

[0068] FIG27 is a schematic diagram of a process for an electronic device to generate subscription information according to an embodiment of the present application;

[0069] FIG28 is a schematic diagram showing the effect of subscription information according to an embodiment of the present application;

[0070] FIG29 is another schematic diagram of a flow chart of a method for performing multi-channel screen projection by an electronic device according to an embodiment of the present application;

[0071] FIG30 is a schematic diagram of a multi-channel screen projection method according to an embodiment of the present application;

[0072] FIG31 is a schematic diagram of a display method according to an embodiment of the present application;

[0073] FIG32 is a schematic diagram of a single-window display according to an embodiment of the present application;

[0074] FIG33 is a schematic diagram of another single-window display according to an embodiment of the present application;

[0075] FIG34 is a schematic diagram of a multi-window display according to an embodiment of the present application;

[0076] FIG35 is a schematic diagram of another display method according to an embodiment of the present application;

[0077] FIG36A is a schematic diagram of another multi-window display according to an embodiment of the present application;

[0078] FIG36B is a schematic diagram of another multi-window display according to an embodiment of the present application;

[0079] FIG37 is a schematic diagram of another display method according to an embodiment of the present application;

[0080] FIG38 is a schematic diagram of yet another multi-window display according to an embodiment of the present application;

[0081] FIG39 is a schematic diagram of yet another multi-window display according to an embodiment of the present application;

[0082] FIG40 is a schematic diagram of yet another multi-window display according to an embodiment of the present application;

[0083] FIG41 is a schematic diagram of another display method according to an embodiment of the present application;

[0084] FIG42 is a schematic diagram of yet another multi-window display according to an embodiment of the present application;

[0085] FIG43 is a schematic diagram of another display method according to an embodiment of the present application;

[0086] FIG44 is a schematic diagram of a focus switching device according to an embodiment of the present application;

[0087] Figure 45 is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0088] To make the purpose, technical solutions and advantages of the embodiments of this application more clear, the technical solutions of the embodiments of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them.

[0089] It should be noted that the brief descriptions of the terms in the examples of the present application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of the present application. Unless otherwise specified, these terms should be understood according to their ordinary and usual meanings.

[0090] In the description and claims of the embodiments of this application and the accompanying drawings, the terms "first," "second," "third," etc. are used to distinguish similar or similar objects or entities, and are not necessarily intended to limit a particular order or precedence, unless otherwise noted. It should be understood that the terms used in this manner are interchangeable where appropriate.

[0091] Figure 1 is a schematic diagram of an operation scenario between an electronic device and a control device according to an embodiment of the present application. As shown in Figure 1 , a user can operate an electronic device 200 through a mobile terminal 300 and a control device 100.

[0092] In some embodiments, the mobile terminal 300 can install software applications on the electronic device 200 and connect and communicate via a network communication protocol to achieve one-to-one control operations and data communication. Audio and video content displayed on the mobile terminal 300 can also be transmitted to the electronic device 200 for synchronized display.

[0093] 1 , the electronic device 200 also communicates data with the server 400 via various communication methods. The electronic device 200 may be allowed to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0094] In addition to providing the broadcast receiving television function, the electronic device 200 may also provide an intelligent network television function with computer support functions, including but not limited to network television, smart TV, Internet Protocol Television (IPTV), etc.

[0095] FIG2 is a hardware configuration block diagram of the electronic device 200 in FIG1 provided in an embodiment of the present application.

[0096] In some embodiments, the electronic device 200 includes at least one of a tuner 210, a communication device 220, a detector 230, an external device interface 240, a processor 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0097] In some embodiments, the detector 230 is used to collect signals from the external environment or from interactions with the external environment.

[0098] In some embodiments, the display 260 includes a display screen component for presenting images, and a driving component for driving image display, a component for receiving image signals output from a processor, and a component for displaying video content, image content, and a menu control interface, as well as a user control UI interface, etc.

[0099] In some embodiments, the communication device 220 is a component for communicating with an external device or server 400 according to various communication protocol types.

[0100] In some embodiments, the processor 250 controls the operation of the electronic device and responds to user operations through various software control programs stored in the memory. The processor 250 controls the overall operation of the electronic device 200.

[0101] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0102] In some embodiments, the user interface 280 is an interface that can be used to receive control input.

[0103] Figure 3 is a block diagram of the hardware configuration of the control device in Figure 1 according to an embodiment of the present application. As shown in Figure 3, the control device 100 includes a processor 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0104] The control device 100 is configured to control the electronic device 200 , and can receive user input operation instructions, and convert the operation instructions into instructions that the electronic device 200 can recognize and respond to, playing the role of an interaction intermediary between the user and the electronic device 200 .

[0105] In some embodiments, the control device 100 may be a smart device. For example, the control device 100 may be installed with various applications for controlling the electronic device 200 according to user needs.

[0106] In some embodiments, as shown in FIG. 1 , the mobile terminal 300 or other intelligent electronic device can perform similar functions to the control device 100 after installing an application for controlling the electronic device 200 .

[0107] The processor 110 includes a processor unit 112, RAM 113, ROM 114, a communication interface 130, and a communication bus. The processor 110 is used to control the operation and operation of the control device 100, as well as the communication and coordination between internal components and external and internal data processing functions.

[0108] Under the control of the processor 110, the communication interface 130 communicates control signals and data signals with the electronic device 200. The communication interface 130 may include at least one of a WiFi chip 131, a Bluetooth module 132, an NFC module 133, or other near field communication modules.

[0109] The user input / output interface 140 includes at least one of a microphone 141 , a touch panel 142 , a sensor 143 , a button 144 and other input interfaces.

[0110] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with the communication interface 130, such as a WiFi, Bluetooth, or NFC module, to encode user input commands via the WiFi protocol, Bluetooth protocol, or NFC protocol and transmit them to the electronic device 200.

[0111] The memory 190 is used to store various operating programs, data, and applications for driving and controlling the control device 100 under the control of the processor. The memory 190 can store various control signal instructions input by the user.

[0112] The power supply 180 is used to provide operating power support for various components of the control device 100 under the control of the processor.

[0113] Figure 4 is a schematic diagram of the software configuration in the electronic device in Figure 1 provided in an embodiment of the present application. In some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (referred to as "application layer"), the application framework layer (referred to as "framework layer"), the system library layer (referred to as "system runtime library layer"), and the kernel layer.

[0114] In some embodiments, at least one application is running in the application layer. These applications may be window programs, system settings programs, clock programs, camera applications, etc. that come with the operating system; or they may be applications developed by third-party developers.

[0115] The framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes predefined functions and acts as a processing center, determining the actions taken by applications in the application layer.

[0116] As shown in Figure 4, the application framework layer in the embodiment of the present application includes managers, content providers, and a view system. The manager includes at least one of the following modules: an activity manager for interacting with all activities running in the system; a location manager for providing system services or applications with access to the system location service; a package manager for retrieving various information related to application packages currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0117] In some embodiments, the activity manager is used to manage the lifecycle of each application and general navigation back functionality.

[0118] In some embodiments, a window manager is used to manage all window programs.

[0119] In some embodiments, the system runtime layer provides support for the upper layer, namely the framework layer. When the framework layer is accessed, the operating system will run the C / C++ library contained in the system runtime layer to implement the functions to be implemented by the framework layer.

[0120] In some embodiments, the kernel layer is a layer between hardware and software. As shown in FIG4 , the kernel layer includes at least one of the following drivers: an audio driver, a display driver, a Bluetooth driver, a camera driver, a Wi-Fi driver, a USB driver, an HDMI driver, and sensor drivers (such as a fingerprint sensor, a temperature sensor, a touch sensor, a pressure sensor, etc.).

[0121] In some embodiments, the core layer further includes a power driver module for performing power management.

[0122] In some embodiments, the software programs and / or modules corresponding to the software architecture in FIG. 4 are stored in the first memory or the second memory shown in FIG. 2 or FIG. 3 .

[0123] The mobile terminal 300 and the electronic device 200 can establish a screen projection connection so that the electronic device 200 can synchronously play the screen content displayed by the mobile terminal 300. In an embodiment of the present application, when the mobile terminal 300 and the electronic device 200 are connected for screen projection, a specific screen projection connection protocol can be used. For example, through Miracast, AirPlay, Digital Living Network Alliance (DLNA), Universal Serial Bus (USB) and other connection protocols, or other self-developed network protocols, etc., this application does not specifically limit this.

[0124] Exemplarily, the electronic device 200 can establish a screen projection connection relationship with the mobile terminal 300 through a communication method such as a wireless local area network, for example, mirror projection. Mirror projection refers to the mobile terminal 300 being connected to the electronic device 200 by wired or wireless means, so that the screen displayed in the mobile terminal 300 is projected to the electronic device 200. When the user performs mirror projection on the mobile terminal 300 and the electronic device 200, the mobile terminal and the electronic device can be connected in advance to a local area network composed of the same router. The mobile terminal 300 needs to record the content displayed on the mobile terminal in real time, and forward the recorded data to the electronic device 200 through the router, and the electronic device 200 decodes and displays it in real time.

[0125] In an electronic device with a multi-window system, the electronic device 200 can display multiple playback windows simultaneously. For example, assuming the electronic device 200 is a smart TV, the smart TV can include two playback windows, the left playback window, which can display the TV homepage or settings page, and the right playback window, which can display the screen projection of the mobile terminal 300. Alternatively, the TV homepage can be displayed full screen on the TV, while the projection screen can be displayed as a floating window on the left or right side of the TV.

[0126] During the screen projection process, the direction of the projected screen will not adjust with the direction of the user's movement. For example, after the projection user establishes the projection connection, the projection screen is in the left window of the electronic device 200, and the user is also in the left window of the electronic device 200, that is, the user is in the same direction as the projection screen, which makes it convenient for the user to view the projection screen. However, when the user's direction changes, such as moving from the left side of the electronic device 200 to the right side of the electronic device 200, the projection screen will not move, and the direction of the user and the projection screen will be inconsistent, which is inconvenient for the user to view the projection screen and affects the user experience.

[0127] In order to facilitate the user to view the projected screen, when the position of the projected screen in the multi-window system electronic device 200 is adjusted, it can be controlled by remote control instructions. In some embodiments, a remote control that is compatible with the electronic device 200 can be used to send instructions for switching positions, such as adjusting the position of the projected screen to the left or right. For example, after the user presses the direction key on the remote control, the remote control will send the specified key value to the electronic device 200. After receiving the key value, the electronic device 200 will control the display to adjust the position of the projected screen in the direction of the target position.

[0128] However, in the above adjustment method, users need to switch frequently, and the projection screen will not be adjusted as the user's direction changes. Therefore, when the direction of the projection screen is inconsistent with the user's direction information, it will be inconvenient for users to view the projection screen and operate the projection screen, thereby affecting the user experience.

[0129] In order to solve the problem that the user experience is affected when the user's direction information is inconsistent with the direction of the projection screen, an embodiment of the present application provides an electronic device 200. The electronic device 200 includes a display 260, a communication device 220, an image acquisition interface 290 and a processor 250. Among them, the communication device 220 is configured to establish a projection connection with the mobile terminal 300, and the image acquisition interface 290 is configured to connect to the camera 2901 to capture the user image. The camera 2901 is set at the center of the upper frame or the lower frame of the display 260. The electronic device 200 can identify the user's direction information relative to the center of the screen of the electronic device 200 through the user image captured by the camera 2901, thereby automatically adjusting the target position of the projection screen sent by the mobile terminal 300 to the electronic device 200.

[0130] To facilitate understanding of the technical solutions in the embodiments of the present application, the following describes each step in detail in conjunction with some specific embodiments and drawings. FIG5 is a flow chart of a method for adjusting the position of a projection screen by an electronic device provided in an embodiment of the present application. As shown in FIG5 , when adjusting the position of a projection screen, the electronic device 200 may include the following steps S1-S4, the specific contents of which are as follows:

[0131] Step S1: In response to the screen projection connection request issued by the mobile terminal 300, the electronic device 200 starts the camera 2901 and obtains the user image captured by the camera 2901.

[0132] In the screen projection scenario between the mobile terminal 300 and the electronic device 200, taking the electronic device 200 of the multi-window system as an example, in response to the screen projection connection request issued by the mobile terminal 300, after the mobile terminal 300 and the electronic device 200 are connected via a network protocol or a USB cable, the camera 2901 located at the center of the upper or lower frame of the display 260 can be started, and the user image captured by the camera 2901 can be obtained. In this way, since the camera 2901 is located at the center of the display 260, when the user image is captured, if the user is at the center of the user image, it means that the user is located at the center of the electronic device 200. In other words, the user's direction information relative to the electronic device 200 can be determined by the user image captured by the camera 2901. After step S1 is completed, the following step S2 can be executed.

[0133] Step S2: The electronic device 200 identifies the direction information of the user relative to the center of the screen through the user image.

[0134] The screen center is the center line position of the electronic device 200 along the vertical direction. In some embodiments, the direction information includes at least a first direction and a second direction, and the first direction and the second direction are opposite.

[0135] FIG6 is a flow chart of an electronic device according to an embodiment of the present application identifying the direction information of a user relative to the center of a screen using a user image. As shown in FIG6 , the electronic device specifically includes the following steps:

[0136] S601: When the electronic device 200 recognizes the direction information of the user relative to the center of the screen through the user image, the center of the screen of the electronic device 200 may be obtained first;

[0137] S602: The electronic device 200 analyzes the user image to obtain a user coordinate set of the user target in the user image.

[0138] S603: After the electronic device 200 obtains the user coordinate set, it can compare the pixel coordinates in the user coordinate set with the screen center coordinates and set comparison rules.

[0139] S604: If the coordinates of the pixel points of a preset number ratio in the user coordinate set are greater than the coordinates of the center of the screen, the electronic device 200 may determine that the direction information is a first direction;

[0140] S605: If the coordinates of the pixel points with a preset ratio in the user coordinate set are smaller than the coordinates of the center of the screen, the electronic device 200 may determine that the direction information is the second direction.

[0141] Exemplarily, FIG7 is a schematic diagram of the relationship between the user direction information and the screen center direction provided by an embodiment of the present application. As shown in FIG7 , the dotted line is the screen center of the electronic device 200. The user coordinate set A of the user target is obtained by parsing the user image. Afterwards, the pixel coordinates in the user coordinate set A are compared with the screen center coordinates. After comparison, if the pixel coordinates of a preset number ratio in the user coordinate set A are greater than the screen center coordinates, it can be determined that the user's direction information is the first direction, that is, the direction indicated by the arrow on the left in FIG7 . On the contrary, if the pixel coordinates of a preset number ratio in the user coordinate set A are less than the screen center coordinates, it can be determined that the user's direction information is the second direction, that is, the direction indicated by the arrow on the right in FIG7 . The first direction is opposite to the second direction. In some embodiments, the preset number ratio can be determined according to actual needs and can be determined in combination with pixel density or other relevant factors. This application does not specifically limit this. After step S2 is completed, the following step S3 can be executed.

[0142] Step S3: The electronic device 200 calculates the target position according to the direction information.

[0143] In order to enhance the user's viewing experience of the projected screen, in some embodiments, the electronic device 200 can calculate the target position based on the user's direction information. Figure 8 is a schematic diagram of the relationship between another user direction information and the direction of the center of the screen provided in an embodiment of the present application. As shown in Figure 8, after the electronic device 200 identifies the direction information of the user relative to the center of the screen, it can determine the target position of the projected screen through the user's direction information. For example, taking the first direction as the left side of the user relative to the center of the screen and the second direction as the right side of the user relative to the center of the screen as an example, in order to make the projected screen consistent with the user's direction information, the direction of the target position of the projected screen should be consistent with the user's direction information. When the user is in the second direction, such as the right direction in Figure 8, the projected screen should also be displayed on the right side of the user interface. After step S3 is executed, the following step S4 can be executed.

[0144] Step S4: The electronic device 200 receives the projection screen sent by the mobile terminal 300 to the electronic device 200, and controls the display 260 to display the projection screen at the target location.

[0145] In order to ensure that the direction of the target location of the projection screen is consistent with the user's direction information, in some embodiments, the electronic device 200 can display the projection screen at the target location.

[0146] FIG9 is a schematic diagram of a process for an electronic device according to an embodiment of the present application to display a projection screen at a target location. As shown in FIG9 , the process specifically includes the following steps:

[0147] S901: The electronic device 200 may first obtain an initial direction of the projection image;

[0148] S902: The electronic device 200 detects the user's direction information.

[0149] S903. After detection, if the initial direction of the projection screen is the same as the user's direction information, the electronic device 200 can control the display 260 to display the projection screen at the default position, that is, there is no need to adjust the position of the projection screen.

[0150] S904. If the initial direction of the projection screen is opposite to the direction information of the user, the electronic device 200 can control the display 260 to display the projection screen at the target position.

[0151] In this way, the direction of the target position of the projection screen should be consistent with the user's direction information, which can facilitate the user to view or control the projection screen and enhance the user experience.

[0152] It should be noted that, in the multi-window system electronic device 200, changing the target position of the projected screen does not affect the originally playing page. For example, the projected screen can be displayed in a split-screen with the original page. Thus, when the projected screen is displayed in a first orientation, the original page can be displayed in a second orientation. Similarly, when the projected screen is displayed in a second orientation, the projected screen can be displayed in the first orientation. In other words, when adjusting the target position of the projected screen, only the display position is adjusted, and the originally playing content on the electronic device 200 is not changed.

[0153] To enable the electronic device 200 to display the projected image at the target location, in some embodiments, the electronic device 200 can first establish a transmission channel for the projected image sent by the mobile terminal 300. After the transmission channel is established, the electronic device 200 can receive the projected image sent by the mobile terminal 300 based on the transmission channel. After that, the electronic device 200 can decode the received projected image and, after decoding, control the display 260 to display the projected image at the target location.

[0154] In some embodiments, the direction information also includes a third direction, which is a direction within a preset range of the center of the screen. FIG10 is a flow chart of determining the third direction of an electronic device provided in an embodiment of the present application, as shown in FIG10 , which specifically includes the following steps:

[0155] S1001: When determining the third direction, the electronic device 200 may first calculate the difference between the coordinates of the pixel points in the user coordinate set and the coordinates of the center position of the screen;

[0156] S1002: If the difference values ​​corresponding to the pixel coordinates of a preset number ratio are within a difference range, the electronic device 200 may determine that the direction information is a third direction.

[0157] S1003. After determining that it is the third direction, in response to the direction information being the third direction, the electronic device 200 can control the display 260 to display the projection screen in a floating manner at a preset position of the user interface.

[0158] For example, by calculating the difference between the coordinates of the pixel points in the user coordinate set and the coordinates of the center of the screen, the distance difference between the coordinates of the pixel points in the user coordinate set and the coordinates of the center of the screen can be obtained. When determining the third direction, a difference range can be pre-set. If the difference between the two is within the difference range, the user's direction information can be determined to be the third direction. In some embodiments, the third direction is a direction near the center of the screen of the electronic device 200, that is, the user is near the middle position of the electronic device 200. For this scenario, the electronic device 200 can control the display 260 to suspend the projection screen at a preset position of the user interface, such as in the upper left corner, upper right corner or other positions.

[0159] The above embodiment is applicable to a scenario where there is only one user in the user image. When there are multiple users captured in the user image, the electronic device 200 can also identify a key user target from the multiple users.

[0160] FIG11 is a schematic diagram of a process for an electronic device according to an embodiment of the present application to identify a key user target from multiple users. As shown in FIG11 , the process specifically includes the following steps:

[0161] S1101: The electronic device 200 may first receive a feature image sent by the mobile terminal 300, which includes feature information of a key user target of the screen projection;

[0162] S1102: The electronic device 200 identifies the number of users in the user image.

[0163] S1103: When the number of users is greater than a preset threshold, the electronic device 200 may identify key user targets in the user image based on the feature image;

[0164] S1104: The electronic device 200 identifies the direction information of the user relative to the center of the screen according to the key user target.

[0165] For example, the preset number threshold can be set to 1. When the number of users is greater than 1, the electronic device 200 can identify the key user target in the user image based on the characteristic image sent by the mobile terminal 300. In some embodiments, the characteristic image may include characteristic information of the key user target of the screen projection. Subsequently, the key user target is identified from multiple users based on the characteristic information of the key user target, that is, the key user target is determined to be the screen projection user, and then the display 260 is controlled to display the screen projection image at the target location based on the direction information of the screen projection user.

[0166] As can be seen from the above technical solution, the above embodiment provides an electronic device 200, which responds to a screen projection connection request sent by a mobile terminal, starts a camera, and obtains a user image captured by the camera; identifies the user's direction information relative to the center of the screen through the user image; wherein the center of the screen is the center line position of the electronic device in the vertical direction; calculates the target position according to the direction information; receives the projection screen sent by the mobile terminal to the electronic device, and controls the display to display the projection screen at the target position. The electronic device 200 can make the direction of the target position of the projection screen consistent with the user's direction information, making it easier for the user to view or manipulate the projection screen, thereby solving the problem of affecting the user experience when the user's direction information is inconsistent with the direction of the projection screen.

[0167] In other embodiments, the present application also provides an electronic device 200. The electronic device 200 includes a display 260, a communication device 220, an image acquisition interface 290 and a processor 250. The communication device 220 is configured to establish a screen projection connection with the mobile terminal 300, and the image acquisition interface 290 is configured to connect to a camera 2901 to capture local video data. The camera is set at the center of the upper or lower frame of the display 260. The electronic device 200 can identify the direction information of the user relative to the center of the screen of the electronic device 200 based on the local video data captured by the camera 2901, thereby automatically adjusting the target position of the projection image sent by the mobile terminal 300 to the electronic device 200.

[0168] To facilitate understanding of the technical solutions in the embodiments of the present application, the following describes each step in detail in conjunction with some specific embodiments and drawings. Figure 12 is a flow chart of a method for adjusting the position of a projection screen by an electronic device provided in some other embodiments of the present application. As shown in Figure 12, when adjusting the position of the projection screen, the electronic device 200 may include the following steps S100-S500, the specific contents of which are as follows:

[0169] Step S100 : In response to a screen projection request for fitness media resources issued by the mobile terminal 300 , the electronic device 200 activates the camera 2901 and obtains local video data captured by the camera 2901 .

[0170] Taking a fitness scenario as an example, in some embodiments, the mobile terminal 300 can play fitness media and project the fitness media onto the electronic device 200. The projected image displayed on the electronic device 200 is the projected image of the fitness media sent by the mobile terminal 300. The user can follow the projected fitness media and view their training progress in the current user interface.

[0171] In response to a request from mobile terminal 300 to project fitness media, after connecting to electronic device 200 via a network protocol or USB cable, mobile terminal 300 can activate camera 2901 located at the center of the upper or lower frame of display 260, obtain local video data captured by camera 2901, and display the captured local video data on electronic device 200. Thus, since camera 2901 is located at the center of display 260, the local video data captured by camera 2901 can be used to determine the user's orientation relative to electronic device 200. After step S100 is completed, step S200 can be executed.

[0172] Step S200: the electronic device 200 controls the display 260 to display the local video data in full screen in the user interface, and controls the display 260 to display the projection screen of the fitness media of the mobile terminal 300 in a floating manner in the user interface.

[0173] In order to enhance the user's viewing experience, in some embodiments, the electronic device 200 can control the display 260 to display local video data in full screen in the user interface, so that the user can see his or her own training situation in full screen. At the same time, the display 260 can be controlled to suspend the display of the projection screen of the fitness media of the mobile terminal 300 in the user interface, and the specific suspension position can be set by yourself. For example, a default position can be set, and when the projection is performed, the projection screen is directly displayed at the default position. In this way, the projection screen can be played in a suspended manner, and the user can follow the fitness media in the projection screen and understand his or her own posture in real time, thereby enhancing the user's viewing experience. After step S200 is completed, the following step S300 can be executed.

[0174] Step S300: The electronic device 200 identifies the direction information of the user's head relative to the center of the screen based on local video data.

[0175] In some embodiments, the screen center is the vertical centerline of the electronic device 200. During the user's follow-up training, if the direction of the projected image is inconsistent with the direction of the user's head, the user needs to turn their head or constantly adjust their posture to adapt to the direction of the projected image, which will also affect the user experience.

[0176] In order to enhance the user's experience during follow-up training, the electronic device 200 can identify the direction information of the user's head relative to the center of the screen based on the local video data. Figure 13 is a flow chart of the process of an electronic device according to an embodiment of the present application identifying the direction information of the user's head relative to the center of the screen based on the local video data. As shown in Figure 13, in some embodiments, the electronic device 200 can first obtain the center of the screen of the electronic device 200, and then extract the joint point information in the local video data to obtain the user's head coordinate set. Finally, the direction information of the user's head relative to the center of the screen is determined based on the head coordinate set and the center of the screen.

[0177] Exemplarily, FIG14 is a schematic diagram of a user follow-up training scenario provided by an embodiment of the present application. As shown in FIG14 , in some embodiments, the joint point information in the local video data can be the user's skeletal point information, and the head coordinate set can be identified through the user's skeletal contour. Thereafter, the direction information of the user's head relative to the center of the screen is determined based on the coordinate set and the center of the screen. In the judgment process, reference can be made to the method for determining the direction information of the user relative to the center of the screen in the aforementioned embodiment, that is, the judgment method in step S2, which will not be repeated here. After step S300 is executed, the following step S400 can be executed.

[0178] Step S400: The electronic device 200 calculates the target position according to the direction information.

[0179] To enhance the user's viewing experience of the projected image, in some embodiments, the electronic device 200 can determine the target position of the projected image based on the user's direction information after identifying the direction information of the user's head relative to the center of the screen. For example, the direction information may include a first direction and a second direction. For example, the first direction is the left side of the user's head relative to the center of the screen, and the second direction is the right side of the user's head relative to the center of the screen. To ensure that the projected image is consistent with the direction information of the user's head, the direction of the target position of the projected image should be consistent with the direction information of the user's head.

[0180] For example, Figure 15 is a schematic diagram of a follow-up training scenario when the user's head is oriented in a first direction, as provided in an embodiment of the present application. Figure 16 is a schematic diagram of a follow-up training scenario when the user's head is oriented in a second direction, as provided in an embodiment of the present application. Combining Figures 15 and 16, when the direction of the projected image is consistent with the direction of the user's head, the user does not need to adjust their posture to accommodate the projected image, thereby improving the user's viewing experience. After step S400 is completed, step S500 may be executed.

[0181] Step S500: the electronic device 200 receives the projection screen of the fitness media resource sent by the mobile terminal 300 to the electronic device 200, and controls the display 260 to display the projection screen at the target location.

[0182] In order to make the direction of the target position of the projection screen consistent with the direction information of the user, in some embodiments, the electronic device 200 can display the projection screen at the target position. The specific display method can refer to the method in step S4 in the aforementioned embodiment, and will not be repeated here. In this way, when the electronic device 200 recognizes that the direction information of the user's head is located to the left of the center of the screen, the projection screen of the fitness media can be controlled to be displayed on the left. When it is recognized that the direction information of the user's head is located to the right of the center of the screen, the projection screen of the fitness media can be controlled to be displayed on the right to enhance the user's experience when following the exercise. The specific display position of the projection screen is not limited in this application. For example, it can be displayed in the upper left corner or upper right corner of the user interface, or other positions.

[0183] In order to further enhance the user's training experience, in some embodiments, the electronic device 200 can also identify changes in the user's posture during fitness and adjust the target position of the projection screen display according to the changes in posture.

[0184] FIG17 is a schematic diagram of a process for an electronic device according to an embodiment of the present application to adjust the position of a projection screen according to the user's posture. As shown in FIG17 , the process specifically includes the following steps:

[0185] S1701: The electronic device 200 may first identify the user's posture height during exercise and set a posture threshold;

[0186] S1702: If the posture height is higher than the posture threshold, control the display 260 to display the projection image at the top of the user interface;

[0187] S1703: If the posture height is lower than the posture threshold, control the display 260 to display the projection image at the bottom of the user interface.

[0188] For example, the user's posture may change during the training process. For example, there are standing postures, squatting postures, and lying postures. The default position of the projection screen can be the upper left corner or the upper right corner, but when the user is in a squatting position, it will be difficult to view the projection screen.

[0189] Therefore, for this scenario, the electronic device 200 can identify the posture height of the user during fitness. When the posture height is higher than the posture threshold, the electronic device 200 can control the display 260 to display the projection screen at the top of the user interface. For example, it can be displayed in the upper left corner or upper right corner of the current user interface. Figure 18 is a schematic diagram of the effect of the electronic device provided in an embodiment of the present application adjusting the projection screen according to the user's posture. As shown in Figure 18, when the posture height is lower than the posture threshold, it means that the user may be in a squatting state or a lying state, etc., and the display 260 can be controlled to display the projection screen at the bottom of the user interface. For example, it can be displayed in the lower left corner or the lower right corner of the current user interface. In this way, the user's line of sight is at the same height as the projection screen, which can further enhance the user's follow-up training experience.

[0190] It should be noted that when the user is in a squatting state or a lying state, if the head is also facing the first direction or the second direction, the direction of the target position of the projection screen will also change. For example, when the user is in a squatting state or a lying state, if the user's head is facing the left, the projection screen can be displayed in the lower left corner of the current user interface, and if the user's head is facing the right, the projection screen can be displayed in the lower right corner of the current user interface. That is to say, in the process of adjusting the target position of the projection screen, on the one hand, it can ensure that the direction of the target position is consistent with the direction of the user's head, and on the other hand, it can ensure that when the user's posture changes, the height of his or her line of sight is as consistent as possible with the height of the line of sight of the projection screen, so as to enhance the user's experience of following the training.

[0191] In some embodiments, when the user is near the center of the screen, the electronic device 200 can control the display 260 to display the projected image in a floating position at a preset position in the user interface, for example, in the upper left corner, upper right corner, or other position. At the same time, the display level of the position can also be set. For example, the upper left corner level can be higher than the upper right corner level. Then, the projected image is displayed according to the priority level of the position based on actual conditions.

[0192] As can be seen from the above technical solution, the electronic device 200 in the above embodiment responds to a screen projection request for fitness media from a mobile terminal, activates the camera, and obtains local video data captured by the camera; controls the display to display the local video data in full screen in the user interface, and controls the display to display the projection of the mobile terminal's fitness media in the user interface in a floating manner; identifies the direction information of the user's head relative to the center of the screen based on the local video data; the center of the screen is the centerline position of the electronic device in the vertical direction; calculates the target position based on the direction information; receives the projection of the fitness media sent by the mobile terminal to the electronic device, and controls the display to display the projection at the target position. The electronic device can identify the direction information of the user relative to the center of the electronic device screen based on the local video data captured by the camera, thereby automatically adjusting the target position of the projection sent by the mobile terminal to the electronic device to enhance the user's training experience.

[0193] Some embodiments of the present application also provide a method for adjusting the position of a projection screen, which is applied to an electronic device 200, wherein the electronic device includes a display 260, a communication device 220, an image acquisition interface 290, and a processor 250. The communication device 220 is configured to establish a projection screen connection with a mobile terminal 300, and the image acquisition interface 290 is configured to connect to a camera 2901 to capture a user image. The camera 2901 is set at the center of the upper or lower frame of the display 260. The method for adjusting the position of the projection screen includes:

[0194] In response to the screen projection connection request sent by the mobile terminal 300, the camera 2901 is activated, and the user image captured by the camera 2901 is obtained. Since the camera 2901 is located at the center of the display 260, when the user is at the center of the user image, it means that the user is at the center of the electronic device 200. In other words, the user's orientation information relative to the electronic device 200 can be determined based on the user image captured by the camera 2901.

[0195] Electronic device 200 uses the user image to identify the user's direction relative to the center of the screen. The center of the screen is the vertical centerline of electronic device 200. In some embodiments, if a predetermined percentage of pixel coordinates in the user coordinate set are greater than the coordinates of the screen center, the direction information may be determined to be a first direction. If a predetermined percentage of pixel coordinates in the user coordinate set are less than the coordinates of the screen center, the direction information may be determined to be a second direction, with the first direction and the second direction being opposite.

[0196] The electronic device 200 calculates the target position based on the direction information. In order to make the projection screen consistent with the user's direction information, the direction of the target position of the projection screen should be consistent with the user's direction information to enhance the user's viewing experience of the projection screen.

[0197] Receive the projection screen sent by the mobile terminal 300 to the electronic device 200, and control the display 260 to display the projection screen at the target position. In some embodiments, the electronic device 200 can display the projection screen at the target position. If the initial direction of the projection screen is the same as the user's direction information, the electronic device 200 can control the display 260 to display the projection screen at the default position, that is, there is no need to adjust the position of the projection screen. If the initial direction of the projection screen is opposite to the user's direction information, the electronic device 200 can control the display 260 to display the projection screen at the target position. In this way, the direction of the target position of the projection screen should be consistent with the user's direction information, which can facilitate the user to view or manipulate the projection screen and enhance the user experience.

[0198] It can be seen from the above technical solution that the method for adjusting the position of the projection screen provided in the above embodiment can make the direction of the target position of the projection screen consistent with the user's direction information, which is convenient for the user to view or control the projection screen, thereby solving the problem of affecting the user experience when the user's direction information is inconsistent with the direction of the projection screen.

[0199] Some embodiments of the present application also provide a method for adjusting the position of a projection screen, which is applied to an electronic device 200, wherein the electronic device includes a display 260, a communication device 220, an image acquisition interface 290, and a processor 250. The communication device 220 is configured to establish a projection screen connection with a mobile terminal 300, and the image acquisition interface 290 is configured to connect to a camera 2901 to capture a user image. The camera 2901 is set at the center of the upper or lower frame of the display 260. The method for adjusting the position of the projection screen includes:

[0200] In response to a request from mobile terminal 300 to project fitness media, electronic device 200 activates camera 2901 and obtains local video data captured by camera 2901. Because camera 2901 is located at the center of display 260, the local video data captured by camera 2901 can be used to determine the user's orientation relative to electronic device 200.

[0201] The electronic device 200 controls the display 260 to display the local video data in full screen in the user interface, and controls the display 260 to display a projection of the fitness media from the mobile terminal 300 in a floating manner in the user interface. To enhance the user's viewing experience, in some embodiments, the electronic device 200 can control the display 260 to display the local video data in full screen in the user interface, so that the user can see their own training progress in full screen. At the same time, the display 260 can be controlled to display a projection of the fitness media from the mobile terminal 300 in a floating manner in the user interface, and the specific floating position can be customized.

[0202] The electronic device 200 identifies the orientation of the user's head relative to the center of the screen based on the local video data. The screen center is the vertical centerline of the electronic device. First, the screen center of the electronic device 200 is obtained. Then, joint point information from the local video data is extracted to obtain the user's head coordinate set. Finally, the orientation of the user's head relative to the screen center is determined based on the head coordinate set and the screen center.

[0203] The electronic device 200 calculates the target position based on the direction information. In order to make the projection screen consistent with the direction information of the user's head, the direction of the target position of the projection screen should be consistent with the direction information of the user's head to enhance the user's viewing experience of the projection screen.

[0204] Electronic device 200 receives a projection of fitness media sent by mobile terminal 300 and controls display 260 to display the projection at a target location. When electronic device 200 identifies that the user's head is to the left of the center of the screen, the projection of the fitness media can be controlled to be displayed on the left. When the user's head is to the right of the center of the screen, the projection of the fitness media can be controlled to be displayed on the right, thereby enhancing the user's workout experience.

[0205] As can be seen from the above technical solution, the method for adjusting the position of the projected screen provided in the above embodiment can identify the user's direction information relative to the center of the electronic device screen based on the local video data captured by the camera, thereby automatically adjusting the target position of the projected screen sent by the mobile terminal to the electronic device. On the one hand, it can ensure that the direction of the target position is consistent with the direction of the user's head. On the other hand, it can ensure that when the user's posture changes, their eye level is as consistent as possible with the eye level of the projected screen, thereby improving the user's training experience.

[0206] The same and similar parts between the various embodiments in this specification can be referenced to each other and will not be repeated here.

[0207] In addition to the above embodiments, this application also provides other embodiments of electronic devices and multi-channel screen projection methods. Based on the large screen size of the electronic device 200, the screen displayed in the screen projection initiating device such as the mobile terminal 300 can be projected to the screen projection receiving device, that is, the electronic device 200. In this way, the electronic device 200 can display the content corresponding to the screen projection data sent by the mobile terminal 300 in real time, that is, the projected content, and improve the user's viewing experience through the large screen.

[0208] For example, 75-inch, 80-inch, and 100-inch TVs are increasingly common in homes. When users watch media on their phones, the displayed image is relatively small. Therefore, to obtain a larger playback image, users can cast content from their phones to the TV, allowing for a larger screen and enhancing the viewing experience.

[0209] In the process of screen casting, the screen casting operation can be implemented in a variety of ways. For example, screen casting can be implemented based on the Digital Living Network Alliance (DLNA) protocol, hereinafter referred to as the DLNA protocol. The main content of the DLNA protocol is to realize the sharing of media between the screen casting initiator device and the screen casting receiver device. For example, some interactive instructions can be used to cast media such as pictures or videos in mobile terminals such as mobile phones to electronic devices such as TVs for display and playback.

[0210] However, the interactive instructions specified by the DLNA protocol are implemented based on Hypertext Transfer Protocol (HTTP) short links. Each interactive instruction executes an HTTP request and response. The HTTP request does not carry the software information used by the screen projection initiating device. In other words, the electronic device 200 cannot distinguish which application software on the mobile terminal 300 sent the screen projection instruction.

[0211] Exemplarily, when the mobile terminal 300 and the electronic device 200 perform screen projection, the DLNA protocol of the electronic device 200 will only start one instance, and the two will distinguish the screen projection session based on IP, that is, from the beginning to the end of a screen projection, the screen is only projected based on IP. In some embodiments, an instance can be understood as a software interaction module, and the DLNA protocol stack can be understood as an execution module of the interactive DLNA protocol. If the execution module is only started once, it is an instance. If the execution module is started twice or more times, it is two or more instances, that is, it can be understood as the number of times the execution module is run. For example, a browser application is a software module. When the browser application is opened multiple times, there will be multiple instances of the browser.

[0212] During the screen projection process, the HTTP request will not carry the software information used by the screen projection initiating device, that is, the electronic device 200 cannot distinguish which software A, B, or C the screen projection instruction is sent from. In this way, when the user sends the projection content through the second application software, the projection content of the first application software will be replaced. Therefore, when the screen projection is executed through the screen projection protocol in the electronic device 200, the purpose of simultaneously projecting multiple application software in the screen projection initiating device cannot be achieved, that is, multi-channel screen projection cannot be achieved for multiple application software on a screen projection initiating device.

[0213] In order to solve the problem that when screen projection is performed through the screen projection protocol, it is impossible to simultaneously project multiple application software in the screen projection initiating device, an embodiment of the present application provides an electronic device 200. The electronic device 200 includes a display 260, a communication device 220 and a processor 250. Among them, the display 260 is configured to display the screen projection interface, the communication device 220 is configured to establish a screen projection connection with the mobile terminal, and the processor 250 is configured to perform multi-way screen projection between the mobile terminal 300 and the electronic device 200. The electronic device 200 can distinguish screen projection sessions based on the HTTP interaction port number, and support DLNA multi-way screen projection for multiple application software in the screen projection initiating device, that is, it can realize multi-way screen projection of multiple application software on a screen projection initiating device.

[0214] To facilitate understanding of the technical solutions in the embodiments of the present application, the following describes each step in detail in conjunction with some specific embodiments and drawings. FIG19 is a flow chart of a method for performing multi-channel screen projection on an electronic device provided in an embodiment of the present application. As shown in FIG19 , when performing multi-channel screen projection, the electronic device 200 may include the following steps S1-S7, the specific contents of which are as follows:

[0215] Step S501: The electronic device 200 configures multiple network interaction port numbers and screen projection device names of the screen projection protocol stack.

[0216] In some embodiments, the screen projection protocol stack can be a DLNA protocol stack, the network interaction port number can be an HTTP interaction port number, and the electronic device 200 will set the maximum number of screen projection channels supported. The following takes the maximum number of screen projection channels as N, the screen projection protocol stack as the DLNA protocol stack, and the network interaction port number as the HTTP interaction port number as an example to illustrate the process of multi-channel screen projection, where N is a positive integer.

[0217] When configuring the network interaction port number and screen projection device name of the screen projection protocol stack, such as the DLNA protocol stack, the electronic device 200 can first configure the network interaction port numbers and screen projection device names of multiple screen projection protocol stacks, and each DLNA protocol stack uses a different screen projection device name and HTTP interaction port number. For example, the screen projection device names can be "DLNA Multi-channel Screen Projection No. 1", "DLNA Multi-channel Screen Projection No. 2", "DLNA Multi-channel Screen Projection No. N", etc., and the HTTP interaction port numbers are 38401, 38402, 3840N, etc. After step S501 is completed, the following step S502 can be executed.

[0218] Step S502: The electronic device 200 establishes a first binding relationship between the network interaction port number and the screen projection device name.

[0219] In order to achieve session differentiation through HTTP interaction port number, in some embodiments, the electronic device 200 can establish a first binding relationship between the network interaction port number and the screen projection device name. For example, the network interaction port number bound to the screen projection device name "DLNA Multi-channel Screen Projection No. 1" can be 38401, the network interaction port number bound to the screen projection device name "DLNA Multi-channel Screen Projection No. 2" can be 38402, and the network interaction port number bound to the screen projection device name "DLNA Multi-channel Screen Projection No. N" can be 3840N. In this way, each screen projection device name will correspond to an HTTP interaction port number, and the mobile terminal 300 can distinguish sessions through the HTTP interaction port number when initiating screen projection.

[0220] In some embodiments, when performing multi-channel screen projection between the mobile terminal 300 and the electronic device 200, the DLNA protocol will first have a device discovery process. Specifically, in a network, when the mobile terminal 300 wants to project the screen to the electronic device 200, it does not know what types of electronic devices 200 are in the network, or what functions the electronic device 200 supports. During the device discovery process, the mobile terminal 300 and the electronic device 200 interact through the network, so that the electronic device 200 will send some information to the mobile terminal 300. For example, information used for screen projection or interaction.

[0221] In some embodiments, the DLNA protocol specifies two types of interactions between the screen projection initiating device and the screen projection receiving device, that is, there are two modes of interaction between the mobile terminal 300 and the electronic device 200. One is in the form of instructions or request control, which is sent by the screen projection initiating device to the screen projection receiving device, that is, the mobile terminal 300 sends an instruction or request to the electronic device 200. Among them, the instruction or request may include the Uniform Resource Locator (URL) address of the screen projection content to be played, that is, the URL address. In addition, it may also include control instructions for the screen projection content, such as play, pause, stop and other control instructions. The other is in the form of notification. The screen projection receiving device will send the playback status to the screen projection initiating device. For example, when the screen projection content is paused, the electronic device 200 will send the paused playback status to the mobile terminal 300, which will be introduced in detail later.

[0222] After the first binding relationship between the network interaction port number and the screen projection device name is established, the electronic device 200 will perform device discovery interaction with the mobile terminal 300, so that the mobile terminal 300 can obtain the network interaction port number of the electronic device. In some embodiments, in the process of performing device discovery, the electronic device 200 can reply to the device description information address to the mobile terminal 300 in accordance with the Simple Service Discovery Protocol (SSDP), or notify the mobile terminal 300 of the address of its corresponding device description information by sending a broadcast packet or multicast packet containing the device description information address to the mobile terminal 300. The address of the device description information may contain relevant information about the network interaction port number of the DLNA protocol. After receiving the address of the device description information, the mobile terminal 300 may send a network acquisition request to the electronic device 200, so that the mobile terminal 300 can obtain the screen projection device name contained in the electronic device 200. Each of them will be introduced in detail below.

[0223] FIG20 is a flow chart of an electronic device and a mobile terminal interacting to obtain an HTTP interaction port number according to an embodiment of the present application. As shown in FIG20 , the mobile terminal 300 may obtain the HTTP interaction port number in the following manner:

[0224] S2001: First, the electronic device 200 may receive a device discovery request for performing multi-channel screen projection sent by the mobile terminal 300;

[0225] S2002 . In response to the device discovery request, the electronic device 200 feeds back a device description information address to the mobile terminal 300 , so that the mobile terminal 300 obtains the network interaction port number of the electronic device 200 by parsing the device description information address.

[0226] For example, after the network interaction port number and the screen projection device name are bound, the mobile terminal 300 and the electronic device 200 can perform protocol interaction based on the bound network interaction port number. In some embodiments, after receiving the device discovery request for performing multi-channel screen projection sent by the mobile terminal 300, the electronic device 200 can feedback the following device description information address to the mobile terminal 300:

[0227] Location: http: / / 192.168.1.101:38400 / MediaServer / rendererdevicedesc.xml;

[0228] 38400 is the network interaction port number of the electronic device 200. After receiving the device description information address, the mobile terminal 300 can parse the content contained therein to obtain the HTTP interaction port number 38400.

[0229] FIG21 is a flow chart of another electronic device and a mobile terminal interacting to obtain an HTTP interaction port number according to an embodiment of the present application. As shown in FIG21 , the mobile terminal 300 may also obtain the HTTP interaction port number in the following manner:

[0230] S2101: In the absence of any request or instruction, for example, when the mobile terminal 300 does not need to send a device discovery request to the electronic device 200, the electronic device 200 may proactively send a broadcast packet or a multicast packet containing a device description information address to the mobile terminal 300;

[0231] S2102: The mobile terminal 300 obtains the network interaction port number of the electronic device 200 by parsing the device description information address in the broadcast packet or the multicast packet.

[0232] For example, without any request or instruction, after the network interaction port number and the screen projection device name are bound, the electronic device 200 can actively send a broadcast packet or multicast packet containing the device description information address to the mobile terminal 300. In some embodiments, the content of the description information address in the broadcast packet or multicast packet can also be:

[0233] Location: http: / / 192.168.1.101:38400 / MediaServer / rendererdevicedesc.xml;

[0234] In this way, upon receiving the device description information address, the mobile terminal 300 can parse the content contained therein and obtain the HTTP interaction port number 38400.

[0235] It should be noted that after the mobile terminal 300 obtains the HTTP interaction port number, it does not know whether the electronic device 200 supports multi-channel screen projection, and it only thinks that the electronic device 200 is a screen projection playback device. Although the above two methods return the same address content, the form is different. One is the method in which the mobile terminal 300 applies to the electronic device 200, and the other is the method in which the electronic device 200 actively sends it to the mobile terminal 300. Regardless of which method is used, the HTTP interaction port number can be obtained.

[0236] FIG22 is a schematic diagram of a process for obtaining the name of a screen projection device according to an embodiment of the present application, as shown in FIG22 :

[0237] S2201. After receiving the device description information address fed back by the electronic device 200, or receiving the broadcast packet or multicast packet containing the device description information address, the mobile terminal 300 also needs to obtain the screen projection device name in the electronic device 200.

[0238] S2202. In some embodiments, the electronic device 200 may first receive a network acquisition request sent by the mobile terminal 300 based on the network interaction port number.

[0239] S2203: Based on the network acquisition request, the electronic device 200 feeds back device description information of the electronic device 200 to the mobile terminal 300;

[0240] S2204. The mobile terminal 300 obtains the screen projection device name of the electronic device 200 that interacts with the mobile terminal 300 based on the screen projection protocol such as the DLNA protocol by parsing the device description information.

[0241] For example, after receiving the device description information address, the mobile terminal 300 will initiate an HTTP Get request to the device description information address http: / / 192.168.1.101:38400 / MediaServer / rendererdevicedesc.xml. The electronic device 200 will respond to the request by feeding back the device description information. The device description information includes the following fields: <friendlyname>SmartTV< / friendlyname> . Among them, SmartTV in the tag is the name displayed externally by the DLNA protocol, that is, the screen projection device name displayed externally by the DLNA protocol stack of the electronic device 200.

[0242] In some embodiments, the electronic device 200 may start multiple instances of the DLNA protocol stack, each instance using a different network interaction port number and screen projection device name. In this way, when the mobile terminal 300 projects the screen to the electronic device 200, it can project the screen to multiple screen projection device names, thereby achieving multi-way screen projection on the electronic device 200. After step S502 is completed, step S503 may be executed.

[0243] Step S503: The electronic device 200 creates multiple playback windows, and establishes a second binding relationship between the playback window and the instance of the screen projection protocol stack.

[0244] After obtaining the network interaction port number and projection device name of the electronic device 200, the electronic device 200 can create multiple playback windows. The number of playback windows is the same as the number of projection protocol stacks. After the playback window is established, a second binding relationship between the playback window and the instance of the projection protocol stack can be established, that is, each playback window is bound to an instance of each protocol stack.

[0245] In some embodiments, the electronic device 200 can also establish a third binding relationship between the playback window and the projection device name, and control the display 260 to display the projection device name at a preset position in the playback window based on the third binding relationship. Figure 23 is a schematic diagram of the effect of binding the playback window and the projection device name provided in an embodiment of the present application. As shown in Figure 23, after the playback window is bound to the projection device name, the projection device name can be displayed at a preset position in the playback window. For example, it can be displayed in the middle of the top of the playback window, or in other positions. In this way, the user can know the projection device name corresponding to each playback window, which is convenient for controlling the projection content played in each window. After step S503 is executed, the following step S504 can be executed.

[0246] Step S504: The electronic device 200 starts the multiple configured screen projection protocol stacks.

[0247] After the binding relationship of the DLNA protocol stack is established, the electronic device 200 can start the multiple screen projection protocol stacks that have been configured. For example, the protocol stack can be started by starting an event processing thread, or by starting it through other interfaces, etc., which is not limited in this application. In this way, each protocol stack will respond to the request or instruction issued by the mobile terminal 300 according to the configured network interaction port number and screen device name. After step S504 is executed, the following step S505 can be executed.

[0248] Step S505: The electronic device 200 receives the screen projection instruction sent by the mobile terminal 300 from the established network interaction port, and parses the screen projection network address information of the mobile terminal from the screen projection instruction.

[0249] After the DLNA protocol stack is started, in some embodiments, the electronic device 200 can receive the screen projection instruction sent by the mobile terminal 300 from the established network interaction port, and parse the screen projection instruction, so that the screen projection network address information of the mobile terminal 300 can be obtained. Exemplarily, each DLNA protocol stack will respond to instructions or event requests sent to its own bound network interaction port number. For example, the screen projection instruction may be SetAVTransportURI, and the electronic device 200 can obtain the screen projection network address information after receiving the screen projection instruction and parsing the screen projection instruction. In this way, the electronic device 200 can obtain the address information of the screen projection content based on the screen projection network address information. After step S505 is completed, the following step S506 can be executed.

[0250] Step S506: In response to the screen projection instruction, the electronic device 200 creates a player and downloads the projection content corresponding to the screen projection network address.

[0251] After receiving the screen projection instruction sent by the mobile terminal 300, the electronic device 200 can create a player for playing the screen projection content in response to the screen projection instruction and play the screen projection content in the playback window bound to the DLNA protocol stack. After step S506 is completed, the following step S507 can be executed.

[0252] Step S507: The electronic device 200 controls the display 260 to play the projection content in the playback window based on the first binding relationship, the second binding relationship and the projection network address information.

[0253] After the electronic device 200 downloads the projection content corresponding to the projection network address, the electronic device 200 can control the display 260 to play the projection content in the playback window based on the first binding relationship, the second binding relationship and the projection network address information.

[0254] Figure 24 is a schematic diagram of the effect of performing multi-channel screen projection based on IP as the dimension provided by an embodiment of the present application. As shown in Figure 24, when the electronic device 200 performs multi-channel screen projection based on IP as the dimension, there is only one DLNA protocol stack in the electronic device 200, and only one HTTP interactive port number is bound. The screen projection initiating device may include device 1 and device 2. Device 1 can play the projection content in window 1 through the DLNA protocol stack. Device 2 can play the projection content in window 2 through the projection protocol stack such as the DLNA protocol stack. However, device 1 and device 2 cannot achieve multi-channel screen projection for multiple APPs.

[0255] Figure 25 is a schematic diagram of the effect of performing multi-channel screen projection based on the network interaction port number, such as the HTTP interaction port number, provided by an embodiment of the present application. As shown in Figure 25, when the electronic device 200 performs multi-channel screen projection based on the HTTP interaction port number, the electronic device 200 can be configured with multiple screen projection protocol stacks, such as the DLNA protocol stack, and correspondingly, multiple HTTP interaction port numbers will also be bound. In some embodiments, the screen projection initiating device still includes device 1 and device 2, and device 1 includes two screen projection application software, namely APP1 and APP2, and device 2 includes one screen projection application software, namely APP3. When performing multi-channel screen projection, the projection content of APP1 in device 1 can be transmitted to the protocol port corresponding to the playback window 1 through the bound DLNA protocol stack 1 and HTTP interaction port number 1, and a binding relationship is established between the protocol port and the playback window 1. In this way, the projection content in APP1 can be transmitted to the playback window 1 through the HTTP interaction port number 1, and the playback and display of the projection content can be completed through the playback window 1.

[0256] Similarly, the screen projection content of APP2 in device 1 can be transmitted to playback window 2 via HTTP interactive port number 2, and the screen projection content of APP3 in device 2 can be transmitted to playback window 3 via HTTP interactive port number 3. By comparing Figures 24 and 25, it can be found that when the electronic device 200 performs multi-way screen projection based on the HTTP interactive port number, the two APPs in device 1 can be simultaneously projected to the electronic device 200, that is, multi-way screen projection of multiple application software on a screen projection initiating device can be achieved, thereby solving the problem that when screen projection is performed through the screen projection protocol, it is impossible to simultaneously project multiple application software in the screen projection initiating device.

[0257] In order to control the playback progress and playback status of the projected content in the playback window on the mobile terminal 300, the electronic device 200 can also perform the following process. Figure 26 is a schematic diagram of the process of controlling the playback progress and playback status of the projected content in the playback window provided by an embodiment of the present application, as shown in Figure 26, which specifically includes the following steps:

[0258] S2601: The electronic device 200 may first receive a control instruction sent by the mobile terminal 300;

[0259] S2602: In response to the control instruction, the electronic device 200 may obtain a playback window corresponding to the control instruction based on a screen projection protocol stack such as a DLNA protocol stack;

[0260] S2603: The electronic device 200 controls the play status and play progress of the play window based on the control instruction.

[0261] In some embodiments, the control instructions may include play, pause, fast forward, etc. When the mobile terminal 300 sends the corresponding control instructions, when the user operates the playback progress or playback status of the projected content on the mobile terminal 300, the various DLNA protocol stacks in the electronic device 200 will respond to the control instructions and control the playback status and playback progress of the bound playback window.

[0262] S2604. When the control instruction is a stop playback instruction, the electronic device 200 can exit the playback window corresponding to the stop playback instruction to release resources of the playback window and wait for the next screen projection.

[0263] In order to notify the mobile terminal 300 of the playback status or playback progress in the playback window of the electronic device 200, in some embodiments, the electronic device 200 may also generate status change information of the projected content and store it as subscription information. Figure 27 is a schematic diagram of the process of generating subscription information by an electronic device according to an embodiment of the present application, as shown in Figure 27, which specifically includes the following steps:

[0264] S2701: The electronic device 200 may first add function controls to the playback window. The function controls include at least a playback control control and a progress control control. The playback control control is used to control the playback status of the projected content in the playback window, and the progress control control is used to control the playback progress of the projected content in the playback window.

[0265] S2702: The electronic device 200 generates state change information of the screen projection content in response to an operation event input by the user based on the playback control control and the progress control control;

[0266] S2703: The electronic device 200 stores the state change information as subscription information.

[0267] When the mobile terminal 300 wants to obtain subscription information, it can send a subscription request for obtaining subscription information to the electronic device 200. The electronic device 200 receives the subscription request sent by the mobile terminal 300 and, in response to the subscription request, can send subscription information to the mobile terminal 300 so that the mobile terminal 300 can obtain status change information of the projected content.

[0268] FIG28 is a schematic diagram of the effect of the subscription information provided in an embodiment of the present application. As shown in FIG28 , after the mobile terminal 300 receives the subscription information, the specific content of the subscription information can be displayed. For example, the content can be, "The projected content has been fast-forwarded. Please be informed," so that the user can understand the current playback status and playback progress of the projected content in the playback window. In some embodiments, the current playback time information can also be displayed, and other information can also be included. The specific setting can be based on actual needs, and this application does not make specific limitations on this.

[0269] After the electronic device 200 is configured as described above, it can support multiple screen projections of different application software on the same device. In this way, when the mobile terminal 300 searches for screen projection devices through the device, it will find multiple screen projection device names that can be projected. For example, the screen projection device name can include "DLNA Multi-channel Screen Projection No. 1", "DLNA Multi-channel Screen Projection No. 2", etc. At the same time, since each playback window in the electronic device 200 has been bound to the screen projection device name, the user can clearly understand the playback window corresponding to each application software. After the user selects the corresponding screen projection device name, the subsequent screen projection, playback progress control, event subscription, etc. of the mobile terminal 300 can be performed on the network interaction port number corresponding to the screen projection device name. At the same time, when the mobile terminal 300 projects another application software, it can select another screen projection device name, and the interaction will be performed on another bound network interaction port number. In this way, the electronic device 200 can distinguish the screen projection of different application software in the same mobile terminal through the HTTP interaction port number, thereby solving the problem that when performing screen projection through the screen projection protocol, it is impossible to realize the simultaneous projection of multiple application software in the screen projection initiating device.

[0270] It can be seen from the above technical solution that the above embodiment provides an electronic device, including configuring a plurality of network interaction port numbers and projection device names of projection protocol stacks; establishing a first binding relationship between the network interaction port number and the projection device name; creating a plurality of playback windows, and establishing a second binding relationship between the playback window and the instance of the projection protocol stack, wherein the number of playback windows is the same as the number of projection protocol stacks; starting the multiple projection protocol stacks that have been configured; receiving the projection instruction sent by the mobile terminal from the established network interaction port, and parsing the projection network address information of the mobile terminal from the projection instruction; creating a player in response to the projection instruction, and downloading the projection content corresponding to the projection network address; controlling the display to play the projection content in the playback window based on the first binding relationship, the second binding relationship and the projection network address information. The electronic device performs multi-channel projection through the network interaction port number as the dimension, thereby solving the problem that when the projection is performed through the projection protocol, it is impossible to realize the simultaneous projection of multiple application software in the projection initiating device.

[0271] FIG29 is another flowchart of a method for performing multi-channel screen projection on an electronic device according to an embodiment of the present application. As shown in FIG29 , when the electronic device 200 performs multi-channel screen projection, it can also be implemented in the following manner:

[0272] S2901: The electronic device 200 first creates a first play window and a first protocol stack, and binds the first play window and the first protocol stack.

[0273] In the aforementioned embodiment, the network interaction port number and the screen projection device name are created in advance, and the binding relationship is established in advance.

[0274] In other embodiments of the present application, it is also possible not to establish a binding relationship in advance. For example, when the user has not yet cast the screen, the electronic device 200 can only create a playback window and start a protocol stack, and establish a binding relationship between the two.

[0275] S2902. The electronic device 200 receives the projection content sent by the mobile terminal 300 based on the first protocol stack.

[0276] After the first playback window and the first protocol stack are established and bound, the electronic device 200 can receive the projection content sent by the mobile terminal 300 based on the first protocol stack, and can display the projection content in the first playback window.

[0277] S2903. In response to an event that the first playback window receives the projection content and the first playback window is occupied, the electronic device 200 creates a second playback window and a second protocol stack, and binds the second playback window and the second protocol stack.

[0278] In some embodiments, if the first playback window is already playing the projection content, that is, it is occupied, a new playback window and a new protocol stack are created, and the relationship between the two brackets is bound. In this way, the user can discover new projection devices through the mobile terminal 300, for example, discover the name of a new projection device.

[0279] S2904. After the first playback window is occupied, when there is new projection content, the electronic device 200 receives the new projection content sent by the mobile terminal 300 based on the new projection request, and controls the display to play the new projection content based on the second playback window and the second protocol stack.

[0280] After the second playback window and the second protocol stack are created and the binding relationship is established, the electronic device 200 can receive the new projection content sent by the mobile terminal 300 based on the new projection request, and control the display to play the new projection content based on the second playback window and the second protocol stack.

[0281] In some embodiments, when the maximum number of projection paths supported by the electronic device 200 is reached, the electronic device 200 may prompt the user, for example, by displaying a prompt message. If a playback window is exited, the playback resources of the corresponding window may be released, for example, by deleting the bound protocol stack. If all playback windows have no projection content to display, the electronic device 200 may retain only one playback window and the corresponding DLNA protocol stack.

[0282] The electronic device 200 provided in the above embodiment can realize multi-channel screen projection of the same device by establishing playback windows and LNA protocol stacks one by one, and perform multi-channel screen projection based on the network interaction port number, thereby solving the problem that when screen projection is performed through the screen projection protocol, it is impossible to realize simultaneous projection of multiple application software in the screen projection initiating device.

[0283] Some embodiments of the present application further provide a multi-channel screen projection method, which can be applied to the electronic device 200 in the above embodiment. FIG30 is a flow chart of the multi-channel screen projection method provided in an embodiment of the present application. As shown in FIG30 , the multi-channel screen projection method may include the following steps S501-S507, the specific contents of which are as follows:

[0284] Step S501: Configure the network interaction port numbers and screen projection device names of multiple screen projection protocol stacks;

[0285] Step S502: Establish a first binding relationship between the network interaction port number and the screen projection device name;

[0286] Step S503: Create multiple playback windows, and establish a second binding relationship between the playback windows and the instances of the screen projection protocol stack; the number of playback windows is the same as the number of screen projection protocol stacks;

[0287] Step S504: Start the multiple projection protocol stacks that have been configured;

[0288] Step S505: receiving a screen projection instruction sent by the mobile terminal 300 from the established network interaction port, and parsing the screen projection network address information of the mobile terminal from the screen projection instruction;

[0289] Step S506: In response to the screen projection instruction, create a player and download the projection content corresponding to the projection network address;

[0290] Step S507: Control the display to play the projection content in the playback window based on the first binding relationship, the second binding relationship and the projection network address information.

[0291] It can be seen from the above technical solution that the multi-channel screen projection method provided in the above embodiment can perform multi-channel screen projection through the network interaction port number as the dimension, thereby solving the problem that when screen projection is performed through the screen projection protocol, it is impossible to realize simultaneous projection of multiple application software in the screen projection initiating device.

[0292] In addition to the above embodiments, the present application also provides other embodiments of electronic devices and display methods. Typically, when the display 260 of the electronic device 200 simultaneously displays user interfaces corresponding to multiple display windows, the user cannot switch the focus between the multiple display windows, nor can the user switch the focus between the applications or controls in each display window. Therefore, the user cannot control the applications or controls in each display window, and thus cannot perform related display settings, which affects the user experience.

[0293] In some examples, when the electronic device 200 exits from multi-window display to single-window display, the user can manipulate the application or control in the single display window by controlling the device 100.

[0294] To address the above-mentioned issues, an embodiment of the present application provides an electronic device 200, comprising a communication device 220, a processor 250, and a display 260. The display 260 is configured to display an image and / or a user interface. The communication device 220 is configured to receive user operations. The processor 250 is configured to receive a multi-window display operation input by the user through the communication device 220 when the display 260 displays a first user interface; in response to the multi-window display operation, control the display 260 to display the first user interface in the first display window, the second user interface in the second display window, and display a focus frame at the first focus position of the first user interface; receive a first focus switching operation input by the user through the communication device 220; and in response to the first focus switching operation, move the focus frame from the first focus position to the second focus position of the first user interface or the third focus position of the second user interface. The electronic device 200 provided in an embodiment of the present application can implement focus switching between the first display window and the second display window, and can also implement focus switching between controls within the first display window or the second display window.

[0295] The display method provided by the embodiment of the present application is described in detail below with reference to the accompanying drawings. It should be noted that the display method in the following embodiment can be implemented in the electronic device 200 having the above hardware structure or software structure.

[0296] Figure 31 is a schematic diagram of a display method provided in an embodiment of the present application. As shown in Figure 31, the display method includes steps S510 to S540 as shown below.

[0297] Step S510: When the display displays the first user interface, a multi-window display operation input by a user is received through a communication device.

[0298] In some examples, the multi-window display operation is used to instruct the display 260 to enter a multi-window display module (also called a split-screen mode), that is, the multi-window display operation is used to instruct the display 260 to simultaneously display user interfaces corresponding to multiple display windows.

[0299] The user can send a multi-window display operation to the electronic device 200 (such as the communication device 220) through a preset interactive action. In some examples, the user can send a multi-window display operation to the electronic device 200 through a split-screen mode entry button set on the control device 100 (such as a remote control) that is matched with the electronic device 200. For example, when the user presses the split-screen mode entry button, the processor 250 controls the electronic device 200 to enter the split-screen mode in response to the operation, and the processor 250 controls the display 260 to perform multi-window display. It should be noted that the buttons corresponding to entering the split-screen mode and exiting the split-screen mode on the control device 100 may be the same button or may not be different buttons, and the embodiments of the present application do not limit this.

[0300] In other examples, the user may also send a multi-window display operation to the electronic device 200 through a voice command to control the electronic device 200 to enter the split-screen mode. For example, after the user inputs a wake-up word (such as "Hi, ×"), the user may input a voice command corresponding to entering the split-screen mode (such as "Switch to split-screen mode") to the electronic device 200. In response to the voice command, the processor 250 may control the display 260 to perform multi-window display.

[0301] In some other examples, the processor 250 can also monitor the user's usage status and determine whether to enter the split-screen mode based on the user's usage status. If the processor 250 determines that the user has a multi-window display requirement based on the user's usage status, it can control the display 260 to enter the split-screen mode, or provide the user with a split-screen mode switching option. For example, when the electronic device 200 is performing a single-window display, if it detects that a projection signal is connected, it can automatically switch to the split-screen mode.

[0302] Step S520 , in response to the multi-window display operation, controlling the display to display the first user interface in the first display window, displaying the second user interface in the second display window, and displaying a focus frame at the first focus position of the first user interface.

[0303] In some examples, if the multiple display windows of the electronic device 200 include a first display window and a second display window, after the processor 250 receives a multi-window display operation, in response to the multi-window display operation, the display 260 can be controlled to enter a two-split screen display, that is, the display 260 can display the first user interface in the first display window while displaying the second user interface in the second display window.

[0304] For example, the first display window and the second display window can respectively display user interfaces corresponding to different external signal sources. For example, the first user interface can be the user interface corresponding to the projection signal source, and the second user interface can be the user interface corresponding to the application, and the first display window and the second display window do not interfere with each other.

[0305] It should be noted that the display 260 may also include a greater number of display windows, such as three display windows, etc. For ease of description, the embodiment of the present application is schematically described by taking the display 260 including two display windows, namely, a first display window and a second display window, as an example.

[0306] For example, the user interface in each display window can be displayed in a horizontal version or a vertical version, and this embodiment of the present application does not limit this.

[0307] In some embodiments, the first display window and the second display window are set with a focus. If the first user interface corresponding to the first display window and the second user interface corresponding to the second display window include at least one control, the at least one control may also be set with a focus. In some examples, the first user interface or the second user interface may not include a control.

[0308] When the electronic device 200 is started, the electronic device 200 can first enter the single-window display mode, such as when the display 260 displays the first user interface in the first display window. In this case, the processor 250 can control the display 260 to display a focus frame at a certain focus position in the first user interface.

[0309] In some examples, the focus frame can be displayed in a highlighted or marked manner. For example, the focus processing module in the processor 250 can mark the focus corresponding to the control in the first user interface and highlight or mark the focus frame in the first user interface.

[0310] In some examples, the processor 250 may determine a starting position of the focus frame and display the focus frame at the starting position (e.g., the starting focus position) before controlling the display 260 to display the focus frame. The processor 250 may then move the focus frame according to a focus switching operation input by the user, thereby switching the focus between the controls in the first user interface.

[0311] In some embodiments, the starting position of the focus frame may be the focus position corresponding to the first control in the first user interface.

[0312] FIG32 is a schematic diagram of a single-window display provided by an embodiment of the present application. As shown in FIG32 , display 260 is currently in a single-window display, with a first user interface displayed in a first display window. The first user interface includes multiple controls, such as three controls (control 601, control 602, and control 603), each of which has a focus. After the starting position of the focus frame is determined in the first user interface, the focus frame can be moved.

[0313] As shown in Figure 32, the starting position of the focus frame in the first user interface may be the first control position in the upper left corner of the first user interface, that is, the position of control 601. The focus frame at the focus position corresponding to control 601 may be displayed in a highlighted manner.

[0314] In other embodiments, the starting position of the focus frame may also be the focus position corresponding to a preset control in the first user interface, wherein the preset control may be the first control in the first user interface or may not be the first control in the first user interface.

[0315] The processor 250 can determine the starting position of the focus frame through a focus search operation. In some examples, the processor 250 can perform a focus search operation in a top-to-bottom and left-to-right order. For example, the processor 250 can search for the first focusable control in the first user interface through restoreDefaultFocus() in DecorView, and can execute the requestFocus() process after the search. The requestFocus() process can include two parameters, direction and previouslyFocusedRect, where the direction parameter can be searched according to the current focus direction, and the previouslyFocusedRect parameter can record the relevant information obtained (such as focus information).

[0316] Figure 33 is a schematic diagram of another single-window display provided in an embodiment of the present application. As shown in Figure 33, when the number of controls in the first user interface is large and their distribution is complex, the controls that are used most frequently in the first user interface can be determined as preset controls; alternatively, preset controls can be set according to other rules, which are not limited in this embodiment of the present application.

[0317] As shown in Figure 33, after the first user interface is laid out and displayed, processor 250 determines that the "Selected" control in the first user interface is the starting focus through a focus search operation. That is, the focus position corresponding to the "Selected" control is the starting position of the focus frame, and then displays the focus frame at the focus position corresponding to the "Selected" control. In other words, the "Selected" control is a preset control. Referring to Figure 33, the starting position of the focus frame is not the focus position corresponding to the first control "My" in the first user interface, but the focus position corresponding to the preset control "Selected".

[0318] Figure 34 is a schematic diagram of a multi-window display provided by an embodiment of the present application. As shown in Figure 34, the first display window 810 can be a projection window on the left side of the display 260, and the second display window 820 can be an application window on the right side. When the display 260 displays a second user interface including multiple applications (or multiple controls) in the right application window, if a multi-window display operation input by the user is received (such as accessing a projection signal source), the multi-window display mode is entered, and the display 260 is controlled to display the first user interface including the projection screen in the left projection window (i.e., the first display window 810).

[0319] After display 260 enters multi-window display, processor 250 needs to re-determine the starting position of the focus frame. In some examples, when display 260 displays only the second user interface, the starting position of the focus frame may be located at a focus position in the second user interface. When display 260 displays the second user interface and the first user interface in the first display window at the same time, the starting position of the focus frame may be determined according to the display order of the first display window and the second display window.

[0320] For example, the second display window 820 (i.e., the right display window) is the main window (also referred to as the preset window) of the display 260. That is, the second display window 820 is displayed before the first display window 810. Therefore, the starting position of the focus frame can be determined to be located at a certain focus position of the second user interface of the second display window 820 that was displayed first. For another example, after the display 260 enters a multi-window display, the starting position of the focus frame can also be located at a certain focus position of the first user interface of the first display window 810 that was displayed later. This embodiment of the present application is not limited to this.

[0321] It should be noted that the process of determining the starting position of the focus frame in the first user interface and the second user interface is similar. To avoid repetition, the embodiment of the present application uses the example of determining the starting position of the focus frame in the first user interface displayed later as an example for illustrative explanation.

[0322] After determining that the starting position of the focus frame is located at the first focus of the first user interface of the first display window displayed later, the position of the first focus of the first user interface can be further determined. The process of displaying the focus frame at the first focus position of the first user interface in an embodiment of the present application is described in detail below with reference to FIG.

[0323] In some embodiments, the first focus includes a first sub-focus and a second sub-focus.

[0324] Figure 35 is a schematic diagram of another display method provided in an embodiment of the present application. As shown in Figure 35, in step S520 of the above embodiment, a focus frame is displayed at the first focus position of the first user interface, which specifically includes steps S5211 to S5213 as described below.

[0325] Step S5211: Determine the priority of the first display window.

[0326] In some examples, the priority of each display window in the plurality of display windows may include a priority attribute of each display window, which may also be referred to as a window attribute, wherein each display window has a pre-set priority attribute, and the first focus may be determined based on the priority attribute of each display window.

[0327] In some examples, the first focus may include a first sub-focus corresponding to the first display window and a second sub-focus corresponding to the first control in the first user interface. In other words, the priority of the first display window may determine whether the starting position of the focus frame is located at the first sub-focus position corresponding to the first display window (also referred to as the focus position corresponding to the large window) or at the second sub-focus position corresponding to the first control in the first user interface (also referred to as the focus position of the sub-window).

[0328] Step S5212: When the priority of the first display window satisfies the first condition, a focus frame is displayed at the first sub-focus position corresponding to the first display window.

[0329] The fact that the priority of the first display window satisfies the first condition may include the fact that the priority attribute of the first display window is window priority. For example, the window attribute of the first display window may be "FOCUS_BLOCK_DESCENDANTS." In other words, when the priority attribute of the first display window is "FOCUS_BLOCK_DESCENDANTS," it indicates that the priority of the first display window satisfies the first condition. In this case, the starting position of the focus frame is located at the first sub-focus position corresponding to the first display window.

[0330] In some examples, after determining that the starting position of the focus frame is located at the first sub-focus position corresponding to the first display window, the processor 250 controls the display 260 to display the focus frame at the first sub-focus position.

[0331] FIG36A is a schematic diagram of another multi-window display provided by an embodiment of the present application. As shown in FIG36A , the display 260 displays a focus frame at the first sub-focus position corresponding to the first display window 810 (such as the left projection window).

[0332] In some examples, as shown in FIG36A , if the priority attribute (e.g., window attribute) of the first display window 810 is determined to be “FOCUS_BLOCK_DESCENDANTS,” the first sub-focus may be acquired in the left projection window mLeftWindowView of the display 260 through a focus search operation. For example, the focus processing module in the processor 250 may mark the first sub-focus corresponding to the first display window 810 as being in focus, and the display management module in the processor 250 may set the display state of the first sub-focus to a displayable state (e.g., a Visible state), and display the focus frame at the first sub-focus position, i.e., set the focus frame to be highlighted or marked at the first focus position.

[0333] Step S5213: When the priority of the first display window satisfies the second condition, a focus frame is displayed at the second sub-focus position corresponding to the first control in the first user interface.

[0334] The priority of the first display window meeting the second condition may include the first display window's priority attribute being control-priority, such as the first display window's window attribute being "FOCUS_AFTER_DESCENDANTS." In other words, when the first display window's priority attribute is "FOCUS_AFTER_DESCENDANTS," it indicates that the first display window's priority meets the second condition. In this case, the focus frame's starting position is located at the second sub-focus position corresponding to the first control in the first user interface.

[0335] In some examples, after determining that the starting position of the focus frame is located at the second sub-focus position corresponding to the first control in the first user interface, the processor 250 controls the display 260 to display the focus frame at the second sub-focus position.

[0336] The first control can be the first control of the first user interface, or it can be a preset control of the first user interface. For example, as shown in Figure 32, the first control is the first control of the first user interface, and the starting position of the focus frame is located at the focus position corresponding to the first control (i.e., control 601) of the first user interface. As shown in Figure 33, the first control is the preset control of the first user interface, and the starting position of the focus frame is located at the focus position corresponding to the preset control (such as the control "Selected") of the first user interface. The embodiment of the present application does not limit the first control. The embodiment of the present application is illustratively described by taking the first control being the first control of the first user interface as an example.

[0337] Figure 36B is a schematic diagram of another multi-window display provided by an embodiment of the present application. As shown in Figure 36B, the display 260 displays a focus frame at the second sub-focus position corresponding to the first control of the first user interface displayed in the first display window 810 (such as the left projection window).

[0338] In some examples, as shown in FIG36B , if the priority attribute (such as the window attribute) of the first display window 810 is determined to be “FOCUS_AFTER_DESCENDANTS”, the second sub-focus can be obtained in the first user interface of the first display window 810 (the left projection window mLeftWindowView) through a focus operation. For example, after the first control mLeftWindwosChildView 0 is determined through a focus operation, it is determined whether the focus corresponding to the first control mLeftWindwosChildView 0 can be obtained; if the focus corresponding to the first control mLeftWindwosChildView 0 can be obtained, a requestfocus() process is performed on the first control mLeftWindwosChildView 0, that is, the display management module in the processor 250 can set the display state of the focus corresponding to the first control mLeftWindwosChildView 0 to a displayable state (such as a Visible state), and highlight or mark the focus frame at the focus position corresponding to the first control mLeftWindwosChildView 0.

[0339] Step S530: receiving a first focus switching operation input by the user through the communication device.

[0340] After the first user interface determines the starting position of the focus frame, the processor 250 may receive a first focus switching operation input by the user. The first focus switching operation is used to instruct the processor 250 to move the focus frame from the first focus position of the first user interface (i.e., the starting position of the focus) to another focus position. The other focus position may be a focus position in the first user interface or a focus position in the second user interface.

[0341] In some examples, the first focus switching operation can be input by the user through a key on the control device 100 (such as a remote control) for indicating a direction. Among them, the keys on the remote control for indicating a direction include direction keys for moving up, moving down, moving left, and moving right. Alternatively, the user can also implement the first focus switching operation by inputting voice commands. It should be noted that the user can also input the first focus switching operation through devices such as a touch screen, mouse, and keyboard. The embodiments of the present application are not limited to this. For example, the processor 250 can switch the focus position, that is, move the focus frame position, according to the first focus switching operation input by the user.

[0342] In some examples, the processor 250 can determine the movement direction and movement order of the focus frame based on the first focus switching operation. For example, as shown in FIG36B , the highlighted focus frame (i.e., the starting position) is located at the focus position corresponding to the first control mLeftWindwosChildView 0 in the first user interface; if the first focus switching operation input by the user is "right move", the processor 250 can move the focus frame from the focus position corresponding to the first control mLeftWindwosChildView 0 to the focus position corresponding to the second control mLeftWindwosChildView 1 to the right of the first control.

[0343] Step S540 : In response to the first focus switching operation, the focus frame is moved from the first focus position to the second focus position of the first user interface or the third focus position of the second user interface.

[0344] In some examples, the processor 250 can switch the focuses within the first user interface or switch the focuses between the first display window (or first user interface) and the second display window (or first user interface) according to the first focus switching operation.

[0345] In some embodiments, the user interface displayed in at least one of the multiple display windows may include at least one control. For example, the first user interface displayed in the first display window may include multiple controls, which may include a first control and a second control; alternatively, the first user interface may include one control; or alternatively, no controls may be provided in the first user interface. It should be noted that the number of controls provided in the second user interface may be the same as or different from the number of controls provided in the first user interface, and this is not limited in the embodiments of the present application.

[0346] The following describes respectively the case of moving the focus frame from the first focus position to the second focus position of the first user interface (i.e., switching the focus within the first user interface) and the case of moving the focus frame from the first focus position to the third focus position of the second user interface (i.e., switching the focus between the first display window and the second display window).

[0347] Figure 37 is a schematic diagram of another display method provided by an embodiment of the present application. In some embodiments, when a focus frame is displayed at a first sub-focus position corresponding to a first display window, and the first user interface includes a first control and a second control, as shown in Figure 37, step S540 includes steps S5411 to S5413 as shown below.

[0348] Step S5411: In response to the first focus switching operation, determine the focus attribute of the first control.

[0349] In some examples, as shown in Figure 36A, when the processor 250 determines that the starting position of the focus frame is the first sub-focus position, that is, determines that the priority attribute of the first display window is "FOCUS_BLOCK_DESCENDANTS", if it is determined that the first user interface includes at least two controls (such as a first control and a second control), the processor 250 responds to the first focus switching operation and can first determine the focus attribute (also called the focus attribute) of the first control.

[0350] In some embodiments, the focus property of a control includes enabled and disabled.

[0351] The focus attribute of a control being valid may include setting the focus attribute of the control to true; when the focus attribute of a control is valid, it indicates that the processor 250 can obtain the focus corresponding to the control and can also display a focus frame at the focus position corresponding to the control. The focus attribute of a control being invalid may include setting the focus attribute of the control to false; when the focus attribute is invalid, it indicates that the processor 250 cannot obtain the focus corresponding to the control and cannot display a focus frame at the focus position corresponding to the control.

[0352] It should be noted that the focus attributes of various controls in a user interface can be the same or different. For example, the focus attributes of multiple controls in a first user interface can be valid or invalid; or, the focus attributes of some controls in the first user interface can be valid, while the focus attributes of other controls can be invalid. This embodiment of the present application is not limited to this.

[0353] The first control of the first user interface may be the first control in the first user interface; for example, the first control may be the first control (i.e., control 601) shown in FIG32. Alternatively, the first control may be a preset control in the first user interface; for example, the first control may be the preset control (i.e., control "Featured") shown in FIG33.

[0354] The embodiment of the present application is exemplified by taking the first control in the first user interface as an example. For example, as shown in FIG36B , the first control may be the first control mleftWindwosChildView 0 in the first user interface.

[0355] Step S5412: When the focus attribute of the first control is valid, the focus frame is moved from the first sub-focus position to the second sub-focus position corresponding to the first control.

[0356] For example, as shown in Figure 36B, when the focus property of the first control (i.e., the first control mLeftWindwosChildView 0) is valid, it indicates that the focus corresponding to the first control can be obtained. Therefore, the processor 250 can move the focus frame from the first sub-focus position to the second sub-focus position corresponding to the first control in response to the first focus switching operation.

[0357] In some examples, when the starting position of the focus frame is at the first sub-focus position corresponding to the first display window, the first focus switching operation can be input through a button of the remote control (such as the confirmation key). That is, when the processor 250 determines that there is a control with a valid focus attribute in the first user interface, it can move the focus frame from the focus position corresponding to the first display window (such as the left projection window mLeftWindowView) to the focus position corresponding to the internal control of the first user interface, and then determine the first control through the focus search operation, and determine whether the second sub-focus corresponding to the first control (such as the first control) can be obtained; when the second sub-focus corresponding to the first control can be obtained, that is, the focus attribute of the first control is true, the first control is processed with requestfocus(), and the display 260 is controlled to display the focus frame at the second sub-focus position corresponding to the first control.

[0358] FIG38 is a schematic diagram of another multi-window display provided by an embodiment of the present application. As shown in (a), (b), and (c) of FIG38 , the first user interface displayed in the first display window 810 includes a first control mLeftWindwosChildView 0, a second control mLeftWindwosChildView 1, a third control mLeftWindwosChildView 2, a fourth control mLeftWindwosChildView 3, a fifth control mLeftWindwosChildView 4, a sixth control mLeftWindwosChildView 5, and a seventh control mLeftWindwosChildView 6. In some examples, if the focus attribute of the first control mLeftWindwosChildView 0 is valid, the processor 250 can move the focus frame from the first sub-focus position corresponding to the first display window 810 shown in FIG38 (a) to the focus position corresponding to the first control mLeftWindwosChildView 0 shown in FIG38 (b).

[0359] Step S5413: When the focus attribute of the first control is invalid, the focus frame is moved from the first sub-focus position to a fourth focus position corresponding to the second control behind the first control.

[0360] The focus property of the second control is valid.

[0361] In some examples, when the focus attribute of the first control is invalid, it indicates that the focus corresponding to the first control cannot be obtained. In this case, the processor 250 can move the focus frame from the first sub-focus position to the fourth focus position corresponding to the second control whose focus attribute is valid after the first control in response to the first focus switching operation.

[0362] That is, when the processor 250 determines that the focus attribute of the first control is invalid, it can continue to determine the focus attribute of the second control after the first control. When the focus attribute of the second control is determined to be valid, the focus frame can be moved from the first sub-focus position corresponding to the first display window to the fourth focus position corresponding to the second control.

[0363] The processor 250 may determine the second control according to a preset focus sequence. For example, the second control may be a control whose focus attribute (such as the first focus attribute) is valid after the first control determined in the order from top to bottom and from left to right.

[0364] In some examples, the processor 250 can determine the focus attributes of each control after the first control in sequence according to a preset focus order until a control with a valid focus attribute in the first user interface is determined, and then move the focus frame from the first sub-focus position to the focus position corresponding to the control with the first valid focus attribute.

[0365] For example, as shown in (a) in Figure 38, when the first control is the first control mLeftWindwosChildView 0 and the focus property of the first control mLeftWindwosChildView 0 is invalid (such as false), the second control can be determined in the order of the second control mLeftWindwosChildView 1, the third control mLeftWindwosChildView 2, the fourth control mLeftWindwosChildView 3, the fifth control mLeftWindwosChildView 4, the sixth control mLeftWindwosChildView 5 and the seventh control mLeftWindwosChildView 6.

[0366] For example, as shown in (a) and (c) of FIG38 , when the focus attribute of the first control mLeftWindwosChildView 0 among the seven controls in the first user interface is invalid, and the focus attributes of the other six controls are valid, the second control may be the second control mLeftWindwosChildView 1 after the first control. The processor 250 may move the focus frame from the first sub-focus position corresponding to the first display window shown in FIG38 (a) to the focus position corresponding to the second control mLeftWindwosChildView1 (i.e., the second control) shown in FIG38 (c).

[0367] For another example, when the focus attributes of the first control mLeftWindwosChildView 0, the second control mLeftWindwosChildView 1, the third control mLeftWindwosChildView 2, and the fourth control mLeftWindwosChildView 3 among the seven controls in the first user interface are invalid, and the focus attributes of the fifth control mLeftWindwosChildView 4, the sixth control mLeftWindwosChildView 5, and the seventh control mLeftWindwosChildView 6 are valid, the determined second control can be the first control with a valid focus attribute after the first control mLeftWindwosChildView 0, that is, the fifth control mLeftWindwosChildView 4.

[0368] It should be noted that the preset focus order may also adopt other rules, which is not limited in the embodiments of the present application.

[0369] In some embodiments, when a focus frame is displayed at a first sub-focus position corresponding to a first display window and the first user interface does not include a control with a valid focus attribute, the focus frame is moved from the first sub-focus position to a third focus position of the second user interface in response to a first focus switching operation.

[0370] That is, when a focus frame is displayed at the first sub-focus position corresponding to the first display window, and the first user interface does not include a control with a valid focus attribute, the processor 250 may move the focus frame from the first display window to the second display window in response to the first focus switching operation, thereby switching the focus between the first display window and the second display window. For example, the user may input the first focus switching operation using a key on the remote control (such as the right shift key) to cause the processor 250 to switch the focus frame from the first display window to the second display window.

[0371] In some examples, that the first user interface does not include a control whose focus attribute is valid may include: the first user interface includes at least one control, and the focus attribute of the at least one control is invalid.

[0372] For example, as shown in Figure 38, when the first user interface includes 7 controls and the focus attributes of the 7 controls are invalid, in this case, the processor 250 moves the focus frame from the first sub-focus position corresponding to the first display window shown in (a) in Figure 38 to the third focus position of the second user interface in the second display window 820.

[0373] In other examples, the first user interface not including a control with a valid focus attribute may also include: the first user interface does not include a control. For example, when the entire first display window displays only the projection image in full screen, the first user interface of the first display window may not include a control.

[0374] Figure 39 is a schematic diagram of another multi-window display provided by an embodiment of the present application. As shown in (a) of Figure 39, the first display window 810 (i.e., the left projection window) displays the first user interface including the projection screen in full screen, and the first user interface does not include other controls. In this case, the processor 250 moves the focus frame from the first sub-focus position corresponding to the first display window to the third focus position of the second user interface in response to the first focus switching operation.

[0375] In some embodiments, the third focus includes a third sub-focus and a fourth sub-focus. The at least one processor 250 is further configured to execute the computer instructions to cause the display device to: determine the priority of the second display window in response to the first focus switching operation; and display a focus frame at the third sub-focus position corresponding to the second display window if the priority of the second display window meets a first condition. If the priority of the second display window meets a second condition, display a focus frame at the fourth sub-focus position corresponding to the third control in the second user interface.

[0376] When it is determined that the first focus switching operation is used to switch the focus frame from the first focus position of the first user interface to the third focus position of the second user interface, the position of the third focus in the second user interface may be determined based on the priority of the second display window. The position of the third focus in the second user interface includes a third sub-focus position corresponding to the second display window or a fourth sub-focus position corresponding to the third control in the second user interface.

[0377] In some examples, when the priority of the second display window satisfies the first condition, that is, the priority attribute of the second display window is "FOCUS_BLOCK_DESCENDANTS", the third focus is the third sub-focus corresponding to the second display window. In this case, the processor 250 can move the focus frame from the first sub-focus position corresponding to the first display window 810 shown in FIG. 39(a) to the third sub-focus position corresponding to the second display window 820 shown in FIG. 39(b).

[0378] In some examples, when the priority of the second display window satisfies the second condition, that is, the priority attribute of the second display window is "FOCUS_AFTER_DESCENDANTS", the third focus is the fourth sub-focus corresponding to the third control in the second user interface. In this case, the processor 250 can move the focus frame from the first sub-focus position corresponding to the first display window 810 shown in (a) of Figure 39 to the fourth sub-focus position corresponding to the third control in the second user interface shown in (c) of Figure 39.

[0379] The third control may be the first control in the second user interface, or may be any control in the second user interface (such as a preset control), which is not limited in this embodiment of the present application. For example, as shown in (c) of Figure 39, the third control is the first control in the second user interface, and the focus attribute of the third control is valid.

[0380] It should be noted that the process of determining the priority of the second display window is similar to the process of determining the priority of the first display window in the above embodiment, and will not be described again here to avoid repetition.

[0381] The above embodiment describes the process by which the processor 250 moves the focus frame in response to a first focus switching operation when the focus frame is displayed at a first sub-focus position corresponding to a first display window. The following describes the process by which the processor 250 moves the focus frame in response to a first focus switching operation when the focus frame is displayed at a second sub-focus position corresponding to a first control in a first user interface.

[0382] It should be noted that the processor 250 controls the display 260 to display a focus frame at the second sub-focus position corresponding to the first control in the first user interface only if the focus attribute of the first control is valid. In other words, the processor 250 can control the display 260 to display a focus frame at the second sub-focus position corresponding to the first control in the first user interface only when the focus attribute of the first control is valid.

[0383] In some embodiments, when a focus frame is displayed at a second sub-focus position corresponding to a first control in a first user interface and the first user interface includes a second control, the processor 250 moves the focus frame from the second sub-focus position to a fourth focus position corresponding to the second control in response to the first focus switching operation.

[0384] The focus property of the second control is valid.

[0385] In some examples, when the processor 250 determines that the starting position of the focus frame is the second sub-focus position corresponding to the first control of the first user interface (such as the first control mLeftWindwosChildView 0 in Figure 36B), that is, when the priority attribute of the first display window is determined to be "FOCUS_AFTER_DESCENDANTS", if it is determined that the first user interface also includes at least one control, the processor 250, in response to the first focus switching operation, can determine the second control according to the focus attribute of the at least one control, and move the focus frame from the second sub-focus position to the fourth focus position corresponding to the second control.

[0386] In some examples, as shown in FIG. 36B , when the first control is determined to be the first control mLeftWindwosChildView 0 in the first user interface, the processor 250 determines the second control according to the first focus switching operation.

[0387] For example, when the first focus switching operation input by the user is a right move operation, the processor 250 may determine whether the focus attribute of the first control to the right of the first control is valid. When the focus attribute of the first control to the right of the first control is valid, the first control to the right of the first control may be determined as the second control. When the focus attribute of the first control to the right of the first control is invalid, the processor 250 may continue to determine whether the focus attribute of the second control to the right of the first control is valid, until it is determined that the first control after the first control has a valid focus attribute, and then determine the control as the second control.

[0388] As shown in FIG36B , when the first focus switching operation input by the user is a right move operation, the processor 250 may determine the focus attributes of the controls in the order of the second control mLeftWindwosChildView 1, the third control mLeftWindwosChildView 2, the fourth control mLeftWindwosChildView 3, the fifth control mLeftWindwosChildView 4, the sixth control mLeftWindwosChildView 5, and the seventh control mLeftWindwosChildView 6. For example, after the processor 250 determines that the first control has a valid focus attribute, it may determine the control as the second control, and there is no need to determine the focus attributes of controls subsequent to the second control. For example, when it is determined that the focus properties of the second control mLeftWindwosChildView 1, the third control mLeftWindwosChildView 2, and the fourth control mLeftWindwosChildView 3 are invalid (e.g., false), and the focus property of the fifth control mLeftWindwosChildView 4 following the fourth control is valid (e.g., true), the processor 250 may determine the fifth control mLeftWindwosChildView 4 as the second control. The processor 250 moves the focus frame from the focus position corresponding to the first control mLeftWindwosChildView 0 to the focus position corresponding to the fifth control mLeftWindwosChildView 4.

[0389] In the embodiment of the present application, the first focus switching operation input by the user is a right move operation as an example to schematically illustrate the focusing order when the processor 250 determines the second control. It should be noted that the first focus switching operation can also be other operations, such as a downward move, upward move or right move operation, etc. When the first focus switching operation is different, the corresponding order of the focusing operations may be different. For example, as shown in Figure 36B, when the first focus switching operation input by the user is a downward move operation, the processor 250 can determine the second control in the first user interface in the order of the fourth control mLeftWindwosChildView 3, the fifth control mLeftWindwosChildView 4, the second control mLeftWindwosChildView 1, the sixth control mLeftWindwosChildView 5, the third control mLeftWindwosChildView 2 and the seventh control mLeftWindwosChildView 6.

[0390] When the focus frame moves to the second control, the processor 250 may continue to receive at least one first focus switching operation input by the user. In response to at least one first focus switching operation, the processor 250 may move the focus frame from the focus position corresponding to the second control to the target position.

[0391] Figure 40 is a schematic diagram of another multi-window display provided by an embodiment of the present application. As shown in Figure 40, an example is given in which the current position of the focus frame is the focus position corresponding to the second control mLeftWindwosChildView 1, and the target position of the focus frame is the focus position corresponding to the fifth control mLeftWindwosChildView 4. For example, the communication device 220 can monitor the user's operation on the remote control in real time. If the user presses the right button of the remote control (i.e., continues to input the first focus switching operation), and determines that the current user's operation direction is Right, the processor 250 performs a focus search operation in the right direction. If the focus attribute of the third control mLeftWindwosChildView 2 to the right of the second control mLeftWindwosChildView 1 is valid, the processor 250 obtains the focus of the third control mLeftWindwosChildView 2 and moves the focus frame from the focus position corresponding to the second control mLeftWindwosChildView 1 to the focus position corresponding to the third control mLeftWindwosChildView 2. If the user continues to press the right button of the remote control, the processor 250 can continue to search for focus in the right direction. Since the right side of the third control mLeftWindwosChildView 2 is the right boundary of the first user interface, there are no other controls, that is, there is no focus that can be focused. Therefore, requestFocusNoSearch can return a prompt message indicating that the switch failed (such as returning false), and the focus frame can continue to remain on the third control mLeftWindwosChildView 2. In this case, the user can press the down button of the remote control, and the processor 250 will continue to search for focus downward. If the processor 250 obtains the focus corresponding to the fourth control mLeftWindwosChildView 3, the focus frame can be moved from the focus position corresponding to the third control mLeftWindwosChildView 2 to the focus position corresponding to the fourth control mLeftWindwosChildView 3. At this time, the user continues to press the right button of the remote control, and the processor 250 continues to perform the focus operation to the right. If the processor 250 obtains the focus corresponding to the fifth control mLeftWindwosChildView 4, the focus frame can be moved from the focus position corresponding to the fourth control mLeftWindwosChildView 3 to the focus position corresponding to the fifth control mLeftWindwosChildView4, thereby realizing the movement of the focus frame from the current position to the target position.

[0392] It should be noted that the user can also move the focus frame from the starting position to the target position through other first focus switching operations (such as pressing the down button light of the remote control twice in succession). This embodiment of the present application does not limit this.

[0393] In other embodiments, when a focus frame is displayed at the second sub-focus position corresponding to the first control in the first user interface, and the first user interface does not include other controls with valid focus attributes except the first control, the focus frame is moved from the second sub-focus position to the first sub-focus position in response to the first focus switching operation.

[0394] That is to say, when a focus frame is displayed at the second sub-focus position corresponding to the first control of the first user interface, and the first user interface does not include other controls with valid focus attributes, the processor 250 can move the focus frame from the focus position corresponding to the control inside the first display window to the focus position corresponding to the first display window in response to the first focus switching operation, thereby realizing the switching of the focus inside the first display window.

[0395] In some examples, the first user interface not including other controls with a valid focus attribute may include: the first user interface includes at least one control other than the first control, and the focus attribute of the at least one control is invalid. In other examples, the first user interface not including other controls with a valid focus attribute may also include: the first user interface includes no other controls other than the first control.

[0396] When the priority of the first display window meets the second condition, that is, the priority attribute of the first display window is "FOCUS_AFTER_DESCENDANTS", and the first user interface does not include controls with valid focus attributes other than the first control, the processor 250 moves the focus frame to the first sub-focus position corresponding to the first display window, and controls the display 260 to display the focus frame at the first sub-focus position.

[0397] For example, when the priority attribute of the first display window is "FOCUS_AFTER_DESCENDANTS", if the first focus switching operation input by the user is received, ViewGroup calls requestFocus() to give priority to the controls (also called subviews) in the first user interface to obtain focus. When the controls in the first user interface cannot become the focus, the focus can be obtained in the first display window, that is, the focus frame is displayed at the focus position corresponding to the first display window.

[0398] In some embodiments, the processor 250 is further configured to: receive a second focus switching operation input by the user through the communication device 220; and move the focus frame from the first sub-focus position to a third focus position of the second user interface in response to the second focus switching operation.

[0399] In the above embodiment, after the processor 250 moves the focus frame from the second sub-focus position to the first sub-focus position in response to the first focus switching operation, the processor 250 may also receive a second focus switching operation input by the user. The input method for the second focus switching operation is similar to the input method for the first focus switching operation, for example, and will not be further described here to avoid repetition.

[0400] In some examples, the second focus switching operation is used to instruct processor 250 to move the position of the focus frame. For example, the second focus switching operation may be the same as or different from the first focus switching operation. When the first focus switching operation is different from the second focus switching operation, the second focus switching operation may instruct processor 250 to move the focus frame, which has been moved by the first focus switching operation, from its current position to another position.

[0401] For example, the second focus switching operation may be to move the focus frame from the first sub-focus position to which the first focus switching operation was moved to the third focus position of the second user interface. In other words, the second focus switching operation may implement a focus switch between the first display window and the second display window.

[0402] Figure 41 is a schematic diagram of another display method provided in an embodiment of the present application. As shown in Figure 41, after step S540 in the above embodiment, the method also includes steps S550 to S570 as described below.

[0403] Step S550: When a focus frame is displayed at a fifth focus position corresponding to the third control in the first user interface, a third focus switching operation input by the user is received through the communication device.

[0404] In some examples, the first user interface further includes a third control, and the focus attribute of the third control is enabled. The third control can be any control in the first user interface. For example, the third control can be the same control as the first control; or the third control can be the same control as the second control; or the third control can be a control other than the first control and the second control. This embodiment of the present application is not limited to this.

[0405] The fifth focus position corresponding to the third control may be the current position of the focus frame.

[0406] In some examples, the third focus switching operation is used to indicate moving the focus frame from its current position to another position. For example, the third focus switching operation may be moving the focus frame from the fifth focus position corresponding to the third control in the first user interface to the third focus position corresponding to the second user interface. In other words, the third focus switching operation can switch the focus between the first display window and the second display window.

[0407] For example, the third focus switching operation may be used to instruct the processor 250 to switch the fifth focus corresponding to the third control in the first user interface to the third focus (including the third sub-focus and the fourth sub-focus) of the second user interface.

[0408] In some examples, the third focus switching operation can be input through a corresponding window switching button, a direction button, or other button on the remote control, or can be input through a voice command.

[0409] Step S560: In response to the third focus switching operation, determine the position of the third control in the first user interface.

[0410] Step S570: When the position of the third control meets the window switching condition, the focus frame is moved from the fifth focus position to the third focus position of the second user interface.

[0411] In some examples, after receiving the third focus switching operation, the processor 250 determines the position of the third control in the first user interface in response to the third focus switching operation, and determines whether the position of the third control meets the window switching condition.

[0412] For example, the position of the third control meeting the window switching condition may include: no other controls are between the position of the third control and a preset boundary in the first user interface. It should be noted that the window switching condition may also include other conditions set as needed, and the specific form of the window switching condition is not limited in this embodiment of the application.

[0413] Figure 42 is a schematic diagram of another multi-window display provided by an embodiment of the present application. As shown in Figure 42, the window switching condition is that there are no other controls between the position of the third control and a preset boundary in the first user interface (such as the right boundary of the first user interface), that is, the third control is adjacent to the right boundary of the first user interface.

[0414] As shown in Figure 42 , the positions of the third control mLeftWindwosChildView 2, the fourth control mLeftWindwosChildView 3, and the seventh control mLeftWindwosChildView 6 in the first user interface are adjacent to the right border of the first user interface. Therefore, the positions of the third control mLeftWindwosChildView 2, the fourth control mLeftWindwosChildView 3, and the seventh control mLeftWindwosChildView 6 meet the window switching condition. That is, in Figure 41 , the third control can be the third control mLeftWindwosChildView 2, the fourth control mLeftWindwosChildView 3, or the seventh control mLeftWindwosChildView 6.

[0415] For example, when the position of the third control meets the window switching condition, the processor 250 can move the focus frame from the fifth focus position corresponding to the third control to the third sub-focus position corresponding to the second display window (the priority of the second display window meets the first condition) or to the fourth sub-focus corresponding to the fourth control in the second user interface (the priority of the second display window meets the second condition) according to the third focus switching operation and the priority of the second display window.

[0416] In some examples, when the user inputs the third focus switching operation, the processor 250 may move the focus frame from the fifth focus position (e.g., the current position) to the third focus position of the second user interface in response to the third focus switching operation; or, in response to the third focus switching operation, the processor 250 may also move the focus frame from the fifth focus position (e.g., the current position) to any focus position in the second user interface. For example, the focus frame may be at the position previously displayed in the second user interface. This is not limited in the embodiments of the present application.

[0417] After the processor 250 controls the display 260 to display the focus frame, it can receive the control operation port input by the control device 100 (remote control), and can display the corresponding secondary or higher-level page for the control corresponding to the focus position of the focus frame.

[0418] It should be noted that the process of switching the focus in the second user interface is similar to the process of switching the focus in the first user interface in the above embodiment, and will not be described again here to avoid repetition.

[0419] The display method provided by the embodiment of the present application sets the focus on multiple display windows and the controls of the user interface corresponding to each display window, and when a multi-window display operation is received, a focus frame is displayed at the first focus position of the first user interface displayed by the first user interface corresponding to the first display window, that is, when the electronic device enters the multi-window display mode, the initial position of the focus frame is determined to be the first focus position. Then, after receiving the first focus switching operation, in response to the first focus switching operation, the focus frame can be moved from the first focus position to the second focus position of the first user interface; or, the focus frame can be moved from the first focus position to the third focus position of the second user interface displayed by the second display window. Therefore, the embodiment of the present application can realize the switching of focus between the first display window and the second display window, and can also realize the switching of focus between the controls within the first display window or the second display window.

[0420] Figure 43 is a schematic diagram of another display method provided in an embodiment of the present application. As shown in Figure 43, the method includes steps S1710 to S1770 as shown below.

[0421] Step S1710: Acquire the window display mode of the electronic device.

[0422] For example, the window display mode of the electronic device 200 includes single window display and multi-window display.

[0423] Step S1720: Determine whether the window display mode is multi-window display.

[0424] For example, if it is determined that the window display mode of the electronic device 200 is multi-window display, step S1730 is executed; if it is determined that the window display mode of the electronic device 200 is single-window display, step S1750 is executed.

[0425] Step S1730: Acquire the focus corresponding to the first display window.

[0426] For example, when it is determined that the display mode of the display window is multi-window display, the focus corresponding to the first display window among the multiple display windows can be obtained, wherein the first display window can be the main display window among the multiple display windows.

[0427] Step S1740: Control the display to display a focus frame at the focus position corresponding to the first display window.

[0428] If the focus corresponding to the first display window is acquired, the display 260 may be controlled to display a focus frame at the focus position of the first display window in a highlighted or marked manner.

[0429] Step S1750: monitoring window changes of the electronic device, internal UI changes of the first display window, and user operations.

[0430] The window change may include an operation corresponding to switching between the first display window and a second display window among the multiple display windows; the change in the UI within the first display window may include a change in each control in the first user interface displayed in the first display window; and the user operation may include a related focus control operation input by the user through the control device 100 (such as a remote control). For example, the user operation may include the first focus switching operation in the above-mentioned embodiment.

[0431] Step S1760: Determine whether there is window switching, internal UI change, or user operation.

[0432] If at least one of window switching, internal UI change or user operation occurs, step 1770 is executed; if there is no window switching, internal UI change or user operation, step S1750 is executed to continue monitoring.

[0433] Step S1770: Process according to the preset focus processing principle.

[0434] If the electronic device 200 displays a single window, the focus of each control in the single window can be determined according to a preset focus order, such as from top to bottom or from left to right, and a focus frame can be displayed at the focus position; alternatively, a focus frame can be displayed at the focus position corresponding to the preset control; if the electronic device 200 displays multiple windows, the focus can be switched between the multiple windows first, and then the focus corresponding to the controls within each window can be switched. The focus between controls within each display window is handled in a similar manner to the focus of controls within a single window.

[0435] It should be noted that steps S1710 to S1770 have been described in detail in steps S520 to S540 of the above embodiment, and will not be repeated here to avoid repetition.

[0436] FIG44 is a schematic diagram of a focus switching device provided by an embodiment of the present application. As shown in FIG44 , the focus switching device includes a receiving module 1810, a control module 1820, and a switching module 1830.

[0437] The receiving module 1810 is configured to receive a multi-window display operation input by a user through a communication device when the display displays the first user interface.

[0438] The control module 1820 is configured to control the display to display the first user interface in the first display window, the second user interface in the second display window, and display a focus frame at the first focus position of the first user interface in response to the multi-window display operation.

[0439] The receiving module 1810 is further configured to receive a first focus switching operation input by a user through a communication device.

[0440] The switching module 1830 is configured to move the focus frame from the first focus position to the second focus position of the first user interface or the third focus position of the second user interface in response to the first focus switching operation.

[0441] It should be noted that the focus switching device provided in the embodiment of the present application includes but is not limited to the above modules.

[0442] Regarding the focus switching device applied to the electronic device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the display method mentioned above, and will not be elaborated here.

[0443] FIG45 is a schematic diagram of an electronic device provided in an embodiment of the present application. As shown in FIG45 , the electronic device 200 includes: a communication device 220, a display 260, a user interface 280, a memory 201, and a processor 250. Among them:

[0444] The display 260 is configured to display images and / or a user interface.

[0445] User interface 280, configured to receive instructions from a user;

[0446] a communication device 220 configured to communicate with an external device according to a predetermined protocol;

[0447] Memory 201, configured to store computer instructions and data associated with the display device;

[0448] At least one processor 250, connected to the display 260, the user interface 280, the communication device 220 and the memory 201, is configured to execute computer instructions to cause the display device to perform:

[0449] When the display 260 displays the first user interface, a multi-window display operation input by the user is received through the communication device 220; in response to the multi-window display operation, the display 260 is controlled to display the first user interface in the first display window, the second user interface in the second display window, and a focus frame is displayed at the first focus position of the first user interface; a first focus switching operation input by the user is received through the communication device 220; in response to the first focus switching operation, the focus frame is moved from the first focus position to the second focus position of the first user interface or the third focus position of the second user interface.

[0450] In some embodiments, the first focus includes a first sub-focus and a second sub-focus; the at least one processor 250 is further configured to execute the computer instructions to cause the display device to execute: determining the priority of the first display window; displaying the focus frame at the first sub-focus position corresponding to the first display window when the priority of the first display window meets the first condition; displaying the focus frame at the second sub-focus position corresponding to the first control in the first user interface when the priority of the first display window meets the second condition.

[0451] In some embodiments, the at least one processor 250 is further configured to execute the computer instructions to cause the display device to perform: displaying a focus frame at a first sub-focus position corresponding to the first display window, and when the first user interface includes a first control and a second control, determining the focus attribute of the first control in response to a first focus switching operation; when the focus attribute of the first control is valid, moving the focus frame from the first sub-focus position to the second sub-focus position corresponding to the first control; when the focus attribute of the first control is invalid, moving the focus frame from the first sub-focus position to a fourth focus position corresponding to the second control behind the first control; wherein, the focus attribute of the second control is valid.

[0452] In some embodiments, the at least one processor 250 is further configured to execute the computer instructions to cause the display device to perform: displaying a focus frame at a first sub-focus position corresponding to the first display window, and when the first user interface does not include a control with a valid focus attribute, in response to the first focus switching operation, moving the focus frame from the first sub-focus position to a third focus position of the second user interface.

[0453] In some embodiments, the at least one processor 250 is further configured to execute the computer instructions to cause the display device to perform: displaying a focus frame at the second sub-focus position corresponding to the first control in the first user interface, and when the first user interface includes a second control, in response to the first focus switching operation, moving the focus frame from the second sub-focus position to the fourth focus position corresponding to the second control; wherein the focus attribute of the second control is valid.

[0454] In some embodiments, the at least one processor 250 is further configured to execute the computer instructions to cause the display device to perform: displaying a focus frame at the second sub-focus position corresponding to the first control in the first user interface, and in response to the first focus switching operation, moving the focus frame from the second sub-focus position corresponding to the first control to the first sub-focus position when the first user interface does not include other controls with valid focus attributes except the first control.

[0455] In some embodiments, the at least one processor 250 is further configured to execute the computer instructions so that the display device performs: receiving a second focus switching operation input by the user through the communication device 220; and moving the focus frame from the first sub-focus position to the third focus position of the second user interface in response to the second focus switching operation.

[0456] In some embodiments, the at least one processor 250 is further configured to execute the computer instructions so that the display device performs: when a focus frame is displayed at the fifth focus position corresponding to the third control in the first user interface, receiving a third focus switching operation input by the user through the communication device 220; in response to the third focus switching operation, determining the position of the third control in the first user interface; and when the position of the third control meets the window switching condition, moving the focus frame from the fifth focus position to the third focus position of the second user interface.

[0457] In some embodiments, the third focus includes a third sub-focus and a fourth sub-focus; the at least one processor 250 is further configured to execute the computer instructions to cause the display device to execute: determining the priority of the second display window; if the priority of the second display window meets the first condition, moving the focus frame to the third sub-focus position corresponding to the second display window; if the priority of the second display window meets the second condition, moving the focus frame to the fourth sub-focus position corresponding to the fourth control in the second user interface.

[0458] An embodiment of the present application also provides an electronic device, comprising: a processor and a memory; the memory is used to store computer instructions, and when the electronic device is running, the processor executes the computer instructions stored in the memory to enable the electronic device to execute the display method provided by the embodiment of the present application.

[0459] An embodiment of the present application also provides a computer non-volatile readable storage medium, which stores computer instructions. When the computer instructions are executed on an electronic device, the electronic device can execute the display method provided by the embodiment of the present application.

[0460] For example, the computer non-volatile readable storage medium may be a ROM, RAM, a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0461] For ease of explanation, the above description has been made with reference to specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations are possible. The above embodiments are selected and described to better explain the principles and practical applications, so that those skilled in the art can better utilize the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. An electronic device, comprising: a display configured to display an image and / or a user interface; A user interface configured to receive instructions from a user; a communication device configured to communicate with an external device according to a predetermined protocol; a memory configured to store computer instructions and data associated with a display device; An image acquisition interface, configured to connect to a camera to acquire a user image, wherein the camera is disposed at a center position of an upper frame or a lower frame of the display; At least one processor, connected to the display, user interface, communicator and memory, is configured to execute computer instructions to cause the display device to perform: In response to a screen projection connection request sent by a mobile terminal, starting the camera, and acquiring a user image captured by the camera; Identify the direction information of the user relative to the center of the screen through the user image; the center of the screen is the center line position of the electronic device along the vertical direction; Calculate the target position according to the direction information; Receive the projection screen sent by the mobile terminal to the electronic device, and control the display to display the projection screen at the target location.

2. The electronic device according to claim 1, wherein the direction information comprises at least a first direction and a second direction; the first direction is opposite to the second direction; the processor performs the step of identifying the direction information of the user relative to the center of the screen through the user image, and is further configured to execute the computer instructions so that the display device performs: Get the screen center position of the electronic device; Parsing the user image to obtain a user coordinate set of a user target in the user image; Comparing the pixel coordinates in the user coordinate set with the screen center coordinates; If the coordinates of the pixel points in a preset ratio in the user coordinate set are greater than the coordinates of the center of the screen, determining that the direction information is a first direction; If the coordinates of the pixel points of a preset number ratio in the user coordinate set are smaller than the coordinates of the center of the screen, it is determined that the direction information is the second direction.

3. The electronic device according to claim 2, wherein the direction information further includes a third direction, the third direction being a direction within a preset range of the center of the screen, and the at least one processor is further configured to execute the computer instruction so that the display device executes: Calculate the difference between the pixel coordinates in the user coordinate set and the center coordinates of the screen; If the difference values ​​corresponding to the pixel point coordinates of a preset number ratio are within a difference range, determining that the direction information is a third direction; In response to the direction information being a third direction, the display is controlled to display the projection image in a floating manner at a preset position of the user interface.

4. The electronic device according to claim 2, wherein the processor executes the step of controlling the display to display the projection image at the target location, and is further configured to execute the computer instructions so that the display device executes: Obtaining the initial direction of the projection image; Detect the user's direction information; If the initial direction is the same as the direction information, controlling the display to display the projection image at a default position; If the initial direction is opposite to the direction information, the display is controlled to display the projection image at the target position.

5. The electronic device of claim 1 , wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: Receiving a feature image sent by a mobile terminal; the feature image includes feature information of a key user target of the screen projection; identifying the number of users in the user image; When the number of users is greater than a preset number threshold, identifying key user targets in the user image according to the feature image; The direction information of the user relative to the center of the screen is identified according to the key user goal.

6. The electronic device of claim 1, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: Establish a transmission channel for the projection image sent by the mobile terminal; Receiving the projection screen through the transmission channel; Decoding the projected screen image; Control the display to display the projection image at the target location.

7. An electronic device comprising: a display configured to display an image and / or a user interface; A user interface configured to receive instructions from a user; a communication device configured to communicate with an external device according to a predetermined protocol; a memory configured to store computer instructions and data associated with a display device; An image acquisition interface, configured to connect to a camera to acquire local video data, wherein the camera is arranged at the center of the upper frame or the lower frame of the display; Processor: At least one processor, connected to the display, user interface, communicator and memory, is configured to execute computer instructions to cause the display device to perform: In response to a screen projection request for fitness media resources issued by a mobile terminal, starting the camera, and acquiring local video data captured by the camera; Controlling the display to display the local video data in full screen in the user interface, and controlling the display to display a projection screen of the fitness media of the mobile terminal in a floating manner in the user interface; Identify the direction information of the user's head relative to the center of the screen according to the local video data; the center of the screen is the center line position of the electronic device along the vertical direction; Calculate the target position according to the direction information; Receive a projection screen of fitness media resources sent by the mobile terminal to the electronic device, and control the display to display the projection screen at the target location.

8. The electronic device according to claim 7, wherein the processor performs the step of identifying the direction information of the user's head relative to the center of the screen according to the local video data, and is further configured to execute the computer instructions so that the display device executes: Get the screen center position of the electronic device; Extracting joint point information from the local video data to obtain a user's head coordinate set; The direction information of the user's head relative to the center position of the screen is determined according to the head coordinate set and the center position of the screen.

9. The electronic device of claim 8, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: Identify the user's posture and height during fitness; If the posture height is higher than the posture threshold, controlling the display to display the projection image at the top of the user interface; If the posture height is lower than the posture threshold, control the display to display the projection image at the bottom of the user interface.

10. A method for adjusting the position of a projection screen, applied to an electronic device, the electronic device comprising a display, a memory, a communication device, a user interface, an image acquisition interface and a processor, the method for adjusting the position of a projection screen comprising: In response to a screen projection connection request sent by a mobile terminal, starting the camera, and acquiring a user image captured by the camera; Identify the direction information of the user relative to the center of the screen through the user image; the center of the screen is the center line position of the electronic device along the vertical direction; Calculate the target position according to the direction information; Receive the projection screen sent by the mobile terminal to the electronic device, and control the display to display the projection screen at the target location.

Images