Human-machine interaction method and apparatus, and vehicle

Abstract

The present application provides a human-machine interaction method and apparatus, and a vehicle. The method can be applied to the field of intelligent cabins. The method comprises: controlling a first display screen in a cabin to display a first interface, the first interface comprising first display content; in response to a first input for the first display content, controlling the first display screen to display a second interface, wherein the second interface comprises a first interaction option and a second interaction option, the first interaction option indicates synchronization of the first display content to a second display screen in the cabin, and the second interaction option indicates transfer of the first display content to the second display screen; and in response to a second input of a user on the second interface, controlling the first display screen and / or the second display screen to display the first display content. The present application can be applied to intelligent vehicles or electric vehicles, and is conducive to avoiding tedious learning and memorization by a user before initiating content co-viewing or content transfer, thereby improving the human-machine interaction experience of the user.

Description

Human-computer interaction methods, devices and vehicles

[0001] This application claims priority to Chinese Patent Application No. 202411738442.6, filed on November 27, 2024, entitled "Human-Computer Interaction Method, Apparatus and Vehicle", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of smart cockpits, and more specifically, to a human-computer interaction method, device, and vehicle. Background Technology

[0003] As the cockpit evolves from a driving space to an entertainment-oriented environment, in-vehicle screens are no longer limited to instrument panel and center console screens; ceiling-mounted screens and projection screens have emerged as a result. Currently, when displaying content on a screen within the cockpit, different interaction paths are required for users to initiate content transfer (moving content from the current screen to another screen) or simultaneous content viewing (synchronizing content from the current screen to other screens for simultaneous viewing). These different interaction paths require users to undergo tedious learning and memorization, resulting in a poor user experience. Summary of the Invention

[0004] This application provides a human-computer interaction method, device, and vehicle that helps avoid the tedious learning and memorization process for users and helps improve the user's human-computer interaction experience.

[0005] In a first aspect, a human-computer interaction method is provided, the method comprising: controlling a first display screen in a cockpit to display a first interface, the first interface including first display content; responding to a first input to the first display content, controlling the first display screen to display a second interface, the second interface including a first interaction option and a second interaction option, the first interaction option instructing the first display content to be synchronized to the second display screen in the cockpit, the second interaction option instructing the first display content to be transferred to the second display screen; and responding to a second input by a user on the second interface, controlling the first display screen and / or the second display screen to display the first display content.

[0006] Based on the above technical solution, after receiving input for the first displayed content, a second interface including a first interactive option and a second interactive option can be displayed. This allows users to conveniently view or switch between content on the same interface, helping to avoid tedious learning and memorization processes and thus improving the user's human-computer interaction experience.

[0007] In some possible implementations, the first displayed content is video.

[0008] In some possible implementations, the first display content can be an image, a document, the desktop displayed on the first screen, or a display interface under a certain tab.

[0009] In some possible implementations, in response to a second input by the user on the second interface, controlling the first display screen and / or the second display screen to display the first display content includes: in response to the user's input on the second interface for the first interactive option, controlling the first display screen and the second display screen to display the first display content; or, in response to the user's input on the second interface for the second interactive option, controlling the first display screen to display the second display content and controlling the second display screen to display the first display content.

[0010] The first interactive option can be used to enable the simultaneous viewing of content, that is, to display the first content on the first display screen and the second display screen; the second interactive option can be used to enable the content transfer function, that is, to transfer the first content displayed on the first display screen to the second display screen.

[0011] The first interactive option above indicates that the first display content is synchronized to the second display screen in the cockpit. It can also be understood that the first interactive option indicates that the first display content is shared to the second display screen in the cockpit, or that the first interactive option indicates that the first display content is shared to the second display screen in the cockpit.

[0012] In some possible implementations, the first input can be a three-finger press input for the first displayed content.

[0013] In some possible implementations, the first input can be an air gesture (e.g., switching from spreading fingers to pinching fingers).

[0014] In some possible implementations, the second display content can be an always-on display (AOD) interface, the parent display interface of the first display content, the desktop, or the display interface of a certain tab.

[0015] In conjunction with the first aspect, in some implementations of the first aspect, before controlling the display device to display the second interface, the method further includes: in response to the first input, controlling the first display screen to display a floating window; and determining the second display screen from a plurality of display screens in the cockpit based on a first movement direction controlled by the user of the floating window.

[0016] Based on the above technical solution, in response to the first input, a floating window can be displayed on the first display screen. By determining the movement direction of the first floating window, the second display screen that the user wants to initiate content viewing or content flow can be determined. This eliminates the need for the user to select a display screen from multiple screens in the cockpit before initiating content viewing or content flow, which helps to avoid cumbersome operations for the user and also helps to improve the intelligence level of the cockpit.

[0017] In some possible implementations, the floating window includes the first displayed content.

[0018] In some possible implementations, determining the second display screen from multiple displays in the cockpit based on a first movement direction controlled by the user of the floating window includes: determining the second display screen from multiple displays in the cockpit based on the first movement direction within a first preset time period, wherein the first preset time period is a preset time period from the detection of the first input, or the first preset time period may be a preset time period from the detection of the user-controlled movement of the floating window.

[0019] In some possible implementations, taking the first input as a three-finger press as an example, the center point of the floating window can be the center point of the area where the three fingers are pressed.

[0020] In some possible implementations, taking the first input as an air gesture (e.g., switching from finger open to finger pinch), the central area of ​​the floating window is the area that the user's finger points to after pinching.

[0021] In conjunction with the first aspect, in some implementations of the first aspect, the second display screen is the display screen of the first mobile terminal. Before determining the second display screen from among multiple display screens in the cockpit according to the first movement direction of the floating window controlled by the user, the method further includes: obtaining the first position of the first mobile terminal in the cockpit; wherein, determining the second display screen from among multiple display screens in the cockpit according to the first movement direction of the floating window controlled by the user includes: determining the display screen of the first mobile terminal as the second display screen when the first sliding direction matches the first position.

[0022] Based on the above technical solution, the second display screen is not limited to the vehicle's display screen, but can also be extended to the display screen of the user's mobile terminal (e.g., mobile phone, tablet, portable laptop, etc.), which allows the user to choose a wider range of display screens. At the same time, when the first sliding direction matches the position of the mobile terminal in the cabin, the display screen of the first mobile terminal can be determined as the second display screen. In this way, the user does not need to manually select the display screen of the mobile terminal, but the floating window intelligently identifies the display screen where the user wants the content to flow or the content to be seen, which helps to avoid cumbersome operations for the user and also helps to improve the intelligence level of the cabin.

[0023] In conjunction with the first aspect, in some implementations of the first aspect, before controlling the first display screen and / or the second display screen to display the first display content in response to a second input from the user on the second interface, the method further includes: in response to obtaining input from the user to control the floating window to move toward a target interactive option, controlling the target interactive option to move toward the floating window, wherein the target interactive option is the first interactive option or the second interactive option.

[0024] Based on the above technical solution, when input indicating that the floating window is moving towards the target interactive option is received, the target interactive option can be controlled to move towards the floating window. This improves the responsiveness of the interactive option and ensures that its movement better matches the user's intent.

[0025] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: in response to receiving input from the user to control the floating window to move toward the target interactive option, controlling the display area corresponding to the target interactive option to increase.

[0026] Based on the above technical solution, when input indicating that the floating window is moving towards the target interactive option is received, the display area of ​​the target interactive option can be enlarged. In this way, by seeing the enlarged display area of ​​the interactive option, the user can confirm whether their operation met expectations.

[0027] In conjunction with the first aspect, in some implementations of the first aspect, controlling the first display screen to display a second interface in response to a first input to the first display content includes: controlling the first display screen to display a third interface in response to the first input, the third interface including a floating window and a display list, the display list including identification information of each of the plurality of displays in the cockpit; and controlling the first display screen to display the second interface in response to the user controlling the floating window to move to at least partially overlap with the display area where the identification information of the second display screen is located.

[0028] Based on the above technical solution, a third interface can be displayed before the second interface. The third interface may include a floating window and a display list, which may include identification information for each of the multiple displays. This allows users to easily select the second display from among the multiple displays on the third interface.

[0029] In some possible implementations, controlling the first display screen to display a second interface in response to a first input to the first display content includes: controlling the first display screen to display a third interface in response to the first input, the third interface including a display list including identification information of each of a plurality of displays in the cockpit; and controlling the first display screen to display the second interface in response to a user selecting the identification information of the second display screen from the display list.

[0030] In conjunction with the first aspect, in some implementations of the first aspect, the plurality of displays does not include the display of the second mobile terminal, and the method further includes: when the second mobile terminal is detected to be located in the cockpit, updating the display list, wherein the updated display list includes identification information of the plurality of displays and the display of the second mobile terminal.

[0031] Based on the above technical solution, when the second mobile terminal is not in the cockpit, the display list may not include the identification information of the second mobile terminal's display screen. When the second mobile terminal is in the cockpit, the display list can be updated, and the updated display list can include the identification information of the multiple display screens and the second mobile terminal's display screen. Thus, by detecting whether the mobile terminal is in the cockpit, the display list can be dynamically updated.

[0032] In conjunction with the first aspect, in certain implementations of the first aspect, in response to a second input by the user on the second interface, controlling the first display screen and / or the second display screen to display the first display content includes: in response to the user controlling the floating window to move to at least partially overlap with the first display area where the first interactive option is located and detecting that the user has switched from touching the first display screen to not touching the first display screen, controlling the first display screen and the second display screen to display the first display content; or, in response to the user controlling the floating window to move to at least partially overlap with the second display area where the second interactive option is located and detecting that the user has switched from touching the first display screen to not touching the first display screen, controlling the first display screen to display the second display content and controlling the second display screen to display the first display content; or, before the floating window at least partially overlaps with the first display area or the second display area, in response to detecting that the user has not touched the first display screen, controlling the first display screen to display the first display content.

[0033] In conjunction with the first aspect, in certain implementations of the first aspect, in response to a user controlling the floating window to move to at least partially overlap with the first display area where the first interactive option is located and detecting that the user switches from touching the first display screen to not touching the first display screen, controlling the first display screen and the second display screen to display the first display content includes: in response to a user controlling the floating window to move to at least partially overlap with the first display area where the first interactive option is located, controlling the first display screen to display a first preview interface where the first display screen and the second display screen display the first display content; and in the first preview interface, detecting that the user switches from touching the first display screen to not touching the first display screen, controlling the first display screen and the second display screen to display the first display content.

[0034] Based on the above technical solution, before finally initiating simultaneous content viewing, a preview interface for simultaneous content viewing on both the first and second displays can be shown on the first display screen. This allows users to determine whether the preview matches their intentions, helping to reduce user errors and thus improving the user's human-computer interaction experience.

[0035] In conjunction with the first aspect, in certain implementations of the first aspect, in response to a user controlling the floating window to move to at least partially overlap with the second display area where the second interactive option is located and detecting that the user switches from touching the first display screen to not touching the first display screen, controlling the first display screen to display second display content and controlling the second display screen to display the first display content includes: in response to a user controlling the floating window to move to at least partially overlap with the first display area where the second interactive option is located, controlling the first display screen to display the second display content and the second display screen to display a second preview interface of the first display content; and detecting that the user switches from touching the first display screen to not touching the first display screen in the second preview interface, controlling the first display screen to display the second display and controlling the second display screen to display the first display content.

[0036] Based on the above technical solution, before finally initiating the content flow, a preview interface for the content flow execution on both the first and second displays can be shown on the first display screen. This allows users to determine whether the content matches their intentions through the preview interface, helping to reduce user errors and thus improving the user's human-computer interaction experience.

[0037] In conjunction with the first aspect, in some implementations of the first aspect, the second interface further includes a floating window, wherein controlling the first display screen to display the second interface in response to a first input to the first display content includes: controlling the first display screen to display a third interface in response to the first input, the third interface including a virtual display screen in the cockpit, the virtual display screen including display areas of multiple displays in the cockpit; and controlling the first display screen to display the second interface in response to the user dragging the floating window to at least partially overlap with the display area of ​​the second display screen in the virtual display screen, the second interface further including the virtual display screen displayed by the first display screen and the second display screen according to preset interaction options, the preset interaction options being the first interaction option or the second interaction option.

[0038] Based on the above technical solution, before finally initiating content transfer or simultaneous viewing, a virtual display screen (i.e., a preview interface displayed according to preset interaction options) can be shown on the first display screen and the second display screen. This allows users to determine whether the virtual display screen matches their intentions, helping to reduce user errors and thus improving the user's human-computer interaction experience.

[0039] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: in response to a user's input of selecting another interactive option, controlling the first display screen to display a virtual display screen shown by the first display screen and the second display screen according to the other interactive option, wherein the other interactive option is an interactive option other than the preset interactive option among the first interactive option and the second interactive option.

[0040] Based on the above technical solution, users can switch interactive options on the virtual display screen, allowing the first display screen to show the virtual display screen displayed on both the first and second display screens according to the other interactive option. This allows users to determine whether the virtual display screen matches their intentions by switching interactive options, helping to reduce user errors and thus improving the user's human-computer interaction experience.

[0041] Secondly, this application provides a human-computer interaction device, comprising: a control unit for controlling a first display screen in a cockpit to display a first interface, the first interface including first display content; an acquisition unit for acquiring a first input to the first display content; the control unit further for controlling the first display screen to display a second interface in response to the acquisition unit acquiring the first input, the second interface including a first interaction option and a second interaction option, the first interaction option indicating that the first display content is synchronized to the second display screen in the cockpit, and the second interaction option indicating that the first display content is transferred to the second display screen; the acquisition unit further for acquiring a second input from a user on the second interface; and the control unit further for controlling the first display screen and / or the second display screen to display the first display content in response to the acquisition unit acquiring the second input.

[0042] In conjunction with the second aspect, in some implementations of the second aspect, the device further includes a determining unit, the control unit being configured to, before controlling the display device to display the second interface, control the first display screen to display a floating window in response to the acquiring unit acquiring a first input from the user on the first display content; the determining unit being configured to determine the second display screen from among a plurality of display screens in the cockpit based on a first movement direction controlled by the user controlling the floating window.

[0043] In conjunction with the second aspect, in some implementations of the second aspect, the second display screen is the display screen of the first mobile terminal, and the acquisition unit is further configured to acquire the first position of the first mobile terminal in the cockpit before the determining unit determines the second display screen from the plurality of display screens in the cockpit; the determining unit is specifically configured to: determine the display screen of the first mobile terminal as the second display screen when the first sliding direction matches the first position.

[0044] In conjunction with the second aspect, in some implementations of the second aspect, the control unit is further configured to, in response to the acquisition unit acquiring user input that the floating window is moving toward a target interactive option, control the target interactive option to move toward the floating window, wherein the target interactive option is the first interactive option or the second interactive option.

[0045] In conjunction with the second aspect, in some implementations of the second aspect, the control unit is further configured to, in response to the acquisition unit acquiring input from the user controlling the floating window to move toward the target interactive option, control the display area corresponding to the target interactive option to increase.

[0046] In conjunction with the second aspect, in some implementations of the second aspect, the control unit is specifically configured to: in response to the acquisition unit acquiring the first input, control the first display screen to display a third interface, the third interface including a floating window and a display list, the display list including identification information of each of the multiple display screens in the cockpit; and in response to the acquisition unit acquiring that the user controls the floating window to move to at least partially overlap with the display area where the identification information of the second display screen is located, control the first display screen to display the second interface.

[0047] In conjunction with the second aspect, in some implementations of the second aspect, the plurality of displays does not include the display of the second mobile terminal. The device further includes a detection unit and an information update unit. The information update unit is used to update the display list when the detection unit detects that the second mobile terminal is located in the cockpit. The updated display list includes the identification information of the plurality of displays and the display of the second mobile terminal.

[0048] In conjunction with the second aspect, in some implementations of the second aspect, the control unit is specifically configured to: control the first display screen and the second display screen to display the first display content in response to the user controlling the floating window to move to at least partially overlap with the first display area where the first interactive option is located and the user switching from touching the first display screen to not touching the first display screen; or, control the first display screen to display the second display content and control the second display screen to display the first display content in response to the user controlling the floating window to move to at least partially overlap with the second display area where the second interactive option is located and the user switching from touching the first display screen to not touching the first display screen; or, before the floating window at least partially overlaps with the first display area or the second display area, control the first display screen to display the first display content in response to the acquisition unit obtaining that the user has not touched the first display screen.

[0049] In conjunction with the second aspect, in some implementations of the second aspect, the control unit is specifically configured to: control the first display screen to display a first preview interface of the first display content when the user controls the floating window to move to at least partially overlap with the first display area where the first interactive option is located; and control the first display screen and the second display screen to display the first display content when the user switches from touching the first display screen to not touching the first display screen in the first preview interface.

[0050] In conjunction with the second aspect, in some implementations of the second aspect, the control unit is specifically configured to: in response to a user controlling the floating window to move to at least partially overlap with the first display area where the second interactive option is located, control the first display screen to display a second preview interface where the first display screen displays the second display content and the second display screen displays the first display content; in the second preview interface, when the user switches from touching the first display screen to not touching the first display screen, control the first display screen to display the second display and control the second display screen to display the first display content.

[0051] In conjunction with the second aspect, in some implementations of the second aspect, the second interface further includes a floating window, wherein the control unit is specifically configured to: in response to the acquisition unit acquiring the first input, control the first display screen to display a third interface, the third interface including a virtual display screen in the cockpit, the virtual display screen including display areas of multiple displays in the cockpit; in response to the user dragging the floating window to at least partially overlap with the display area of ​​the second display screen in the virtual display screen, control the first display screen to display the second interface, the second interface further including the virtual display screen displayed by the first display screen and the second display screen according to preset interaction options, the preset interaction options being the first interaction option or the second interaction option.

[0052] In conjunction with the second aspect, in some implementations of the second aspect, the control unit is further configured to, in response to receiving input from the user selecting another interactive option, control the first display screen to display a virtual display screen that is displayed on the first display screen and the second display screen according to the other interactive option, wherein the other interactive option is an interactive option other than the preset interactive option among the first interactive option and the second interactive option.

[0053] In conjunction with the second aspect, in some implementations of the second aspect, the control unit is specifically configured to: control the first display screen and the second display screen to display the first display content in response to user input on the second interface for the first interactive option; or, control the first display screen to display the second display content and control the second display screen to display the first display content in response to user input on the second interface for the second interactive option.

[0054] In conjunction with the second aspect, in some implementations of the second aspect, the first displayed content is a video.

[0055] Thirdly, this application provides a human-computer interaction device, which includes a processor and a memory, wherein the memory is used to store instructions, and the processor executes the instructions stored in the memory to cause the device to perform any of the possible methods in the first aspect.

[0056] Fourthly, this application provides a human-computer interaction system, which includes a computing platform and multiple displays, wherein the computing platform includes any of the possible devices in the second or third aspect.

[0057] Fifthly, this application provides a vehicle that includes any of the possible devices of the second or third aspect, or includes the system described in the fourth aspect.

[0058] In a sixth aspect, this application provides a computer program product comprising: computer program code, which, when executed on a computer, causes the computer to perform any of the possible methods described in the first aspect above.

[0059] It should be noted that the above-mentioned computer program code can be stored in whole or in part on the first storage medium, wherein the first storage medium can be packaged together with the processor or packaged separately from the processor. This application embodiment does not specifically limit this.

[0060] In a seventh aspect, this application provides a computer-readable medium storing program code that, when run on a computer, causes the computer to perform any of the possible methods described in the first aspect above.

[0061] Eighthly, this application provides a chip system including a processor for calling a computer program or computer instructions stored in a memory to cause the processor to perform any of the possible methods in the first aspect above.

[0062] In conjunction with the eighth aspect, in one possible implementation, the processor is coupled to the memory via an interface.

[0063] In conjunction with the eighth aspect, in one possible implementation, the chip system also includes a memory in which computer programs or computer instructions are stored.

[0064] Ninthly, this application provides a chip system including circuitry for performing any of the possible methods described in the first aspect above. Attached Figure Description

[0065] Figure 1 is a functional block diagram of the vehicle provided in an embodiment of this application.

[0066] Figure 2 is a schematic diagram of a vehicle cabin scenario provided in an embodiment of this application.

[0067] Figures 3A-3H are a set of GUIs provided in the embodiments of this application.

[0068] Figures 4A-4C show another set of GUIs provided in the embodiments of this application.

[0069] Figures 5A-5B show another set of GUIs provided in the embodiments of this application.

[0070] Figures 6A-6C show another set of GUIs provided in the embodiments of this application.

[0071] Figures 7A-7F show another set of GUIs provided in the embodiments of this application.

[0072] Figures 8A-8E show another set of GUIs provided in the embodiments of this application.

[0073] Figures 9A-9G show another set of GUIs provided in the embodiments of this application.

[0074] Figures 10A-10G are another set of GUIs provided in the embodiments of this application.

[0075] Figures 11A-11C are another set of GUIs provided in the embodiments of this application.

[0076] Figure 12 is a schematic diagram of the structure of the display system provided in an embodiment of this application.

[0077] Figure 13 is a schematic diagram of the structure of a display system with one chip and multiple screens provided in an embodiment of this application.

[0078] Figure 14 is a schematic diagram of the structure of the display system provided in an embodiment of this application.

[0079] Figure 15 is a schematic flowchart of the human-computer interaction method provided in an embodiment of this application.

[0080] Figure 16 is a schematic diagram of the location of the identification device provided in an embodiment of this application.

[0081] Figure 17 is a schematic block diagram of the human-computer interaction device provided in an embodiment of this application. Detailed Implementation

[0082] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0083] Figure 1 is a functional block diagram of a vehicle provided in an embodiment of this application. As shown in Figure 1, the vehicle 100 may include a display device 130 and a computing platform 150. The display device 130 in the cabin is mainly divided into two categories: the first is an in-vehicle display screen; the second is a projection display screen, such as a head-up display (HUD). An in-vehicle display screen is a physical display screen and an important component of the in-vehicle infotainment system. Multiple displays can be installed in the cabin, such as a digital instrument cluster display screen, a central control screen, a display screen in front of the front passenger (also known as the front row passenger), a display screen in front of the left rear passenger, and a display screen in front of the right rear passenger; even the car window can be used as a display screen. A head-up display, also known as a head-up display system, is mainly used to display driving information such as speed and navigation on a display device (e.g., the windshield) in front of the driver. This reduces the driver's eye movement time, avoids pupil changes caused by eye movement, and improves driving safety and comfort. HUDs include, for example, combiner-HUD (C-HUD) systems, windshield-HUD (W-HUD) systems, and augmented reality HUD (AR-HUD) systems.

[0084] Some or all of the functions of vehicle 100 can be controlled by computing platform 150. Computing platform 150 may include processors 151 to 15n. A processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction read and execute capabilities, such as a central processing unit (CPU), microprocessor, graphics processing unit (GPU) (which can be understood as a type of microprocessor), or digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a field-programmable gate array (FPGA). In reconfigurable hardware circuits, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement some or all of the functions of the aforementioned units. Furthermore, the processor can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), tensor processing unit (TPU), deep learning processing unit (DPU), etc. In addition, the computing platform 150 may also include a memory for storing instructions. Some or all of the processors 151 to 15n can call the instructions in the memory to implement the corresponding functions.

[0085] Optionally, the structure of the vehicle 100 described above is merely illustrative. In actual applications, various components of the vehicle 100 may be added or removed as needed.

[0086] Figure 2 is a schematic diagram of a vehicle cockpit scenario provided in an embodiment of this application. The smart cockpit is equipped with one or more in-vehicle displays (or in-vehicle screens), including but not limited to display screen 201 (or central control screen), display screen 202 (or passenger entertainment screen), display screen 203 (or driver's headrest rear screen), display screen 204 (or passenger headrest rear screen), display screen 205 (or second-row entertainment screen) mounted on the cockpit ceiling, and an instrument panel. Further, displays 201 to 205 can display a graphical user interface (GUI), which may include icons for one or more applications and / or one or more cards. Exemplarily, the display device 130 shown in Figure 1 can be one or more of displays 201 to 205. In some possible implementations, display screen 201 can also be a long, continuous screen extending to the passenger area. Additionally, display screen 205 can also be a projection screen associated with a projector, which can be associated with a desktop launcher to manage applications projected onto the projection screen.

[0087] As shown in Figure 2, one or more cameras can also be installed in the cockpit to capture images inside or outside the cockpit. These could include cameras from a driver monitor system (DMS), a cabin monitor system (CMS), or a dashcam. The cameras used to capture images inside and outside the cockpit can be the same camera or different cameras. In addition, one or more pressure sensors and acoustic sensors are installed in the cockpit to monitor the presence and location of users.

[0088] It should be understood that the control display method in the following embodiments is illustrated using a 5-seat vehicle as shown in Figure 2 as an example, and the embodiments of this application are not limited to this. For example, for a 7-seat sport / suburban utility vehicle (SUV), the cabin may include a central control screen, a passenger entertainment screen, a screen behind the driver's headrest, a screen behind the passenger's headrest, entertainment screens in the left area of ​​the third row, and entertainment screens in the right area of ​​the third row. As another example, for a bus, the cabin may include front and rear entertainment screens; or, the cabin may include a display screen in the driver's area and an entertainment screen in the passenger area. Furthermore, the following embodiments use a left-hand drive vehicle (i.e., the driver is located on the left side of the vehicle) as an example for illustration; in actual implementation, the vehicle may also be a right-hand drive vehicle (i.e., the driver is located on the right side of the vehicle).

[0089] Figures 3A-3H illustrate a set of GUIs provided in embodiments of this application.

[0090] As shown in Figure 3A, the vehicle displays a video playback interface on screen 201, on which video frame 301 is playing. In response to a user's three-finger press input on the video frame 301, the vehicle can display a GUI as shown in Figure 3B on screen 201.

[0091] As shown in Figure 3B, in response to the three-finger press input, the vehicle can display a floating window 302, interaction options 303, and interaction options 304 on the display screen 201. For example, interaction option 303 can be a content viewing control, with the text explanation below it stating "Synchronize to the passenger screen, displaying simultaneously on the central control screen and the passenger screen." For example, interaction option 304 can be a content flow control, with the text explanation below it stating "Flow to the passenger screen, flowing from the central control screen to the passenger screen."

[0092] For example, a frame of image can be displayed in the floating window 302. This frame of image can be a frame of image displayed on the video playback interface when the user presses three fingers on the display screen 201.

[0093] For example, the video playback interface can continue to be displayed in the floating window 302.

[0094] For example, as shown in Figure 3B, the GUI also includes a prompt message below the display screen 201: "Slide the floating window with three fingers to the edge of the screen area, and release after sensing."

[0095] Optionally, before displaying the GUI as shown in Figure 3B on display screen 201, the vehicle can first determine the second display screen from among multiple displays in the cabin based on the movement direction 1 of the three-finger press. For example, when a user's three-finger press is detected on the video playback interface shown in Figure 3A and the movement direction of the three-finger press is movement direction 1, it can be determined that the user wishes to stream or synchronize the video to display screen 202 and the GUI as shown in Figure 3B can be displayed.

[0096] For example, the interactive option 303 may be located in the upper right corner of the display screen 201, and the interactive option 304 may be located in the lower right corner of the display screen 201.

[0097] Optionally, in response to the three-finger press input and when it is detected that the three fingers are not moving on the display screen, the vehicle can display the floating window 302 on the display screen 201 without displaying the interaction options 303 and 304. When it is detected that the user's three-finger press on the floating window 302 and moves it along the movement direction 1, the interaction options 303 and 304 can then be displayed on the display screen 201.

[0098] Optionally, the time period from detecting a user's three-finger press on the floating window 302 to detecting that the floating window 302 has moved a first preset distance can be used as the time period for determining the display screen the user wishes to select. The vehicle can determine the display screen 202 from multiple displays in the cabin based on the direction of the first preset distance.

[0099] As shown in Figure 3C, when it is determined that the user wants to stream or synchronize the video feed to the passenger-side screen, the vehicle can continue to detect the direction of movement of the user's three-finger press on the floating window 302. For example, when it is detected that the user's three-finger press on the floating window 302 along the movement direction 2, the vehicle can control the interactive option 303 to move toward the floating window 302 and control the display area of ​​the interactive option 303 to increase.

[0100] Optionally, as the distance between the interactive option 303 and the floating window 302 gradually decreases, the display area of ​​the interactive option 303 can gradually increase.

[0101] As shown in Figure 3D, when the display areas of the floating window 302 and the interactive option 303 are detected to overlap at least partially, a preview interface for simultaneous viewing can be displayed on the display screen 201. This preview interface includes display windows 305 and 306. Display window 305 can indicate the display screen 201, and display window 306 can indicate the display screen 202. When the user switches from touching the display screen 201 to not touching it (or when the user's three fingers leave the display screen 201) in this preview interface, the vehicle can control the display screen 201 to display the GUI shown in Figure 3E and control the display screen 202 to display the GUI shown in Figure 3F.

[0102] As shown in Figures 3E and 3F, the vehicle can play the video 301 through displays 201 and 202.

[0103] Optionally, before displaying the video feed 301 on the display screen 202, the vehicle can also display a prompt message on the display screen 202. For example, the prompt message could be "A synchronization request for the content displayed on the central control screen has been detected. Do you agree to the request?", an agree control, and a refuse control. In response to detecting user input by clicking the agree control, the vehicle can play the video feed 301 on both the display screens 201 and 202. In response to detecting user input by clicking the agree control, the vehicle can continue playing the video feed 301 on the display screen 201 while controlling the display screen 202 not to display the video feed 301.

[0104] For example, Figure 3G shows a schematic diagram of the display area division on display screen 201. The vehicle can determine the display screen to which the user wants content to be synchronized or flowed based on this display area division method and the direction of movement of the user's three fingers pressing the screen. Assume that point O is the center point of the area touched by the user's three fingers on display screen 201. After detecting that the user's three fingers have moved the floating window 302 a first preset distance (to point O'), the display screen 202 can be determined from among multiple displays in the cabin based on the direction 1 of the user's three-finger press on the floating window 302 and the aforementioned display area division method.

[0105] For example, the first preset distance can be 2cm.

[0106] The area division shown in Figure 3G is merely illustrative. This division may include more or fewer displays. For example, this division may exclude the instrument panel. Alternatively, it may include the display of a mobile terminal (e.g., a mobile phone) within the cabin. Furthermore, when a mobile phone is detected to be in the cabin and connected to the vehicle, an area corresponding to the mobile phone's display can be added to this division. For instance, if the mobile phone is located in the second row, right-side area of ​​the cabin, then the area corresponding to the mobile phone's display and display 204 can be the same area.

[0107] For example, Figure 3H shows another schematic diagram of the display area division on display screen 201. After identifying display screen 202 from among multiple displays in the cockpit, the vehicle can continue to detect the direction of movement of the user's three-finger press on the floating window 302. For example, when the direction of movement 2 of the three-finger press on the floating window 302 is towards area 1, it can be determined that the user wishes to move the floating window 302 to overlap with the interaction option 303.

[0108] In one embodiment, as shown in FIG3D, when the display areas where the floating window 302 and the interactive option 303 are located are detected to overlap at least partially and the user keeps pressing the display screen 201 with three fingers, in addition to displaying a preview interface of the content to be viewed on the display screen 201, the display screen 202 can also be controlled to display a frame image. This frame image can be a frame image of the video screen 301 obtained when the user presses the display screen 201 with three fingers.

[0109] In one embodiment, the vehicle displays a video playback interface on display screen 201, on which video frame 301 is playing. In response to detecting a user's air gesture (e.g., switching from open to pinch), the vehicle can display a GUI as shown in FIG. 3B on display screen 201. The direction of movement of the floating window 302 can then be determined by the direction of movement after the finger pinches. As shown in FIG. 3D, when another air gesture (e.g., switching from pinch to open) is detected in the preview interface, the vehicle can control display screen 201 to display the GUI as shown in FIG. 3E and control display screen 202 to display the GUI as shown in FIG. 3F.

[0110] Figures 4A-4C illustrate another set of GUIs provided in embodiments of this application.

[0111] As shown in Figure 4A, when the movement direction 2 is towards region 2, it can be determined that the user wishes to move the floating window 302 to the interactive option 304. When it is detected that the display areas where the floating window 302 and the interactive option 304 are located at least partially overlap, a preview interface of the content flow can be displayed on the display screen 201. This preview interface includes a display window 307 and a display window 308. The display window 307 can indicate the display screen 201, and the display window 308 can indicate the display screen 202.

[0112] When the preview interface detects that the user has switched from touching the display screen 201 to not touching the display screen 201 (or the user's three fingers have left the display screen 201), the vehicle can control the display screen 201 to display the GUI as shown in Figure 4B and control the display screen 202 to display the GUI as shown in Figure 4C.

[0113] As shown in Figures 4B and 4C, the vehicle can display an off-screen display interface on display screen 201 and display the video image 301 on display screen 202.

[0114] Based on the above technical solution, after receiving input for the video screen 301, an interface including interactive options 303 and 304 can be displayed. This allows users to conveniently view or switch between content on the same interface, helping to avoid tedious learning and memorization processes and thus improving the user's human-computer interaction experience. Figures 5A-5B illustrate another set of GUIs provided in embodiments of this application.

[0115] As shown in Figures 5A and 5B, when the user presses three fingers on the video screen 301, a floating window 502 can be displayed.

[0116] After detecting that the user has moved the floating window 502 by pressing it with three fingers a first preset distance, the display screen 204 can be determined from among the multiple displays in the cabin based on the movement direction 3 (moving downward to the right) of the floating window 502 and the display area division method shown in Figure 3G above. At this time, the vehicle can display interactive controls 503 and 504. For example, the interactive option 503 can be a content viewing control, which includes the text explanation "synchronized to the second-row right-side entertainment screen, displayed simultaneously on the central control screen and the second-row right-side entertainment screen". For example, the interactive option 504 can be a content flow control, which includes the text explanation "flows to the second-row right-side entertainment screen, from the central control screen to the second-row right-side entertainment screen".

[0117] In this embodiment of the application, in response to the user's three-finger pressing input, a floating window 502 can be displayed on the display screen 201. By judging the movement direction of the floating window 502, the display screen to which the user wants to view content simultaneously or to which the content flows can be determined. This eliminates the need for the user to select a display screen from multiple screens in the cockpit before initiating content viewing or content flow, which helps to avoid cumbersome operations for the user and also helps to improve the intelligence level of the cockpit.

[0118] Figures 6A-6C illustrate another set of GUIs provided in embodiments of this application.

[0119] As shown in Figure 6A, the vehicle displays a video playback interface on screen 201, on which video frame 601 is playing. In response to a user's three-finger press input on the video frame 601, the vehicle can display a GUI as shown in Figure 6B on screen 201.

[0120] As shown in Figure 6B, in response to the input of the three-finger press, the vehicle can display a floating window 602 and a display list 603 on the display screen 201. The display list 603 includes identification information of multiple displays in the cabin, such as digital light processing (DLP) headlights, HUD, passenger screen, second-row left-side entertainment screen, second-row right-side entertainment screen and screen.

[0121] Optionally, the display list 603 can be displayed in the form of a ring, and the display area where the ring is located can be divided into multiple areas, each area corresponding to the area where the identification information of a display screen is located.

[0122] When the system detects that the user's three-finger press on the floating window 602 has moved to at least partially overlap with the area where the identification information on the passenger screen is located, the GUI shown in Figure 6C can be displayed on the display screen 202. As shown in Figure 6C, in response to the floating window 602 at least partially overlapping with the area where the identification information on the passenger screen is located, the vehicle can display interactive controls 604 and 605 on the display screen 201. For example, the interactive option 603 can be a content viewing control, which includes the text explanation "Synchronize to the passenger screen, display simultaneously on the central control screen and the passenger screen". For example, the interactive option 605 can be a content flow control, which includes the text explanation "Flow to the passenger screen, flow from the central control screen to the passenger screen".

[0123] For example, the interactive options 604 and 605 may be located in another ring outside the ring containing the display list 603.

[0124] Optionally, when the display areas where the floating window 602 and the interactive option 604 are located are detected to overlap at least partially, the vehicle can display the video image 601 through the display screen 201 and the display screen 202.

[0125] Optionally, when the display areas where the floating window 602 and the interactive option 605 are located are detected to overlap at least partially, the vehicle can display the off-screen display interface on the display screen 201 and display the video image 601 on the display screen 202.

[0126] Optionally, when it is detected that the display areas of the floating window 602 and the interactive option 604 at least partially overlap, a preview interface for viewing the same content can be displayed on the display screen 201. When it is detected that the user switches from touching the display screen 201 to not touching the display screen 201 (or, the user's three fingers leave the display screen 201) in the preview interface, the vehicle can display the video image 601 on both the display screen 201 and the display screen 202.

[0127] Optionally, when it is detected that the display areas where the floating window 602 and the interactive option 605 are located at least partially overlap, a preview interface of the content flow can be displayed on the display screen 201. When it is detected that the user switches from touching the display screen 201 to not touching the display screen 201 (or, the user's three fingers leave the display screen 201) in the preview interface, the vehicle can display a screen-off display interface on the display screen 201 and display the video image 601 on the display screen 202.

[0128] Figures 7A-7F illustrate another set of GUIs provided in embodiments of this application.

[0129] As shown in Figure 7A, the vehicle displays a video playback interface on display screen 201, on which video frame 701 is playing. In response to a user's three-finger press input on video frame 701, the vehicle can display a GUI as shown in Figure 7B on display screen 201.

[0130] As shown in Figure 7B, in response to the three-finger press input, the vehicle can display a virtual display screen in the cockpit, as well as a floating window 702, an interactive control 704, and a confirmation control 705 on the display screen 201. For example, the interactive control 704 includes a simultaneous viewing option and a flow option. The virtual display screen in the cockpit includes information from multiple displays. For example, the display area of ​​the display screen 204 in the virtual display screen is area 703.

[0131] Optionally, the vehicle may not display the interactive control 704 and the confirmation control 705 until the floating window 702 overlaps with the display area of ​​any display screen. When the floating window 702 at least partially overlaps with the area 703, the vehicle may display the preview interface 1, the interactive control 704, and the confirmation control 705 through preset interactive options (e.g., viewing options).

[0132] Optionally, before the floating window 702 at least partially overlaps with the display area of ​​any display screen, the vehicle may also display the interactive control 704 and the confirmation control 705. At this time, it can be assumed that the user has selected the "view content together" option in the interactive control 704. When the floating window 702 at least partially overlaps with the area 703, the vehicle can display the preview interface 1, the interactive control 704, and the confirmation control 705 through the "view content together" interactive option.

[0133] As shown in Figure 7C, in response to the floating window 702 at least partially overlapping with the area 703, the vehicle can display another virtual display screen through the display screen 201. This other virtual display screen includes the preview interface 1 of the video screen 701 displayed on the display screen 201 and the display screen 204 in the cockpit, the interactive control 704, and the confirmation control 705.

[0134] As shown in Figure 7D, when the user selects a flow option in the interactive control 704, another virtual display screen can be displayed on the display screen 201. This other virtual display screen includes the cockpit display screen 201 displaying a screen-off interface and the display screen 204 displaying a preview interface 2 of the video screen 701. When the user clicks the confirm control 705, the vehicle can control the display screen 201 to display the GUI shown in Figure 7E and control the display screen 204 to display the GUI shown in Figure 7F.

[0135] As shown in Figures 7E and 7F, the vehicle can display an off-screen display interface on display screen 201 and display the video image 701 on display screen 204.

[0136] Figures 8A-8E illustrate another set of GUIs provided in embodiments of this application.

[0137] As shown in Figure 8A, the vehicle displays a video playback interface on the display screen 201, on which video frame 801 is being played.

[0138] As shown in Figure 8B, user C in the right area of ​​the second row is using a mobile phone, which can display the phone's desktop.

[0139] Alternatively, the phone and the vehicle can establish a connection via wireless communication (e.g., Wi-Fi, Bluetooth, or Starlight). The vehicle can then determine the location of the phone within the cabin.

[0140] As shown in Figure 8C, when the input of a user pressing three fingers on the video screen 801 is detected, the vehicle can display a floating window 802.

[0141] When the vehicle detects that a user presses three fingers on the floating window 802 and moves it a first preset distance, with the direction 4 of the movement pointing towards the right side of the second row, the vehicle can determine that the user wishes to synchronize or transfer the video screen 801 to the display screen in the right side of the second row. Since the right side of the second row includes the display screen 204 and the mobile phone screen, the vehicle can display the floating window 802 and interactive options 803-806 on the display screen 201. For example, interactive option 803 can be a content viewing control, with the text explanation "synchronize to the entertainment screen on the right side of the second row" below it. For example, interactive option 804 can be a content transfer control, with the text explanation "transfer to the entertainment screen on the right side of the second row" below it. For example, interactive option 805 can be a content viewing control, with the text explanation "synchronize to the mobile phone" below it. For example, interactive option 806 can be a content transfer control, with the text explanation "transfer to the mobile phone" below it.

[0142] As shown in Figures 8D and 8E, when the vehicle detects that the user presses the floating window 302 with three fingers and continues to move along the moving direction 5 and the moving direction 5 points to the interactive option 806, the vehicle can control the interactive option 806 to move toward the floating window 802 and control the display area where the interactive option 806 is located to increase.

[0143] Optionally, as the distance between the interactive option 806 and the floating window 802 gradually decreases, the display area of ​​the interactive option 806 can gradually increase.

[0144] Figures 9A-9G illustrate another set of GUIs provided in embodiments of this application.

[0145] As shown in Figure 9A, the user in the right-hand area of ​​the second row is using a mobile phone, which can display a video playback interface, on which a video is playing.

[0146] As shown in Figure 9B, when the phone detects that the user is pressing the video screen with three fingers, it can display a floating window 902.

[0147] When the vehicle detects that the user presses three fingers on the floating window 902 and moves it a second preset distance, with the direction 6 of the movement pointing towards the passenger screen, the vehicle can determine that the user wishes to synchronize or transfer the video feed 901 to the passenger screen. At this time, the mobile phone can display interaction options 903 and 904 on its screen. For example, interaction option 903 can be a content synchronization control, with the text explanation "synchronize to passenger screen" below it. For example, interaction option 904 can be a content transfer control, with the text explanation "transfer to passenger screen" below it.

[0148] In the scenario shown in Figure 9 above, the vehicle may only be equipped with display screen 201 and display screen 202.

[0149] Optionally, the phone can have the location information of multiple displays in the vehicle pre-set. Simultaneously, the phone can have a pre-set display area division method similar to that shown in Figure 3G. In this way, after detecting the movement direction of a three-finger press on the video screen a first preset distance, the phone can determine, based on the display area division method, which screen the user wishes to synchronize or stream the video screen to.

[0150] As shown in Figure 9C, when the vehicle detects that the user presses the floating window 902 with three fingers and continues to move along the moving direction 7 and the moving direction 7 points to the interactive option 903, the vehicle can control the interactive option 903 to move toward the floating window 902 and control the display area where the interactive option 903 is located to increase.

[0151] Optionally, as the distance between the interactive option 903 and the floating window 902 gradually decreases, the display area of ​​the interactive option 903 can gradually increase.

[0152] When the display areas where the floating window 902 and the interactive option 903 are located are detected to overlap at least partially and when the user switches from touching the phone's display to not touching the phone's display (or when the user's three fingers leave the phone's display), the vehicle can control the display screen 202 to display the GUI as shown in Figure 9D and the phone can control the phone's display screen to display the GUI as shown in Figure 9E.

[0153] As shown in Figures 9D and 9E, the vehicle can display video footage on the display screen 202, and the mobile phone can also display video footage on the display screen.

[0154] Optionally, the mobile phone can obtain its position within the cockpit, and the mobile phone can divide the display area of ​​multiple displays based on its position within the cockpit.

[0155] For example, Figures 9F and 9G show schematic diagrams of the division of display areas on a mobile phone screen.

[0156] Assuming the phone detects that it is currently located in the right-hand area of ​​the second row, the display area can be divided as shown in Figure 9F. If the vehicle is only equipped with displays 201 and 202, when the center point O of the three-finger press is detected sliding towards area 3, it can be determined that the display screen the user wants the video to be synchronized or flow to is display 202; or, when the center point O of the three-finger press is detected sliding towards area 4, it can be determined that the display screen the user wants the video to be synchronized or flow to is display 201.

[0157] Assuming the phone detects that it is currently located in the left area of ​​the second row, the display area can be divided as shown in Figure 9G. If the vehicle is only equipped with display screens 201 and 202, when the center point O of the three-finger press is detected sliding towards area 5, it can be determined that the display screen the user wants the video to be synchronized or flow to is display screen 201; or, when the center point O of the three-finger press is detected sliding towards area 6, it can be determined that the display screen the user wants the video to be synchronized or flow to is display screen 202.

[0158] In this embodiment of the application, the user can conveniently synchronize or transfer the content displayed on the mobile terminal to a certain display screen in the cockpit, making it convenient for the user to synchronize or transfer content to the display screens in other areas of the cockpit through the mobile terminal.

[0159] Figures 10A-10G illustrate another set of GUIs provided in embodiments of this application.

[0160] As shown in Figures 10A and 10B, when the vehicle is displaying video screen 1001 on display screens 201 and 202, it detects the user's three-finger press input on display screen 202 for video screen 1001, and can display the GUI shown in Figure 10C on display screen 202.

[0161] For example, when the vehicle detects that a user presses three fingers on the floating window 1001 and moves it a first preset distance, with the direction 8 of the movement pointing towards the display screen 204, the vehicle can determine that the user wishes to synchronize or transfer the video screen 1001 to the display screen 204. The vehicle can display the floating window 1002, interaction option 1003, and interaction option 1004 on the display screen 202. For example, interaction option 1003 can be a content viewing control, with a text explanation below it: "Synchronize to the second-row right-side entertainment screen, and display simultaneously on the passenger screen and the second-row right-side entertainment screen." For example, interaction option 1004 can be a content transfer control, with a text explanation below it: "Transfer to the second-row right-side entertainment screen, or transfer from the passenger screen to the second-row right-side entertainment screen."

[0162] As shown in Figure 10D, when the system detects that the user presses the floating window 1002 with three fingers and continues to move along the moving direction 9 and the moving direction 9 points to the interactive option 1004, the vehicle can control the interactive option 1004 to move toward the floating window 1002 and control the display area where the interactive option 1004 is located to become larger.

[0163] Optionally, when it is detected that the display areas where the floating window 1002 and the interactive option 1004 are located at least partially overlap, the GUI shown in Figures 10E, 10F and 10G can be displayed on the display screens 201, 202 and 204 respectively.

[0164] As shown in Figures 10E, 10F and 10G, the vehicle can display the video image 1001 through display screens 201 and 204 and display the off-screen display interface through display screen 202.

[0165] In this embodiment, the user can continue to perform content synchronization or content transfer operations on the display screen 202 that is being synchronized or transferred. This allows users in the cabin to easily initiate content synchronization or content transfer to other display screens according to their needs, helping to improve the user's human-computer interaction experience.

[0166] Figures 11A-11C illustrate another set of GUIs provided in embodiments of this application.

[0167] As shown in Figure 11A, when the display areas where the floating window 302 and the interactive option 303 are located are detected to overlap at least partially, a preview interface for viewing the content can be displayed on the display screen 201.

[0168] As shown in Figure 11B, when the user presses the floating window 302 with three fingers and moves it to the left, the preview interface for viewing the content can be stopped and the floating window 302, interaction options 303 and interaction options 304 can be displayed.

[0169] As shown in Figure 11C, when the system detects that the user presses three fingers on the floating window 302 and moves it along the movement direction 10, it can be determined that the user wishes to move the floating window 302 to overlap with the interactive option 304. The vehicle can then control the interactive option 304 to move toward the floating window 302 and control the display area of ​​the interactive option 304 to increase.

[0170] The human-computer interaction method provided in the embodiments of this application has been described above with reference to the GUI shown in Figures 3 to 11. The following describes some possible implementation processes in the embodiments of this application with reference to Figures 12 to 15.

[0171] Figure 12 shows a schematic diagram of the structure of the display system 1200 provided in an embodiment of this application. As shown in Figure 12, the system includes a multi-user center module, an application deployment decision module, a system-on-a-chip (SOC) 1, and displays 201 to 204. The multi-user center module can store the identity information of one or more users. For example, a vehicle can store the biometrics entered by user A in the multi-user center module, thereby establishing a correspondence between user A's identity information and user A's biometrics. Similarly, a vehicle can store the biometrics entered by user B in the multi-user center module, thereby establishing a correspondence between user B's identity information and user B's biometrics.

[0172] The application deployment decision module can store the correspondence between users and their associated interface elements. Figure 12 illustrates this using the application as an example. Table 1 shows a correspondence between different users and their associated interface elements.

[0173] Table 1

[0174] It should be understood that the correspondence shown in Table 1 above is merely illustrative. Different users may have completely identical, partially identical, or completely different applications. This application embodiment does not limit this.

[0175] It should also be understood that the application deployment decision module can store the mapping between users and their associated different types of interface elements. For example, the application deployment decision module can store the mapping between users and their associated wallpapers, animations, and applications. For example, Table 2 shows another type of mapping between users and their associated interface elements.

[0176] Table 2

[0177] In one embodiment, the application deployment decision module can determine the correspondence between a user and its associated interface elements based on the user's settings. For example, when it is detected that user A has selected application 1, application 2, and application 3 as their preferred applications, a correspondence can be established between user A's identity information and application 1, application 2, and application 3.

[0178] In one embodiment, the application deployment decision module can determine the correspondence between a user and its associated interface elements based on the frequency with which the user uses applications over a recent period (e.g., a week or a month). For example, if user A's most frequently used applications in the past week are application 1, application 2, and application 3, then the application deployment decision module can establish a correspondence between user A's identity information and application 1, application 2, and application 3.

[0179] SOC1 is used to acquire data collected by sensors in the vehicle and determine whether a user is present in different areas of the vehicle. For example, SOC1 can acquire data from the pressure sensor under the seat to determine whether a user is present in each area. When SOC1 determines that the pressure value acquired by the pressure sensor under the driver's seat is greater than or equal to a preset pressure threshold, it can determine that a user is present in the driver's area. At this time, SOC1 can continue to control the cameras in the cabin (e.g., cameras of the driver monitor system (DMS) or cabin monitor system (CMS)) to start and acquire images of the driver's area, thereby determining the user's identity information in the driver's area based on the images acquired by the cameras and the user's facial features stored in the multi-user center module. For example, it can be determined that the user in the driver's area is user A.

[0180] For example, when SOC1 determines that the pressure value obtained by the pressure sensor under the passenger seat is greater than or equal to a preset pressure threshold, it can determine that a user is present in the passenger area. At this point, SOC1 can continue to control the in-cabin camera (e.g., DMS or CMS) to activate and capture images of the passenger area, thereby determining the user's identity information based on the images captured by the camera and the user's facial features stored in the multi-user center module. For instance, it can be determined that the user in the passenger area is user B.

[0181] When SOC1 determines that the user in the driver's area is user A and the user in the passenger's area is user B, SOC1 can control the display screen 201 in the driver's area to display information of application 1, application 2 and application 3, and control the display screen 202 in the passenger's area to display information of application 1, application 4 and application 5, according to the correspondence stored in the application deployment decision module.

[0182] In one embodiment, SOC1 can also display user-related interface elements based on user actions. For example, when SOC1 determines that the user in the driver's area is user A and detects that user A has not clicked on display screen 201, it can display wallpaper 1 or animation 1 on display screen 201. When it detects that user A has clicked on display screen 201, it can control display screen 201 to switch from displaying wallpaper 1 or animation 1 to displaying information about application 1, application 2, and application 3.

[0183] It should be understood that the above implementation is illustrated by SOC1 controlling different in-vehicle displays (e.g., display 201 and display 202) separately, and the embodiments of this application are not limited thereto. For example, SOC1 can also control different display areas of the same in-vehicle display for display.

[0184] The architecture of the display system shown in Figure 12 can also be called a one-chip multi-screen display system, that is, multiple in-vehicle displays are controlled by a single chip (e.g., SOC1). These multiple in-vehicle displays include the instrument panel and the aforementioned displays 201 to 204.

[0185] Taking the scenario shown in Figure 3 as an example, as shown in Figure 3A, SOC1 can play the video 301 through the video application deployed on the display screen 201. When the system detects that the user is pressing the video 301 with three fingers, it can pause the playback of the video 301 and record the playback progress at this time. For example, the total playback time of the video 301 is 1 hour, and the current playback progress is 39 minutes and 30 seconds.

[0186] As shown in Figures 3E and 3F, in response to the user switching from touching the display screen 201 to not touching the display screen 201, if the video application is also deployed on the display screen 202, then the SOC1 can control the display screen 201 to continue playing the video playback screen 301 at the current playback progress and control the video application deployed on the display screen 202 to play the video screen 301 at the current playback progress.

[0187] If the video application is not deployed on the display screen 202, the SOC1 can use screen casting technology (e.g., Miracast screen casting technology, Digital Living Network Alliance (DLNA) screen casting technology) to cast the video image 301 on the display screen 201 to the display screen 202 and control the display screen 201 to continue playing the video image 301 at the current playback progress.

[0188] Taking the scenario shown in Figure 8 as an example, as shown in Figure 8A, SOC1 can play the video screen 801 through the video application deployed on the display screen 201. When the system detects that the user presses three fingers on the video screen 301, it can pause the playback of the display screen 801 and record the playback progress at this time.

[0189] As shown in Figure 8E, in response to the user switching from touching the display screen 201 to not touching the display screen 201, if the video application is also deployed on the mobile phone, then the SOC1 can control the display screen 201 to continue playing the video playback screen 301 at the current playback progress and control the communication module in the cockpit (such as a Bluetooth communication module, Wi-Fi communication module, or Starlight communication module) to instruct the mobile phone to use the video application to play the video screen 801 at the current playback progress. In response to receiving an instruction from the vehicle, the mobile phone can open the video application and play the video screen 801 at the current playback progress.

[0190] If the video application is not installed on the mobile phone, the SOC1 can use screen casting technology (e.g., Miracast screen casting technology, DLNA screen casting technology) to cast the video image 801 on the display screen 201 to the mobile phone's display screen and control the display screen 201 to continue playing the video image 801 at the current playback progress.

[0191] Figure 13 shows a schematic diagram of the display system under a single chip and multiple screens provided in an embodiment of this application. This display system can simulate the functions of the hardware systems required by different types of operating systems through multiple virtual machines, enabling the running of different types of operating systems within the display system 1300. Furthermore, the management of these multiple operating systems can be achieved through a virtual machine manager. For example, virtual machine 1 can run a real-time operating system (RTOS), and virtual machine 2 can run a guest Linux operating system. Based on a lightweight framework / library, application domain 1 can run on virtual machine 1. Application domain 1 can include applications for the instrument cluster, such as those displayed on the instrument panel. Based on the framework / library, application domain 2 can run on virtual machine 2. Application domain 2 can include in-vehicle infotainment (IVI) applications, such as those displayed on displays 201, 202, 203, and 204.

[0192] The technical solutions of this application embodiment can be implemented through a single-chip multi-screen display system or through a multi-chip multi-screen display system. Figure 14 shows a structural schematic diagram of the display system 1400 provided in this application embodiment. Unlike the structural schematic diagram shown in Figure 12, the structural schematic diagram shown in Figure 14 also includes a SOC2, wherein SOC1 is used to control the display of the display screen 201, and SOC2 is used to control the display of the display screens 202 to 204.

[0193] For example, when SOC1 determines that the pressure value obtained by the pressure sensor under the driver's seat is greater than or equal to a preset pressure threshold, it can determine that a user is present in the driver's area. At this time, SOC1 can continue to control the in-cabin camera (e.g., DMS or CMS) to start and collect images of the driver's area, thereby determining the user's identity information based on the images collected by the camera and the user's facial features stored in the multi-user center module. For example, it can be determined that the user in the driver's area is user A. When SOC1 determines that the user in the driver's area is user A, SOC1 can control the display screen 201 in the driver's area to display application 1, application 2, and application 3 according to the correspondence stored in the application deployment decision module.

[0194] For example, when SOC2 determines that the pressure value obtained by the pressure sensor under the passenger seat is greater than or equal to a preset pressure threshold, the pressure value obtained by the pressure sensor under the seat on the left side of the second row is greater than or equal to the preset pressure threshold, and the pressure value obtained by the pressure sensor under the seat on the right side of the second row is less than the preset pressure threshold, it can be determined that there are users in the passenger seat area and the left side of the second row, but no users in the right side of the second row. At this time, SOC2 can continue to control the in-cabin camera to start and collect images of the passenger seat area and the left side of the second row, thereby determining the identity information of the user in the passenger seat area and the user in the left side of the second row based on the images collected by the camera and the user's facial features stored in the multi-user center module. For example, it can be determined that the user in the passenger seat area is user B and the user in the left side of the second row is user N. When SOC2 determines that the user in the passenger seat area is user B and the user in the left side of the second row is user N, SOC2 can, according to the correspondence stored in the application deployment decision module, control the display screen 202 in the passenger seat area to display information of application 1, application 4, and application 5, and control the display screen 203 in the left side of the second row to display information of application 6, application 7, and application 8.

[0195] It should be understood that Figure 14 above illustrates an example where SOC1 controls the display of screen 201 and SOC2 controls the display of screens 202 to 204, and the embodiments of this application are not limited thereto. For example, the display system 1400 may also include SOC3 and SOC4, wherein SOC1 controls the display of screen 201, SOC2 controls the display of screen 202, SOC3 controls the display of screen 203, and SOC4 controls the display of screen 204.

[0196] Taking the scenario shown in Figure 3 as an example, as shown in Figure 3A, SOC1 can play the video 301 through the video application deployed on the display screen 201. When the system detects that the user presses the video 301 with three fingers, it can pause the playback of the video 301 and record the playback progress at this time (for example, the total playback time of the video 301 is 1 hour, and the current playback progress is 39 minutes and 30 seconds).

[0197] As shown in Figures 3E and 3F, in response to the user switching from touching display screen 201 to not touching display screen 201, if SOC1 determines that the video application is also deployed on display screen 202, then SOC1 can send an instruction message to SOC2. This instruction message instructs SOC2 to control display screen 202 to play the video frame 301 at the current playback progress. The instruction message includes the playback progress. In response to receiving this instruction message, SOC2 can open the video application in the background and open the video frame 301. After opening the video frame 301, SOC2 can control display screen 202 to play the video frame 301 at the current playback progress and send a response message to SOC1. This response message instructs SOC1 to continue controlling display screen 201 to play the video frame 301 at the current playback progress. In response to receiving this response message, SOC1 can continue controlling display screen 201 to play the video frame 301 at the current playback progress.

[0198] If the video application is not deployed on display 202, SOC1 can continue to control display 201 to play the video frame 301 at the current playback progress, and SOC1 can send the screen projection data at the current playback progress to SOC2. In response to receiving the screen projection data, SOC2 can control display 202 to play the video frame 301 at the current playback progress.

[0199] Figure 15 shows a schematic flowchart of the human-computer interaction method 1500 provided in an embodiment of this application. The method 1500 can be executed by the vehicle 100; or, the method 1500 can be executed by the computing platform 150; or, the method 1500 can also be executed by a processor, circuit, or chip in the computing platform 150; or, the method 1500 can also be executed by the display system 1200 or the display system 1400. The method 1500 includes:

[0200] S1510, the first display screen in the control cockpit displays a first interface, which includes first display content.

[0201] Optionally, the first displayed content is a video.

[0202] Optionally, the first displayed content can be an image, a document, the desktop displayed on the first display screen, or a display interface under a certain tab.

[0203] For example, the first display screen may be display screen 201, and the first interface may be the display interface shown in FIG3A.

[0204] S1520, in response to a first input to the first display content, control the first display screen to display a second interface, the second interface including a first interaction option and a second interaction option, the first interaction option instructing the first display content to be synchronized to the second display screen in the cockpit, and the second interaction option instructing the first display content to be transferred to the second display screen.

[0205] For example, the second display screen can be display screen 202, the second interface can be as shown in FIG3B, the first interaction option can be interaction option 303, and the second interaction option can be interaction option 304.

[0206] For example, the second display screen can be display screen 202, the second interface can be as shown in FIG6C, the first interaction option can be interaction option 604, and the second interaction option can be interaction option 605.

[0207] For example, the second display screen can be display screen 202, the second interface can be as shown in FIG7C, the first interactive option can be the "watch together option" in interactive control 704, and the second interactive option can be the "flow option" in interactive control 704.

[0208] Optionally, before controlling the display device to display the second interface, the method 1500 further includes: in response to a first input, controlling the first display screen to display a floating window; and determining the second display screen from a plurality of display screens in the cockpit based on a first movement direction controlled by the user of the floating window.

[0209] For example, as shown in Figure 3A, when a user presses three fingers on the display screen 201, the video playback 301 can be paused and a floating window 302 can be displayed. After detecting that the user has moved the floating window 302 a first preset distance by pressing three fingers, the display screen 202 can be identified from multiple displays in the cockpit based on the movement direction 1 of the user's three-finger press on the floating window 302 a first preset distance and the above-mentioned display area division method.

[0210] For example, the second display screen may also include multiple displays. For instance, the second display screen may include display screen 204 and a mobile phone display screen. As shown in Figures 8C and 8D, when the vehicle detects that a user presses three fingers on the floating window 802 and moves it a first preset distance, with the direction 4 of the first preset distance pointing towards the right side area of ​​the second row, the vehicle can determine that the user wishes to synchronize or stream the video screen 801 to the display screen in the right side area of ​​the second row. Since the right side area of ​​the second row includes the entertainment screen and the mobile phone display screen at this time, the vehicle can display the floating window 802 and interactive options 803-806 through display screen 201.

[0211] Optionally, the second display screen is the display screen of the first mobile terminal. Before determining the second display screen from multiple display screens in the cockpit according to the first movement direction controlled by the user of the floating window, the method 1500 further includes: obtaining the first position of the first mobile terminal in the cockpit; wherein, determining the second display screen from multiple display screens in the cockpit according to the first movement direction controlled by the user of the floating window includes: determining the display screen of the first mobile terminal as the second display screen when the first sliding direction matches the first position.

[0212] For example, the vehicle may only be equipped with displays 201 and 204. After the mobile phone and the vehicle establish a connection, the vehicle can determine the position of the mobile phone in the vehicle cabin according to the orientation detection method, for example, it can determine that the mobile phone is located in the right-hand area of ​​the second row in the cabin. When the user presses three fingers on the display screen 201, the video 301 can be paused and a floating window 302 can be displayed. After the user moves the floating window 302 a first preset distance by pressing three fingers, the direction of movement of the floating window 302 by pressing three fingers (e.g., moving to the lower right of the display screen 201) and the above-mentioned display area division method can be used to determine from multiple displays in the cabin which the user wants to synchronize or stream the video 301 to the mobile phone's display screen.

[0213] For example, Figure 16 shows a schematic diagram of the orientation of the identification device provided in an embodiment of this application. As shown in Figure 16, the vehicle has a Bluetooth / Wi-Fi antenna array (or the vehicle has the ability to calculate the angle of arrival (AOA)), while the mobile phone does not have a Bluetooth / Wi-Fi antenna array (or the mobile phone does not have the ability to calculate AOA). The vehicle can calculate the orientation of the mobile phone, and the vehicle's Bluetooth / Wi-Fi antenna array can receive the wireless signal of the mobile phone. The orientation of the mobile phone in the cabin is calculated according to formulas (1) and (2):

[0214] Where d is the distance between the vehicle's Bluetooth / Wi-Fi antenna array and the phone's Bluetooth / Wi-Fi antenna. λ represents the phase difference between the vehicle's Bluetooth / Wi-Fi antenna array and the mobile phone's Bluetooth / Wi-Fi antenna, λ represents the wavelength of the signal transmitted by the mobile phone, and θ represents the angle of arrival. It should be understood that in this embodiment, the vehicle's calculation of the mobile phone's orientation within the cabin can also be understood as the vehicle's calculation of the orientation of the connection between the vehicle's Bluetooth / Wi-Fi antenna array and the mobile phone's Bluetooth / Wi-Fi antenna.

[0215] The above-described method for detecting the position of a mobile phone in the cabin is merely illustrative and is not specifically limited in this application.

[0216] Optionally, in response to a first input to the first display content, controlling the first display screen to display a second interface includes: in response to the first input, controlling the first display screen to display a third interface, the third interface including a floating window and a display list, the display list including identification information of each of the multiple displays in the cockpit; and in response to the user controlling the floating window to move to at least partially overlap with the display area where the identification information of the second display screen is located, controlling the first display screen to display the second interface.

[0217] For example, the third interface can be a GUI as shown in Figure 6B, and the second interface can be a GUI as shown in Figure 6C. In response to the three-finger press input, the vehicle can display a floating window 602 and a display list 603 on the display screen 201. The identification information of multiple displays in the display list 603 can be located in a ring as shown in Figure 6B. In response to the floating window 602 at least partially overlapping with the area where the identification information of the passenger-side screen is located, the vehicle can display interactive controls 604 and 605 on the display screen 201.

[0218] Optionally, in response to a first input to the first display content, controlling the first display screen to display a second interface includes: in response to the first input, controlling the first display screen to display a third interface, the third interface including a display list, the display list including identification information of each display screen among a plurality of display screens in the cockpit; and in response to the user selecting the identification information of the second display screen from the display list, controlling the first display screen to display the second interface.

[0219] For example, in response to the three-finger press input, the vehicle can display a display list 603 on the display screen 201 without displaying the floating window 602. The display list 603 includes identification information for multiple displays within the cabin, such as, but not limited to, DLP headlights, a HUD, a passenger-side screen, a second-row left-side entertainment screen, a second-row right-side entertainment screen, and a screen. When user input is detected clicking on the identification information of the passenger-side screen, the vehicle can display interactive controls 604 and 605 on the display screen 201.

[0220] Optionally, the plurality of displays does not include the display of the second mobile terminal. The method 1500 further includes: when the second mobile terminal is detected to be in the cockpit, updating the display list, wherein the updated display list includes the identification information of the plurality of displays and the display of the second mobile terminal.

[0221] For example, the display list 603 shown in FIG6B does not include the identification information of the mobile terminal. When the mobile phone and the vehicle establish a connection and the mobile phone is located in the cabin, the vehicle can update the display list 603. The updated display list includes the identification information of the mobile phone's display screen. Thus, in response to the three-finger press input, the vehicle can display a floating window 602 and a display list 603 on the display screen 201, which may include the identification information of the DLP headlights, HUD, passenger screen, second-row left entertainment screen, second-row right entertainment screen, screen, and mobile phone.

[0222] Optionally, the second interface further includes a floating window, wherein, in response to a first input to the first display content, controlling the first display screen to display the second interface includes: in response to the first input, controlling the first display screen to display a third interface, the third interface including a virtual display screen in the cockpit, the virtual display screen including display areas of multiple displays in the cockpit; in response to the user dragging the floating window to at least partially overlap with the display area of ​​the second display screen in the virtual display screen, controlling the first display screen to display the second interface, the second interface also including the virtual display screen displayed by the first display screen and the second display screen according to preset interaction options, the preset interaction options being the first interaction option or the second interaction option.

[0223] For example, the first interface can be a GUI as shown in Figure 7A, the second interface can be a GUI as shown in Figure 7B, the first interactive option can be the "watch together" option in the interactive control 704, and the second interactive option can be the "flow" option in the interactive control 704. As shown in Figure 7B, in response to the three-finger press input, the vehicle can display a virtual display screen in the cockpit, as well as a floating window 702, interactive control 704, and confirmation control 705 on the display screen 201. For example, the interactive control 704 includes a "watch together" option and a "flow" option. The virtual display screen in the cockpit includes information from multiple displays. For example, the display area where the display screen 204 is located in the virtual display screen in the cockpit is area 703.

[0224] The preset interactive option can be a simultaneous viewing option, as shown in Figure 7C. In response to the floating window 702 and the area 703 at least partially overlapping, the vehicle can display another virtual display screen through the display screen 201. This other virtual display screen includes the display screen 201 in the cockpit and the preview interface 1 of the video screen 701 displayed on the display screen 204.

[0225] Optionally, the method 1500 further includes: in response to the user's input of selecting another interaction option, controlling the first display screen to display a virtual display screen that is displayed by the first display screen and the second display screen according to the other interaction option, wherein the other interaction option is an interaction option other than the preset interaction option among the first interaction option and the second interaction option.

[0226] For example, the other interactive option can be a flow option. As shown in Figure 7D, when the user selects the flow option in the interactive control 704, another virtual display screen can be displayed on the display screen 201. This other virtual display screen includes the cockpit display screen 201 displaying an off-screen display interface and the display screen 204 displaying a preview interface 2 of the video screen 701.

[0227] S1530, in response to a second input by the user on the second interface, control the first display screen and / or the second display screen to display the first display content.

[0228] Optionally, in response to a second input by the user on the second interface, controlling the first display screen and / or the second display screen to display the first display content includes: in response to the user's input on the second interface for the first interactive option, controlling the first display screen and the second display screen to display the first display content; or, in response to the user's input on the second interface for the second interactive option, controlling the first display screen to display the second display content and controlling the second display screen to display the first display content.

[0229] As shown in Figure 6C, the first display screen can be display screen 201, the second display screen can be display screen 202, the first interaction option can be interaction option 604, and the second interaction option can be interaction option 605. When the display areas of the floating window 602 and the interaction option 604 are detected to overlap at least partially, the vehicle can display the video image 601 through display screens 201 and 202. Alternatively, when the display areas of the floating window 602 and the interaction option 605 are detected to overlap at least partially, the vehicle can display a screen-off interface through display screen 201 and display the video image 601 through display screen 202.

[0230] Optionally, before controlling the first display screen and / or the second display screen to display the first display content in response to a second input from the user on the second interface, the method 1500 further includes: in response to obtaining input from the user to control the floating window to move toward a target interactive option, controlling the target interactive option to move toward the floating window, wherein the target interactive option is the first interactive option or the second interactive option.

[0231] For example, as shown in Figure 3C, when a three-finger press is detected and the user's three-finger press moves the floating window 302 to the upper right, the vehicle can control the interaction option 303 to move toward the floating window 302.

[0232] Optionally, the method 1500 further includes: in response to receiving input from the user to control the floating window to move toward the target interactive option, controlling the display area corresponding to the target interactive option to increase.

[0233] For example, as shown in Figure 3C, when a three-finger press is detected and the user presses the floating window 302 to the upper right, the vehicle can control the interactive option 303 to move toward the floating window 302 and control the display area where the interactive option 303 is located to increase.

[0234] Optionally, as the distance between the interactive option 303 and the floating window 302 gradually decreases, the display area of ​​the interactive option 303 can gradually increase.

[0235] Optionally, in response to a second input by the user on the second interface, controlling the first display screen and / or the second display screen to display the first display content includes: in response to the user controlling the floating window to move to at least partially overlap with the first display area where the first interactive option is located and detecting that the user switches from touching the first display screen to not touching the first display screen, controlling the first display screen and the second display screen to display the first display content; or, in response to the user controlling the floating window to move to at least partially overlap with the second display area where the second interactive option is located and detecting that the user switches from touching the first display screen to not touching the first display screen, controlling the first display screen to display the second display content and controlling the second display screen to display the first display content; or, before the floating window at least partially overlaps with the first display area or the second display area, in response to detecting that the user has not touched the first display screen, controlling the first display screen to display the first display content.

[0236] For example, as shown in FIG3C, when it is detected that the display area where the user control floating window 302 and the interactive option 303 are located overlaps at least partially, the display screen 201 and the display screen 202 can be directly controlled to display the GUI shown in FIG3E and FIG3F respectively.

[0237] Optionally, in response to the user controlling the floating window to move to at least partially overlap with the first display area where the first interactive option is located and detecting that the user switches from touching the first display screen to not touching the first display screen, controlling the first display screen and the second display screen to display the first display content includes: in response to the user controlling the floating window to move to at least partially overlap with the first display area where the first interactive option is located, controlling the first display screen to display a first preview interface of the first display content displayed on the first display screen and the second display screen; and in the first preview interface, detecting that the user switches from touching the first display screen to not touching the first display screen, controlling the first display screen and the second display screen to display the first display content.

[0238] For example, the first display screen is display screen 201, the floating window can be floating window 302, and the first interactive option can be interactive option 303. As shown in Figure 3D, when it is detected that the display areas of the floating window 302 and the interactive option 303 at least partially overlap, a preview interface for viewing the same content can be displayed on display screen 201. This preview interface includes display window 305 and display window 306. Display window 305 can indicate display screen 201, and display window 306 can indicate display screen 202. When it is detected that the user switches from touching display screen 201 to not touching display screen 201 (or, the user's three fingers leave display screen 201) in the preview interface, the vehicle can control display screen 201 and display screen 202 to display video image 301.

[0239] Optionally, in response to the user controlling the floating window to move to at least partially overlap with the second display area where the second interactive option is located and detecting that the user switches from touching the first display screen to not touching the first display screen, controlling the first display screen to display the second display content and controlling the second display screen to display the first display content includes: in response to the user controlling the floating window to move to at least partially overlap with the first display area where the second interactive option is located, controlling the first display screen to display the second display content and the second display screen to display a second preview interface of the first display content; and detecting that the user switches from touching the first display screen to not touching the first display screen in the second preview interface, controlling the first display screen to display the second display and controlling the second display screen to display the first display content.

[0240] For example, the first display screen is display screen 201, the floating window can be floating window 302, and the first interactive option can be interactive option 303. As shown in Figure 4A, when it is detected that the display areas of floating window 302 and interactive option 304 at least partially overlap, a preview interface of content flow can be displayed on display screen 201. The preview interface of content flow includes display window 307 and display window 308. Display window 307 can indicate display screen 201, and display window 308 can indicate display screen 202. When it is detected that the user switches from touching display screen 201 to not touching display screen 201 (or, the user's three fingers leave display screen 201) in the preview interface, the vehicle can control display screen 201 to display a screen-off display interface and control display screen 202 to display video image 301.

[0241] Figure 17 shows a schematic block diagram of a human-computer interaction device 1700 provided in an embodiment of this application. The device 1700 includes: a control unit 1710, configured to control a first display screen in a cockpit to display a first interface, the first interface including first display content; an acquisition unit 1720, configured to acquire a first input to the first display content; the control unit 1710 is further configured to, in response to the acquisition unit acquiring the first input, control the first display screen to display a second interface, the second interface including a first interaction option and a second interaction option, the first interaction option indicating that the first display content is synchronized to the second display screen in the cockpit, and the second interaction option indicating that the first display content is transferred to the second display screen; the acquisition unit 1720 is further configured to acquire a second input from the user on the second interface; the control unit 1710 is further configured to, in response to the acquisition unit acquiring the second input, control the first display screen and / or the second display screen to display the first display content.

[0242] Optionally, the device 1700 further includes a determining unit, the control unit 1710, which is further configured to control the first display screen to display a floating window in response to the acquisition unit acquiring a first input before controlling the display device to display a second interface; the determining unit is configured to determine the second display screen from a plurality of display screens in the cockpit according to a first movement direction controlled by the user controlling the floating window.

[0243] Optionally, the second display screen is the display screen of the first mobile terminal. The acquisition unit 1720 is further configured to acquire the first position of the first mobile terminal in the cockpit before the determining unit determines the second display screen from the plurality of display screens in the cockpit. The determining unit is specifically configured to: determine the display screen of the first mobile terminal as the second display screen when the first sliding direction matches the first position.

[0244] Optionally, the control unit 1710 is further configured to, in response to the acquisition unit acquiring an input from the user to control the floating window to move toward the target interactive option, control the target interactive option to move toward the floating window, wherein the target interactive option is the first interactive option or the second interactive option.

[0245] Optionally, the control unit 1710 is further configured to, in response to the acquisition unit acquiring an input from the user to control the floating window to move toward the target interactive option, control the display area corresponding to the target interactive option to increase.

[0246] Optionally, the control unit 1710 is specifically configured to: in response to the acquisition unit 1720 acquiring the first input, control the first display screen to display a third interface, the third interface including a floating window and a display list, the display list including the identification information of each of the multiple display screens in the cockpit; and in response to the acquisition unit 1720 acquiring that the user controls the floating window to move to at least partially overlap with the display area where the identification information of the second display screen is located, control the first display screen to display the second interface.

[0247] Optionally, the plurality of displays does not include the display of the second mobile terminal. The device 1700 further includes a detection unit and an information update unit. The information update unit is used to update the display list when the detection unit detects that the second mobile terminal is located in the cockpit. The updated display list includes the identification information of the plurality of displays and the display of the second mobile terminal.

[0248] Optionally, the control unit 1710 is specifically configured to: control the first display screen and the second display screen to display the first display content in response to the user controlling the floating window to move to at least partially overlap with the first display area where the first interactive option is located and the user switching from touching the first display screen to not touching the first display screen; or, control the first display screen to display the second display content and control the second display screen to display the first display content in response to the user controlling the floating window to move to at least partially overlap with the second display area where the second interactive option is located and the user switching from touching the first display screen to not touching the first display screen; or, before the floating window at least partially overlaps with the first display area or the second display area, control the first display screen to display the first display content in response to the acquisition unit 1720 acquiring that the user has not touched the first display screen.

[0249] Optionally, the control unit 1710 is specifically configured to: in response to a user controlling the floating window to move to at least partially overlap with the first display area where the first interactive option is located, control the first display screen to display a first preview interface of the first display content displayed on the first display screen and the second display screen; and when the user switches from touching the first display screen to not touching the first display screen in the first preview interface, control the first display screen and the second display screen to display the first display content.

[0250] Optionally, the control unit 1710 is specifically configured to: in response to a user controlling the floating window to move to at least partially overlap with the first display area where the second interactive option is located, control the first display screen to display a second preview interface where the first display screen displays the second display content and the second display screen displays the first display content; in the second preview interface, when the user switches from touching the first display screen to not touching the first display screen, control the first display screen to display the second display and control the second display screen to display the first display content.

[0251] Optionally, the second interface further includes a floating window, wherein the control unit 1710 is specifically configured to: in response to the acquisition unit 1720 acquiring the first input, control the first display screen to display a third interface, the third interface including a virtual display screen in the cockpit, the virtual display screen including the display area of ​​multiple displays in the cockpit; in response to the user dragging the floating window to at least partially overlap with the display area of ​​the second display screen in the virtual display screen, control the first display screen to display the second interface, the second interface further including the virtual display screen displayed by the first display screen and the second display screen according to preset interaction options, the preset interaction options being the first interaction option or the second interaction option.

[0252] Optionally, the control unit 1710 is further configured to, in response to receiving input from the user selecting another interactive option, control the first display screen to display a virtual display screen that is displayed on the first display screen and the second display screen according to the other interactive option, wherein the other interactive option is an interactive option other than the preset interactive option among the first interactive option and the second interactive option.

[0253] Optionally, the control unit 1710 is specifically configured to: control the first display screen and the second display screen to display the first display content in response to the user's input on the second interface for the first interactive option; or, control the first display screen to display the second display content and control the second display screen to display the first display content in response to the user's input on the second interface for the second interactive option.

[0254] Optionally, the first displayed content is a video.

[0255] It should be understood that the division of units in the above device is only a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, the units in the device can be implemented by a processor calling software; for example, the device includes a processor connected to memory, which stores instructions. The processor calls the instructions stored in memory to implement any of the above methods or to implement the functions of each unit in the device. The processor can be, for example, a general-purpose processor, such as a CPU or microprocessor, and the memory can be internal or external to the device. Alternatively, the units in the device can be implemented as hardware circuits. The functions of some or all units can be implemented through the design of the hardware circuits, which can be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all units are implemented through the design of the logical relationships between the components within the circuit. In another implementation, the hardware circuit can be implemented using a PLD, such as an FPGA, which can include a large number of logic gates. The connection relationships between the logic gates are configured through configuration files, thereby implementing the functions of some or all units. All units of the above devices can be implemented entirely through processor calling software, or entirely through hardware circuits, or partially through processor calling software with the remaining parts implemented through hardware circuits.

[0256] In this application embodiment, a processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a CPU, microprocessor, GPU, or DSP. In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented as an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the processor loading instructions to implement the functions of some or all of the above units. Furthermore, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, or DPU.

[0257] As can be seen, each unit in the above device can be one or more processors (or processing circuits) configured to implement the above methods, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0258] Furthermore, the units in the above devices can be integrated in whole or in part, or they can be implemented independently. In one implementation, these units are integrated together and implemented as a System-on-Chip (SoC). The SoC may include at least one processor for implementing any of the above methods or implementing the functions of the units in the device. The at least one processor may be of different types, such as CPU and FPGA, CPU and AI processor, CPU and GPU, etc.

[0259] This application also provides a human-computer interaction device, which includes a processing unit and a storage unit. The storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit to enable the device to perform the methods or steps described in the above embodiments.

[0260] Optionally, if the human-machine interface device is located in a vehicle, the processing unit may be the processor 151-15n shown in Figure 1.

[0261] This application also provides a human-computer interaction system, which may include a computing platform and multiple displays. The computing platform may include the aforementioned human-computer interaction device 1700.

[0262] This application also provides a vehicle that may include the human-computer interaction device 1700 or the human-computer interaction system described above.

[0263] This application also provides a computer program product, which includes computer program code that, when run on a computer, causes the computer to perform the methods described in the above embodiments.

[0264] This application also provides a computer-readable medium storing program code that, when run on a computer, causes the computer to perform the methods described in the above embodiments.

[0265] This application also provides a chip, which includes a circuit for performing the methods described in the above embodiments.

[0266] In implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software. The method disclosed in the embodiments of this application can be directly implemented by a hardware processor, or by a combination of hardware and software modules within the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, power-on erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, detailed descriptions are omitted here.

[0267] It should be understood that in the embodiments of this application, the memory may include read-only memory and random access memory, and provides instructions and data to the processor.

[0268] It should also be understood that, in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0269] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0270] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0271] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0272] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0273] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0274] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0275] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be covered. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

A human-machine interaction method, characterized in that, The method comprises: controlling a first display screen in a cockpit to display a first interface, the first interface comprising first display content; in response to a first input directed to the first display content, controlling the first display screen to display a second interface, the second interface comprising a first interaction option and a second interaction option, the first interaction option indicating synchronization of the first display content to a second display screen in the cockpit, the second interaction option indicating streaming of the first display content to the second display screen; in response to a second input of a user on the second interface, controlling the first display screen and / or the second display screen to display the first display content. The method of claim 1, wherein Before the controlling the first display screen to display the second interface, the method further comprises: in response to the first input, controlling the first display screen to display a floating window; determining the second display screen from a plurality of display screens in the cockpit according to a first movement direction of the floating window controlled by the user. The method according to claim 2, characterized in that The second display screen is a display screen of a first mobile terminal, and before the determining the second display screen from the plurality of display screens in the cockpit according to the first movement direction of the floating window controlled by the user, the method further comprises: obtaining a first position of the first mobile terminal in the cockpit; wherein the determining the second display screen from the plurality of display screens in the cockpit according to the first movement direction of the floating window controlled by the user comprises: when the first movement direction matches the first position, determining the display screen of the first mobile terminal as the second display screen. The method according to claim 2 or 3, characterized in that Before the in response to the second input of the user on the second interface, controlling the first display screen and / or the second display screen to display the first display content, the method further comprises: in response to obtaining an input of the user controlling the floating window to move towards a target interaction option, controlling the target interaction option to move towards the floating window, the target interaction option being the first interaction option or the second interaction option. The method according to claim 4, characterized in that The method further comprises: in response to obtaining the input of the user controlling the floating window to move towards the target interaction option, controlling a display area corresponding to the target interaction option to become larger. The method of claim 1, wherein The controlling the first display screen to display the second interface in response to the first input directed to the first display content comprises: in response to the first input, controlling the first display screen to display a third interface, the third interface comprising a floating window and a display list, the display list comprising identification information of each display screen in a plurality of display screens in the cockpit; in response to the user controlling the floating window to move to at least partially overlap with a display area in which the identification information of the second display screen is located, controlling the first display screen to display the second interface. The method according to claim 6, characterized in that The plurality of display screens does not comprise a display screen of a second mobile terminal, and the method further comprises: when detecting that the second mobile terminal is located in the cockpit, updating the display list, the updated display list comprising identification information of the plurality of display screens and the display screen of the second mobile terminal. The method according to any one of claims 2 to 7, characterized in that The response to the second input of the user on the second interface, the first display screen and / or the second display screen display the first display content, including: In response to the user controlling the floating window to move to at least partially overlap with the first display area where the first interaction option is located and detecting that the user switches from touching the first display screen to not touching the first display screen, the first display screen and the second display screen display the first display content; or, In response to the user controlling the floating window to move to at least partially overlap with the second display area where the second interaction option is located and detecting that the user switches from touching the first display screen to not touching the first display screen, the first display screen displays the second display content and the second display screen displays the first display content; or, Before the floating window at least partially overlaps with the first display area or the second display area, in response to detecting that the user does not touch the first display screen, the first display screen displays the first display content. The method of claim 8, wherein The response to the second input of the user on the second interface, the first display screen and / or the second display screen display the first display content, including: In response to the user controlling the floating window to move to at least partially overlap with the first display area where the first interaction option is located, the first display screen displays the first preview interface of the first display screen and the second display screen displaying the first display content; In response to detecting that the user switches from touching the first display screen to not touching the first display screen under the first preview interface, the first display screen and the second display screen display the first display content. The method of claim 8, wherein The response to the second input of the user on the second interface, the first display screen and / or the second display screen display the first display content, including: In response to the user controlling the floating window to move to at least partially overlap with the first display area where the first interaction option is located, the first display screen displays the first preview interface of the first display screen and the second display screen displaying the first display content; In response to detecting that the user switches from touching the first display screen to not touching the first display screen under the second preview interface, the first display screen displays the second display content and the second display screen displays the first display content. The method of claim 1, wherein The second interface further includes a floating window, The response to the first input of the first display content, the first display screen displays the second interface, including: in response to the first input, control the first display screen to display a third interface, the third interface comprising a virtual display picture in the cockpit, the virtual display picture comprising display regions of a plurality of display screens in the cockpit; in response to the user dragging the floating window to at least partially overlap with a display region of a second display screen in the virtual display picture, control the first display screen to display the second interface, the second interface further comprising a virtual display picture of the first display screen and the second display screen displayed according to a preset interaction option, the preset interaction option being the first interaction option or the second interaction option. The method of claim 11, wherein The method further comprises: in response to an input of the user selecting another interaction option, control the first display screen to display a virtual display picture of the first display screen and the second display screen displayed according to the another interaction option, the another interaction option being an interaction option other than the preset interaction option among the first interaction option and the second interaction option. The method according to any one of claims 1 to 12, characterized in that The response to the second input of the user on the second interface, control the first display screen and / or the second display screen to display the first display content, comprises: in response to an input of the user on the second interface for the first interaction option, control the first display screen and the second display screen to display the first display content; or in response to an input of the user on the second interface for the second interaction option, control the first display screen to display second display content and control the second display screen to display the first display content. The method according to any one of claims 1 to 13, characterized in that The first display content is a video. A human-machine interaction device, characterized in that, Comprise: a control unit configured to control a first display screen in a cockpit to display a first interface, the first interface comprising first display content; an acquisition unit configured to acquire a first input for the first display content; the control unit is further configured to, in response to the acquisition unit acquiring the first input, control the first display screen to display a second interface, the second interface comprising a first interaction option and a second interaction option, the first interaction option indicating synchronizing the first display content to a second display screen in the cockpit, and the second interaction option indicating transferring the first display content to the second display screen; the acquisition unit is further configured to acquire a second input of the user on the second interface; the control unit is further configured to, in response to the acquisition unit acquiring the second input, control the first display screen and / or the second display screen to display the first display content. The apparatus of claim 15, wherein The device further comprises a determination unit, the control unit is further configured to, before controlling the first display screen to display a second interface, in response to the acquisition unit acquiring a first input of the user on the first display content, control the first display screen to display a floating window; the determination unit is configured to determine the second display screen from a plurality of display screens in the cockpit according to a first moving direction of the user controlling the floating window. The apparatus of claim 16, wherein The second display screen is a display screen of a first mobile terminal, The acquisition unit is further configured to acquire a first position of the first mobile terminal in the cabin before the determination unit determines the second display screen from the plurality of display screens in the cabin. The determination unit is specifically configured to determine the display screen of the first mobile terminal as the second display screen when the first moving direction matches the first position. The apparatus according to claim 16 or 17, wherein The control unit is further configured to, in response to the acquisition unit acquiring the input of the user controlling the floating window to move towards the target interaction option, control the target interaction option to move towards the floating window, the target interaction option being the first interaction option or the second interaction option. The apparatus according to claim 18, wherein The control unit is further configured to, in response to the acquisition unit acquiring the input of the user controlling the floating window to move towards the target interaction option, control a display area corresponding to the target interaction option to become larger. The apparatus of claim 15, wherein The control unit is specifically configured to: in response to the acquisition unit acquiring the first input, control the first display screen to display a third interface, the third interface including a floating window and a display list, the display list including identification information of each display screen of the plurality of display screens in the cabin; in response to the acquisition unit acquiring the input of the user controlling the floating window to move to at least partially overlap with a display area in which the identification information of the second display screen is located, control the first display screen to display the second interface. The apparatus of claim 20, wherein The plurality of display screens do not include a display screen of a second mobile terminal, and the apparatus further includes a detection unit and an information updating unit, The information updating unit is configured to, in response to the detection unit detecting that the second mobile terminal is located in the cabin, update the display list, the updated display list including identification information of the plurality of display screens and the display screen of the second mobile terminal. The apparatus of any one of claims 16 to 21, wherein The control unit is specifically configured to: in response to the user controlling the floating window to move to at least partially overlap with a first display area in which the first interaction option is located and the user switching from touching the first display screen to not touching the first display screen, control the first display screen and the second display screen to display the first display content; or in response to the user controlling the floating window to move to at least partially overlap with a second display area in which the second interaction option is located and the user switching from touching the first display screen to not touching the first display screen, control the first display screen to display second display content and control the second display screen to display the first display content; or before the floating window at least partially overlaps with the first display area or the second display area, in response to the acquisition unit acquiring that the user does not touch the first display screen, control the first display screen to display the first display content. The control unit is specifically configured to: ​ The apparatus of claim 22, wherein ​ In response to a user controlling the floating window to move to at least partially overlap with the first display area where the first interactive option is located, the first display screen is controlled to display a first preview interface of the first display content displayed on the first display screen and the second display screen; When the user switches from touching the first display screen to not touching the first display screen in the first preview interface, the first display screen and the second display screen are controlled to display the first display content. The apparatus of claim 22, wherein The control unit is specifically used for: In response to a user controlling the floating window to move to at least partially overlap with the first display area where the second interactive option is located, the first display screen is controlled to display a second preview interface of the first display content and the second display screen displays the first display content; In the second preview interface, the user switches from touching the first display screen to not touching the first display screen, controls the first display screen to display the second display, and controls the second display screen to display the first display content. The apparatus of claim 15, wherein The second interface also includes a floating window. Specifically, the control unit is used for: In response to the acquisition unit acquiring the first input, the first display screen is controlled to display a third interface, the third interface including a virtual display screen in the cockpit, the virtual display screen including the display area of ​​multiple display screens in the cockpit; In response to a user dragging the floating window to at least partially overlap with the display area of ​​the second display screen in the virtual display screen, the first display screen is controlled to display the second interface. The second interface also includes a virtual display screen displayed by the first display screen and the second display screen according to preset interaction options, wherein the preset interaction options are either the first interaction option or the second interaction option. The apparatus according to claim 25 is characterized in that, The control unit is further configured to, in response to receiving input from the user selecting another interactive option, control the first display screen to display a virtual display screen that is displayed on the first display screen and the second display screen according to the other interactive option, wherein the other interactive option is an interactive option other than the preset interactive option among the first interactive option and the second interactive option. The apparatus of any one of claims 15 to 26, wherein The control unit is specifically used for: In response to user input on the second interface regarding the first interactive option, control the first and second displays to show the first displayed content; or, In response to the user's input on the second interface for the second interactive option, the system controls the first display screen to display the second display content and controls the second display screen to display the first display content. The apparatus of any one of claims 15 to 27, wherein The first displayed content is a video. A human-machine interaction device, characterized in that, include: Memory, used to store computer programs; A processor for executing a computer program stored in the memory to cause the apparatus to perform the method as described in any one of claims 1 to 14. A human-machine interaction system, characterized in that It includes multiple displays and a computing platform, the computing platform including the apparatus as described in any one of claims 15 to 29. A vehicle characterized by comprising: comprising the apparatus of any one of claims 15 to 29, or the system of claim 30. A computer-readable storage medium, characterized by having instructions stored thereon, which when executed by a processor, cause the processor to implement the method of any one of claims 1 to 14. A computer program product, characterized in that The computer program product comprises computer program code which, when the computer program code is run on a computer, causes the computer to implement the method of any one of claims 1 to 14. A chip characterized by The chip comprises a circuit for performing the method of any one of claims 1 to 14.

Images