A human-computer interaction method, device and electronic equipment

Abstract

The application provides a human-computer interaction method and device and electronic equipment, which can be applied to the field of intelligent cockpit. The method comprises the following steps: controlling a display device to display a first interface, wherein the first interface comprises a plurality of interface elements in a reduced display state, and each interface element in the plurality of interface elements corresponds to a service, an application or an atomic function; and according to a first input, controlling an interface element in a first interface element set corresponding to the first input to switch from the reduced display state to an expanded display state, wherein the plurality of interface elements comprise the first interface element set. The application can be applied to a smart car or an electric car, and helps to improve the user experience during human-computer interaction.

Description

Technical Field

[0001] This application relates to the field of human-computer interaction, and more specifically, to a human-computer interaction method, apparatus, and electronic device. Background Technology

[0002] Users are interacting with electronic devices more and more frequently. Current interaction methods involve displaying an interaction model (e.g., a voice assistant) in a fixed area. Because this interaction model is static and can only interact with users through voice or text, the user experience is often poor. Summary of the Invention

[0003] This application provides a human-computer interaction method, apparatus, and electronic device that helps improve the user experience during human-computer interaction.

[0004] In a first aspect, a human-computer interaction method is provided, the method comprising: controlling a display device to display a first interface, the first interface including multiple interface elements in a minimized display state, each of the multiple interface elements corresponding to a service, application or atomic function; and controlling an interface element in a set of first interface elements corresponding to the first input to switch from a minimized display state to an expanded display state according to a first input, the multiple interface elements including the set of first interface elements.

[0005] Based on the above technical solution, by displaying multiple interface elements in a minimized state, users can experience the feeling of being watched by multiple services, applications, or atomic functions within an electronic device. Upon receiving the first input, the expanded display state of the interface element corresponding to that input can be displayed, providing users with the expanded display state of the applications, services, or atomic functions they expect to see under the first input. This facilitates user operation and improves the user experience during human-computer interaction.

[0006] In some possible implementations, this first interface can also be called the Starry Sky Desktop.

[0007] In some possible implementations, the first interface element set includes first interface elements. Controlling the interface elements in the first interface element set corresponding to the first input to switch from a minimized display state to an expanded display state includes: controlling the display device to switch from displaying the first interface elements in the minimized display state to displaying the application icon and service card corresponding to the first interface elements in the expanded display state. The service card includes services or atomic functions related to the application in the expanded display state.

[0008] In some possible implementations, the arrangement of multiple interface elements in the minimized display state under the first interface can be arranged according to preset rules.

[0009] For example, the graphics created by the animation effects of multiple interface elements in a minimized state are related to the weather conditions of the current scene. If the current weather is rainy, the shapes of the minimized interface elements can be raindrops. Similarly, if the current weather is snowing, the shapes of the minimized interface elements can be snowflakes.

[0010] In conjunction with the first aspect, in some implementations of the first aspect, before controlling the interface elements in the first set of interface elements corresponding to the first input to switch from a minimized display state to an expanded display state, the method further includes: controlling the interface elements in the second set of interface elements on the first interface to switch from a minimized display state to an expanded display state based on the user's usage frequency and / or usage duration of each interface element among the plurality of interface elements; and controlling the interface elements in the second set of interface elements to switch from an expanded display state to a minimized display state when the duration of the expanded display state of the interface elements in the second set of interface elements continues for a first preset duration.

[0011] Based on the above technical solution, frequently used services, applications, or atomic functions can be switched to an expanded display state on the first interface and maintained for a first preset duration. This allows users to easily view their frequently used services, applications, or atomic functions through the first interface. When users do not have a specific task-oriented dialogue need, they can select their frequently used services, applications, or atomic functions to quickly access the relevant interface.

[0012] In conjunction with the first aspect, in some implementations of the first aspect, the plurality of interface elements includes a third set of interface elements, the usage frequency and / or usage duration of the plurality of interface elements in the third set of interface elements meet preset conditions, wherein controlling the display device to display the first interface includes: controlling the display device to display the application icon corresponding to each interface element in the third set of interface elements, and controlling the display device to display a plurality of preset shapes, each of the plurality of preset shapes corresponding one-to-one with each interface element in a fourth set of interface elements, the fourth set of interface elements being the interface elements other than the third set of interface elements among the plurality of interface elements.

[0013] Based on the above technical solution, the first interface can display icons of frequently used applications, services, or atomic functions that are in a minimized display state, as well as multiple preset shapes. This allows users to see the icons of their frequently used applications on the first interface.

[0014] In conjunction with the first aspect, in some implementations of the first aspect, the preset shape is a dot, a star, or a sphere.

[0015] In some possible implementations, the preset shape is either raindrop-shaped or snowflake-shaped.

[0016] In conjunction with the first aspect, in some implementations of the first aspect, controlling the display device to display the first interface includes: controlling the display device to display the first interface according to the area where the user is located in the cockpit.

[0017] Based on the above technical solution, the display device can be controlled to show the first interface according to the area where the user is located in the cabin. In this way, the first interface can be displayed in combination with the distribution of users in the cabin, which helps to further improve the user experience when conducting human-computer interaction.

[0018] In conjunction with the first aspect, in some implementations of the first aspect, the display device is a central control screen. Depending on the area where the user is located in the cabin, the display device is controlled to display the first interface, including: when a user is present in the driver's seat area, controlling a first area of ​​the central control screen to display interface elements related to vehicle control and navigation among the multiple interface elements, the first area being the display area of ​​the central control screen closest to the driver's seat area; and / or, when a user is present in the passenger seat area, controlling a second area of ​​the central control screen to display interface elements related to entertainment and social interaction among the multiple interface elements, the second area being the display area of ​​the central control screen closest to the passenger seat area.

[0019] Based on the above technical solution, interface elements related to vehicle control and navigation can be displayed in the display area near the driver's seat, and / or interface elements related to entertainment and social networking can be displayed in the display area near the passenger seat. This allows users in the driver's seat to easily view interface elements related to vehicle control and navigation on the first screen, and also allows users in the passenger seat to easily view interface elements related to entertainment and social networking on the first screen.

[0020] In some possible implementations, the method further includes: when a user is present in the driver's area, controlling the first area of ​​the central control screen to display interface elements related to faults and prompts among the multiple interface elements.

[0021] In some possible implementations, the display device is controlled to display the first interface according to the area where the user is located in the cabin, including: when the vehicle is in a fault state, controlling the first area to display the atomic function corresponding to the fault state in an expanded display state; controlling the atomic function corresponding to the fault state to switch from a minimized display state to an expanded display state.

[0022] In some possible implementations, the display device is controlled to display the first interface according to the area where the user is located in the cabin, including: when the vehicle needs to provide certain prompts to the user, controlling the display area near the user to display the atomic function corresponding to the prompt in an expanded display state; and controlling the atomic function corresponding to the prompt to switch from a minimized display state to an expanded display state.

[0023] For example, when the vehicle needs to remind the passenger in the front passenger area to fasten their seat belt, the second area can be controlled to display the atomic function corresponding to the seat belt in the unfolded display state.

[0024] In conjunction with the first aspect, in some implementations of the first aspect, the first input is the user's input, and the display device is controlled to display the first interface according to the area where the user is located in the cockpit, including: when the user is located in a third area in the cockpit, the display device is controlled to display a three-dimensional display of the plurality of interface elements; wherein, in the three-dimensional display, the normal of the plane on which the plurality of interface elements are located points to the third area, or the normal of the plane on which the plurality of interface elements are located points to the user's line of sight.

[0025] Based on the above technical solution, by pointing the normal of the plane where the first interface is located to the area where the user is located or the direction of the user's line of sight in the three-dimensional display, the electronic device can dynamically adjust the normal of the plane where multiple interface elements on the first interface are in a scaled-down display state based on the user's information, which helps to further improve the user experience during human-computer interaction.

[0026] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: controlling at least some of the interface elements in the minimized display state to move in a preset direction.

[0027] Based on the above technical solution, at least some of the interface elements on the first interface that are in a minimized display state can move in a preset direction, so that the user can see a dynamic first interface, which helps to improve the user's visual experience.

[0028] In some possible implementations, when the electronic device detects that the environment is rainy or snowy, the preset directional movement can be along the direction of gravity. This can give the user the sensation of raindrops falling.

[0029] In some possible implementations, taking a vehicle as an example, when a left turn is detected, the preset directional movement could be to the right. This gives the user the feeling that the interface elements are moving in the opposite direction of the vehicle's travel due to inertia.

[0030] In some possible implementations, taking a vehicle as an example where the first interface is a 3D display, when the vehicle accelerates rapidly, the preset directional movement can move in the opposite direction of the vehicle's movement. This gives the user the feeling that the interface elements are moving in the opposite direction of the vehicle's travel due to inertia.

[0031] In conjunction with the first aspect, in some implementations of the first aspect, before controlling the display device to display the first interface, the method includes: controlling the display device to display a second interface, the second interface including a fifth set of interface elements in an expanded display state, the plurality of interface elements including the fifth set of interface elements; wherein, controlling the display device to display the first interface includes: acquiring a second input from a user on the second interface; in response to the second input, controlling the interface elements in the fifth set of interface elements to gradually shrink and move in a preset direction to display the first interface.

[0032] Based on the above technical solution, when a second user input is detected on the second interface, the set of elements on the fifth interface can be controlled to gradually change from an expanded display state to a minimized display state. This creates an animation effect for the user, transitioning from the currently displayed second interface to the first interface, thus improving the user experience during human-computer interaction.

[0033] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: responding to the second input, controlling the display device to display a voice assistant in a wake-up state on the first interface; wherein, according to the first input, controlling the interface elements in the first interface element set corresponding to the first input to switch from a minimized display state to an expanded display state includes: according to the user's first voice command, controlling the interface elements in the first interface element set corresponding to the first voice command to switch from a minimized display state to an expanded display state.

[0034] Based on the above technical solution, after displaying the first interface through the second input, the voice assistant in a woke-up state can be displayed. Through the user's voice commands, the display device can be controlled to display the set of elements of the first interface in an expanded display state.

[0035] In conjunction with the first aspect, in some implementations of the first aspect, where the first input is a second voice command, before controlling the display device to display the first interface, the method further includes: controlling the display device to display a third interface; while the display device displays the third interface, acquiring the second voice command, the second voice command including a wake-up word and the user's intent; wherein controlling the display device to display the first interface includes: in response to acquiring the wake-up word, controlling the display device to display the first interface; and controlling the interface elements in the first interface element set corresponding to the first input to switch from a minimized display state to an expanded display state according to the first input, including: according to the user's intent, controlling the interface elements in the first interface element set corresponding to the first input to switch from a minimized display state to an expanded display state.

[0036] Based on the above technical solution, when a user's voice command is detected, the first interface can be displayed based on the wake word, and based on the user's intention, the display device can be controlled to display the set of first interface elements in the expanded display state.

[0037] In some possible implementations, before controlling the display device to display the first interface in response to obtaining the wake word, the method further includes: determining that the second voice command is a voice command in a preset set of voice commands.

[0038] For example, the preset set of voice commands may include voice commands for open-domain dialogue.

[0039] For example, voice commands in open-domain dialogue can also be called open-ended casual conversation, which is a dialogue primarily aimed at chatting and entertainment. Open-ended casual conversation is an important way for voice assistants to portray user personas and express emotions, helping to maintain users' social relationships and set appropriate user expectations. Therefore, open-ended casual conversation, based on authenticity and credibility, can encourage user input, cater to user emotions, and make the conversation interesting and natural. For example, voice commands in open-domain dialogue include "Generate some pictures for me" and "Generate a text describing the scenery outside the car."

[0040] In conjunction with the first aspect, in some implementations of the first aspect, the first input is the input of one or more sensors in the vehicle, and controlling the display device to display the first interface includes: controlling the display device to display the first interface based on the input of the one or more sensors.

[0041] Based on the above technical solution, the trigger condition for displaying the first interface can also be sensor input. In this way, the first interface can be displayed when sensor input is received, without requiring user voice input.

[0042] In some possible implementations, the one or more sensors include a collision sensor, and controlling the interface elements in the first set of interface elements corresponding to the first input to switch from a zoomed-out display state to an expanded display state includes: when the collision signal collected by the collision sensor meets preset conditions, controlling one or more of the interface elements related to vehicle status self-check, interface elements related to roadside assistance, interface elements related to insurance, and interface elements related to collision accident scene photos to switch from a zoomed-out display state to an expanded display state.

[0043] In some possible implementations, the one or more sensors include a positioning sensor, and controlling the interface elements in the first set of interface elements corresponding to the first input to switch from a minimized display state to an expanded display state includes: when the data collected by the positioning sensor indicates that the electronic device is in a tourist destination, controlling one or more of the interface elements related to food in the tourist destination, the interface elements related to attractions in the tourist destination, and the interface elements related to tourist routes in the tourist destination to switch from a minimized display state to an expanded display state.

[0044] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: controlling the plurality of interface elements in a minimized display state to flash and / or change size at a preset frequency.

[0045] Based on the above technical solution, when displaying the first interface, multiple interface elements in a shrunk state can be controlled to flash and / or change size at a preset frequency. This allows the user to see the animation effect of the first interface, which helps to improve the user experience during human-computer interaction.

[0046] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: determining the first set of interface elements based on the first input and the machine learning model.

[0047] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: sending first instruction information to a cloud server based on the first input, the first instruction information being used to indicate the first input; and receiving information about the first set of interface elements sent by the cloud server.

[0048] Secondly, a human-computer interaction device is provided, the device including an acquisition unit and a control unit, wherein the control unit is used to control a display device to display a first interface, the first interface including multiple interface elements in a minimized display state, each of the multiple interface elements corresponding to a service, application or atomic function; the control unit is also used to control the interface elements in a first set of interface elements corresponding to the first input to switch from a minimized display state to an expanded display state according to a first input acquired by the acquisition unit, the multiple interface elements including the first set of interface elements.

[0049] In conjunction with the second aspect, in some implementations of the second aspect, before the control unit controls the interface elements in the first set of interface elements corresponding to the first input to switch from a minimized display state to an expanded display state, the control unit is further configured to control the interface elements in the second set of interface elements on the first interface to switch from a minimized display state to an expanded display state based on the user's usage frequency and / or usage duration of each interface element among the plurality of interface elements; and when the duration of controlling the interface elements in the second set of interface elements to be in the expanded display state continues for a first preset duration, the control unit controls the interface elements in the second set of interface elements to switch from the expanded display state to the minimized display state.

[0050] In conjunction with the second aspect, in some implementations of the second aspect, the plurality of interface elements includes a third set of interface elements, wherein the usage frequency and / or usage duration of the plurality of interface elements in the third set of interface elements meet preset conditions. Specifically, the control unit is used to: control the display device to display the application icon corresponding to each interface element in the third set of interface elements, and control the display device to display a plurality of preset shapes, wherein each preset shape corresponds one-to-one with each interface element in a fourth set of interface elements, and the fourth set of interface elements consists of the interface elements other than the third set of interface elements among the plurality of interface elements.

[0051] In conjunction with the second aspect, in some implementations of the second aspect, the preset shape is a dot, a star, or a sphere.

[0052] In conjunction with the second aspect, in some implementations of the second aspect, the control unit is specifically used to: control the display device to display the first interface according to the area where the user is located in the cockpit.

[0053] In conjunction with the second aspect, in some implementations of the second aspect, the display device is a central control screen, and the control unit is specifically used to: when a user is present in the driver's seat area, control the first area of ​​the central control screen to display interface elements related to vehicle control and navigation among the plurality of interface elements, the first area being the display area of ​​the central control screen closest to the driver's seat area; and / or, when a user is present in the passenger seat area, control the second area of ​​the central control screen to display interface elements related to entertainment and social interaction among the plurality of interface elements, the second area being the display area of ​​the central control screen closest to the passenger seat area.

[0054] In conjunction with the second aspect, in some implementations of the second aspect, the first input is the user's input, and the control unit is specifically used to: control the display device to display a three-dimensional display of the plurality of interface elements when the user is located in the third area of ​​the cockpit; wherein, in the three-dimensional display, the normal of the plane on which the plurality of interface elements are located points to the third area, or the normal of the plane on which the plurality of interface elements are located points to the user's line of sight.

[0055] In conjunction with the second aspect, in some implementations of the second aspect, the control unit is also used to control at least some of the interface elements in the scaled-down display state to move in a preset direction.

[0056] In conjunction with the second aspect, in some implementations of the second aspect, the control unit is further configured to control the display device to display a second interface before controlling the display device to display the first interface, the second interface including a fifth interface element set in an expanded display state, the plurality of interface elements including the fifth interface element set; to acquire a second input from the user on the second interface; and in response to the second input, to control the interface elements in the fifth interface element set to gradually shrink and move in a preset direction to display the first interface.

[0057] In conjunction with the second aspect, in some implementations of the second aspect, the control unit is configured to, in response to the second input, control the display device to display the voice assistant in a woke-up state on the first interface; and, according to the user's first voice command, control the interface elements in the first interface element set corresponding to the first voice command to switch from a minimized display state to an expanded display state.

[0058] In conjunction with the second aspect, in some implementations of the second aspect, the first input is a second voice command. The control unit is configured to control the display device to display a third interface before controlling the display device to display the first interface; when the display device displays the third interface, acquire the second voice command, which includes a wake-up word and the user's intent; in response to acquiring the wake-up word, control the display device to display the first interface; and according to the user's intent, control the interface elements in the first interface element set corresponding to the first input to switch from a minimized display state to an expanded display state.

[0059] In conjunction with the second aspect, in some implementations of the second aspect, the first input is the input of one or more sensors in the vehicle, and the control unit is specifically used to: control the display device to display the first interface based on the input of the one or more sensors.

[0060] In conjunction with the second aspect, in some implementations of the second aspect, the control unit is also used to control the plurality of interface elements in the reduced display state to flash at a preset frequency and / or change size.

[0061] In conjunction with the second aspect, in some implementations of the second aspect, the apparatus further includes: a determining unit, configured to determine the first set of interface elements based on the first input and the machine learning model.

[0062] In conjunction with the second aspect, in some implementations of the second aspect, the device further includes: a sending unit, configured to send first instruction information to a cloud server based on the first input, the first instruction information being used to indicate the first input; and a receiving unit, configured to receive information about the first set of interface elements sent by the cloud server.

[0063] Thirdly, this application provides a human-computer interaction device, which includes a processor and a memory, wherein the memory is used to store instructions, and the processor executes the instructions stored in the memory to cause the device to perform any of the possible methods in the first aspect.

[0064] Fourthly, this application provides a human-computer interaction system, which includes a computing platform and a display device, wherein the computing platform includes any of the possible devices in the second or third aspect.

[0065] Fifthly, this application provides an electronic device that includes any of the possible means of the second or third aspect, or includes the system described in the fourth aspect.

[0066] In conjunction with the fifth aspect, in some implementations of the fifth aspect, the electronic device is a vehicle.

[0067] In a sixth aspect, this application provides a computer program product comprising: computer program code, which, when executed on a computer, causes the computer to perform any of the possible methods described in the first aspect above.

[0068] It should be noted that the above-mentioned computer program code can be stored in whole or in part on the first storage medium, wherein the first storage medium can be packaged together with the processor or packaged separately from the processor. This application embodiment does not specifically limit this.

[0069] In a seventh aspect, this application provides a computer-readable medium storing program code that, when run on a computer, causes the computer to perform any of the possible methods described in the first aspect above.

[0070] Eighthly, this application provides a chip system including a processor for calling a computer program or computer instructions stored in a memory to cause the processor to perform any of the possible methods in the first aspect above.

[0071] In conjunction with the eighth aspect, in one possible implementation, the processor is coupled to the memory via an interface.

[0072] In conjunction with the eighth aspect, in one possible implementation, the chip system also includes a memory in which computer programs or computer instructions are stored.

[0073] Ninthly, this application provides a chip system including circuitry for performing any of the possible methods described in the first aspect above. Attached Figure Description

[0074] Figure 1 This is a functional block diagram of the vehicle provided in the embodiments of this application.

[0075] Figure 2 This is a schematic diagram of a vehicle cabin scenario provided in an embodiment of this application.

[0076] Figure 3 This is a set of GUIs provided in the embodiments of this application.

[0077] Figure 4 This is another set of GUIs provided in the embodiments of this application.

[0078] Figure 5 This is another GUI provided in the embodiments of this application.

[0079] Figure 6 This is another GUI provided in the embodiments of this application.

[0080] Figure 7 This is another set of GUIs provided in the embodiments of this application.

[0081] Figure 8 This is another set of GUIs provided in the embodiments of this application.

[0082] Figure 9 This is another set of GUIs provided in the embodiments of this application.

[0083] Figure 10 This is another GUI provided in the embodiments of this application.

[0084] Figure 11 This is another set of GUIs provided in the embodiments of this application.

[0085] Figure 12 This application provides an illustrative flowchart of a human-computer interaction method.

[0086] Figure 13 This application provides a schematic block diagram of a human-computer interaction device. Detailed Implementation

[0087] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0088] Figure 1 This is a functional block diagram of a vehicle provided in an embodiment of this application. For example... Figure 1 As shown, the vehicle 100 may include a display device 130 and a computing platform 150. The display device 130 in the cabin is mainly divided into two categories: the first is an in-vehicle display screen; the second is a projection display screen, such as a head-up display (HUD). An in-vehicle display screen is a physical display screen and an important component of the in-vehicle infotainment system. Multiple displays can be installed in the cabin, such as a digital instrument cluster display, a central control screen, a display screen in front of the front passenger (also known as the front-seat passenger), a display screen in front of the left rear passenger, and a display screen in front of the right rear passenger; even the vehicle windows can be used as displays. A head-up display, also known as a head-up display system, is mainly used to display driving information such as speed and navigation on a display device in front of the driver (such as the windshield). This reduces the driver's eye-shifting time, avoids pupil changes caused by eye-shifting, and improves driving safety and comfort. HUDs include, for example, combiner-HUD (C-HUD) systems, windshield-HUD (W-HUD) systems, and augmented reality HUD (AR-HUD) systems.

[0089] Some or all of the functions of vehicle 100 can be controlled by computing platform 150. Computing platform 150 may include processors 151 to 15n. A processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction read and execute capabilities, such as a central processing unit (CPU), microprocessor, graphics processing unit (GPU) (which can be understood as a type of microprocessor), or digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a field-programmable gate array (FPGA). In reconfigurable hardware circuits, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement some or all of the functions of the aforementioned units. Furthermore, the processor can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), tensor processing unit (TPU), deep learning processing unit (DPU), etc. In addition, the computing platform 150 may also include a memory for storing instructions. Some or all of the processors 151 to 15n can call the instructions in the memory to implement the corresponding functions.

[0090] Optionally, the structure of the vehicle 100 described above is merely illustrative. In actual applications, various components of the vehicle 100 may be added or removed as needed.

[0091] Figure 2This is a schematic diagram of a vehicle cockpit scenario provided in an embodiment of this application. The smart cockpit includes one or more in-vehicle displays (or in-vehicle screens), including but not limited to display screen 201 (or central control screen), display screen 202 (or passenger entertainment screen), display screen 203 (or driver's headrest rear screen), display screen 204 (or passenger headrest rear screen), display screen 205 (or second-row entertainment screen) mounted on the cockpit ceiling, and an instrument panel. Further, displays 201 to 205 can display a graphical user interface (GUI), which may include icons for one or more applications, and / or one or more cards. For example, Figure 1 The display device 130 shown may be one or more of displays 201 to 205. In some possible implementations, display 201 may also be a long screen extending into the passenger area. In addition, display 205 may also be a projection screen associated with a projector, which may be associated with a desktop launcher, through which applications projected by the projector onto the projection screen can be managed.

[0092] Figure 2 The cockpit can also be equipped with one or more cameras to capture images inside or outside the cockpit, such as cameras from a driver monitor system (DMS), a cabin monitor system (CMS), and a dashcam. These cameras can be the same or different cameras. In addition, one or more pressure sensors and acoustic sensors are installed in the cockpit to monitor the presence and location of users.

[0093] It should be understood that the method for controlling the display in the following embodiments is based on... Figure 2 The embodiments shown are based on a 5-seater vehicle, but are not limited to this. For example, for a 7-seater sport / suburban utility vehicle (SUV), the cabin may include a central control screen, a passenger entertainment screen, a screen behind the driver's headrest, a screen behind the passenger's headrest, entertainment screens in the left-hand area of ​​the third row, and entertainment screens in the right-hand area of ​​the third row. As another example, for a bus, the cabin may include front and rear entertainment screens; or, the cabin may include a display screen in the driver's area and an entertainment screen in the passenger area. Furthermore, the following embodiments use a left-hand drive vehicle (i.e., the driver is on the left side of the vehicle) as an example; in actual implementation, the vehicle may also be a right-hand drive vehicle (i.e., the driver is on the right side of the vehicle).

[0094] Figure 3 A set of GUIs provided in embodiments of this application is shown.

[0095] like Figure 3 As shown in (a), the vehicle can display a desktop on the display screen 201, which includes a service card 301 for the settings application, a service card 302 for the music application, a service card 303 for the vehicle's remaining battery power and remaining driving range, a service card 304 for the owner's guide application, an icon 305 for the gallery application, an icon 306 for the clock application, an icon 307 for the music application, and an icon 308 for the video application.

[0096] For example, the icons 305-308 of the application can have the same size; for example, the length of the icon can be a1.

[0097] For example, service cards 301-304 can have the same size; for instance, the length of service cards 301-304 can be b1.

[0098] like Figure 3 As shown in (b), in response to detecting a user swiping from the left side of the desktop to the right, the changes in service cards 301-304 and application icons 305-308 can be displayed on the screen.

[0099] For example, the change process includes the service cards 301-304 and the application icons 305-308 gradually shrinking and the service cards 301-304 and the application icons 305-308 moving in the direction of movement shown by the dotted lines in the figure.

[0100] For example, such as Figure 3 As shown in (b), during the process of service cards 301-304 gradually shrinking, the length of service cards 301-304 gradually shrinks from b1 to b2, and the length of application icons 305-308 gradually shrinks from a1 to a2.

[0101] In one embodiment, the movement direction of service cards 301-304 and application icons 305-308 can be preset, or it can be determined according to the distribution of users in the cabin.

[0102] For example, when a user is present in the driver's area of ​​the cabin, service cards 301, 303, and 304 related to vehicle control can be moved to the display area of ​​the display screen 201 near the driver's area, and service cards 302, application icons 305, 307, and 308 related to entertainment can be moved to the display area of ​​the display screen 201 near the passenger's area.

[0103] In one embodiment, in response to detecting a user's two-finger swipe, three-finger swipe, or entry into the negative one screen on the desktop, the change process of service cards 301-304 and application icons 305-308 can be displayed on the screen.

[0104] In one embodiment, since the service card 302 is the service card corresponding to the music application, the icon 307 and the service card 302 can be moved to the same location during the movement.

[0105] In one embodiment, since the remaining battery power and remaining driving range displayed in the service card 303 are a service under the settings application, the service card 301 and the service card 303 can be moved to the same location during the movement.

[0106] like Figure 3 As shown in (c), when the change process of service cards 301-304 and application icons 305-308 ends, a starry sky interface can be displayed on the display screen 201. This starry sky interface includes multiple interface elements, including an icon 301 for the settings application (in a minimized state), an icon 305 for the gallery application, an icon 307 for the music application, an icon 308 for the video application, and an icon 309 for the navigation application. In addition to displaying the icons of the minimized applications, the starry sky interface can also display multiple preset shapes (e.g., dots). Each dot can correspond to an application, service, or atomic function. For example, dot 304 can correspond to the driver's guide application, dot 306 can correspond to the clock application, and dot 310 can correspond to the atomic function of adjusting seat heating in the settings application.

[0107] In one embodiment, at the end of the change process, the length of the applied icon is finally reduced to a3.

[0108] The dots 310 and 301 mentioned above can correspond to the same application (e.g., the settings application), but they can correspond to different services or atomic functions. For example, icon 301 can correspond to the service of the vehicle's remaining battery power and remaining driving range, while dot 310 can correspond to the atomic function of adjusting the seat heating.

[0109] The above atomic functions can refer to a specific function in the application.

[0110] Taking the Settings app as an example, the Settings app can include multiple atomic functions. For example, atomic function 1 can be the function of selecting driving mode in the Settings app, atomic function 2 can be the function of selecting air suspension height in the Settings app, and atomic function 3 can be the function of adjusting air conditioning temperature in the Settings app.

[0111] Taking a music application as an example, a music application can include multiple atomic functions. For example, atomic function 4 can be the playback interface of a song in the music application, including playback controls, spaces for switching to the previous song, and spaces for switching to the next song. Atomic function 5 can be the display interface of the daily recommended songs in the music application.

[0112] Taking instant messaging applications as an example, an instant messaging application can include multiple atomic functions. For instance, atomic function 6 could be the chat interface with contact 'a', and atomic function 7 could be the chat interface with contact 'b'.

[0113] The above services may refer to information comprised of certain information that users care about within an application.

[0114] Taking the settings app as an example, users might be more concerned about the vehicle's remaining battery power and remaining driving range. This information constitutes Service 1 of the settings app, which can be provided by, for example... Figure 3 The service card 303 shown is displayed.

[0115] Taking shopping apps as an example, users may be more concerned about the delivery time and tracking number of the package. This information constitutes Service 2 of the shopping app. Service 2 can be displayed by a service card, such as "Pickup code: 123456; Package has arrived at xx courier station; Arrival time: xx year xx month xx day".

[0116] The above services can be customized by the user or obtained from information that the user frequently views when using the application.

[0117] The above service cards can be used to display atomic functions or services.

[0118] like Figure 3 As shown in (d), a certain interface element under the starry sky interface can be switched from a minimized display state to an expanded display state via the display screen 201. For example, 5 seconds after entering the starry sky interface, the display can switch from the minimized icon 307 to the expanded icon 307 and the expanded service card 302 corresponding to the music application (the service card includes the playback interface of a song in the music application), and the service card 302 corresponds to an atomic function of the music application.

[0119] like Figure 3As shown in (e), after displaying the expanded icon 307 and the expanded service card 302 of the music application for 3 seconds, the expanded icon 307 can be switched to a minimized state and the expanded service card 302 of the music application can be hidden. Simultaneously, the display can switch from showing the minimized icon 309 to showing the expanded icon 309 and the expanded service card 311 of the navigation application.

[0120] like Figure 3 As shown in (f), when the user's voice command "Activate nap mode" is detected, the voice assistant can respond with "A set of service cards related to nap mode has been generated for you" and display the service cards related to nap mode on the starry sky interface. For example, the starry sky interface can be displayed on the display screen 201, showing icons 312, 314, and 316 in an expanded state, as well as service cards 313, 315, and 317 in an expanded state. Icon 312 is the icon for the clock application, and service card 313 includes an atomic function for timing nap mode. Icon 314 is the icon for volume adjustment, and service card 315 includes an atomic function for selecting background sound and noise reduction modes. Icon 316 is the icon for seat adjustment, and service card 317 is the atomic function corresponding to seat adjustment.

[0121] Figure 4 Another set of GUIs provided in the embodiments of this application is shown.

[0122] like Figure 4 As shown in (a), the vehicle can display a desktop on the display screen 201, which includes a service card 301 for the settings application, a service card 302 for the music application, a service card 303 for the vehicle's remaining battery power and remaining driving range, a service card 304 for the owner's guide application, an icon 305 for the gallery application, an icon 306 for the clock application, an icon 307 for the music application, and an icon 308 for the video application.

[0123] like Figure 4 As shown in (b), in response to detecting a user's two-finger or three-finger swipe operation on the desktop, the display screen can be controlled to show the changes in service cards 301-304 and application icons 305-308 according to the area where the user is located in the cockpit.

[0124] For example, when the image captured by the camera determines that a user is present in the driver's seat area and the passenger seat area, the change process includes moving service cards 301 and 303 related to vehicle control towards the display area of ​​the screen 201 closer to the driver's seat area, and moving icons or service cards related to entertainment towards the display area of ​​the screen 201 closer to the passenger seat area. During the movement, service cards 301-304 and application icons 305-308 gradually shrink.

[0125] like Figure 4 As shown in (c), the display screen 201 includes area 1 and area 2. Area 1 includes icons 301 (settings application) and 309 (navigation application) in a minimized display state, and includes multiple dots, each of which corresponds to an application, service, or atomic function related to vehicle control or fault management. Area 2 includes icons 305 (gall application), 307 (music application), and 308 (video application) in a minimized display state, and includes multiple dots, each of which corresponds to an application, service, or atomic function related to entertainment or social networking. For example, dot 312 may correspond to an atomic function in an instant messaging application, or dot 312 may correspond to a service associated with a shopping application.

[0126] Figure 5 This illustrates another GUI provided in an embodiment of this application. When displaying a starry sky interface on the display screen 201, at least some of the multiple interface elements can be controlled to change size at a preset frequency. For example... Figure 5 As shown, after displaying the starry sky interface, during the time period from T1 to T2, the dots in the minimized display state can be minimized and then enlarged; during the time period from T3 to T4, the icon 307 in the minimized display state can be minimized and then enlarged. In this way, users can see the starry sky interface in a "breathing state," which helps to improve the user experience during human-computer interaction.

[0127] In one embodiment, during the time period from time T1 to time T2, in addition to controlling the dot in the minimized display state to shrink and then enlarge, the dot can also be controlled to blink.

[0128] The above controls include adjusting the brightness of the dot. This creates a "twinkling starlight" effect for the user, enhancing the user experience during human-computer interaction.

[0129] Figure 6Another GUI provided in this application embodiment is shown. The vehicle can display a three-dimensional display of the starry sky interface on the display screen 201. The normal of the two-dimensional plane on which the starry sky interface is located in the three-dimensional display can point to the area where the user who triggered the entry into the starry sky desktop is located, or it can point to the direction of the user's line of sight.

[0130] For example, when the system detects that a user in the driver's area is swiping with two or three fingers on the desktop, or triggering input to enter the negative one screen, the display screen 201 can be controlled to display the three-dimensional display screen. In this three-dimensional display screen, the normal direction of the plane containing the starry sky interface points towards the user in the driver's area, or towards the direction of the user's gaze. By controlling the starry sky interface to align with the user, the user in the driver's area can feel as if they are interacting face-to-face with the starry sky interface, which helps to further improve the user experience during human-computer interaction.

[0131] Figure 7 Another set of GUIs provided in the embodiments of this application is shown.

[0132] like Figure 7 As shown in (a), the vehicle can display a desktop on display screen 201. When the system detects a voice command from a user in the driver's area, "Hey A, turn on nap mode," the display screen 201 can show a desktop such as... Figure 7 The GUI shown in (b) is shown in the image.

[0133] like Figure 7 As shown in (b), in response to the voice command, the vehicle can display the voice image as multiple star points in area 3 of the display screen 201.

[0134] The process of the voice image transforming into multiple star points can be understood as the process by which the vehicle determines the icons and service cards displayed to the user in an expanded state on the starry sky interface.

[0135] like Figure 7 As shown in (c), after the voice image is transformed into multiple star points, the vehicle can display a starry sky interface on the display screen 201 and display icons 701, 703, and 705 in an expanded display state, as well as service cards 702, 704, and 706 in an expanded display state. Among them, icon 701 is the icon for the clock application, and service card 702 includes the atomic function of timing the nap mode; icon 703 is the icon for volume adjustment, and service card 704 includes the atomic function of selecting background sound and noise reduction mode; icon 705 is the icon for seat adjustment, and service card 706 is the atomic function corresponding to seat adjustment.

[0136] In one embodiment, the voice command includes the wake word "Xiao A Xiao A" and the user's intent "Activate nap mode". In response to the wake word "Xiao A Xiao A", the vehicle can display a voice image transformed into multiple star points in area 3 of the display screen 201. After the voice image is transformed into multiple star points, the vehicle can display a starry sky interface on the display screen 201. In response to the user's intent "Activate nap mode", the vehicle can display icons 701, 703, and 705 in an expanded state, as well as service cards 702, 704, and 706 in an expanded state on the display screen 201.

[0137] Figure 8 Another set of GUIs provided in the embodiments of this application is shown.

[0138] like Figure 8 As shown in (a), when a collision is detected between the vehicle and the vehicle in front, the vehicle can display a starry sky interface on the display screen 201 and control the voice assistant to enter the wake-up state. At this time, the voice assistant can issue the voice message "Developing a solution for you".

[0139] In one embodiment, detecting a collision between a vehicle and a vehicle in front includes: detecting that the collision signal collected by the collision sensor meets a preset collision condition.

[0140] like Figure 8 As shown in (b), after formulating a solution for the current scenario, the vehicle can display icons 801, 803, and 805 in an expanded state, as well as service cards 802, 804, and 806 in an expanded state on display screen 201. Icon 801 is the icon for the settings application, and service card 802 includes the detection results of the current vehicle damage status, such as "front view camera malfunction, front tire pressure normal, rear tire pressure normal...", a save log control, and a cancel control. Icon 803 is the icon for the phone application, and service card 804 includes information about the roadside assistance phone number, an immediate dial control, and a cancel control. Icon 805 is the icon for the insurance application, and service card 806 includes the phone number of the claims specialist, an immediate dial control, and a cancel control.

[0141] In this embodiment, upon detecting sensor input, the display screen can immediately switch to a starry sky interface and prompt the user that a solution for a collision scenario is being developed. After the solution is developed, multiple icons and service cards can be displayed on the display screen 201. Thus, after a vehicle collision, the user can see multiple icons and service cards related to the collision scenario on the display screen 201, helping them quickly access the functions or services that might be needed in the collision scenario, thus improving the human-computer interaction experience. This is especially helpful for novice drivers who may experience a mental blank during a collision and not know what to do. The above technical solution helps novice drivers immediately understand what to do in a collision scenario, and the multiple icons and service cards related to the collision scenario also help stabilize the user's emotions and mindset.

[0142] The above example illustrates how the display screen can be controlled to show a starry sky interface after the sensor detects the input. In this scenario, the user does not need to control the display screen to show the starry sky interface by swiping with two fingers, three fingers, or by voice input.

[0143] In one embodiment, when a user is detected entering a tourist destination A (e.g., the vehicle is registered in city B and the user is detected driving into tourist destination A), the vehicle can display a starry sky interface on display screen 201. Under this starry sky interface, one or more application icons and one or more service cards in an expanded state are displayed. For example, the one or more icons may include icons for food-related applications, and the service card associated with the food-related application icon may include a list of recommended restaurants in tourist destination A. As another example, the one or more icons may include icons for travel-related applications, and the service card associated with the travel-related application icon may include a list of recommended attractions in tourist destination A.

[0144] In one embodiment, when the vehicle is detected to switch from a locked state to an unlocked state and a user is detected opening a door, the vehicle can display a starry sky interface on the display screen 201, and display icons of one or more applications and one or more service cards in an expanded state on the starry sky interface. Optionally, the icons of one or more applications and the one or more service cards can be determined based on the atomic functions selected by the user after getting into the vehicle over a period of time (e.g., a week or a month). For example, if the user frequently sets the driving mode and seat ventilation function after getting into the vehicle over a period of time, then the icons of one or more applications may include the icon of the setting application, and the service card associated with the icon of the setting application may include the atomic function of selecting the driving mode. As another example, the one or more icons may include the icon of the seat function, and the service card associated with the icon of the seat function may include the atomic function corresponding to the seat ventilation.

[0145] Figure 9 Another set of GUIs provided in the embodiments of this application is shown.

[0146] like Figure 9 As shown in (a), when Tom, the user on the right side of the second row, triggers input to display the starry sky interface on the display screen 204, the vehicle can display the starry sky interface on the display screen 204 and control the voice assistant to be in a wake-up state. At this time, the voice assistant can issue the voice message "I am listening...". The starry sky interface includes multiple icons 901-904 in a minimized display state and multiple dots. Among them, icon 901 is the icon corresponding to the gallery application, icon 902 is the icon corresponding to the instant messaging application, icon 903 is the icon corresponding to the music application, and icon 904 is the icon corresponding to the video application.

[0147] For example, icons 901-904 could be icons for multiple applications that user Tom uses frequently.

[0148] For example, when the system detects that a user on the right side of the second row triggers input to display the starry sky interface on the display screen 204, the in-cabin camera can be controlled to collect the user's feature information (e.g., facial information). This feature information can be used to identify the user on the right side of the second row as user Tom. Based on user Tom's operations under his vehicle's in-vehicle account, the vehicle can identify several applications that user Tom has used frequently over a period of time, as well as the atomic functions or services that each of these applications has used frequently.

[0149] like Figure 9As shown in (b), a certain interface element under the starry sky interface can be switched from a minimized display state to an expanded display state via the display screen 204. For example, 5 seconds after entering the starry sky interface, the display can switch from the minimized icon 901 to the expanded icon 901, and display the service cards 905 and 906 corresponding to the gallery application in the expanded state. Service card 905 includes a service of photos related to user Tom from this day last year, and service card 906 includes a service of a photo of user Tom and user Lily.

[0150] above Figures 3 to 9 The example given is of multiple icons and service cards displayed on a starry sky interface that do not overlap; however, this embodiment is not limited to this. For instance, the multiple icons and service cards can also be displayed overlapping.

[0151] Figure 10 Another GUI provided by an embodiment of this application is shown.

[0152] like Figure 10 As shown, after formulating a solution for the current scenario, the vehicle can display an icon 801 in an expanded state and service cards 802, 804, and 806 in an expanded state, stacked together, on the display screen 201. For example, if icon 801 and service card 802 are displayed for 5 seconds without any user input (clicking service card 802), the display can switch to an icon 803 in an expanded state and service cards 804, 806, and 802 in an expanded state, stacked together.

[0153] Figure 11 Another set of GUIs provided in the embodiments of this application is shown.

[0154] like Figure 11 As shown, in the starry sky interface, in addition to controlling the icon of an application to switch from a minimized display state to an expanded display state based on the frequency and / or number of times the user uses the application, service, or atomic function, the icon of an application can also be controlled to switch from a minimized display state to an expanded display state based on fault information and / or prompt information.

[0155] For example, such as Figure 11As shown in (a), when the starry sky interface is displayed on the display screen 201, low oil pressure is detected. At this time, the vehicle can control the setting application icon to switch from a minimized display state to an expanded display state and display the expanded service card 1101. The service card 1101 includes the prompt message "Low oil pressure, there is a risk of damage to the range extender. Click to view".

[0156] For example, such as Figure 11 As shown in (b), when the vehicle is in motion and the passenger in the front seat is not wearing a seatbelt while the starry sky interface is displayed on the display screen 201, the vehicle can control the display screen 201 to switch from displaying the dot 1102 to displaying the icon of the settings application in the expanded display state and the service card 1103 in the expanded display state. The service card 1103 includes the prompt message "The passenger in the front seat is not wearing a seatbelt. For your driving safety, please fasten your seatbelt".

[0157] Optionally, if the user in the front passenger area is not detected to have fastened their seatbelt within a preset time (e.g., 10 seconds) starting from the display of the Settings app icon and Service Card 1103 in the expanded display state, the Settings app icon can be controlled to display in red and a voice announcement can be made: "The user in the front passenger area is not fastening their seatbelt. For your driving safety, please fasten your seatbelt."

[0158] Figure 12 A schematic flowchart illustrating a human-computer interaction method 1200 provided in an embodiment of this application is shown. The method 1200 can be executed by the vehicle 100; or, the method 1200 can be executed by the computing platform 120; or, the method 1200 can be executed by a processor, chip, or circuit in the computing platform. The method 1200 includes:

[0159] S1210, the control display device displays a first interface, which includes multiple interface elements in a minimized display state, and each of the multiple interface elements corresponds to a service, application or atomic function.

[0160] For example, this first interface can also be called the Starry Sky Desktop.

[0161] Optionally, the first interface can be as follows: Figure 3 The starry sky interface is shown in (c). The starry sky interface includes multiple interface elements in a minimized display state, each of which can correspond to a service, application, or atomic function.

[0162] S1220, according to the first input, control the interface elements in the first interface element set corresponding to the first input to switch from a minimized display state to an expanded display state, the plurality of interface elements including the first interface element set.

[0163] Optionally, the first interface element set includes first interface elements. Controlling the interface elements in the first interface element set corresponding to the first input to switch from a minimized display state to an expanded display state includes: controlling the display device to switch from displaying the first interface elements in the minimized display state to displaying the icon and service card of the application corresponding to the first interface elements in the expanded display state. The service card includes services or atomic functions related to the application in the expanded display state.

[0164] For example, the vehicle displays such as on display screen 201 Figure 3 The starry sky interface shown in (c) is triggered by the user's voice command "Activate nap mode". The vehicle can control the display of one or more service cards in an expanded state on this starry sky interface. For example, see reference Figure 3 As shown in (f), when the user's voice command "Start nap mode" is detected, a combination of one or more service cards, including service card 312, service card 314, and service card 316, can be displayed on the expanded screen 201. The above first interface element set may include, for example, Figure 3 The icons 312, 314, and 316 shown in (f) are in an expanded state, as are the service cards 313, 315, and 317, which are also in an expanded state.

[0165] Optionally, the arrangement of multiple interface elements in the minimized state on the first interface can be arranged according to preset rules. For example, the graphic formed by the animation effect of the multiple interface elements in the minimized state is related to the weather conditions of the current scene where the vehicle is located. If the current weather condition is rainy, the shape of the multiple interface elements in the minimized state can be raindrop-shaped. Or, if the current weather condition is snowy, the shape of the multiple interface elements in the minimized state can be snowflake-shaped.

[0166] Optionally, before controlling the interface elements in the first set of interface elements corresponding to the first input to switch from a minimized display state to an expanded display state, the method 1200 further includes: controlling the interface elements in the second set of interface elements on the first interface to switch from a minimized display state to an expanded display state based on the user's usage frequency and / or usage duration of each interface element among the plurality of interface elements; and controlling the interface elements in the second set of interface elements to switch from an expanded display state to a minimized display state when the duration of the expanded display state of the interface elements in the second set of interface elements continues for a first preset duration.

[0167] For example, such as Figure 3 As shown in (d) and (e) in the diagram, the second set of interface elements may include a music application and a navigation application. When it is detected that the user's frequency and / or duration of use of the music application and navigation application has been high over a past period, the display can switch from showing the minimized icon 307 to showing the expanded icon 307 and the expanded service card 302 corresponding to the music application in the starry sky interface. After the expanded icon 307 and service card 302 have been displayed for 3 seconds, the display can switch from showing the expanded icon 307 to showing the minimized icon 307 and hide the expanded service card 302. Simultaneously, the display can switch from showing the minimized icon 309 to showing the expanded icon 309 and the expanded service card 311 corresponding to the navigation application.

[0168] Optionally, the plurality of interface elements includes a third set of interface elements, wherein the usage frequency and / or usage duration of the plurality of interface elements in the third set of interface elements meet preset conditions. The control of the display device to display the first interface includes: controlling the display device to display the application icon corresponding to each interface element in the third set of interface elements, and controlling the display device to display a plurality of preset shapes, wherein each preset shape in the plurality of preset shapes corresponds one-to-one with each interface element in a fourth set of interface elements, wherein the fourth set of interface elements consists of the interface elements in the plurality of interface elements other than the third set of interface elements.

[0169] For example, the preset shape is a dot, a star, or a sphere.

[0170] For example, such as Figure 3 As shown in (c), the third interface element set can include a settings app, a video app, a gallery app, a music app, and a navigation app. In this starry sky interface, icons of the settings app, video app, gallery app, music app, and navigation app in a minimized state can be displayed, along with multiple dots.

[0171] For example, the preset shape is a raindrop or a snowflake.

[0172] Optionally, controlling the display device to display the first interface includes: controlling the display device to display the first interface according to the area where the user is located in the cabin.

[0173] Optionally, the display device is a central control screen. Depending on the area where the user is located in the cabin, the display device is controlled to display the first interface, including: when a user is present in the driver's seat area, the first area of ​​the central control screen is controlled to display interface elements related to vehicle control and navigation among the multiple interface elements, and the first area is the display area of ​​the central control screen closest to the driver's seat area; and / or, when a user is present in the passenger seat area, the second area of ​​the central control screen is controlled to display interface elements related to entertainment and social interaction among the multiple interface elements, and the second area is the display area of ​​the central control screen closest to the passenger seat area.

[0174] For example, such as Figure 4 As shown in (c), the first area can be area 1, and the second area can be area 2. The display screen 201 includes area 1 and area 2. Area 1 includes icons 301 (settings application) and 309 (navigation application) in a minimized state. Area 2 includes icons 305 (gall application), 307 (music application), and 308 (video application) in a minimized state. Area 2 also includes multiple dots, each corresponding to an entertainment or social application, service, or functional element. For example, dot 312 can correspond to the display interface associated with an instant messaging application. This allows for the display of vehicle control and navigation-related interface elements in the area near the driver's seat and / or entertainment and social interface elements in the area near the passenger seat. This facilitates the selection of the corresponding application, service, or functional element by the user in the driver's seat using the first interface.

[0175] Optionally, depending on the area where the user is located in the cabin, the display device is controlled to display the first interface, including: when the vehicle malfunctions, the first area is controlled to display the atomic function corresponding to the malfunction that is in an unfolded display state.

[0176] For example, such as Figure 11 As shown in (a), the fault condition can be low oil pressure. When low oil pressure is detected while the starry sky interface is displayed on display screen 201, the vehicle can control the settings application icon to switch from a minimized display state to an expanded display state and display the expanded service card 1101, which includes the prompt message "Low oil pressure, there is a risk of damage to the range extender, click to view".

[0177] Optionally, depending on the area where the user is located in the cabin, the display device is controlled to display the first interface, including: when the vehicle needs to provide the user with certain prompts, the display area near the user is controlled to display the atomic function corresponding to the prompt in an expanded display state.

[0178] For example, such as Figure 11 As shown in (b), the prompt message can be a reminder to fasten your seatbelt. When the vehicle is in motion and the passenger in the front seat is not fastened, the vehicle can control the display screen 201 to switch from displaying the dot 1102 to displaying the icon of the settings application and the service card 1103, which are both in an expanded state. The service card 1103 includes the prompt message "The passenger in the front seat is not fastening their seatbelt. For your driving safety, please fasten your seatbelt."

[0179] Optionally, the first input is user input. Depending on the area where the user is located in the cabin, the display device is controlled to display the first interface, including: when the user is located in the third area of ​​the cabin, the display device is controlled to display a three-dimensional display of the multiple interface elements; wherein, in the three-dimensional display, the normal of the plane where the multiple interface elements are located points to the third area, or the normal of the plane where the multiple interface elements are located points to the user's line of sight.

[0180] For example, the three-dimensional display screen can be as follows: Figure 6 The three-dimensional display shown is illustrated below. For example, if the first input is triggered by a user in the driver's area (e.g., voice input), the display screen 201 can be controlled to display the three-dimensional display, wherein the normal direction of the plane containing the starry sky interface in the three-dimensional display points towards the user in the driver's area, or towards the user's line of sight. In this way, by controlling the starry sky interface to align with the user, it can provide the user with a feeling of face-to-face interaction with the starry sky interface, helping to further improve the user experience during human-computer interaction.

[0181] Optionally, the method 1200 further includes: controlling at least some of the interface elements in the minimized display state to move in a preset direction. For example, such as... Figure 3 As shown in (c), dot 304 can move in a preset direction. For example, 10 seconds after entering the starry sky interface, dot 304 can move upward to the position of dot 306 and dot 306 can move downward to the position of dot 304.

[0182] For example, when the electronic device detects that the environment is rainy or snowy, the preset directional movement can be along the direction of gravity. This gives the user the sensation of raindrops falling.

[0183] For example, taking a vehicle as an electronic device, when a left turn is detected, the preset directional movement can be to the right. This gives the user the feeling that the interface elements are moving in the opposite direction of the vehicle's travel due to inertia.

[0184] For example, taking a vehicle as an electronic device and a three-dimensional display screen as an example, when the vehicle is detected to be accelerating rapidly, the preset directional movement can move in the opposite direction of the vehicle's movement. In this way, the user can feel that the interface elements are moving in the opposite direction of the vehicle's movement due to inertia.

[0185] Optionally, before controlling the display device to display the first interface, the method 1200 further includes: controlling the display device to display a second interface, the second interface including a fifth interface element set in an expanded display state, the plurality of interface elements including the fifth interface element set; wherein, controlling the display device to display the first interface includes: acquiring a second input from the user on the second interface; in response to the second input, controlling the interface elements in the fifth interface element set to gradually shrink and move in a preset direction to display the first interface.

[0186] For example, the second interface can be as follows: Figure 3 The desktop shown in (a) allows the second input to be a user swiping from left to right on the desktop. The desktop includes service cards 301-304 in an expanded state and application icons 305-308. In response to detecting a user swiping from left to right on the desktop, the display screen can show the changing process of service cards 301-304 and application icons 305-308. This changing process includes the service cards 301-304 and application icons 305-308 gradually shrinking and moving in the direction shown by the dotted lines, ultimately forming the starry sky interface.

[0187] Optionally, the method 1200 further includes: responding to the second input, controlling the display device to display the voice assistant in a wake-up state on the first interface; wherein, according to the first input, controlling the interface elements in the first interface element set corresponding to the first input to switch from a minimized display state to an expanded display state includes: according to the user's first voice command, controlling the interface elements in the first interface element set corresponding to the first voice command to switch from a minimized display state to an expanded display state.

[0188] For example, such as Figure 3 As shown in (c), in response to detecting a user swiping from the left to the right on the desktop, the voice assistant can be activated. At this time, the voice assistant is activated and its current status can be displayed as "I'm listening...". For example, the first voice command can be as follows: Figure 3 The voice command "Start nap mode" is shown in (f) in the image.

[0189] Optionally, the first input is a second voice command. Before controlling the display device to display the first interface, the method further includes: controlling the display device to display a third interface; when the display device displays the third interface, acquiring the second voice command, the second voice command including a wake-up word and the user's intent; wherein controlling the display device to display the first interface includes: in response to acquiring the wake-up word, controlling the display device to display the first interface; according to the first input, controlling the interface elements in the first interface element set corresponding to the first input to switch from a minimized display state to an expanded display state includes: according to the user's intent, controlling the interface elements in the first interface element set corresponding to the first input to switch from a minimized display state to an expanded display state.

[0190] For example, the third interface can be as follows: Figure 7 As shown in (a) of the desktop, the second voice command can be the user's voice command "Xiao A Xiao A, turn on nap mode". The voice command includes the wake word "Xiao A Xiao A" and the user's intention "turn on nap mode".

[0191] Optionally, in response to obtaining the wake word and before controlling the display device to display the first interface, the method 1200 further includes: determining that the second voice command is a voice command in a preset voice command set.

[0192] Optionally, the preset set of voice commands may include voice commands for open-domain dialogue.

[0193] For example, the voice commands for this open-domain dialogue could be "Hey A, help me generate some pictures" or "Hey A, help me generate a blessing message," etc.

[0194] For example, voice commands in open-domain dialogue can also be called open-ended chat, which is a dialogue primarily aimed at chatting and entertainment. Open-ended chat is an important way for voice assistants to portray user personas and express emotions, helping to maintain users' social relationships and set appropriate user expectations. Therefore, open-ended chat, based on authenticity and credibility, can encourage user input, cater to user emotions, and make the dialogue interesting and natural.

[0195] Optionally, the preset set of voice commands may include preset task-oriented dialogues.

[0196] For example, task-oriented dialogues have a clear goal. The voice assistant obtains the necessary slots through slot filling to help the user complete the task. Task-oriented dialogues aim for effectiveness and efficiency. Users express relatively low levels of emotion.

[0197] For example, a preset task-oriented dialogue includes "Hey A, Hey A, turn on the nap mode".

[0198] For example, the preset task-oriented dialogues do not include dialogues that open car doors or windows. For instance, when a user issues a voice command "Hey A, open the window for me," it can be determined that the voice command does not belong to the preset task-oriented dialogues. In this case, the vehicle can directly execute the task of opening the window corresponding to the voice command without controlling the display device to show the starry sky interface.

[0199] Optionally, the first input is the input of one or more sensors in the vehicle, and controlling the display device to display the first interface includes: controlling the display device to display the first interface based on the input of the one or more sensors.

[0200] For example, such as Figure 8 As shown, the multiple sensors include a collision sensor, and the first set of interface elements may include atomic functions corresponding to vehicle damage self-check, calling roadside assistance in a telephone application, and calling a claims specialist in an insurance application.

[0201] Optionally, the method 1200 further includes: controlling the plurality of interface elements in the minimized display state to flash and / or change size at a preset frequency.

[0202] For example, such as Figure 5 As shown, after displaying the starry sky interface, during the time period from T1 to T2, the dot in the minimized state can be minimized and then enlarged; during the time period from T3 to T4, the icon 307 in the minimized display state can be minimized and then enlarged. Optionally, in addition to controlling the minimized dot to shrink and then enlarge, the dot can also be controlled to blink.

[0203] Optionally, the method 1200 further includes: determining the first set of interface elements based on the first input and the machine learning model.

[0204] In this embodiment, the process of determining the first set of interface elements can be implemented on the vehicle side. For example, the machine learning model can be a large language model (LLM).

[0205] For example, the input to this LLM can be the user's intent in a voice command, such as "turn on nap mode," and the LLM can then output information about applications, services, or atomic functions related to that intent. As another example, the input to this LLM can be text content converted from data collected by sensors.

[0206] For example, when the collision signal collected by the vehicle's collision sensor meets the preset conditions, the collision signal can be converted into the text content "vehicle collision has occurred" and the text content can be input into the LLM. The LLM can then output information about applications, services, or atomic functions related to the text content.

[0207] For example, when a vehicle detects that a user is driving into a tourist destination A, it can combine map information and data collected by positioning sensors to convert it into text content "Recommendations of attractions and restaurants in tourist destination A" and input this text content into an LLM. The LLM can then output information about applications, services, or atomic functions related to the recommendations of attractions and restaurants in tourist destination A.

[0208] Optionally, the method 1200 further includes: sending first indication information to a cloud server based on the first input, the first indication information being used to indicate the first input; and receiving information about the first set of interface elements sent by the cloud server.

[0209] In this embodiment, the process of determining the first set of interface elements can be implemented on a cloud server. The process implemented on the cloud server can be referenced from the process implemented in the electronic device described above, and will not be repeated here.

[0210] Figure 13 A schematic block diagram of a human-computer interaction device 1300 provided in an embodiment of this application is shown. The device 1300 includes an acquisition unit 1310 and a control unit 1320. The control unit 1320 is configured to control a display device to display a first interface, the first interface including multiple interface elements in a minimized display state, each of the multiple interface elements corresponding to a service, application, or atomic function. The control unit 1320 is also configured to, based on a first input acquired by the acquisition unit 1310, control the interface elements in a first set of interface elements corresponding to the first input to switch from a minimized display state to an expanded display state, the multiple interface elements including the first set of interface elements.

[0211] Optionally, before the control unit 1320 controls the interface elements in the first set of interface elements corresponding to the first input to switch from a minimized display state to an expanded display state, the control unit 1320 is further configured to control the interface elements in the second set of interface elements on the first interface to switch from a minimized display state to an expanded display state based on the user's usage frequency and / or usage duration of each interface element among the plurality of interface elements; and when the duration of the expanded display state of the interface elements in the second set of interface elements continues for a first preset duration, the control unit 1320 controls the interface elements in the second set of interface elements to switch from an expanded display state to a minimized display state.

[0212] Optionally, the plurality of interface elements includes a third set of interface elements, wherein the usage frequency and / or usage duration of the plurality of interface elements in the third set of interface elements meet preset conditions. Specifically, the control unit 1320 is used to: control the display device to display the application icon corresponding to each interface element in the third set of interface elements, and control the display device to display a plurality of preset shapes, wherein each preset shape corresponds one-to-one with each interface element in a fourth set of interface elements, and the fourth set of interface elements consists of the interface elements other than the third set of interface elements among the plurality of interface elements.

[0213] Optionally, the preset shape can be a dot, a star, or a sphere.

[0214] Optionally, the control unit 1320 is specifically used to control the display device to display the first interface according to the area where the user is located in the cabin.

[0215] Optionally, the display device is a central control screen, and the control unit 1320 is specifically used to: when a user is present in the driver's seat area, control the first area of ​​the central control screen to display interface elements related to vehicle control and navigation among the multiple interface elements, the first area being the display area of ​​the central control screen closest to the driver's seat area; and / or, when a user is present in the passenger seat area, control the second area of ​​the central control screen to display interface elements related to entertainment and social interaction among the multiple interface elements, the second area being the display area of ​​the central control screen closest to the passenger seat area.

[0216] Optionally, the first input is the user's input, and the control unit 1320 is specifically used to: when the user is in the third area inside the cockpit, control the display device to display a three-dimensional display of the plurality of interface elements; wherein, in the three-dimensional display, the normal of the plane on which the plurality of interface elements are located points to the third area, or the normal of the plane on which the plurality of interface elements are located points to the user's line of sight.

[0217] Optionally, the control unit 1320 is also used to control at least some of the interface elements in the scaled-down display state to move in a preset direction.

[0218] Optionally, the control unit 1320 is further configured to control the display device to display a second interface before controlling the display device to display the first interface, the second interface including a fifth interface element set in an expanded display state, the plurality of interface elements including the fifth interface element set; to obtain a second input from the user on the second interface; and in response to the second input, to control the interface elements in the fifth interface element set to gradually shrink and move in a preset direction to display the first interface.

[0219] Optionally, the control unit 1320 is configured to, in response to the second input, control the display device to display the voice assistant in a woke-up state on the first interface; and, according to the user's first voice command, control the interface elements in the first interface element set corresponding to the first voice command to switch from a minimized display state to an expanded display state.

[0220] Optionally, the first input is a second voice command. The control unit 1320 is configured to control the display device to display a third interface before controlling the display device to display the first interface; when the display device displays the third interface, acquire the second voice command, which includes a wake-up word and the user's intent; in response to acquiring the wake-up word, control the display device to display the first interface; and according to the user's intent, control the interface elements in the first interface element set corresponding to the first input to switch from a minimized display state to an expanded display state.

[0221] Optionally, the first input is the input of one or more sensors in the vehicle, and the control unit 1320 is specifically used to control the display device to display the first interface based on the input of the one or more sensors.

[0222] Optionally, the control unit 1320 is also used to control the plurality of interface elements in the reduced display state to flash at a preset frequency and / or change size.

[0223] In conjunction with the second aspect, in some implementations of the second aspect, the apparatus further includes: a determining unit, configured to determine the first set of interface elements based on the first input and the machine learning model.

[0224] Optionally, the device further includes: a sending unit, configured to send first instruction information to a cloud server based on the first input, the first instruction information being used to indicate the first input; and a receiving unit, configured to receive information about the first set of interface elements sent by the cloud server.

[0225] It should be understood that the division of units in the above device is only a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, the units in the device can be implemented by a processor calling software; for example, the device includes a processor connected to memory, which stores instructions. The processor calls the instructions stored in memory to implement any of the above methods or to implement the functions of each unit in the device. The processor can be, for example, a general-purpose processor, such as a CPU or microprocessor, and the memory can be internal or external to the device. Alternatively, the units in the device can be implemented as hardware circuits. The functions of some or all units can be implemented through the design of the hardware circuits, which can be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all units are implemented through the design of the logical relationships between the components within the circuit. In another implementation, the hardware circuit can be implemented using a PLD, such as an FPGA, which can include a large number of logic gates. The connection relationships between the logic gates are configured through configuration files, thereby implementing the functions of some or all units. All units of the above devices can be implemented entirely through processor calling software, or entirely through hardware circuits, or partially through processor calling software with the remaining parts implemented through hardware circuits.

[0226] In this application embodiment, a processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a CPU, microprocessor, GPU, or DSP. In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented as an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the processor loading instructions to implement the functions of some or all of the above units. Furthermore, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, or DPU.

[0227] As can be seen, each unit in the above device can be one or more processors (or processing circuits) configured to implement the above methods, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0228] Furthermore, the units in the above devices can be integrated in whole or in part, or they can be implemented independently. In one implementation, these units are integrated together as a System-on-a-Chip (SoC). The SoC may include at least one processor for implementing any of the above methods or implementing the functions of the units in the device. The at least one processor may be of different types, such as CPU and FPGA, CPU and AI processor, CPU and GPU, etc.

[0229] This application also provides a human-computer interaction device, which includes a processing unit and a storage unit. The storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit to enable the device to perform the methods or steps described in the above embodiments.

[0230] Optionally, if the human-machine interface device is located in a vehicle, the aforementioned processing unit may be... Figure 1 The processors shown are 151-15n.

[0231] This application also provides a human-computer interaction system, which may include a computing platform and a display screen. The computing platform may include the aforementioned human-computer interaction device 1300.

[0232] This application also provides a vehicle that may include the human-computer interaction device 1300 or the human-computer interaction system described above.

[0233] This application also provides a computer program product, which includes computer program code that, when run on a computer, causes the computer to perform the methods described in the above embodiments.

[0234] This application also provides a computer-readable medium storing program code that, when run on a computer, causes the computer to perform the methods described in the above embodiments.

[0235] This application also provides a chip, which includes a circuit for performing the methods described in the above embodiments.

[0236] In implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software. The method disclosed in the embodiments of this application can be directly implemented by a hardware processor, or by a combination of hardware and software modules within the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, power-on erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, detailed descriptions are omitted here.

[0237] It should be understood that in the embodiments of this application, the memory may include read-only memory and random access memory, and provides instructions and data to the processor.

[0238] It should also be understood that, in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0239] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0240] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0241] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0242] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0243] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0244] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0245] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be covered. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A human-computer interaction method, characterized in that, include: The control display device displays a first interface, which includes multiple interface elements in a minimized display state. Each of the multiple interface elements corresponds to a service, application, or atomic function. Based on the first input, control the interface elements in the first set of interface elements corresponding to the first input to switch from a minimized display state to an expanded display state, and control the interface elements other than the first set of interface elements to remain in a minimized display state, wherein the multiple interface elements include the first set of interface elements. The plurality of interface elements includes a third set of interface elements, wherein the usage frequency and / or usage duration of the plurality of interface elements in the third set of interface elements meet preset conditions. The control display device displays a first interface, including: The display device is controlled to display the application icon corresponding to each interface element in the third interface element set, and the display device is also controlled to display multiple preset shapes. Each preset shape corresponds one-to-one with each interface element in the fourth interface element set. The fourth interface element set consists of interface elements other than the third interface element set. The preset shapes are dots, stars, or spheres.

2. The method according to claim 1, characterized in that, Before the interface elements in the first set of interface elements corresponding to the first input are switched from a minimized display state to an expanded display state, the method further includes: Based on the frequency and / or duration of user use of each of the multiple interface elements, control the interface elements in the second set of interface elements on the first interface to switch from the minimized display state to the expanded display state. When the duration of the expanded display state of the interface elements in the second set of interface elements is maintained for a first preset duration, the interface elements in the second set of interface elements are controlled to switch from the expanded display state to the minimized display state.

3. The method according to claim 1 or 2, characterized in that, The control display device displays a first interface, including: The display device is controlled to display the first interface based on the area where the user is located in the cabin.

4. The method according to claim 3, characterized in that, The display device is a central control screen. Controlling the display device to show the first interface based on the user's location within the cabin includes: When a user is present in the driver's area, the first area of ​​the central control screen is controlled to display interface elements related to vehicle control and navigation from among the plurality of interface elements; the first area is the display area of ​​the central control screen closest to the driver's area; and / or When a user is present in the passenger seat area, the second area of ​​the central control screen is controlled to display the interface elements related to entertainment and social networking among the multiple interface elements. The second area is the display area of ​​the central control screen that is close to the passenger seat area.

5. The method according to claim 3, characterized in that, The first input is user input, and the step of controlling the display device to display the first interface based on the user's location within the cockpit includes: When the user is located in the third area of ​​the cockpit, the display device is controlled to display a three-dimensional display of the multiple interface elements; In the three-dimensional display, the normal to the plane on which the plurality of interface elements are located points to the third region, or the normal to the plane on which the plurality of interface elements are located points to the user's line of sight.

6. The method according to any one of claims 1, 2, 4 and 5, characterized in that, The method further includes: Control at least some of the interface elements in the minimized display state to move in a preset direction.

7. The method according to any one of claims 1, 2, 4 and 5, characterized in that, Before the control display device displays the first interface, the method includes: The display device is controlled to display a second interface, the second interface including a fifth interface element set in an expanded display state, and the plurality of interface elements including the fifth interface element set; The control display device displays a first interface, including: Obtain the user's second input on the second interface; In response to the second input, the interface elements in the fifth set of interface elements are controlled to gradually shrink and move in a preset direction to display the first interface.

8. The method according to claim 7, characterized in that, The method further includes: In response to the second input, the display device is controlled to display the voice assistant in a wake-up state on the first interface; The step of controlling the interface elements in the first set of interface elements corresponding to the first input to switch from a minimized display state to an expanded display state according to the first input includes: Based on the user's first voice command, control the interface elements in the first set of interface elements corresponding to the first voice command to switch from a minimized display state to an expanded display state.

9. The method according to any one of claims 1, 2, 4, 5 and 8, characterized in that, The method further includes the following steps: The first input is a second voice command. Before the control display device displays the first interface, the method includes: Control the display device to display the third interface; When the display device displays the third interface, the second voice command is acquired, the second voice command including a wake word and the user's intent; The control display device displays a first interface, including: In response to receiving the wake word, the display device is controlled to display the first interface; The step of controlling the interface elements in the first set of interface elements corresponding to the first input to switch from a minimized display state to an expanded display state according to the first input includes: Based on the user's intention, the interface elements in the first set of interface elements corresponding to the first input are controlled to switch from a minimized display state to an expanded display state.

10. The method according to any one of claims 1, 2, 4, 5 and 8, characterized in that, The first input is the input from one or more sensors in the vehicle, and the control display device displays a first interface, including: Based on the input from one or more sensors, the display device is controlled to display the first interface.

11. The method according to any one of claims 1, 2, 4, 5 and 8, characterized in that, The method further includes: Control the multiple interface elements in the minimized display state to flash and / or change size at a preset frequency.

12. The method according to any one of claims 1, 2, 4, 5 and 8, characterized in that, The method further includes: Based on the first input and the machine learning model, the first set of interface elements is determined.

13. The method according to any one of claims 1, 2, 4, 5 and 8, characterized in that, The method further includes: Based on the first input, a first instruction message is sent to the cloud server, the first instruction message being used to indicate the first input; Receive information about the first set of interface elements sent by the cloud server.

14. A human-computer interaction device, characterized in that, It includes an acquisition unit and a control unit, wherein, The control unit is used to control the display device to display a first interface, the first interface including multiple interface elements in a minimized display state, each of the multiple interface elements corresponding to a service, application or atomic function; The control unit is further configured to control the interface elements in the first interface element set corresponding to the first input to switch from a minimized display state to an expanded display state according to the first input obtained by the acquisition unit, and to control the interface elements other than the first interface element set among the plurality of interface elements to remain in a minimized display state, wherein the plurality of interface elements includes the first interface element set. The plurality of interface elements includes a third set of interface elements, wherein the usage frequency and / or usage duration of the plurality of interface elements in the third set of interface elements meet preset conditions. Specifically, the control unit is used for: The display device is controlled to display the application icon corresponding to each interface element in the third interface element set, and the display device is also controlled to display multiple preset shapes. Each preset shape corresponds one-to-one with each interface element in the fourth interface element set. The fourth interface element set consists of interface elements other than the third interface element set. The preset shapes are dots, stars, or spheres.

15. The apparatus according to claim 14, characterized in that, Before the control unit controls the interface elements in the first set of interface elements corresponding to the first input to switch from a minimized display state to an expanded display state, ... The control unit is also configured to control the interface elements in the second set of interface elements on the first interface to switch from the minimized display state to the expanded display state based on the frequency and / or duration of the user's use of each interface element among the plurality of interface elements. When the duration of the expanded display state of the interface elements in the second set of interface elements is maintained for a first preset duration, the interface elements in the second set of interface elements are controlled to switch from the expanded display state to the minimized display state.

16. The apparatus according to claim 14 or 15, characterized in that, The control unit is specifically used for: The display device is controlled to display the first interface based on the area where the user is located in the cabin.

17. The apparatus according to claim 16, characterized in that, The display device is a central control screen, and the control unit is specifically used for: When a user is present in the driver's area, the first area of ​​the central control screen is controlled to display interface elements related to vehicle control and navigation among the multiple interface elements. The first area is the display area of ​​the central control screen that is close to the driver's area. and / or When a user is present in the passenger seat area, the second area of ​​the central control screen is controlled to display the interface elements related to entertainment and social networking among the multiple interface elements. The second area is the display area of ​​the central control screen that is close to the passenger seat area.

18. The apparatus according to claim 16, characterized in that, The first input is user input, and the control unit is specifically used for: When the user is located in the third area of ​​the cockpit, the display device is controlled to display a three-dimensional display of the multiple interface elements; In the three-dimensional display, the normal to the plane on which the plurality of interface elements are located points to the third region, or the normal to the plane on which the plurality of interface elements are located points to the user's line of sight.

19. The apparatus according to any one of claims 14, 15, 17, and 18, characterized in that, The control unit is also used to control at least some of the interface elements in the multiple interface elements in the minimized display state to move in a preset direction.

20. The apparatus according to any one of claims 14, 15, 17, and 18, characterized in that, The control unit is further configured to control the display device to display a second interface before controlling the display device to display the first interface, the second interface including a fifth interface element set in an expanded display state, and the plurality of interface elements including the fifth interface element set; Obtain the user's second input on the second interface; In response to the second input, the interface elements in the fifth set of interface elements are controlled to gradually shrink and move in a preset direction to display the first interface.

21. The apparatus according to claim 20, characterized in that, The control unit is configured to respond to the second input and control the display device to display the voice assistant in a wake-up state on the first interface. Based on the user's first voice command, control the interface elements in the first set of interface elements corresponding to the first voice command to switch from a minimized display state to an expanded display state.

22. The apparatus according to any one of claims 14, 15, 17, 18, and 21, characterized in that, The first input is the second voice command. The control unit is used to control the display device to display a third interface before controlling the display device to display the first interface; When the display device displays the third interface, the second voice command is acquired, the second voice command including a wake word and the user's intent; In response to receiving the wake word, the display device is controlled to display the first interface; Based on the user's intention, the interface elements in the first set of interface elements corresponding to the first input are controlled to switch from a minimized display state to an expanded display state.

23. The apparatus according to any one of claims 14, 15, 17, 18, and 21, characterized in that, The first input is the input from one or more sensors in the vehicle, and the control unit is specifically used for: Based on the input from one or more sensors, the display device is controlled to display the first interface.

24. The apparatus according to any one of claims 14, 15, 17, 18, and 21, characterized in that, The control unit is also used to control the plurality of interface elements in the reduced display state to flash at a preset frequency and / or change size.

25. The apparatus according to any one of claims 14, 15, 17, 18, and 21, characterized in that, The device further includes: The determining unit is used to determine the first set of interface elements based on the first input and the machine learning model.

26. The apparatus according to any one of claims 14, 15, 17, 18, and 21, characterized in that, The device further includes: The sending unit is configured to send first indication information to the cloud server based on the first input, wherein the first indication information is used to indicate the first input; The receiving unit is used to receive information about the first set of interface elements sent by the cloud server.

27. A human-computer interaction device, characterized in that, include: Memory, used to store computer programs; A processor for executing a computer program stored in the memory to cause the apparatus to perform the method as described in any one of claims 1 to 13.

28. A human-computer interaction system, characterized in that, It includes a display device and a computing platform, wherein the computing platform includes the device as described in any one of claims 14 to 27.

29. An electronic device, characterized in that, Includes the apparatus as described in any one of claims 14 to 27, or the system as described in claim 28.

30. The electronic device according to claim 29, characterized in that, The electronic device is a vehicle.

31. A computer-readable storage medium, characterized in that, It stores instructions that, when executed by a processor, cause the processor to implement the method as described in any one of claims 1 to 13.

32. A computer program product, characterized in that, The computer program product includes computer program code that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 13.

33. A chip, characterized in that, The chip includes circuitry for performing the method as described in any one of claims 1 to 13.

Images