Human-computer interaction method and device and electronic equipment

Abstract

The invention provides a man-machine interaction method, a man-machine interaction device and electronic equipment, which can be applied to the field of intelligent cabins. The method comprises the steps that a display device is controlled to display a first interface, the first interface comprises a plurality of interface elements in a reduced display state, and each interface element in the interface elements corresponds to a service, an application or an atomic function; according to the first input, the interface elements in the first interface element set corresponding to the first input are controlled to be switched from the reduced display state to the unfolded display state, and the multiple interface elements comprise the first interface element set. The method can be applied to an intelligent automobile or an electric automobile, and the user experience during man-machine interaction can be improved.

Description

Technical Field

[0001] The present application relates to the field of human-computer interaction, and more specifically, to a human-computer interaction method, device, and electronic device. Background Art

[0002] Users are increasingly interacting with electronic devices. In current interaction methods, electronic devices display an interaction model (e.g., a voice assistant) in a fixed area. Because the interaction model is fixed and can only interact with users through voice or text, the user experience when interacting with the electronic device is poor. Summary of the Invention

[0003] The present application provides a human-computer interaction method, device, and electronic device, which help to improve the user experience during human-computer interaction.

[0004] In a first aspect, a human-computer interaction method is provided, which includes: controlling a display device to display a first interface, the first interface including multiple interface elements in a reduced display state, each of the multiple interface elements corresponding to a service, application or atomic function; according to a first input, controlling the interface elements in a first interface element set corresponding to the first input to switch from a reduced display state to an expanded display state, the multiple interface elements including the first interface element set.

[0005] Based on the above technical solution, by displaying multiple interface elements in a reduced display state, the user can be given the feeling of being watched by multiple services, applications, or atomic functions in the electronic device. After obtaining a first input, the expanded display state of the interface element corresponding to the first input can be displayed, and the user can be provided with the application, service, or atomic function in the expanded display state that the user expected to see under the first input, thereby facilitating user operation and helping to improve the user experience during human-computer interaction.

[0006] In some possible implementations, the first interface may also be called a starry sky desktop.

[0007] In some possible implementations, the first interface element set includes a first interface element, and controlling the interface elements in the first interface element set corresponding to the first input to switch from a reduced display state to an expanded display state includes: controlling the display device to switch from displaying the first interface element in a reduced display state to displaying the icon and service card of the application corresponding to the first interface element in an expanded display state, wherein the service card includes services or atomic functions related to the application in the expanded display state.

[0008] In some possible implementations, the arrangement of the multiple interface elements in the zoomed-out display state on the first interface may be arranged according to a preset rule.

[0009] For example, the graphics formed by the animation effects of multiple interface elements in the zoomed-out state may be related to the weather conditions in the scene the vehicle is currently in. For example, if the weather condition in the current scene is rainy, the shapes of the multiple interface elements in the zoomed-out state may be raindrop-shaped. For another example, if the weather condition in the current scene is snowy, the shapes of the multiple interface elements in the zoomed-out state may be snowflake-shaped.

[0010] In combination with the first aspect, in certain implementations of the first aspect, before controlling the interface elements in the first interface element set corresponding to the first input to switch from a reduced display state to an expanded display state, the method also includes: controlling the interface elements in the second interface element set on the first interface to switch from the reduced display state to an expanded display state based on the user's frequency of use and / or duration of use of each of the multiple interface elements; and controlling the interface elements in the second interface element set to switch from the expanded display state to the reduced display state when the duration for which the interface elements in the second interface element set are controlled to be in the expanded display state continues for a first preset duration.

[0011] Based on the above technical solution, the services, applications, or atomic functions frequently used by the user can be switched to an expanded display state on the first interface for a first preset duration, so that the user can easily view the services, applications, or atomic functions they frequently use through the first interface. When the user does not have a clear task-based dialogue requirement, they can select the services, applications, or atomic functions they frequently use to quickly enter the relevant interface.

[0012] In combination with the first aspect, in certain implementations of the first aspect, the multiple interface elements include a third interface element set, and the usage frequency and / or usage duration of the multiple interface elements in the third interface element set meet preset conditions, wherein controlling the display device to display the first interface includes: controlling the display device to display the icon of the application corresponding to each interface element in the third interface element set, and controlling the display device to display multiple preset shapes, each preset shape in the multiple preset shapes corresponds one-to-one to each interface element in a fourth interface element set, and the fourth interface element set is the interface elements in the multiple interface elements other than the third interface element set.

[0013] Based on the above technical solution, icons of applications, services or atomic functions that are frequently used by users and multiple preset shapes can be displayed in a reduced display state on the first interface, so that users can view the icons of applications they frequently use on the first interface.

[0014] In combination with the first aspect, in some implementations of the first aspect, the preset shape is a dot shape, a star shape, or a sphere shape.

[0015] In some possible implementations, the preset shape is a raindrop shape or a snowflake shape.

[0016] In combination with the first aspect, in some implementations of the first aspect, controlling the display device to display the first interface includes: controlling the display device to display the first interface according to the area where the user is located in the cabin.

[0017] Based on the above technical solution, the display device can be controlled to display the first interface based on the area where the user is located in the cabin. In this way, the first interface can be displayed in combination with the distribution of users in the cabin, which helps to further improve the user experience during human-computer interaction.

[0018] In combination with the first aspect, in certain implementations of the first aspect, the display device is a central control screen, and the display device is controlled to display the first interface according to the area where the user is located in the cockpit, including: when there is a user in the main driving area, controlling the first area in the central control screen to display interface elements related to vehicle control and navigation among the multiple interface elements, and the first area is the display area of ​​the central control screen close to the main driving area; and / or, when there is a user in the co-pilot area, controlling the second area in the central control screen to display interface elements related to entertainment and social categories among the multiple interface elements, and the second area is the display area of ​​the central control screen close to the co-pilot area.

[0019] Based on the above technical solution, interface elements related to vehicle control and navigation can be displayed in the display area near the main driver's seat, and / or interface elements related to entertainment and social networking can be displayed in the display area near the passenger seat. In this way, it is convenient for the user in the main driver's seat to view interface elements related to vehicle control and navigation in the first interface, and it is also convenient for the user in the passenger seat to view interface elements related to entertainment and social networking in the first interface.

[0020] In some possible implementations, the method further includes: when there is a user in the main driving area, controlling the first area in the central control screen to display interface elements related to the fault type and the prompt type among the multiple interface elements.

[0021] In some possible implementations, the display device is controlled to display the first interface according to the area where the user is located in the cockpit, including: when a fault state occurs in the vehicle, controlling the first area to display the atomic function corresponding to the fault state in an expanded display state; controlling the atomic function corresponding to the fault state to switch from a reduced display state to an expanded display state.

[0022] In some possible implementations, the display device is controlled to display the first interface according to the area where the user is located in the cabin, including: when the vehicle needs to prompt the user with certain prompt information, controlling the display area close to the user to display the atomic function corresponding to the prompt information in an expanded display state; controlling the atomic function corresponding to the prompt information to switch from a reduced display state to an expanded display state.

[0023] For example, when the vehicle needs to remind the user in the passenger seat to fasten the seat belt, the second area can be controlled to display the atomic function corresponding to the seat belt in an expanded display state.

[0024] In combination with the first aspect, in certain implementations of the first aspect, the first input is the user's input, and the display device is controlled to display the first interface according to the area in which the user is located in the cabin, including: when the user is located in a third area in the cabin, the display device is controlled to display a three-dimensional display screen of the multiple interface elements; wherein, in the three-dimensional display screen, the normal of the plane in which the multiple interface elements are located points to the third area, or the normal of the plane in which the multiple interface elements are located points to the direction of the user's line of sight.

[0025] Based on the above technical solution, by pointing the normal of the plane where the first interface is located to the area where the user is located or the direction of the user's line of sight in the three-dimensional display screen, the electronic device can dynamically adjust the normal of the plane where multiple interface elements in the reduced display state on the first interface are located based on the user's information, which helps to further improve the user experience during human-computer interaction.

[0026] In combination with the first aspect, in some implementations of the first aspect, the method further includes: controlling at least some of the multiple interface elements in the zoomed-out display state to move in a preset direction.

[0027] Based on the above technical solution, at least part of the interface elements in the zoomed-out state on the first interface can be moved in a preset direction, so that the user can see the dynamic first interface, which helps to improve the user's visual experience.

[0028] In some possible implementations, when it is detected that the electronic device is located in a rainy or snowy environment, the preset direction of movement may be movement along the direction of gravity, so as to give the user a feeling of falling raindrops.

[0029] In some possible implementations, for example, if the electronic device is a vehicle, when the vehicle is detected to be turning left, the preset direction of movement may be rightward. This can give the user the feeling that the interface element is moving in the opposite direction of the vehicle's travel due to inertia.

[0030] In some possible implementations, for example, where the electronic device is a vehicle and the first interface is a three-dimensional display, upon detecting a sudden acceleration of the vehicle, the predetermined direction of movement may be in the opposite direction of the vehicle's movement. This can give the user the impression that the interface element is moving in the opposite direction of the vehicle's travel due to inertia.

[0031] In combination with the first aspect, in certain implementations of the first aspect, before controlling the display device to display the first interface, the method includes: controlling the display device to display the second interface, the second interface including a fifth interface element set in an expanded display state, and the multiple interface elements including the fifth interface element set; wherein, controlling the display device to display the first interface includes: obtaining the user's second input on the second interface; in response to the second input, controlling the interface elements in the fifth interface element set to gradually shrink and move in a preset direction to display the first interface.

[0032] Based on the above technical solution, when the user's second input is detected on the second interface, the fifth interface element set on the second interface can be controlled to gradually change from an expanded display state to a reduced display state. This can create an animation effect for the user when switching from the currently displayed second interface to the first interface, helping to improve the user experience during human-computer interaction.

[0033] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: in response to the second input, controlling the display device to display the voice assistant in the awakened state on the first interface; wherein, according to the first input, controlling the interface elements in the first interface element set corresponding to the first input to switch from a reduced display state to an expanded display state, including: according to the user's first voice command, controlling the interface elements in the first interface element set corresponding to the first voice command to switch from a reduced display state to an expanded display state.

[0034] Based on the above technical solution, after the first interface is displayed through the second input, the voice assistant in the awakened state can be displayed. Through the user's voice command, the display device can be controlled to display the first interface element set in the expanded display state.

[0035] In combination with the first aspect, in certain implementations of the first aspect, the first input is a second voice instruction, and before controlling the display device to display the first interface, the method also includes: controlling the display device to display the third interface; when the display device displays the third interface, obtaining the second voice instruction, the second voice instruction including a wake-up word and the user's intention; wherein, controlling the display device to display the first interface includes: in response to obtaining the wake-up word, controlling the display device to display the first interface; according to the first input, controlling the interface elements in the first interface element set corresponding to the first input to switch from a reduced display state to an expanded display state, including: according to the user's intention, controlling the interface elements in the first interface element set corresponding to the first input to switch from a reduced display state to an expanded display state.

[0036] Based on the above technical solution, when a user's voice command is detected, the first interface can be displayed based on the wake-up word and based on the user's intention, the display device can be controlled to display a set of first interface elements in an expanded display state.

[0037] In some possible implementations, in response to obtaining the wake-up word, before controlling the display device to display the first interface, the method further includes: determining that the second voice instruction is a voice instruction in a preset voice instruction set.

[0038] Exemplarily, the preset voice instruction set may include voice instructions for open domain dialogue.

[0039] For example, voice commands in open-domain conversations, also known as open small talk, are primarily designed for entertainment and conversation. Open small talk is an important way for voice assistants to portray character settings and allow users to express their emotions. It helps maintain social relationships and set the right expectations. Therefore, open small talk, while maintaining authenticity, encourages user input, caters to user emotions, and maintains an engaging and natural flow. Example voice commands for open-domain conversations include "Help me generate some pictures" and "Help me generate a text describing the beautiful scenery outside the car."

[0040] In combination with the first aspect, in certain implementations of the first aspect, the first input is the input of one or more sensors in the vehicle, and controlling the display device to display the first interface includes: controlling the display device to display the first interface based on the input of the one or more sensors.

[0041] Based on the above technical solution, the trigger condition for displaying the first interface can also be the input of the sensor. In this way, the user does not need to perform voice input, and the first interface can be displayed when the input of the sensor is obtained.

[0042] In some possible implementations, the one or more sensors include a collision sensor, which controls the interface elements in the first interface element set corresponding to the first input to switch from a reduced display state to an expanded display state, including: when the collision signal collected by the collision sensor meets a preset condition, controlling one or more of the interface elements related to vehicle status self-inspection, interface elements related to road rescue, interface elements related to insurance, and interface elements related to photos of the collision accident scene to switch from a reduced display state to an expanded display state.

[0043] In some possible implementations, the one or more sensors include a positioning sensor, which controls the interface elements in the first interface element set corresponding to the first input to switch from a reduced display state to an expanded display state, including: when the data collected by the positioning sensor indicates that the electronic device is at a certain tourist location, controlling one or more of the interface elements related to food in the tourist location, interface elements related to attractions in the tourist location, and interface elements related to tourist routes in the tourist location to switch from a reduced display state to an expanded display state.

[0044] In combination with the first aspect, in some implementations of the first aspect, the method further includes: controlling the multiple interface elements in the reduced display state to flash and / or change size according to a preset frequency.

[0045] Based on the above technical solution, when displaying the first interface, multiple interface elements in the reduced state can be controlled to flash and / or change size according to a preset frequency, so that the user can see the animation effect of the first interface, which helps to improve the user experience during human-computer interaction.

[0046] In combination with the first aspect, in some implementations of the first aspect, the method further includes: determining the first interface element set based on the first input and the machine learning model.

[0047] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: sending first indication information to a cloud server based on the first input, the first indication information being used to indicate the first input; and receiving information of the first interface element set sent by the cloud server.

[0048] In a second aspect, a human-computer interaction device is provided, which includes an acquisition unit and a control unit, wherein the control unit is used to control the display device to display a first interface, the first interface including multiple interface elements in a reduced display state, each of the multiple interface elements corresponding to a service, application or atomic function; the control unit is also used to control the interface elements in a first interface element set corresponding to the first input obtained by the acquisition unit to switch from a reduced display state to an expanded display state, the multiple interface elements including the first interface element set.

[0049] In combination with the second aspect, in certain implementations of the second aspect, before the control unit controls the interface elements in the first interface element set corresponding to the first input to switch from a reduced display state to an expanded display state, the control unit is also used to control the interface elements in the second interface element set on the first interface to switch from the reduced display state to the expanded display state based on the user's frequency of use and / or usage duration of each of the multiple interface elements; when the duration for which the interface elements in the second interface element set are controlled to be in the expanded display state continues for a first preset duration, the interface elements in the second interface element set are controlled to switch from the expanded display state to the reduced display state.

[0050] In combination with the second aspect, in certain implementations of the second aspect, the multiple interface elements include a third interface element set, and the usage frequency and / or usage duration of the multiple interface elements in the third interface element set meet preset conditions, wherein the control unit is specifically used to: control the display device to display the icon of the application corresponding to each interface element in the third interface element set, and control the display device to display multiple preset shapes, each preset shape in the multiple preset shapes corresponds one-to-one to each interface element in the fourth interface element set, and the fourth interface element set is the interface elements in the multiple interface elements other than the third interface element set.

[0051] In combination with the second aspect, in some implementations of the second aspect, the preset shape is a dot shape, a star shape, or a sphere shape.

[0052] In combination with the second aspect, in certain implementations of the second aspect, the control unit is specifically configured to: control the display device to display the first interface according to the area where the user is located in the cabin.

[0053] In combination with the second aspect, in certain implementations of the second aspect, the display device is a central control screen, and the control unit is specifically used to: when there is a user in the main driver's area, control the first area in the central control screen to display interface elements related to vehicle control and navigation among the multiple interface elements, and the first area is the display area of ​​the central control screen close to the main driver's area; and / or, when there is a user in the co-driver's area, control the second area in the central control screen to display interface elements related to entertainment and social categories among the multiple interface elements, and the second area is the display area of ​​the central control screen close to the co-driver's area.

[0054] In combination with the second aspect, in certain implementations of the second aspect, the first input is the user's input, and the control unit is specifically used to: when the user is located in the third area in the cabin, control the display device to display a three-dimensional display screen of the multiple interface elements; wherein, in the three-dimensional display screen, the normal of the plane in which the multiple interface elements are located points to the third area, or the normal of the plane in which the multiple interface elements are located points to the direction of the user's line of sight.

[0055] In combination with the second aspect, in some implementations of the second aspect, the control unit is further used to control at least some of the multiple interface elements in the reduced display state to move in a preset direction.

[0056] In combination with the second aspect, in certain implementations of the second aspect, the control unit is further used to control the display device to display a second interface before controlling the display device to display the first interface, the second interface including a fifth interface element set in an expanded display state, the multiple interface elements including the fifth interface element set; obtain the user's second input on the second interface; in response to the second input, control the interface elements in the fifth interface element set to gradually shrink and move in a preset direction to display the first interface.

[0057] In combination with the second aspect, in certain implementations of the second aspect, the control unit is used to control the display device to display the voice assistant in the awakened state on the first interface in response to the second input; and according to the user's first voice command, control the interface elements in the first interface element set corresponding to the first voice command to switch from a reduced display state to an expanded display state.

[0058] In combination with the second aspect, in certain implementations of the second aspect, the first input is a second voice instruction, and the control unit is used to control the display device to display a third interface before controlling the display device to display the first interface; when the display device displays the third interface, obtain the second voice instruction, and the second voice instruction includes a wake-up word and the user's intention; in response to obtaining the wake-up word, control the display device to display the first interface; according to the user's intention, control the interface elements in the first interface element set corresponding to the first input to switch from a reduced display state to an expanded display state.

[0059] In combination with the second aspect, in certain implementations of the second aspect, the first input is the input of one or more sensors in the vehicle, and the control unit is specifically used to control the display device to display the first interface based on the input of the one or more sensors.

[0060] In combination with the second aspect, in some implementations of the second aspect, the control unit is further used to control the multiple interface elements in the reduced display state to flash and / or change in size according to a preset frequency.

[0061] In combination with the second aspect, in some implementations of the second aspect, the device further includes: a determination unit, configured to determine the first set of interface elements based on the first input and the machine learning model.

[0062] In combination with the second aspect, in certain implementations of the second aspect, the device also includes: a sending unit, used to send first indication information to the cloud server based on the first input, and the first indication information is used to indicate the first input; and a receiving unit, used to receive information of the first interface element set sent by the cloud server.

[0063] In a third aspect, the present application provides a human-computer interaction device, which includes a processor and a memory, wherein the memory is used to store instructions, and the processor executes the instructions stored in the memory to enable the device to perform any possible method in the first aspect.

[0064] In a fourth aspect, the present application provides a human-computer interaction system, which includes a computing platform and a display device, wherein the computing platform includes any possible device in the second aspect or the third aspect.

[0065] In a fifth aspect, the present application provides an electronic device, which includes any possible device in the second aspect or the third aspect, or includes the system described in the fourth aspect.

[0066] In combination with the fifth aspect, in some implementations of the fifth aspect, the electronic device is a vehicle.

[0067] In a sixth aspect, the present application provides a computer program product, comprising: a computer program code, which, when executed on a computer, enables the computer to execute any possible method in the first aspect.

[0068] It should be noted that the above-mentioned computer program code can be stored in whole or in part on the first storage medium, wherein the first storage medium can be packaged together with the processor or separately packaged with the processor, and the embodiments of the present application do not specifically limit this.

[0069] In a seventh aspect, the present application provides a computer-readable medium storing a program code, which enables the computer to execute any possible method in the first aspect when the computer program code is run on the computer.

[0070] In an eighth aspect, the present application provides a chip system comprising a processor for calling a computer program or computer instruction stored in a memory so that the processor executes any possible method in the above-mentioned first aspect.

[0071] In combination with the eighth aspect, in a possible implementation, the processor is coupled to the memory through an interface.

[0072] In combination with the eighth aspect, in one possible implementation, the chip system also includes a memory, in which a computer program or computer instructions are stored.

[0073] In a ninth aspect, the present application provides a chip system comprising a circuit for executing any possible method in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 It is a functional block diagram of the vehicle provided in an embodiment of the present application.

[0075] Figure 2 This is a schematic diagram of a vehicle cabin scene provided in an embodiment of the present application.

[0076] Figure 3 It is a set of GUIs provided in the embodiments of this application.

[0077] Figure 4 This is another set of GUIs provided in an embodiment of the present application.

[0078] Figure 5 This is another GUI provided by an embodiment of the present application.

[0079] Figure 6 This is another GUI provided by an embodiment of the present application.

[0080] Figure 7 This is another set of GUIs provided in an embodiment of the present application.

[0081] Figure 8 This is another set of GUIs provided in an embodiment of the present application.

[0082] Figure 9 This is another set of GUIs provided in an embodiment of the present application.

[0083] Figure 10 This is another GUI provided by an embodiment of the present application.

[0084] Figure 11 This is another set of GUIs provided in an embodiment of the present application.

[0085] Figure 12 The embodiment of the present application provides a schematic flowchart of a human-computer interaction method.

[0086] Figure 13 The embodiment of the present application provides a schematic block diagram of a human-computer interaction device. DETAILED DESCRIPTION

[0087] The technical solution in this application will be described below with reference to the accompanying drawings.

[0088] Figure 1 This is a functional block diagram of a vehicle provided in an embodiment of the present application. Figure 1 As shown, the vehicle 100 may include a display device 130 and a computing platform 150, wherein the display device 130 in the cockpit is mainly divided into two categories, the first category is the vehicle-mounted display screen; the second category is a projection display screen, such as a head-up display (HUD). The vehicle-mounted display screen is a physical display screen and is an important part of the vehicle infotainment system. Multiple display screens can be set up in the cockpit, such as a digital instrument display screen, a central control screen, a display screen in front of the passenger on the front passenger seat (also called the front passenger), a display screen in front of the left rear passenger, and a display screen in front of the right rear passenger. Even the windows can be used as display screens for display. Head-up display, also known as head-up display system. It is mainly used to display driving information such as speed and navigation on a display device in front of the driver (such as a windshield). This can reduce the driver's line of sight diversion time, avoid pupil changes caused by the driver's line of sight diversion, and improve driving safety and comfort. For example, HUD includes a combiner-HUD (C-HUD) system, a windshield-HUD (W-HUD) system, and an augmented reality HUD (AR-HUD) system.

[0089] Some or all functions of the vehicle 100 may be controlled by a computing platform 150. The computing platform 150 may include processors 151 to 15n. A processor is a circuit capable of processing signals. In one implementation, the processor may be a circuit capable of reading and executing instructions, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), or a digital signal processor (DSP). In another implementation, the processor may implement certain functions through the logical relationships of a hardware circuit. The logical relationships of the hardware circuit may be fixed or reconfigurable. For example, the processor may be a hardware circuit implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a field programmable gate array (FPGA). In a reconfigurable hardware circuit, the process of the processor loading a configuration file to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, the processor may also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), etc. In addition, the computing platform 150 may also include a memory for storing instructions, and some or all of the processors 151 to 15n may call the instructions in the memory to implement corresponding functions.

[0090] Optionally, the structure of the above vehicle 100 is merely illustrative, and in actual applications, various components in the above vehicle 100 may be added or deleted according to actual needs.

[0091] Figure 2A schematic diagram of a vehicle cockpit scene provided for an embodiment of the present application. One or more vehicle-mounted display screens (or vehicle-mounted screens) are arranged inside the smart cockpit, including but not limited to display screen 201 (or central control screen), display screen 202 (or co-pilot entertainment screen), display screen 203 (or screen behind the main driver's headrest), display screen 204 (or screen behind the co-pilot's headrest), display screen 205 (or second-row entertainment screen) arranged on the top of the cockpit, and instrument screen. Furthermore, display screens 201 to 205 can display a graphical user interface (GUI), which can include one or more application icons, and / or one or more cards. For example, Figure 1 The display device 130 shown may display one or more of the screens 201 to 205. In some possible implementations, the screen 201 may be a long screen extending to the passenger seat area. In addition, the screen 205 may be a projection screen associated with a projector. The projector may be associated with a desktop launcher, and the desktop launcher may be used to manage applications projected onto the projection screen.

[0092] Figure 2 The cockpit may also be equipped with one or more cameras to capture images inside or outside the cabin, such as a driver monitoring system (DMS) camera, a cabin monitoring system (CMS) camera, and a dashcam camera. The cameras used to capture images inside and outside the cabin can be the same or different. Furthermore, the cabin may also be equipped with one or more pressure sensors and acoustic wave sensors to monitor the presence and location of a user.

[0093] It should be understood that the method for controlling the display in the following embodiments is based on Figure 2 The five-seater vehicle shown is used as an example for illustration, and the embodiments of the present application are not limited thereto. For example, for a seven-seater sport / suburban utility vehicle (SUV), the cockpit may include a central control screen, a co-pilot entertainment screen, a screen behind the driver's headrest, a screen behind the co-pilot's headrest, entertainment screens in the three-row left area, and entertainment screens in the three-row right area. For another example, for a passenger car, the cockpit may include front-row entertainment screens and rear-row entertainment screens; or, the cockpit may include a display screen in the driving area and an entertainment screen in the passenger area. In addition, the following embodiments are described using a left-hand drive vehicle (i.e., the driver is on the left side of the vehicle) as an example. In actual implementation, the vehicle may also be a right-hand drive vehicle (i.e., the driver is on the right side of the vehicle).

[0094] Figure 3 A set of GUIs provided by an embodiment of the present application is shown.

[0095] like Figure 3 As shown in (a) in the figure, the vehicle can display a desktop through the display screen 201, wherein the desktop includes a service card 301 corresponding to the setting application, a service card 302 corresponding to the music application, a service card 303 corresponding to the vehicle's remaining power and remaining mileage, a service card 304 corresponding to the owner's guide application, an icon 305 for the gallery application, an icon 306 for the clock application, an icon 307 for the music application, and an icon 308 for the video application.

[0096] For example, the sizes of the application icons 305 - 308 may be the same, for example, the length of the icon may be a1.

[0097] Illustratively, the service cards 301 - 304 may have the same size. For example, the length of the service cards 301 - 304 may be b1.

[0098] like Figure 3 As shown in (b) in FIG, in response to detecting the user sliding from the left side of the desktop to the right, the change process of the service cards 301-304 and the application icons 305-308 can be displayed on the display screen.

[0099] Illustratively, the change process includes the service cards 301 - 304 and the application icons 305 - 308 gradually shrinking and the displayed service cards 301 - 304 and the application icons 305 - 308 moving in the direction indicated by the dotted line in the figure.

[0100] For example, Figure 3 As shown in (b) in FIG, in the process of gradually shrinking the service cards 301 - 304 , the lengths of the service cards 301 - 304 gradually shrink from b1 to b2 , and the lengths of the application icons 305 - 308 gradually shrink from a1 to a2 .

[0101] In one embodiment, the movement directions of the service cards 301 - 304 and the application icons 305 - 308 may be preset, or may be determined according to the distribution of users in the cabin.

[0102] For example, when there is a user in the main driving area of ​​the cockpit, the service card 301, service card 303 and service card 304 related to vehicle control can be moved to the display area of ​​the display screen 201 close to the main driving area, and the service card 302, application icon 305, application icon 307 and application icon 308 related to entertainment can be moved to the display area of ​​the display screen 201 close to the co-pilot area.

[0103] In one embodiment, in response to detecting a two-finger slide, three-finger slide, or entry into the negative one screen by the user on the desktop, the change process of the service cards 301-304 and the application icons 305-308 can be displayed on the display screen.

[0104] In one embodiment, since the service card 302 is a service card corresponding to a music application, during the movement process, the icon 307 and the service card 302 may be moved to the same position.

[0105] In one embodiment, since the remaining battery power and remaining mileage of the vehicle displayed in the service card 303 is a service under the settings application, during the movement, the service card 301 and the service card 303 can be moved to the same position.

[0106] like Figure 3 As shown in (c), when the change process of the service cards 301-304 and the application icons 305-308 is completed, the starry sky interface can be displayed through the display screen 201, wherein the starry sky interface includes multiple interface elements, including the icon 301 of the settings application in a reduced display state, the icon 305 of the gallery application, the icon 307 of the music application, the icon 308 of the video application, and the icon 309 of the navigation application. In addition to displaying the icons of applications in a reduced display state, the starry sky interface can also display multiple preset shapes (for example, dots), each of the multiple dots can correspond to an application, service or atomic function. For example, dot 304 can correspond to the owner's guide application, dot 306 can correspond to the clock application, and dot 310 can correspond to the atomic function of adjusting the seat heating in the settings application.

[0107] In one embodiment, at the end of the transformation process, the length of the application icon is finally reduced to a3.

[0108] The dot 310 and icon 301 may correspond to the same application (e.g., the Settings application), but may correspond to different services or atomic functions. For example, icon 301 may correspond to the service of the vehicle's remaining battery and remaining range, while dot 310 may correspond to the atomic function of adjusting the seat heating.

[0109] The above atomic function can refer to a specific function in the application.

[0110] Taking the Settings app as an example, the Settings app can include multiple atomic functions. For example, Atomic Function 1 can be the function of selecting the driving mode in the Settings app, Atomic Function 2 can be the function of selecting the height of the air suspension in the Settings app, and Atomic Function 3 can be the function of adjusting the air conditioning temperature in the Settings app.

[0111] Taking the music app as an example, a music app can include multiple atomic functions. For example, atomic function 4 can be the playback interface of a song in the music app, including playback controls, a space for switching to the previous song, and a space for switching to the next song. Atomic function 5 can be the display interface of the music app's daily recommended songs.

[0112] Taking instant messaging applications as an example, the instant messaging category may include multiple atomic functions. For example, atomic function 6 may be a chat interface with contact a, and atomic function 7 may be a chat interface with contact b.

[0113] The above services may refer to information composed of information that certain users care about in the application.

[0114] Taking the settings application as an example, users may be more concerned about the information of the vehicle's remaining power and remaining mileage in the settings application. This information constitutes the service 1 of the settings application. The service 1 can be composed of Figure 3 The service card 303 is shown.

[0115] Taking shopping apps as an example, users may be more concerned about information such as the delivery time and courier number of the courier in the shopping app. This information constitutes Service 2 of the shopping app. Service 2 can be displayed by a service card. For example, the service card can display "Pickup code: 123456; The express has arrived at xx courier station; Arrival time: xx / xx / xx".

[0116] The above services can be obtained by user customization or by the information that the user frequently views when using the application.

[0117] The above service cards can be used to display atomic functions or services.

[0118] like Figure 3 As shown in (d) in FIG, a certain interface element under the starry sky interface can be displayed on the display screen 201, switching from a reduced display state to an expanded display state. For example, 5 seconds after entering the starry sky interface, the starry sky interface can switch from displaying the icon 307 in a reduced display state to displaying the icon 307 in an expanded display state and displaying a service card 302 corresponding to a music application in an expanded display state (the service card includes a playback interface of a song in the music application), and the service card 302 corresponds to an atomic function of the music application.

[0119] like Figure 3As shown in (e) of FIG. 3 , after the icon 307 in the expanded display state and the service card 302 corresponding to the music application in the expanded display state are displayed for 3 seconds, the icon 307 in the expanded display state can be switched to a reduced display state, and the service card 302 corresponding to the music application in the expanded display state can be hidden. At the same time, the icon 309 in the reduced display state can be switched to the icon 309 in the expanded display state and the service card 311 corresponding to the navigation application in the expanded display state can be displayed.

[0120] like Figure 3 As shown in (f), when the voice command "turn on nap mode" issued by the user is detected, the voice assistant can make a voice reply "A set of service cards related to nap mode has been generated for you" and the service cards related to nap mode can be displayed under the starry sky interface. Exemplarily, the starry sky interface can be displayed by the display screen 201 and icons 312, 314 and 316 in an expanded display state can be displayed, as well as service cards 313, 315 and 317 in an expanded display state can be displayed, wherein icon 312 is an icon for a clock application and service card 313 includes an atomic function for timing the nap mode, icon 314 is an icon for sound adjustment and service card 315 includes an atomic function for selecting background sound and noise reduction mode, icon 316 is an icon for seat adjustment and service card 317 is an atomic function corresponding to seat adjustment.

[0121] Figure 4 Another set of GUIs provided by an embodiment of the present application is shown.

[0122] like Figure 4 As shown in (a) in the figure, the vehicle can display a desktop through the display screen 201, wherein the desktop includes a service card 301 corresponding to the setting application, a service card 302 corresponding to the music application, a service card 303 corresponding to the vehicle's remaining power and remaining mileage, a service card 304 corresponding to the owner's guide application, an icon 305 for the gallery application, an icon 306 for the clock application, an icon 307 for the music application, and an icon 308 for the video application.

[0123] like Figure 4 As shown in (b), in response to detecting a two-finger or three-finger sliding operation of the user on the desktop, the display screen can be controlled to display the changes of service cards 301-304 and application icons 305-308 according to the area where the user is located in the cabin.

[0124] For example, when it is determined through camera images that a user is currently present in the driver's seat and the passenger seat, the change process includes moving vehicle control-related service cards 301 and 303 toward the display area of ​​display screen 201 near the driver's seat, and moving entertainment-related icons or service cards toward the display area of ​​display screen 201 near the passenger seat. During the movement, the service cards 301-304 and application icons 305-308 gradually shrink.

[0125] like Figure 4 As shown in (c), it can be seen that the display screen 201 includes area 1 and area 2, wherein area 1 includes an icon 301 of a setting application and an icon 309 of a navigation application in a reduced display state, and area 1 includes multiple dots, each of which corresponds to an application, service or atomic function related to vehicle control or fault; area 2 includes an icon 305 of a gallery application, an icon 307 of a music application and an icon 308 of a video application in a reduced display state, and area 2 includes multiple dots, each of which corresponds to an application, service or atomic function related to entertainment or social networking, such as dot 312 can correspond to an atomic function in an instant messaging application, or dot 312 can correspond to a service associated with a shopping application.

[0126] Figure 5 Another GUI provided by an embodiment of the present application is shown. When displaying the starry sky interface through the display screen 201, at least some of the multiple interface elements can be controlled to change in size according to a preset frequency. Figure 5 As shown, after the starry sky interface is displayed, during the time period from T1 to T2, the dot in the reduced display state can be first reduced and then increased; during the time period from T3 to T4, the icon 307 in the reduced display state can be first reduced and then increased. In this way, the user can see the starry sky interface in a "breathing state", which helps to improve the user experience during human-computer interaction.

[0127] In one embodiment, during the time period from time T1 to time T2, in addition to controlling the dot in the reduced display state to first shrink and then grow, the dot can also be controlled to flash.

[0128] The above-mentioned control of the dot flashing includes: controlling the brightness of the dot. This can create a "twinkling starlight" feeling for the user, which helps to improve the user experience during human-computer interaction.

[0129] Figure 6Another GUI provided by an embodiment of the present application is shown. The vehicle can display a three-dimensional display screen of the starry sky interface through the display screen 201. The normal of the two-dimensional plane where the starry sky interface is located in the three-dimensional display screen can point to the area where the user who triggered the starry sky desktop is located, or can point to the direction of the user's line of sight.

[0130] For example, when a two-finger or three-finger swipe or input to enter the negative first screen is detected on the desktop by the user in the main driver's seat, the display screen 201 can be controlled to display the three-dimensional display screen, wherein the normal direction of the plane where the starry sky interface is located in the three-dimensional display screen points toward the user in the main driver's seat, or points in the direction of the user's line of sight in the main driver's seat. In this way, by controlling the starry sky interface to align with the user, the user in the main driver's seat can be given the feeling of face-to-face interaction with the starry sky interface, which helps to further enhance the user experience during human-computer interaction.

[0131] Figure 7 Another set of GUIs provided by an embodiment of the present application is shown.

[0132] like Figure 7 As shown in (a) of FIG, the vehicle can display the desktop through the display screen 201. When it is detected that the user in the main driving area issues a voice command "Xiao A Xiao A, turn on the nap mode", the display screen 201 can display the desktop. Figure 7 The GUI shown in (b) in FIG.

[0133] like Figure 7 As shown in (b) in FIG. 2 , in response to the voice command, the vehicle can display the voice image in area 3 of the display screen 201 and transform into multiple star points.

[0134] The above process of the voice image being transformed into multiple star points can be understood as the process of the vehicle determining the icons and service cards in the expanded display state to be displayed to the user in the starry sky interface.

[0135] like Figure 7 As shown in (c), after the displayed voice image is transformed into multiple star points, the vehicle can display the starry sky interface through the display screen 201 and display icons 701, 703 and 705 in the expanded display state and service cards 702, 704 and 706 in the expanded display state, wherein icon 701 is an icon for the clock application and service card 702 includes the atomic function of nap mode timing, icon 703 is an icon for sound adjustment and service card 704 includes the atomic function of selecting background sound and noise reduction mode, icon 705 is an icon for seat adjustment and service card 706 is the atomic function corresponding to seat adjustment.

[0136] In one embodiment, the voice command includes the wake-up word "Xiao A Xiao A" and the user's intention to "turn on nap mode." In response to the wake-up word "Xiao A Xiao A," the vehicle may display a voice image transformed into a plurality of star points in area 3 of the display screen 201. After the voice image is transformed into a plurality of star points, the vehicle may display a starry sky interface through the display screen 201. In response to the user's intention to "turn on nap mode," the vehicle may display icons 701, 703, and 705 in an expanded display state, as well as service cards 702, 704, and 706 in an expanded display state, through the display screen 201.

[0137] Figure 8 Another set of GUIs provided by an embodiment of the present application is shown.

[0138] like Figure 8 As shown in (a), when a collision between the vehicle and the vehicle in front is detected, the vehicle can display a starry sky interface through the display screen 201 and control the voice assistant to enter the wake-up state. At this time, the voice assistant can send a voice message "A solution is being developed for you."

[0139] In one embodiment, detecting that the vehicle collides with a preceding vehicle includes: detecting that a collision signal collected by a collision sensor satisfies a preset collision condition.

[0140] like Figure 8 As shown in (b), after the solution for the current scenario is formulated, the vehicle can display icons 801, 803 and 805 in the expanded display state and service cards 802, 804 and 806 in the expanded display state through the display screen 201, wherein icon 801 is an icon for the setting application and service card 802 includes the detection results of the current vehicle damage status, such as "front camera failure, front tire pressure is normal, rear tire pressure is normal...", save log control and cancel control, icon 803 is an icon for the phone application and service card 804 includes road rescue phone information, call now control and cancel control, icon 805 is an icon for the insurance application and service card 806 is the claim specialist's phone number, call now control and cancel control.

[0141] In an embodiment of the present application, when the input of the sensor is detected, the display screen can be immediately switched to the starry sky interface and the user can be prompted that a solution to the collision scenario is being formulated for the user. After the solution is formulated, multiple icons and multiple service cards can be displayed through the display screen 201. In this way, after a vehicle collision, the user can see multiple icons and service cards related to the collision scenario through the display screen 201, which can help the user quickly call up functions or services that may be used in the collision scenario, and help improve the experience during human-computer interaction. Especially for some novice drivers, their brains may go blank when the vehicle collides, and they don’t know what to do in the scenario. The above technical solution can help novice drivers immediately understand what should be done in the collision scenario. At the same time, through multiple icons and service cards related to the collision scenario, it can also play a role in stabilizing the user's emotions and mentality.

[0142] The above is an example of controlling the display screen to display the starry sky interface after detecting the input collected by the sensor. In this scenario, the user does not need to control the display screen to display the starry sky interface through two-finger sliding, three-finger sliding, and voice input.

[0143] In one embodiment, when a user is detected entering a certain tourist location A (for example, the vehicle is registered in city B and the user is detected driving the vehicle into the tourist location A), the vehicle may display a starry sky interface through the display screen 201 and display one or more application icons in an expanded display state under the starry sky interface and one or more service cards in an expanded display state. For example, the one or more icons may include an icon of a food application, and the service card associated with the icon of the food application may include a recommended food list in the tourist location A. For another example, the one or more icons may include an icon of a tourism application, and the service card associated with the icon of the tourism application may include a recommended attraction list of A in the tourist location.

[0144] In one embodiment, when it is detected that the vehicle switches from a locked state to an unlocked state and detects the operation of the user opening the door, the vehicle can display a starry sky interface through the display screen 201 and display one or more application icons in an expanded display state and one or more service cards in an expanded display state under the starry sky interface. Optionally, the one or more application icons and the one or more service cards can be determined based on the atomic functions selected by the user after getting on the car in the past period of time (for example, a week or a month). For example, the frequency of setting the driving mode and seat ventilation function after the user gets on the car in the past period of time is relatively high, then the one or more application icons may include an icon for setting the application, and the service card associated with the icon for setting the application may include the atomic function of selecting the driving mode. For another example, the one or more icons may include an icon for a seat function, and the service card associated with the icon for the seat function may include the atomic function corresponding to seat ventilation.

[0145] Figure 9 Another set of GUIs provided by an embodiment of the present application is shown.

[0146] like Figure 9 As shown in (a) of FIG, upon detecting that user Tom in the second row, right, triggers an input on display screen 204 to display a starry sky interface, the vehicle can display the starry sky interface on display screen 204 and control the voice assistant to be in an awake state, at which point the voice assistant can issue a voice message "I'm listening..." The starry sky interface includes multiple icons 901-904 in a reduced display state and multiple dots. Icon 901 corresponds to a gallery application, icon 902 corresponds to an instant messaging application, icon 903 corresponds to a music application, and icon 904 corresponds to a video application.

[0147] For example, the icons 901 - 904 may be icons of multiple applications that are frequently used by the user Tom.

[0148] For example, upon detecting that the user in the second row, right, has triggered an input on display screen 204 to display a starry sky interface, the camera in the cabin may be controlled to collect characteristic information (e.g., facial information) of the user in the second row, right, and based on this characteristic information, the user in the second row, right, may be identified as user Tom. Based on the operations performed by user Tom under his vehicle-mounted account, the vehicle may determine multiple applications that user Tom has used most frequently over the past period of time, as well as the atomic functions or services within each of the multiple applications that user Tom has used most frequently.

[0149] like Figure 9As shown in (b) in FIG, a certain interface element under the starry sky interface can be displayed on the display screen 204 to switch from a reduced display state to an expanded display state. For example, 5 seconds after entering the starry sky interface, the icon 901 in the reduced display state can be switched to the icon 901 in the expanded display state and the service cards 905 and 906 corresponding to the gallery application in the expanded display state can be displayed, wherein the service card 905 includes a service of photos related to the user Tom on this day last year, and the service card 906 includes a service of photos of the user Tom and the user Lily.

[0150] above Figures 3 to 9 The example of multiple icons and multiple service cards displayed in the starry sky interface without overlapping is used for illustration, but the embodiment of the present application is not limited thereto. For example, the multiple icons and multiple service cards can also be displayed in an overlapping manner.

[0151] Figure 10 Another GUI provided by an embodiment of the present application is shown.

[0152] like Figure 10 As shown, after a solution for the current scenario is formulated, the vehicle can display an icon 801 in an expanded display state and service cards 802, 804, and 806 in an expanded display state and stacked together in sequence through the display screen 201. For example, when the icon 801 and service card 802 are displayed for 5 seconds and no user click input on the service card 802 is detected, the display can be switched to displaying an icon 803 in an expanded display state and service cards 804, 806, and 802 in an expanded display state and stacked together in sequence.

[0153] Figure 11 Another set of GUIs provided by an embodiment of the present application is shown.

[0154] like Figure 11 As shown, in the starry sky interface, in addition to controlling the icon of an application to switch from a reduced display state to an expanded display state according to the frequency and / or number of times the user uses the application, service or atomic function, the icon of an application can also be controlled to switch from a reduced display state to an expanded display state based on fault information and / or prompt information.

[0155] For example, Figure 11As shown in (a), when the starry sky interface is displayed on the display screen 201, low oil pressure is detected. At this time, the vehicle can control the icon of the setting application to switch from a reduced display state to an expanded display state and display a service card 1101 in an expanded display state. The service card 1101 includes a prompt message "Low oil pressure, there is a risk of damage to the range extender, click to view".

[0156] For example, Figure 11 As shown in (b), when the starry sky interface is displayed through the display screen 201, it is detected that the vehicle is in driving state and the user in the passenger seat is not fastening the seat belt. At this time, the vehicle can control the display screen 201 to switch from displaying the dot 1102 to displaying the icon of the setting application in the expanded display state and the service card 1103 in the expanded display state. The service card 1103 includes a prompt message "The user in the passenger seat is not fastening the seat belt. For your driving safety, please fasten your seat belt."

[0157] Optionally, if the user in the passenger seat is not detected as fastening his seat belt within a preset time (for example, 10 seconds) from the display of the icon of the settings application and the service card 1103 in the expanded display state, the icon of the settings application can be controlled to display red and a voice announcement can be made, "The user in the passenger seat is not fastening his seat belt. For your driving safety, please fasten your seat belt."

[0158] Figure 12 1 shows a schematic flow chart of a human-computer interaction method 1200 provided in an embodiment of the present application. The method 1200 may be executed by the vehicle 100; or, the method 1200 may be executed by the computing platform 120; or, the method 1200 may be executed by a processor, chip, or circuit in the computing platform. The method 1200 includes:

[0159] S1210: Control the display device to display a first interface, where the first interface includes a plurality of interface elements in a reduced display state, each of the plurality of interface elements corresponding to a service, an application, or an atomic function.

[0160] Exemplarily, the first interface may also be referred to as a starry sky desktop.

[0161] Optionally, the first interface can be as follows Figure 3 The starry sky interface shown in (c) of FIG. The starry sky interface includes multiple interface elements in a reduced display state, each of which can correspond to a service, application or atomic function.

[0162] S1220: According to a first input, control interface elements in a first interface element set corresponding to the first input to switch from a reduced display state to an expanded display state, the plurality of interface elements including the first interface element set.

[0163] Optionally, the first interface element set includes a first interface element, and controlling the interface elements in the first interface element set corresponding to the first input to switch from a reduced display state to an expanded display state includes: controlling the display device to switch from displaying the first interface element in the reduced display state to displaying the icon and service card of the application corresponding to the first interface element in the expanded display state, the service card including services or atomic functions related to the application in the expanded display state.

[0164] For example, the vehicle displays the following on the display screen 201: Figure 3 The starry sky interface shown in (c) is a combination of one or more service cards in an expanded display state displayed under the starry sky interface when the user's voice command "turn on nap mode" is detected. For example, you can refer to Figure 3 As shown in (f) of FIG, when a voice command "turn on nap mode" issued by a user is detected, a combination of one or more service cards in an expanded display state on the display screen 201 may include service card 312, service card 314 and service card 316. The above first interface element set may include the following: Figure 3 (f) in the figure shows icons 312, 314, and 316 in an expanded display state, and service cards 313, 315, and 317 in an expanded display state.

[0165] Optionally, the arrangement of the multiple interface elements in the reduced display state in the first interface can be arranged according to a preset rule. For example, the graphics formed by the animation effects of the multiple interface elements in the reduced display state are related to the weather conditions in the scene currently located by the vehicle. If the weather condition in the current scene is rainy, the shape of the multiple interface elements in the reduced display state can be raindrop-shaped. For another example, if the weather condition in the current scene is snowy, the shape of the multiple interface elements in the reduced display state can be snowflake-shaped.

[0166] Optionally, before controlling the interface elements in the first interface element set corresponding to the first input to switch from a reduced display state to an expanded display state, the method 1200 also includes: controlling the interface elements in the second interface element set on the first interface to switch from the reduced display state to the expanded display state based on the user's frequency of use and / or duration of use of each of the multiple interface elements; when the duration for which the interface elements in the second interface element set are controlled to be in the expanded display state continues for a first preset duration, controlling the interface elements in the second interface element set to switch from the expanded display state to the reduced display state.

[0167] For example, Figure 3 As shown in (d) and (e) in the figure, the second set of interface elements may include a music application and a navigation application. When it is detected that the frequency and / or duration of use of the music application and the navigation application by the user is high over the past period of time, the display of the icon 307 in the reduced display state can be switched to the display of the icon 307 in the expanded display state and the service card 302 corresponding to the music application in the expanded display state in the starry sky interface. After the duration of displaying the icon 307 and the service card 302 in the expanded display state reaches 3 seconds, the icon 307 in the expanded display state can be switched to the display of the icon 307 in the reduced display state and the service card 302 in the expanded display state can be hidden. At the same time, the display of the icon 309 in the reduced display state can be switched to the display of the icon 309 in the expanded display state and the service card 311 corresponding to the navigation application in the expanded display state.

[0168] Optionally, the multiple interface elements include a third interface element set, and the usage frequency and / or usage duration of the multiple interface elements in the third interface element set meet preset conditions, wherein controlling the display device to display the first interface includes: controlling the display device to display the icon of the application corresponding to each interface element in the third interface element set, and controlling the display device to display multiple preset shapes, each preset shape in the multiple preset shapes corresponds one-to-one to each interface element in the fourth interface element set, and the fourth interface element set is the interface elements in the multiple interface elements other than the third interface element set.

[0169] Exemplarily, the preset shape is a dot shape, a star shape or a sphere shape.

[0170] For example, Figure 3 As shown in (c) in FIG, the third set of interface elements may include a settings application, a video application, a gallery application, a music application, and a navigation application. In the starry sky interface, icons of the settings application, video application, gallery application, music application, and navigation application in a reduced display state may be displayed, as well as multiple dots.

[0171] Exemplarily, the preset shape is a raindrop shape or a snowflake shape.

[0172] Optionally, controlling the display device to display the first interface includes: controlling the display device to display the first interface according to the area where the user is located in the cabin.

[0173] Optionally, the display device is a central control screen, and according to the area where the user is located in the cockpit, the display device is controlled to display the first interface, including: when there is a user in the main driving area, controlling the first area in the central control screen to display interface elements related to vehicle control and navigation among the multiple interface elements, and the first area is the display area of ​​the central control screen close to the main driving area; and / or, when there is a user in the co-pilot area, controlling the second area in the central control screen to display interface elements related to entertainment and social categories among the multiple interface elements, and the second area is the display area of ​​the central control screen close to the co-pilot area.

[0174] For example, Figure 4 As shown in (c), the first area can be the above-mentioned area 1, and the second area can be the above-mentioned area 2. The display screen 201 includes area 1 and area 2, wherein area 1 includes an icon 301 of the settings application and an icon 309 of the navigation application in a reduced display state, and area 2 includes an icon 305 of the gallery application, an icon 307 of the music application, and an icon 308 of the video application in a reduced display state, and area 2 includes a plurality of dots, each of which corresponds to an application, service or atomic function related to entertainment or social networking, such as dot 312 can correspond to a display interface associated with an instant messaging application. In this way, interface elements related to vehicle control and navigation can be displayed in the display area near the main driving area and / or interface elements related to entertainment and social networking can be displayed in the display area near the co-pilot area. In this way, it is convenient for users in the main driving area to select corresponding applications, services or atomic functions under the first interface.

[0175] Optionally, the display device is controlled to display the first interface according to the area where the user is located in the cockpit, including: when a fault state occurs in the vehicle, the first area is controlled to display the atomic function corresponding to the fault state in an expanded display state.

[0176] For example, Figure 11 As shown in (a) of FIG. , the fault state may be low oil pressure. When low oil pressure is detected while the starry sky interface is displayed on display screen 201, the vehicle can control the setting application icon to switch from a reduced display state to an expanded display state and display a service card 1101 in the expanded display state. Service card 1101 includes a prompt message "Low oil pressure, there is a risk of damage to the range extender, click to view."

[0177] Optionally, according to the area where the user is located in the cabin, the display device is controlled to display the first interface, including: when the vehicle needs to prompt the user with certain prompt information, controlling the display area close to the user to display the atomic function corresponding to the prompt information in the expanded display state.

[0178] For example, Figure 11 As shown in (b) in FIG, the prompt information may be a message to fasten the seat belt. When the starry sky interface is displayed on the display screen 201, if it is detected that the vehicle is in driving state and the user in the passenger seat is not fastening the seat belt, the vehicle can control the display screen 201 to switch from displaying the dot 1102 to displaying the icon of the setting application in the expanded display state and the service card 1103 in the expanded display state. The service card 1103 includes the prompt information "The user in the passenger seat is not fastening the seat belt. For your driving safety, please fasten your seat belt."

[0179] Optionally, the first input is the user's input, and the display device is controlled to display the first interface according to the area in which the user is located in the cockpit, including: when the user is located in a third area in the cockpit, the display device is controlled to display a three-dimensional display screen of the multiple interface elements; wherein, in the three-dimensional display screen, the normal of the plane in which the multiple interface elements are located points to the third area, or the normal of the plane in which the multiple interface elements are located points to the direction of the user's line of sight.

[0180] For example, the three-dimensional display can be as follows Figure 6 The three-dimensional display screen shown. For example, if the first input is an input triggered by the user in the main driving area (for example, voice input), the display screen 201 can be controlled to display the three-dimensional display screen, wherein the normal direction of the plane where the starry sky interface is located in the three-dimensional display screen points to the user in the main driving area, or points to the direction of the line of sight of the user in the main driving area. In this way, by controlling the starry sky interface to align with the user, the user can be provided with a sense of face-to-face interaction with the starry sky interface, which helps to further enhance the user experience during human-computer interaction.

[0181] Optionally, the method 1200 further includes: controlling at least some of the multiple interface elements in the zoomed-out display state to move in a preset direction. Figure 3 As shown in (c), dot 304 can move in a preset direction. For example, 10 seconds after entering the starry sky interface, dot 304 can move upward to the position of dot 306 and dot 306 can move downward to the position of dot 304.

[0182] For example, when it is detected that the electronic device is located in a rainy or snowy environment, the preset direction of movement may be movement along the direction of gravity, so as to give the user a feeling of falling raindrops.

[0183] For example, if the electronic device is a vehicle, when it is detected that the vehicle is turning left, the preset direction of movement may be rightward movement. In this way, the user may feel that the interface element is moving in the opposite direction of the vehicle's direction due to inertia.

[0184] For example, if the electronic device is a vehicle and the first interface is a three-dimensional display, when a sudden acceleration of the vehicle is detected, the predetermined direction of movement can be in the opposite direction of the vehicle's movement. This can give the user the impression that the interface element is moving in the opposite direction of the vehicle's movement due to inertia.

[0185] Optionally, before controlling the display device to display the first interface, the method 1200 also includes: controlling the display device to display a second interface, the second interface including a fifth interface element set in an expanded display state, the multiple interface elements including the fifth interface element set; wherein, controlling the display device to display the first interface includes: obtaining the user's second input on the second interface; in response to the second input, controlling the interface elements in the fifth interface element set to gradually shrink and move in a preset direction to display the first interface.

[0186] For example, the second interface may be as follows: Figure 3 For the desktop shown in (a) in the figure, the second input can be an operation of the user sliding from the left side of the desktop to the right. The desktop includes service cards 301-304 and application icons 305-308 in an expanded display state. In response to detecting the operation of the user sliding from the left side of the desktop to the right, the change process of the service cards 301-304 and the application icons 305-308 can be displayed on the display screen. The change process includes the service cards 301-304 and the application icons 305-308 gradually shrinking and the service cards 301-304 and the application icons 305-308 moving in the direction of movement shown by the dotted line in the figure, and finally forming the starry sky interface.

[0187] Optionally, the method 1200 further includes: in response to the second input, controlling the display device to display the voice assistant in the awake state on the first interface; wherein, according to the first input, controlling the interface elements in the first interface element set corresponding to the first input to switch from a reduced display state to an expanded display state, including: according to the user's first voice command, controlling the interface elements in the first interface element set corresponding to the first voice command to switch from a reduced display state to an expanded display state.

[0188] For example, Figure 3 As shown in (c) in FIG, in response to detecting the user sliding from the left side of the desktop to the right, the voice assistant can be awakened. At this time, the voice assistant is in the awake state and the current state of the voice assistant can be displayed as "I am listening...". For example, the first voice instruction can be as follows Figure 3 (f) in the figure shows the voice command issued by the user: "Turn on nap mode."

[0189] Optionally, the first input is a second voice command, and before controlling the display device to display the first interface, the method also includes: controlling the display device to display a third interface; when the display device displays the third interface, obtaining the second voice command, the second voice command including a wake-up word and the user's intention; wherein, controlling the display device to display the first interface includes: in response to obtaining the wake-up word, controlling the display device to display the first interface; according to the first input, controlling the interface elements in the first interface element set corresponding to the first input to switch from a reduced display state to an expanded display state, including: according to the user's intention, controlling the interface elements in the first interface element set corresponding to the first input to switch from a reduced display state to an expanded display state.

[0190] For example, the third interface may be as follows: Figure 7 For the desktop shown in (a) in FIG, the second voice command can be a voice command issued by the user, "Xiao A Xiao A, turn on nap mode." The voice command includes the wake-up word "Xiao A Xiao A" and the user's intention, "turn on nap mode."

[0191] Optionally, in response to obtaining the wake-up word, before controlling the display device to display the first interface, the method 1200 further includes: determining that the second voice instruction is a voice instruction in a preset voice instruction set.

[0192] Optionally, the preset voice instruction set may include voice instructions for open domain dialogue.

[0193] Exemplarily, the voice command of the open domain dialogue may be “Xiao A Xiao A, help me generate some pictures” or “Xiao A Xiao A, help me generate a blessing message” and the like.

[0194] For example, voice commands in open-domain conversations, also known as open small talk, are primarily designed for entertainment and conversation. Open small talk is an important way for voice assistants to portray character settings and allow users to express their emotions. It helps maintain social relationships and set the right expectations. Therefore, open small talk, while maintaining authenticity and trust, encourages user input, caters to user emotions, and ensures that the conversation is engaging and natural.

[0195] Optionally, the preset voice instruction set may include a preset task-based dialogue.

[0196] For example, task-based conversations have clear goals. Voice assistants fill in slots to help users complete tasks. Task-based conversations aim for effectiveness and efficiency. Users display less emotional expression.

[0197] Exemplarily, the preset task-based dialogue includes “Xiao A Xiao A, turn on nap mode.”

[0198] For example, the preset task-based dialogue does not include dialogues for opening doors and windows. For example, if a user issues a voice command "Xiao A Xiao A, help me open the window," the vehicle may determine that the voice command does not belong to the preset task-based dialogue. In this case, the vehicle may not control the display device to display the starry sky interface, but may directly execute the task of opening the window corresponding to the voice command.

[0199] Optionally, the first input is input from one or more sensors in the vehicle, and controlling the display device to display the first interface includes: controlling the display device to display the first interface based on the input from the one or more sensors.

[0200] For example, Figure 8 As shown, the one or more sensors include a collision sensor, and the first interface element set may include an atomic function corresponding to a self-check of a vehicle's damaged status, an atomic function corresponding to calling roadside assistance in a telephone application, and an atomic function corresponding to calling a claims specialist in an insurance application.

[0201] Optionally, the method 1200 further includes: controlling the multiple interface elements in the zoomed-out display state to flash and / or change in size according to a preset frequency.

[0202] For example, Figure 5 As shown, after the starry sky interface is displayed, during the time period from time T1 to time T2, the dot in the reduced state can be first reduced and then increased; during the time period from time T3 to time T4, the icon 307 in the reduced display state can be first reduced and then increased. Optionally, in addition to controlling the dot in the reduced state to first reduce and then increase, the dot can also be controlled to flash.

[0203] Optionally, the method 1200 further includes: determining the first set of interface elements based on the first input and the machine learning model.

[0204] In the embodiment of the present application, the process of determining the first set of interface elements can be implemented on the vehicle side. Exemplarily, the machine learning model can be a large language model (LLM).

[0205] For example, the LLM input can be the user's intention in a voice command, such as "turn on nap mode," and the LLM can output information about applications, services, or atomic functions related to the user's intention. For another example, the LLM input can be text content converted from data collected by sensors.

[0206] For example, when the collision signal collected by the vehicle's collision sensor meets the preset conditions, the collision signal can be converted into the text content "The vehicle has collided" and the text content can be input into the LLM, so that the LLM can output information about applications, services or atomic functions related to the text content.

[0207] For example, when the vehicle detects that the user is driving the vehicle into a tourist destination A, it can combine the map information and the data collected by the positioning sensor to convert it into text content "Attractions and food recommendations in tourist destination A" and input the text content into the LLM, so that the LLM can output information about applications, services or atomic functions related to the attractions and food recommendations in the tourist destination A.

[0208] Optionally, the method 1200 further includes: sending first indication information to a cloud server according to the first input, where the first indication information is used to indicate the first input; and receiving information of the first interface element set sent by the cloud server.

[0209] In the embodiment of the present application, the process of determining the first interface element set can be implemented in the cloud server. The process implemented by the cloud server can refer to the process implemented in the electronic device, which will not be repeated here.

[0210] Figure 13 A schematic block diagram of a human-computer interaction device 1300 provided in an embodiment of the present application is shown. The device 1300 includes an acquisition unit 1310 and a control unit 1320, wherein the control unit 1320 is configured to control a display device to display a first interface, wherein the first interface includes multiple interface elements in a reduced display state, each of the multiple interface elements corresponding to a service, application, or atomic function; and the control unit 1320 is further configured to control, based on a first input acquired by the acquisition unit 1310, an interface element in a first interface element set corresponding to the first input to switch from a reduced display state to an expanded display state, wherein the multiple interface elements include the first interface element set.

[0211] Optionally, before the control unit 1320 controls the interface elements in the first interface element set corresponding to the first input to switch from a reduced display state to an expanded display state, the control unit 1320 is also used to control the interface elements in the second interface element set on the first interface to switch from the reduced display state to the expanded display state based on the user's frequency of use and / or usage duration of each of the multiple interface elements; when the duration for which the interface elements in the second interface element set are controlled to be in the expanded display state continues for a first preset duration, the interface elements in the second interface element set are controlled to switch from the expanded display state to the reduced display state.

[0212] Optionally, the multiple interface elements include a third interface element set, and the usage frequency and / or usage duration of the multiple interface elements in the third interface element set meet preset conditions, wherein the control unit 1320 is specifically used to: control the display device to display the icon of the application corresponding to each interface element in the third interface element set, and control the display device to display multiple preset shapes, each of the multiple preset shapes corresponds one-to-one to each interface element in the fourth interface element set, and the fourth interface element set is the interface elements in the multiple interface elements other than the third interface element set.

[0213] Optionally, the preset shape is a dot shape, a star shape or a sphere shape.

[0214] Optionally, the control unit 1320 is specifically configured to control the display device to display the first interface according to the area where the user is located in the cabin.

[0215] Optionally, the display device is a central control screen, and the control unit 1320 is specifically used to: when there is a user in the main driving area, control the first area in the central control screen to display interface elements related to vehicle control and navigation among the multiple interface elements, and the first area is the display area of ​​the central control screen close to the main driving area; and / or, when there is a user in the co-pilot area, control the second area in the central control screen to display interface elements related to entertainment and social categories among the multiple interface elements, and the second area is the display area of ​​the central control screen close to the co-pilot area.

[0216] Optionally, the first input is the user's input, and the control unit 1320 is specifically used to: when the user is located in the third area in the cabin, control the display device to display a three-dimensional display screen of the multiple interface elements; wherein, in the three-dimensional display screen, the normal of the plane where the multiple interface elements are located points to the third area, or the normal of the plane where the multiple interface elements are located points to the user's line of sight.

[0217] Optionally, the control unit 1320 is further configured to control at least some of the multiple interface elements in the zoomed-out display state to move in a preset direction.

[0218] Optionally, the control unit 1320 is also used to control the display device to display a second interface before controlling the display device to display the first interface, the second interface including a fifth interface element set in an expanded display state, the multiple interface elements including the fifth interface element set; obtain the user's second input on the second interface; in response to the second input, control the interface elements in the fifth interface element set to gradually shrink and move in a preset direction to display the first interface.

[0219] Optionally, the control unit 1320 is used to control the display device to display the voice assistant in the awakened state on the first interface in response to the second input; and according to the user's first voice command, control the interface elements in the first interface element set corresponding to the first voice command to switch from a reduced display state to an expanded display state.

[0220] Optionally, the first input is a second voice command, and the control unit 1320 is used to control the display device to display a third interface before controlling the display device to display the first interface; when the display device displays the third interface, obtain the second voice command, and the second voice command includes a wake-up word and the user's intention; in response to obtaining the wake-up word, control the display device to display the first interface; according to the user's intention, control the interface elements in the first interface element set corresponding to the first input to switch from a reduced display state to an expanded display state.

[0221] Optionally, the first input is input from one or more sensors in the vehicle, and the control unit 1320 is specifically configured to control the display device to display the first interface according to the input from the one or more sensors.

[0222] Optionally, the control unit 1320 is further configured to control the multiple interface elements in the zoomed-out display state to flash and / or change in size according to a preset frequency.

[0223] In combination with the second aspect, in some implementations of the second aspect, the device further includes: a determination unit, configured to determine the first set of interface elements based on the first input and the machine learning model.

[0224] Optionally, the device further includes: a sending unit, used to send first indication information to the cloud server based on the first input, wherein the first indication information is used to indicate the first input; and a receiving unit, used to receive information of the first interface element set sent by the cloud server.

[0225] It should be understood that the division of the various units in the above device is merely a division of logical functions. In actual implementation, they may be fully or partially integrated into a single physical entity, or they may be physically separated. Furthermore, the units in the device may be implemented in the form of a processor calling software; for example, the device includes a processor connected to a memory storing instructions, and the processor calls the instructions stored in the memory to implement any of the above methods or the functions of the various units in the device, where the processor is, for example, a general-purpose processor such as a CPU or a microprocessor, and the memory is a memory within the device or a memory external to the device. Alternatively, the units in the device may be implemented in the form of hardware circuits, and the functions of some or all of the units may be implemented through the design of the hardware circuits. The hardware circuits may be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units may be implemented through the design of the logical relationships between the components within the circuits. In another implementation, the hardware circuit may be implemented using a PLD, such as an FPGA, which may include a large number of logic gate circuits, and the connections between the logic gate circuits may be configured using a configuration file to implement the functions of some or all of the above units. All units of the above apparatus may be implemented entirely in the form of software called by a processor, or entirely in the form of hardware circuits, or partially in the form of software called by a processor and the rest in the form of hardware circuits.

[0226] In an embodiment of the present application, a processor is a circuit with the ability to process signals. In one implementation, the processor may be a circuit with the ability to read and execute instructions, such as a CPU, a microprocessor, a GPU, or a DSP. In another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit may be fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the configuration of the hardware circuit can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, DPU, etc.

[0227] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0228] In addition, the various units in the above apparatus may be fully or partially integrated together, or may be implemented independently. In one implementation, these units are integrated together and implemented in the form of a system-on-chip (SoC). The SoC may include at least one processor for implementing any of the above methods or implementing the functions of the various units of the apparatus. The at least one processor may be of different types, for example, including a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0229] An embodiment of the present application also provides a human-computer interaction device, which includes a processing unit and a storage unit, wherein the storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit to enable the device to execute the method or steps executed in the above embodiment.

[0230] Alternatively, if the human-computer interaction device is located in a vehicle, the processing unit may be Figure 1 Processors 151-15n are shown.

[0231] An embodiment of the present application further provides a human-computer interaction system, which may include a computing platform and a display screen, and the computing platform may include the above-mentioned human-computer interaction device 1300.

[0232] An embodiment of the present application also provides a vehicle, which may include the above-mentioned human-computer interaction device 1300 or human-computer interaction system.

[0233] An embodiment of the present application further provides a computer program product, which includes: computer program code, which enables the computer to execute the method in the above embodiment when the computer program code is run on a computer.

[0234] An embodiment of the present application further provides a computer-readable medium, wherein the computer-readable medium stores a program code. When the computer program code runs on a computer, the computer executes the method in the above embodiment.

[0235] An embodiment of the present application further provides a chip, which includes a circuit, and the circuit is used to execute the method in the above embodiment.

[0236] During implementation, each step of the above method can be completed by an integrated logic circuit of the hardware in the processor or by instructions in the form of software. The method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or a power-on erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0237] It should be understood that in the embodiment of the present application, the memory may include a read-only memory and a random access memory, and provide instructions and data to the processor.

[0238] It should also be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0239] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0240] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0241] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0242] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0243] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0244] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0245] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be covered and fall within the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A human-computer interaction method, characterized in that: include: Controlling a display device to display a first interface, the first interface including a plurality of interface elements in a reduced display state, each of the plurality of interface elements corresponding to a service, an application, or an atomic function; According to a first input, interface elements in a first interface element set corresponding to the first input are controlled to switch from a reduced display state to an expanded display state, the plurality of interface elements including the first interface element set.

2. The method according to claim 1, characterized in that Before controlling the interface elements in the first interface element set corresponding to the first input to switch from a reduced display state to an expanded display state, the method further includes: Controlling the interface elements in the second set of interface elements on the first interface to switch from the reduced display state to the expanded display state based on the frequency and / or duration of use of each interface element in the plurality of interface elements by the user; When the duration of controlling the interface elements in the second interface element set to be in the expanded display state continues for a first preset duration, the interface elements in the second interface element set are controlled to switch from the expanded display state to the reduced display state.

3. The method according to claim 2, characterized in that The plurality of interface elements include a third interface element set, and the usage frequency and / or usage duration of the plurality of interface elements in the third interface element set meet a preset condition, The controlling the display device to display the first interface includes: Control the display device to display the icon of the application corresponding to each interface element in the third interface element set, and control the display device to display multiple preset shapes, each of the multiple preset shapes corresponds one-to-one to each interface element in the fourth interface element set, and the fourth interface element set is the interface elements in the multiple interface elements other than the third interface element set.

4. The method according to claim 3, characterized in that The preset shape is a dot shape, a star shape or a sphere shape.

5. The method according to any one of claims 1 to 4, characterized in that The controlling the display device to display the first interface includes: The display device is controlled to display the first interface according to the area where the user is located in the cabin.

6. The method according to claim 5, characterized in that The display device is a central control screen, and controlling the display device to display the first interface according to the area where the user is located in the cabin includes: When a user is in the main driving area, controlling the first area of ​​the central control screen to display interface elements related to vehicle control and navigation among the multiple interface elements, the first area being the display area of ​​the central control screen close to the main driving area; and / or When there is a user in the co-pilot area, the second area in the central control screen is controlled to display interface elements related to entertainment and social networking among the multiple interface elements. The second area is the display area of ​​the central control screen close to the co-pilot area.

7. The method according to claim 5, characterized in that The first input is a user input, and controlling the display device to display the first interface according to the area where the user is located in the cabin includes: When the user is located in a third area in the cabin, controlling the display device to display a three-dimensional display screen of the plurality of interface elements; In the three-dimensional display screen, the normals of the plane where the multiple interface elements are located point to the third area, or the normals of the plane where the multiple interface elements are located point to the direction of the user's line of sight.

8. The method according to any one of claims 1 to 7, characterized in that The method further comprises: Control at least some of the multiple interface elements in the zoomed-out display state to move in a preset direction.

9. The method according to any one of claims 1 to 8, characterized in that Before controlling the display device to display the first interface, the method includes: Controlling the display device to display a second interface, where the second interface includes a fifth interface element set in an expanded display state, and the plurality of interface elements include the fifth interface element set; The controlling the display device to display the first interface includes: Obtaining a second input from the user on the second interface; In response to the second input, the interface elements in the fifth interface element set are controlled to gradually shrink and move in a preset direction to display the first interface.

10. The method according to claim 9, characterized in that The method further comprises: In response to the second input, controlling the display device to display the voice assistant in the awake state on the first interface; The step of controlling, based on the first input, the interface elements in the first interface element set corresponding to the first input to switch from a reduced display state to an expanded display state includes: According to a first voice instruction of the user, the interface elements in the first interface element set corresponding to the first voice instruction are controlled to switch from a reduced display state to an expanded display state.

11. The method according to any one of claims 1 to 8, characterized in that The first input is a second voice command, and before controlling the display device to display the first interface, the method further includes: Controlling the display device to display a third interface; When the display device displays the third interface, obtaining the second voice instruction, where the second voice instruction includes a wake-up word and a user's intention; The controlling the display device to display the first interface includes: In response to acquiring the wake-up word, controlling the display device to display the first interface; The controlling, according to the first input, the interface elements in the first interface element set corresponding to the first input to switch from a reduced display state to an expanded display state includes: According to the user's intention, the interface elements in the first interface element set corresponding to the first input are controlled to switch from a reduced display state to an expanded display state.

12. The method according to any one of claims 1 to 8, characterized in that The first input is input from one or more sensors in the vehicle, and the control display device displays a first interface, including: According to input from the one or more sensors, the display device is controlled to display the first interface.

13. The method according to any one of claims 1 to 12, characterized in that The method further comprises: The multiple interface elements in the zoomed-out display state are controlled to flash and / or change in size according to a preset frequency.

14. The method according to any one of claims 1 to 13, characterized in that The method further comprises: Determine the first set of interface elements based on the first input and the machine learning model.

15. The method according to any one of claims 1 to 13, characterized in that The method further comprises: Sending first indication information to a cloud server according to the first input, where the first indication information is used to indicate the first input; Receive information about the first interface element set sent by the cloud server.

16. A human-computer interaction device, characterized in that: It includes an acquisition unit and a control unit, wherein, The control unit is configured to control the display device to display a first interface, wherein the first interface includes a plurality of interface elements in a reduced display state, each of the plurality of interface elements corresponding to a service, an application, or an atomic function; The control unit is further configured to control, based on the first input acquired by the acquisition unit, the interface elements in the first interface element set corresponding to the first input, to switch from a reduced display state to an expanded display state, wherein the multiple interface elements include the first interface element set.

17. The device according to claim 16, characterized in that The control unit controls the interface elements in the first interface element set corresponding to the first input to switch from a reduced display state to an expanded display state. The control unit is further configured to control the interface elements in the second set of interface elements on the first interface to switch from the reduced display state to the expanded display state based on the frequency and / or duration of use of each interface element in the plurality of interface elements by the user; When the duration of controlling the interface elements in the second interface element set to be in the expanded display state continues for a first preset duration, the interface elements in the second interface element set are controlled to switch from the expanded display state to the reduced display state.

18. The device according to claim 17, characterized in that The plurality of interface elements include a third interface element set, and the usage frequency and / or usage duration of the plurality of interface elements in the third interface element set meet a preset condition, Wherein, the control unit is specifically used for: Control the display device to display the icon of the application corresponding to each interface element in the third interface element set, and control the display device to display multiple preset shapes, each of the multiple preset shapes corresponds one-to-one to each interface element in the fourth interface element set, and the fourth interface element set is the interface elements in the multiple interface elements other than the third interface element set.

19. The device according to claim 18, characterized in that The preset shape is a dot shape, a star shape or a sphere shape.

20. The device according to any one of claims 16 to 19, characterized in that The control unit is specifically used for: The display device is controlled to display the first interface according to the area where the user is located in the cabin.

21. The device according to claim 20, characterized in that The display device is a central control screen, and the control unit is specifically used to: When a user is in the main driving area, controlling a first area of ​​the central control screen to display interface elements related to vehicle control and navigation among the multiple interface elements, the first area being a display area of ​​the central control screen close to the main driving area; and / or When there is a user in the co-pilot area, the second area in the central control screen is controlled to display interface elements related to entertainment and social networking among the multiple interface elements. The second area is the display area of ​​the central control screen close to the co-pilot area.

22. The device according to claim 20, characterized in that The first input is a user input, and the control unit is specifically configured to: When the user is located in a third area in the cabin, controlling the display device to display a three-dimensional display screen of the plurality of interface elements; In the three-dimensional display screen, the normals of the plane where the multiple interface elements are located point to the third area, or the normals of the plane where the multiple interface elements are located point to the direction of the user's line of sight.

23. The device according to any one of claims 16 to 22, characterized in that The control unit is further configured to control at least some of the multiple interface elements in the zoomed-out display state to move in a preset direction.

24. The device according to any one of claims 16 to 23, characterized in that The control unit is further configured to control the display device to display a second interface before controlling the display device to display the first interface, wherein the second interface includes a fifth interface element set in an expanded display state, and the plurality of interface elements include the fifth interface element set; Obtaining a second input from the user on the second interface; In response to the second input, the interface elements in the fifth interface element set are controlled to gradually shrink and move in a preset direction to display the first interface.

25. The device according to claim 24, characterized in that The control unit is configured to control the display device to display the voice assistant in the awake state on the first interface in response to the second input; According to a first voice instruction of the user, the interface elements in the first interface element set corresponding to the first voice instruction are controlled to switch from a reduced display state to an expanded display state.

26. The device according to any one of claims 16 to 23, characterized in that The first input is a second voice command, The control unit is configured to control the display device to display a third interface before controlling the display device to display the first interface; When the display device displays the third interface, obtaining the second voice instruction, where the second voice instruction includes a wake-up word and a user's intention; In response to acquiring the wake-up word, controlling the display device to display the first interface; According to the user's intention, the interface elements in the first interface element set corresponding to the first input are controlled to switch from a reduced display state to an expanded display state.

27. The device according to any one of claims 16 to 23, characterized in that The first input is input from one or more sensors in the vehicle, and the control unit is specifically configured to: According to input from the one or more sensors, the display device is controlled to display the first interface.

28. The device according to any one of claims 16 to 27, characterized in that The control unit is further configured to control the multiple interface elements in the reduced display state to flash and / or change in size according to a preset frequency.

29. The device according to any one of claims 16 to 28, characterized in that The device further comprises: A determination unit is used to determine the first set of interface elements based on the first input and the machine learning model.

30. The device according to any one of claims 16 to 28, characterized in that The device further comprises: a sending unit, configured to send first indication information to a cloud server according to the first input, wherein the first indication information is used to indicate the first input; A receiving unit is used to receive information about the first interface element set sent by the cloud server.

31. A human-computer interaction device, characterized in that: include: memory for storing computer programs; A processor, configured to execute the computer program stored in the memory, so that the apparatus performs the method according to any one of claims 1 to 15.

32. A human-computer interaction system, characterized in that: The device comprises a display device and a computing platform, wherein the computing platform comprises the device according to any one of claims 16 to 31.

33. An electronic device, characterized in that: Comprising the apparatus of any one of claims 16 to 31, or the system of claim 32.

34. The electronic device according to claim 33, wherein: The electronic device is a vehicle.

35. A computer-readable storage medium, characterized in that Instructions are stored thereon, and when the instructions are executed by a processor, the processor is caused to implement the method according to any one of claims 1 to 15.

36. A computer program product, characterized in that The computer program product comprises a computer program code, which, when run on a computer, causes the computer to implement the method according to any one of claims 1 to 15 .

37. A chip, characterized in that: The chip comprises a circuit for performing the method according to any one of claims 1 to 15 .

Images