Human-machine interaction apparatus, system and method, program product, and vehicle

Abstract

A human-machine interaction apparatus, system and method, a program product, and a vehicle. The human-machine interaction apparatus is used for acquiring driving environment information and / or intelligent driving control information of a vehicle, and constructing an intelligent driving scenario image according to the driving environment information and / or the intelligent driving control information. The human-machine interaction apparatus is further configured for sending the intelligent driving scenario image to a display apparatus, so that the intelligent driving scenario image is displayed on the display apparatus.

Description

Human-computer interaction devices, systems, methods, programs, products and vehicles

[0001] Cross-reference to related applications

[0002] This disclosure claims priority to Chinese Patent Application No. 202411182079.4, filed on August 27, 2024, entitled "Human-computer interaction device, system, method, program product and vehicle", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to the field of vehicle technology, and more specifically, to a human-computer interaction device, system, method, program product, and vehicle. Background Technology

[0004] In the development of new energy vehicles, especially autonomous vehicles, the role of the Human-Machine Interface (HMI) has become particularly important. It not only needs to provide traditional vehicle control functions, but also needs to support interactions related to the autonomous driving system.

[0005] In related technologies, after receiving intelligent driving control information transmitted by the human-machine interface (HMI) and environmental perception information transmitted by the perception fusion module, the instrument panel or central control screen usually needs to process the intelligent driving control information and environmental perception information to obtain intelligent driving scene images. Summary of the Invention

[0006] The purpose of this disclosure is to provide a human-computer interaction device, system, method, program product, and vehicle to improve the user experience.

[0007] To achieve the above objectives, the first aspect of this disclosure provides a human-computer interaction device, which is used to acquire vehicle driving environment information and / or intelligent driving control information, and construct an intelligent driving scene image based on the driving environment information and / or the intelligent driving control information;

[0008] The human-computer interaction device is also used to send the intelligent driving scene image to the display device so that the intelligent driving scene image is displayed on the display device.

[0009] Optionally, the human-computer interaction device includes:

[0010] The human-computer interaction engine module and the human-computer interaction agent module are connected;

[0011] The human-machine interaction engine module is used to acquire the driving environment information and / or the intelligent driving control information, construct an intelligent driving scene image based on the driving environment information and / or the intelligent driving control information, and send the intelligent driving scene image to the human-machine interaction agent module;

[0012] The human-computer interaction agent module is connected to the display device and is used to send the intelligent driving scene image to the display device so that the intelligent driving scene image is displayed on the display device.

[0013] Optionally, the human-computer interaction agent module is further configured to acquire the vehicle body status information and send the vehicle body status information to the display device so that the vehicle body status information is displayed on the display device.

[0014] Optionally, the human-computer interaction engine module includes:

[0015] A perception information fusion unit and a scene construction unit, wherein the perception information fusion unit is connected to the scene construction unit;

[0016] The perception information fusion unit is used to acquire the driving environment information, process the driving environment information to obtain perception fusion data, and send the perception fusion data to the scene construction unit.

[0017] The scene construction unit is used to acquire the intelligent driving control information, construct the intelligent driving scene image based on the perception fusion data and the intelligent driving control information, and send the intelligent driving scene image to the human-machine interaction agent module.

[0018] Optionally, the human-computer interaction engine module further includes:

[0019] An intelligent driving status acquisition unit, which is connected to the scene construction unit;

[0020] The intelligent driving status acquisition unit is used to acquire the current intelligent driving status data of the vehicle and send the current intelligent driving status data to the scene construction unit;

[0021] The scenario construction unit is used to obtain the intelligent driving control information corresponding to the current intelligent driving state data in the intelligent driving system based on the current intelligent driving state data.

[0022] Optionally, the scene construction unit is further configured to send the current intelligent driving status data to the human-machine interaction agent module, and send the current intelligent driving status data to the display device through the human-machine interaction agent module, so that the current intelligent driving status data is displayed on the display device.

[0023] Optionally, the human-computer interaction agent module includes: an information encapsulation unit and a first information transmission interface;

[0024] The information encapsulation unit is connected to the human-machine interaction engine module and is used to receive the intelligent driving scene image and / or the current intelligent driving status data sent by the human-machine interaction engine module, encapsulate the intelligent driving scene image and / or the current intelligent driving status data to obtain encapsulation information, and send the encapsulation information to the display device via the first information transmission interface.

[0025] Optionally, the human-computer interaction agent module further includes a second information transmission interface;

[0026] The second information transmission interface is connected to the information encapsulation unit and is used to acquire the vehicle body status information and send the vehicle body status information to the information encapsulation unit;

[0027] The information encapsulation unit is used to encapsulate the intelligent driving scene image and / or the current intelligent driving state data and / or the vehicle body state information to obtain encapsulation information, and send the encapsulation information to the display device via the first information transmission interface.

[0028] A second aspect of this disclosure provides a human-computer interaction system, the human-computer interaction system including a human-computer interaction device and a display device as provided in the first aspect of this disclosure;

[0029] The human-computer interaction device is used to acquire vehicle driving environment information and / or intelligent driving control information, and to construct an intelligent driving scene image based on the driving environment information and / or the intelligent driving control information.

[0030] The human-computer interaction device is connected to the display device and is used to send the intelligent driving scene image to the display device so that the intelligent driving scene image is displayed on the display device.

[0031] Optionally, the human-computer interaction system further includes:

[0032] An environmental sensing device is provided, and the human-computer interaction device is connected to the environmental sensing device to acquire the driving environment information of the vehicle detected by the environmental sensing device.

[0033] Optionally, the human-computer interaction system includes:

[0034] An intelligent driving system, wherein the human-machine interaction device is connected to the intelligent driving system and is used to obtain intelligent driving control information from the intelligent driving system.

[0035] Optionally, the intelligent driving system includes multiple intelligent driving function modules, each of which corresponds to an intelligent driving state;

[0036] The human-machine interaction device is used to acquire the current intelligent driving status data of the vehicle, as well as the intelligent driving control information in the intelligent driving function module corresponding to the current intelligent driving status data.

[0037] Optionally, the intelligent driving system further includes an intelligent driving state detection module;

[0038] The intelligent driving status detection module is connected to the human-machine interaction device and is used to detect the current intelligent driving status data of the vehicle and send the current intelligent driving status data to the human-machine interaction device.

[0039] Optionally, the human-computer interaction device is further configured to acquire the vehicle body status information and send the vehicle body status information to the display device so that the vehicle body status information is displayed on the display device.

[0040] A third aspect of this disclosure provides a human-computer interaction method applied to a human-computer interaction device as described in the first aspect of this disclosure, the human-computer interaction method comprising:

[0041] Acquire vehicle driving environment information and / or intelligent driving control information;

[0042] Construct an intelligent driving scene image based on the driving environment information and / or the intelligent driving control information;

[0043] The intelligent driving scene image is sent to the display device so that the intelligent driving scene image is displayed on the display device.

[0044] Optionally, the human-computer interaction method further includes:

[0045] Obtain the vehicle's body status information;

[0046] The vehicle body status information is sent to the display device so that the vehicle body status information is displayed on the display device.

[0047] Optionally, intelligent driving control information is obtained, including:

[0048] Obtain the current intelligent driving status data of the vehicle;

[0049] Based on the current intelligent driving status data, obtain the intelligent driving control information corresponding to the current intelligent driving status data in the intelligent driving system.

[0050] Optionally, the human-computer interaction method further includes:

[0051] The current intelligent driving status data is sent to the display device so that the current intelligent driving status data is displayed on the display device.

[0052] This fourth aspect of the disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the third aspects of the disclosure.

[0053] The fifth aspect of this disclosure provides a vehicle including a human-machine interaction system as described in any one of the second aspects of this disclosure.

[0054] By employing the above technical solution, the human-machine interface device acquires the vehicle's driving environment information and / or intelligent driving control information, and constructs an intelligent driving scene image based on this information. Then, it sends the autonomous driving scene image to the display device for display. In this way, the human-machine interface device processes the driving environment information and / or intelligent driving control information to obtain the intelligent driving scene image, eliminating the need for rendering on the display device. This effectively reduces the latency of rendering the intelligent driving scene image, thereby improving the user experience.

[0055] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0056] The accompanying drawings are provided to further understand the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof.

[0057] Figure 1A is a schematic diagram of the human-computer interaction system in the related technology of this application.

[0058] Figure 1B is a schematic diagram of the environmental perception information transmission channel in the related technology of this application.

[0059] Figure 2 is a block diagram of a human-computer interaction system according to an exemplary embodiment of this application.

[0060] Figure 3 is a block diagram of another human-computer interaction system according to an exemplary embodiment of this application.

[0061] Figure 4 is a schematic diagram of a human-computer interaction system according to an exemplary embodiment of this application.

[0062] Figure 5 is a flowchart illustrating a human-computer interaction method according to an exemplary embodiment of this application. Detailed Implementation

[0063] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.

[0064] In related technologies, because instrument panels or central control screens are typically located within the vehicle's infotainment domain, and the computing power of the vehicle's domain controller is limited, rendering delays occur, resulting in a poor user experience. Furthermore, the Human-Machine Interface (HMI) primarily transmits two types of information: vehicle status information and intelligent driving control information. With the development of automotive intelligence, the transmission of environmental perception information has become an indispensable part of intelligent driving systems. Currently, the channel for transmitting environmental perception information to the instrument panel or central control screen is independent of the HMI channel.

[0065] For example, Figure 1A is a schematic diagram of a human-machine interaction system in the related technology. As shown in Figure 1A, the human-machine interaction device acquires vehicle status information and intelligent driving control information, and transmits the vehicle status information and intelligent driving control information to the instrument panel or central control screen. Figure 1B is a schematic diagram of an environmental perception information transmission channel in the related technology. As shown in Figure 1B, the perception fusion module acquires environmental perception information and performs fusion processing, and then transmits the processed environmental perception information to the instrument panel or central control screen. In this way, on the one hand, the instrument panel or central control screen needs to set up two sets of transmission interfaces to receive the information transmitted by the human-machine interaction device and the information transmitted by the perception fusion module respectively, which is not conducive to the unified management and iteration of the interfaces. On the other hand, after receiving the intelligent driving control information transmitted by the human-machine interaction device and the environmental perception information transmitted by the perception fusion module, the instrument panel or central control screen usually needs to reconstruct the intelligent driving control information and environmental perception information to obtain an intelligent driving scene image. However, since the instrument panel or central control screen is usually located in the vehicle domain, and the computing performance of the vehicle domain controller is limited, it will cause rendering delay, resulting in a poor user experience.

[0066] In view of this, the present disclosure provides a human-computer interaction device, system, method, program product and vehicle, which uses the human-computer interaction device to process driving environment information and / or intelligent driving control information to obtain intelligent driving scene images without the need for rendering processing by a display device, effectively reducing the latency of rendering intelligent driving scene images, thereby improving the user experience.

[0067] For ease of description, the human-computer interaction system provided in this disclosure is described below.

[0068] Figure 2 is a block diagram illustrating a human-computer interaction system according to an exemplary embodiment. As shown in Figure 2, the human-computer interaction system 200 may include a human-computer interaction device 201 and a display device 204.

[0069] The human-machine interface device 201 is used to acquire vehicle driving environment information and / or intelligent driving control information, and to construct an intelligent driving scene image based on the driving environment information and / or intelligent driving control information. The human-machine interface device 201 is connected to the display device 204 and is used to send the intelligent driving scene image to the display device 204 so that the intelligent driving scene image is displayed on the display device 204.

[0070] By employing the above technical solution, the human-machine interface device acquires the vehicle's driving environment information and / or intelligent driving control information, and constructs an intelligent driving scene image based on this information. Then, it sends the autonomous driving scene image to the display device for display. In this way, the human-machine interface device processes the driving environment information and / or intelligent driving control information to obtain the intelligent driving scene image, eliminating the need for rendering on the display device. This effectively reduces the latency of rendering the intelligent driving scene image, thereby improving the user experience.

[0071] In one embodiment, the human-computer interaction system 200 may further include an environmental sensing device 203. FIG3 is a block diagram of another human-computer interaction system according to an exemplary embodiment. As shown in FIG3, the human-computer interaction device 201 is connected to the environmental sensing device 203 and is used to acquire vehicle driving environment information detected by the environmental sensing device 203.

[0072] The environmental perception device 203 may include, but is not limited to, cameras, millimeter-wave radar, and lidar. Accordingly, the vehicle's driving environment information may include, but is not limited to, raw information collected by cameras, raw information collected by millimeter-wave radar, and raw information collected by lidar. This information can characterize the relationship between the vehicle and its surrounding environment, such as vehicle position and attitude information, road lane line information, obstacle information, indicator light and road sign information, parking space information, etc.

[0073] In another embodiment, the human-machine interface system 200 may further include an intelligent driving system 202. As shown in FIG3, the human-machine interface device 201 may also be connected to the intelligent driving system 202 to acquire intelligent driving control information. The intelligent driving system 202 may include multiple intelligent driving function modules, such as, but not limited to: a Remote Parking Assist (RPA) function module, an Auto Parking Assist (APA) function module, a Navigate On Autopilot (NOA) function module, and an Advanced Driver Assistance System (ADAS) function module. Correspondingly, the intelligent driving control information may include intelligent driving control information from the Remote Parking Assist function module, the Auto Parking Assist function module, the Navigate On Autopilot function module, and the Advanced Driver Assistance System (ADAS) function module, etc.

[0074] For example, the intelligent driving control information of the automatic parking function module includes, but is not limited to: parking path, parking direction, current vehicle speed, vehicle position and target parking space, etc. The intelligent navigation function module includes, but is not limited to: lane keeping, lane selection, lane change instruction, target vehicle following reminder, etc.

[0075] It should be understood that multiple intelligent driving function modules may only have one function module in an active state at the same time, and the human-machine interaction device can only obtain the intelligent driving control information of the function module in an active state.

[0076] For example, the intelligent driving system 202 includes multiple intelligent driving function modules, each corresponding to an intelligent driving state. The human-machine interaction device 201 is used to acquire the vehicle's current intelligent driving state data, as well as the intelligent driving control information in the intelligent driving function module corresponding to the current intelligent driving state data.

[0077] In addition, the intelligent driving system 202 may also include an intelligent driving state detection module, which is connected to the human-machine interaction device 201 and is used to detect the current intelligent driving state data of the vehicle and send the current intelligent driving state data to the human-machine interaction device 201.

[0078] The intelligent driving state may include, but is not limited to, one of the following: remote parking state, automatic parking state, intelligent navigation state, and advanced driver assistance state. For example, if multiple intelligent driving function modules may include a remote parking function module, an automatic parking function module, an intelligent navigation function module, and an advanced driver assistance system function module, then the intelligent driving state may include one of the following: remote parking state, automatic parking state, intelligent navigation state, and advanced driver assistance state.

[0079] For example, if the intelligent driving state is remote parking, the human-machine interface device 201 obtains the intelligent driving control information of the remote parking function module; if the intelligent driving state is automatic parking, the human-machine interface device 201 obtains the intelligent driving control information of the automatic parking function module; if the intelligent driving state is intelligent navigation, the human-machine interface device 201 obtains the intelligent driving control information of the intelligent navigation function module; if the intelligent driving state is advanced driver assistance state, the human-machine interface device 201 obtains the intelligent driving control information of the advanced driver assistance system function module.

[0080] In this way, the human-machine interaction device can obtain the vehicle's current intelligent driving status data, and then obtain the intelligent driving control information in the intelligent driving function module corresponding to the current intelligent driving status data, thereby improving the reliability of the human-machine interaction device in obtaining intelligent driving control information.

[0081] The human-machine interaction device 201 can construct an intelligent driving scene image based on driving environment information and / or intelligent driving control information in at least one of the following ways: the human-machine interaction device 201 constructs an intelligent driving scene image based on driving environment information; the human-machine interaction device 201 constructs an intelligent driving scene image based on intelligent driving control information; the human-machine interaction device 201 constructs an intelligent driving scene image based on driving environment information and intelligent driving control information.

[0082] The specific implementation method of the human-computer interaction device 201 constructing an intelligent driving scene image based on driving environment information and intelligent driving control information is as follows: the human-computer interaction device 201 synchronizes the driving environment information and intelligent driving control information in time, and then constructs an intelligent driving scene image based on the time-synchronized driving environment information and intelligent driving control information. The techniques for synchronizing driving environment information and intelligent driving control information in time, and for constructing an intelligent driving scene image based on the time-synchronized driving environment information and intelligent driving control information, are relatively mature technologies, and this disclosure does not specifically limit them.

[0083] For example, the human-machine interface device 201 may be located within the intelligent driving domain of the vehicle. The display device 204 is located within the vehicle's infotainment domain. For instance, the display device 204 may include an in-vehicle display screen and / or an in-vehicle instrument cluster.

[0084] Since no display device is needed to construct the intelligent driving scene image, the coupling between the vehicle infotainment domain (where the display device resides) and the intelligent driving domain is effectively reduced. The vehicle infotainment domain only needs to provide a unified interface for the human-machine interaction device, eliminating the need for specific processing steps such as information collection, time synchronization, and intelligent driving scene construction within the intelligent driving domain, thus achieving decoupling. This reduces the impact of vehicle software and hardware iteration updates on vehicle infotainment rendering, and also reduces the latency of vehicle infotainment rendering by reducing the computational load on the vehicle infotainment domain.

[0085] By employing the above technical solution, the human-machine interface device acquires the vehicle's driving environment information and / or intelligent driving control information, and constructs an intelligent driving scene image based on this information. Then, it sends the autonomous driving scene image to the display device for display. In this way, the human-machine interface device processes the driving environment information and / or intelligent driving control information to obtain the intelligent driving scene image, eliminating the need for rendering on the display device. This effectively reduces the latency of rendering the intelligent driving scene image, thereby improving the user experience.

[0086] Furthermore, compared to existing technologies, this disclosure integrates independent environmental perception information pathways into the human-machine interaction system, enabling driving environment information and intelligent driving control information to be transmitted to the display device through the human-machine interaction device. This achieves unified interface design and management, reduces the coupling between the display device and the vehicle's intelligent driving system, and improves the security and stability of interface iteration management.

[0087] The human-computer interaction device 201 will now be described in detail with reference to the accompanying drawings.

[0088] Figure 4 is a schematic diagram of a human-computer interaction system according to an exemplary embodiment. As shown in Figure 4, the human-computer interaction device 201 may include a human-computer interaction engine module 2011 and a human-computer interaction agent module 2012. The human-computer interaction engine module 2011 is connected to the human-computer interaction agent module 2012. The human-computer interaction engine module 2011 is used to acquire driving environment information and / or intelligent driving control information, construct an intelligent driving scene image based on the driving environment information and / or intelligent driving control information, and send the intelligent driving scene image to the human-computer interaction agent module 2012.

[0089] For example, as shown in Figure 4, the human-machine interaction engine module 2011 is connected to the human-machine interaction agent module 2012, the environmental perception device 203, and the intelligent driving system 202, respectively. It is used to acquire vehicle driving environment information and intelligent driving control information, construct an intelligent driving scene image based on the driving environment information and intelligent driving control information, and send the intelligent driving scene image to the human-machine interaction agent module 2012. The intelligent driving system 202 can be located within the intelligent driving domain.

[0090] The human-computer interaction agent module 2012 is connected to the display device 204 and is used to send the intelligent driving scene image to the display device 204 when it receives the intelligent driving scene image, so that the intelligent driving scene image is displayed on the display device 204.

[0091] In other words, in this disclosure, the human-machine interaction engine module 2011 is used to receive raw driving environment information and / or intelligent driving control information, create a complete intelligent driving scene image based on the raw driving environment information and / or intelligent driving control information, and send it to the human-machine interaction agent module 2012. The human-machine interaction agent module 2012 is used for information collection and forwarding, for example, receiving the intelligent driving scene image from the human-machine interaction engine module 2011 and forwarding the intelligent driving scene image to the display device 204.

[0092] This approach employs a composite modular design for the human-computer interaction device 201, decoupling its interface encapsulation and information collection and processing functions into a human-computer interaction engine module 2011 and a human-computer interaction proxy module 2012, respectively. This enables partial iterative updates to the human-computer interaction device 201, effectively reducing the risks and difficulties associated with updating the device and contributing to the stability of its updates.

[0093] For example, when the display device is an in-vehicle display screen and / or an in-vehicle instrument panel, when the vehicle-to-machine communication interface is updated, only the human-machine interaction agent module 2012 needs to be updated to match the update of the vehicle-to-machine interface. When the interface for obtaining driving environment information or the interface for obtaining intelligent driving control information is modified, only the human-machine interaction engine module 2011 needs to be updated to match the modification of the interface for obtaining driving environment information or the interface for obtaining intelligent driving control information.

[0094] Furthermore, in this disclosure, the human-machine interface device 201 can also send vehicle status information to the display device 204 so that the vehicle status information is displayed on the display device 204, making it easier for the driver to better understand the vehicle status. For example, as shown in FIG4, the human-machine interface agent module 2012 is also used to acquire vehicle status information and send it to the display device 204 to display the vehicle status information on the display device 204.

[0095] As shown in Figure 4, vehicle status information may include, but is not limited to, chassis status control information, cabin status control information, door status information, and driving status control information. Chassis status information may include powertrain status information, wheel suspension status information, and braking system status information. Cabin status control information may include information on whether the user is wearing a seatbelt. Door status information may include information on whether the vehicle body is closed. Driving status control information may include vehicle speed information and driving mode status information.

[0096] For example, in a high-speed driving scenario, the human-machine interaction agent module 2012 can receive vehicle speed information and fault status information of various chassis components from the chassis, receive information on whether passengers are wearing seat belts from the cabin, and receive information on whether doors and / or trunks are open from the vehicle body. After collecting all the above information, it encapsulates the message according to the vehicle communication interface format through the Controller Area Network (CAN) protocol and sends it to the display device at a predetermined frequency for display.

[0097] Using the above technical solution, the human-machine interaction engine module 2011 sends the intelligent driving scene image it constructs to the human-machine interaction agent module 2012, which then sends it to the display device 204. In addition, the human-machine interaction agent module 2012 can also send the vehicle status information it acquires to the display device 204. Thus, both the intelligent driving scene image and the vehicle status information are sent to the display device 204 via the human-machine interaction agent module 2012.

[0098] Thus, on the one hand, the human-machine interface device can simultaneously receive and process three types of information: vehicle status information, driving environment information, and intelligent driving control information. It can also communicate with the display device located in the vehicle's domain controller via a unified, encapsulated communication interface, achieving decoupling between the vehicle domain controller and other domain controllers such as chassis domain controller, cockpit domain controller, and intelligent driving domain controller. On the other hand, compared to the prior art shown in Figure 1A, this disclosure uses different modules to acquire vehicle status information and intelligent driving control information. That is, the vehicle status information processing function and the intelligent driving control information processing function are set on different modules. This ensures that the normal display of vehicle status information is not affected when the human-machine interface engine module 2011 malfunctions, improving the robustness of the human-machine interface device. Furthermore, during iterative updates, the human-machine interface device can achieve partial iterations, effectively reducing the difficulty of updating and iterating the device, and contributing to the stability of the device's version updates.

[0099] It should be understood that the human-computer interaction device can send vehicle status information to the display device separately, or send intelligent driving scene images to the display device separately, or send vehicle status information and intelligent driving scene images to the display device simultaneously. This disclosure does not make any specific limitations in this regard.

[0100] In one embodiment, as shown in FIG4, the human-computer interaction engine module 2011 may include a perception information fusion unit 20111 and a scene construction unit 20112. The perception information fusion unit 20111 and the scene construction unit 20112 are connected. The perception information fusion unit 20111 is used to acquire vehicle driving environment information, process the driving environment information to obtain perception fusion data, and send the perception fusion data to the scene construction unit 20112.

[0101] As shown in Figure 4, the perception information fusion unit 20111 is connected to the environmental perception device 203 and is used to acquire vehicle driving environment information from the environmental perception device 203. Then, the acquired vehicle driving environment information is processed to obtain perception fusion data. For example, the driving environment information is filtered, compared, and fused to convert the original driving environment information collected by the environmental perception device 203 into driving environment information with higher confidence, i.e., perception fusion data. The perception fusion data may include, but is not limited to: vehicle pose information, obstacle information, road and lane line information, street light and sign information, parking space information, etc. Finally, the perception information fusion unit 20111 sends the driving environment information with higher confidence, i.e., the perception fusion data, to the scene construction unit 20112 for the construction of intelligent driving scenarios.

[0102] The scene construction unit 20112 acquires the perception fusion data sent by the perception information fusion unit 20111 and connects with the intelligent driving system 202 to obtain intelligent driving control information from the intelligent driving system 202. Then, it performs time synchronization processing on the acquired perception fusion data and intelligent driving control information, and after synchronization processing, it constructs a real-time intelligent driving scene image.

[0103] In another embodiment, the human-machine interaction engine module 2011 may further include an intelligent driving state acquisition unit 20113. The intelligent driving state acquisition unit 20113 can be connected to the intelligent driving system 202. Furthermore, as shown in FIG4, the intelligent driving state acquisition unit 20113 is also connected to the intelligent driving state detection module 2021 in the intelligent driving system 202, for acquiring the current intelligent driving state data of the intelligent driving system 202. For example, the intelligent driving state represented by the current intelligent driving state data includes, but is not limited to, remote parking state, automatic parking state, intelligent navigation state, and advanced driver assistance state. It should be understood that when the intelligent driving state is remote parking state or automatic parking state, the intelligent driving state may further include parking entry state or parking exit state.

[0104] The scenario construction unit 20112 is used to obtain intelligent driving control information corresponding to the current intelligent driving state data in the intelligent driving system. For example, as shown in Figure 4, the scenario construction unit 20112 is connected to the remote parking function module 2022, the automatic parking function module 2023, the intelligent navigation function module 2024, and the advanced driving assistance system function module 2025 included in the intelligent driving system 202. When the intelligent driving state represented by the current intelligent driving state data is a remote parking state, the scenario construction unit 20112 obtains the intelligent driving control information of the remote parking function module 2022. Alternatively, when the intelligent driving state represented by the current intelligent driving state data is an advanced driving assistance state, the scenario construction unit 20112 obtains the intelligent driving control information of the advanced driving assistance system function module 2025.

[0105] After acquiring driving environment information and intelligent driving control information, the scene construction unit 20112 performs time synchronization processing on the acquired perception fusion data and intelligent driving control information, and then constructs a real-time intelligent driving scene image.

[0106] For example, when the intelligent driving state is in intelligent navigation state, the function flag of the intelligent driving state detection module 2021 switches to intelligent navigation state and synchronizes this state to the intelligent driving state acquisition unit 20113, thereby enabling the scene construction unit 20112 to obtain the current intelligent driving state data. Based on the intelligent navigation function requirements, the scene construction unit 20112 obtains intelligent navigation state control information such as lane keeping, lane selection, lane change instruction, and target vehicle following reminder from the intelligent navigation function module 2024. It then performs time synchronization processing on this intelligent navigation state control information with lane line information, indicator light information, road sign information, surrounding vehicle information, and other obstacle information from the perception information fusion unit 20111 to create a real-time intelligent navigation scene image, which is then forwarded to the human-machine interaction agent module 2012.

[0107] For example, when the intelligent driving state is in automatic parking or remote parking mode, the scene construction unit 20112, upon acquiring intelligent driving control information, can also highlight the target parking space and number it from 1 to 6. Then, it processes the acquired perception fusion data to construct a complete parking scene image including the highlighted target parking space, which is then displayed on the display device. Furthermore, if the user updates the target parking space, the scene construction unit 20112 can also obtain the updated target parking space from the intelligent driving system 202 and highlight the updated target parking space when constructing the parking scene image.

[0108] Furthermore, in this embodiment, the scene construction unit 20112 can obtain the current intelligent driving status data from the intelligent driving status acquisition unit 20113. In this way, the scene construction unit 20112 can also send the current intelligent driving status data to the human-machine interaction agent module 2012, so that the human-machine interaction agent module 2012 can send the current intelligent driving status data to the display device 204, so that the current intelligent driving status data can be displayed on the display device 204, making it convenient for the people in the vehicle to know the intelligent driving status of the vehicle.

[0109] It should be understood that, in this disclosure, the human-machine interaction device may send at least one of the vehicle status information, intelligent driving scene image and current intelligent driving status data to the display device according to actual needs, and the display device may then display at least one of the vehicle status information, intelligent driving scene image and current intelligent driving status data.

[0110] The structure of the Human-Computer Interaction Agent Module 2012 is described below.

[0111] In one embodiment, the human-computer interaction agent module 2012 may include an information encapsulation unit 20121 and a first information transmission interface 20122. The information encapsulation unit 20121 is connected to the human-computer interaction engine module 2011 and is used to receive intelligent driving scene images and / or current intelligent driving status data sent by the human-computer interaction engine module 2011, encapsulate the intelligent driving scene images and / or current intelligent driving status data to obtain encapsulated information, and send the encapsulated information to the display device via the first information transmission interface 20122.

[0112] As shown in Figure 4, the information encapsulation unit 20121 can be connected to the scene construction unit 20112 in the human-computer interaction engine module 2011. It receives intelligent driving scene images sent by the scene construction unit 20112, performs format conversion and encapsulation processing on the intelligent driving scene images according to a preset communication protocol, and sends the encapsulated intelligent driving scene images to the display device 204 via the first information transmission interface 20122. And / or, the information encapsulation unit 20121 receives current intelligent driving status data sent by the scene construction unit 20112, performs format conversion and encapsulation processing on the current intelligent driving status data according to a preset communication protocol, and sends the encapsulated current intelligent driving status data to the display device 204 via the first information transmission interface 20122.

[0113] Furthermore, as shown in Figure 4, the human-machine interaction agent module 2012 may also include a second information transmission interface 20123. The second information transmission interface 20123 is connected to the information encapsulation unit 20121 and is used to acquire vehicle body status information and send the vehicle body status information to the information encapsulation unit 20121. The information encapsulation unit 20121 is used to encapsulate intelligent driving scene images and / or current intelligent driving status data and / or vehicle body status information to obtain encapsulated information, and sends the encapsulated information to the display device 204 via the first information transmission interface 20122.

[0114] For example, the information encapsulation unit 20121 performs format conversion and encapsulation processing on at least one of the intelligent driving scene image, current intelligent driving status data and vehicle status information according to the communication protocol, and sends the encapsulated information to the display device 204 via the first information transmission interface 20122.

[0115] In this way, vehicle status information, intelligent driving scene images, and current intelligent driving status data can all be transmitted to the display device 204 through a unified first information transmission interface 20122. This eliminates the need for the display device to have two sets of transmission interfaces, facilitating unified interface management and iteration. Furthermore, integrating the independent perception environment information pathway into the human-machine interaction system allows driving environment information and intelligent driving control information to be transmitted to the display device through a unified interface. This achieves unified interface design and management, reduces the coupling between the display device and the vehicle's intelligent driving system, and improves the security and stability of interface iteration management.

[0116] Based on the same inventive concept, this disclosure also provides a human-computer interaction method. Figure 5 is a flowchart illustrating a human-computer interaction method according to an exemplary embodiment, which can be applied to the human-computer interaction device provided in this disclosure. As shown in Figure 5, the human-computer interaction method may include the following steps.

[0117] In step S51, vehicle driving environment information and / or intelligent driving control information are acquired.

[0118] In step S52, an intelligent driving scenario image is constructed based on driving environment information and / or intelligent driving control information.

[0119] In step S53, an intelligent driving scene image is sent to a display device so that the intelligent driving scene image is displayed on the display device.

[0120] By employing the above technical solution, the human-machine interface device acquires the vehicle's driving environment information and / or intelligent driving control information, constructs an intelligent driving scene image based on this information, and then sends the intelligent driving scene image to the display device for display. In this way, the intelligent driving scene image is obtained by processing the driving environment information and / or intelligent driving control information using the human-machine interface device, eliminating the need for rendering by the display device. This effectively reduces the latency of rendering the intelligent driving scene image, thereby improving the user experience.

[0121] Optionally, the human-computer interaction method further includes:

[0122] Obtain the vehicle's body status information;

[0123] The vehicle body status information is sent to the display device so that the vehicle body status information is displayed on the display device.

[0124] Optionally, intelligent driving control information is obtained, including:

[0125] Obtain the current intelligent driving status data of the vehicle;

[0126] Based on the current intelligent driving status data, obtain the intelligent driving control information in the intelligent driving system corresponding to the current intelligent driving status data.

[0127] Optionally, the human-computer interaction method further includes:

[0128] The current intelligent driving status data is sent to the display device so that the current intelligent driving status data is displayed on the display device.

[0129] Regarding the human-computer interaction method in the above embodiments, the specific manner of each step has been described in detail in the embodiments of the human-computer interaction device and the human-computer interaction system, and will not be elaborated here.

[0130] Based on the same inventive concept, this disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the human-computer interaction method provided in this disclosure.

[0131] Based on the same inventive concept, this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the human-computer interaction method provided in this disclosure.

[0132] Based on the same inventive concept, this disclosure also provides a vehicle, such as the human-computer interaction system provided in this disclosure.

[0133] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.

[0134] It should also be noted that the various specific technical features described in the above embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.

[0135] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. A human-computer interaction device (201), wherein, The human-computer interaction device (201) is used to acquire vehicle driving environment information and / or intelligent driving control information, and to construct an intelligent driving scene image based on the driving environment information and / or the intelligent driving control information; The human-computer interaction device (201) is also used to send the intelligent driving scene image to the display device (204) so ​​that the intelligent driving scene image is displayed on the display device (204).

2. The human-computer interaction device (201) according to claim 1, wherein, The human-computer interaction device (201) includes: Human-computer interaction engine module (2011) and human-computer interaction agent module (2012), wherein the human-computer interaction engine module (2011) and the human-computer interaction agent module (2012) are connected; The human-machine interaction engine module (2011) is used to acquire the driving environment information and / or the intelligent driving control information, construct an intelligent driving scene image based on the driving environment information and / or the intelligent driving control information, and send the intelligent driving scene image to the human-machine interaction agent module (2012). The human-computer interaction agent module (2012) is connected to the display device (204) and is used to send the intelligent driving scene image to the display device (204) so ​​that the intelligent driving scene image is displayed on the display device (204).

3. The human-computer interaction device (201) according to claim 2, wherein, The human-computer interaction agent module (2012) is also used to obtain the vehicle body status information and send the vehicle body status information to the display device (204) so ​​that the vehicle body status information is displayed on the display device (204).

4. The human-computer interaction device (201) according to claim 2 or 3, wherein, The human-computer interaction engine module (2011) includes: The system includes a perception information fusion unit (20111) and a scene construction unit (20112), wherein the perception information fusion unit (20111) and the scene construction unit (20112) are connected. The perception information fusion unit (20111) is used to acquire the driving environment information, process the driving environment information to obtain perception fusion data, and send the perception fusion data to the scene construction unit (20112). The scene construction unit (20112) is used to acquire the intelligent driving control information, construct the intelligent driving scene image based on the perception fusion data and the intelligent driving control information, and send the intelligent driving scene image to the human-machine interaction agent module (2012).

5. The human-computer interaction device (201) according to claim 4, wherein, The human-computer interaction engine module (2011) also includes: Intelligent driving state acquisition unit (20113), which is connected to the scene construction unit (20112); The intelligent driving status acquisition unit (20113) is used to acquire the current intelligent driving status data of the vehicle and send the current intelligent driving status data to the scene construction unit (20112). The scenario construction unit (20112) is used to obtain the intelligent driving control information in the intelligent driving system (202) corresponding to the current intelligent driving state data based on the current intelligent driving state data.

6. The human-computer interaction device (201) according to claim 5, wherein, The scene construction unit (20112) is also used to send the current intelligent driving status data to the human-machine interaction agent module (2012), and send the current intelligent driving status data to the display device (204) through the human-machine interaction agent module (2012), so that the current intelligent driving status data is displayed on the display device (204).

7. The human-computer interaction device (201) according to any one of claims 2-6, wherein, The human-computer interaction agent module (2012) includes: an information encapsulation unit (20121) and a first information transmission interface (20122). The information encapsulation unit (20121) is connected to the human-computer interaction engine module (2011) and is used to receive the intelligent driving scene image and / or current intelligent driving status data sent by the human-computer interaction engine module (2011), encapsulate the intelligent driving scene image and / or the current intelligent driving status data to obtain encapsulation information, and send the encapsulation information to the display device (204) via the first information transmission interface (20122).

8. The human-computer interaction device (201) according to claim 7, wherein, The human-computer interaction agent module (2012) also includes a second information transmission interface (20123); The second information transmission interface (20123) is connected to the information encapsulation unit (20121) and is used to obtain the vehicle body status information and send the vehicle body status information to the information encapsulation unit (20121). The information encapsulation unit (20121) is used to encapsulate the intelligent driving scene image and / or the current intelligent driving state data and / or the vehicle body state information to obtain encapsulation information, and send the encapsulation information to the display device (204) via the first information transmission interface (20122).

9. A human-computer interaction system (200), wherein, The human-computer interaction system (200) includes the human-computer interaction device (201) and the display device (204) as described in any one of claims 1-8; The human-computer interaction device (201) is used to acquire vehicle driving environment information and / or intelligent driving control information, and to construct an intelligent driving scene image based on the driving environment information and / or the intelligent driving control information. The human-computer interaction device (201) is connected to the display device (204) and is used to send the intelligent driving scene image to the display device so that the intelligent driving scene image is displayed on the display device (204).

10. The human-computer interaction system (200) according to claim 9, wherein, The human-computer interaction system (200) also includes: An environmental sensing device (203) is provided, wherein the human-computer interaction device (201) is connected to the environmental sensing device (203) and is used to acquire the driving environment information of the vehicle detected by the environmental sensing device (203).

11. The human-computer interaction system (200) according to claim 9 or 10, wherein, The human-computer interaction system (200) includes: Intelligent driving system (202), wherein the human-machine interaction device (201) is connected to the intelligent driving system (202) and is used to obtain intelligent driving control information from the intelligent driving system (202).

12. The human-computer interaction system (200) according to claim 11, wherein, The intelligent driving system (202) includes multiple intelligent driving function modules, each of which corresponds to an intelligent driving state; The human-machine interaction device (201) is used to acquire the current intelligent driving status data of the vehicle, and the intelligent driving control information in the intelligent driving function module corresponding to the current intelligent driving status data.

13. The human-computer interaction system (200) according to claim 12, wherein, The intelligent driving system (202) also includes an intelligent driving state detection module; The intelligent driving state detection module is connected to the human-machine interaction device (201) and is used to detect the current intelligent driving state data of the vehicle and send the current intelligent driving state data to the human-machine interaction device (201).

14. The human-computer interaction system (200) according to any one of claims 9-13, wherein, The human-computer interaction device (201) is also used to acquire the vehicle body status information and send the vehicle body status information to the display device (204) so ​​that the vehicle body status information is displayed on the display device (204).

15. A human-computer interaction method, wherein, The human-computer interaction method, applied to the human-computer interaction device (201) as described in any one of claims 1-8, comprises: Acquire vehicle driving environment information and / or intelligent driving control information; Construct an intelligent driving scene image based on the driving environment information and / or the intelligent driving control information; The intelligent driving scene image is sent to the display device (204) so ​​that the intelligent driving scene image is displayed on the display device (204).

16. The human-computer interaction method according to claim 15, wherein, The human-computer interaction method further includes: Obtain the vehicle's body status information; The vehicle body status information is sent to the display device (204) so ​​that the vehicle body status information is displayed on the display device (204).

17. The human-computer interaction method according to claim 15 or 16, wherein, Obtain intelligent driving control information, including: Obtain the current intelligent driving status data of the vehicle; Based on the current intelligent driving state data, obtain the intelligent driving control information in the intelligent driving system (202) that corresponds to the current intelligent driving state data.

18. The human-computer interaction method according to claim 17, wherein, The human-computer interaction method further includes: The current intelligent driving status data is sent to the display device (204) so ​​that the current intelligent driving status data is displayed on the display device (204).

19. A computer program product, wherein, Includes a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 15-18.

20. A vehicle, wherein, Includes the human-computer interaction system (200) as described in any one of claims 9-14.

Images