Human-computer interaction device, system, method, program product and vehicle
Through human-computer interaction devices, the intelligent driving scene images are obtained and constructed in autonomous vehicles and sent directly to the display device, which solves the problem of rendering delay in autonomous vehicles, improves user experience and simplifies interface management.
Patent Information
- Application Number
- CN202411182079.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-27
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-08-27
AI Technical Summary
In autonomous driving cars, due to limited computing performance of instruments or central control screens, the delay in rendering images of intelligent driving scenes, affecting the user experience.
The driving environment information and intelligent driving control information are obtained through the human-computer interactive device, an intelligent driving scene image is constructed, and directly sent to the display device to avoid rendering processing by the display device.
It effectively reduces the delay in rendering images of intelligent driving scenes, improves user experience, and realizes unified interface design and management, reducing the complexity of vehicle software and hardware iteration.
Smart Images

Figure CN118701087B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of vehicle technology, and in particular, to a human-computer interaction device, system, method, program product and vehicle. Background Art
[0002] In the development of new energy vehicles, especially autonomous vehicles, the role of the human-machine interface (HMI) has become particularly important. It not only needs to provide traditional vehicle control functions, but also needs to support interactions related to the autonomous driving system.
[0003] In the related art, after receiving the intelligent driving control information transmitted by the human-machine interface HMI and the environmental perception information transmitted by the perception fusion module, the instrument or central control screen usually needs to process the intelligent driving control information and environmental perception information to obtain the intelligent driving scene image. However, since the instrument or central control screen is usually set in the vehicle domain, and the vehicle domain controller has limited computing performance, it will cause image rendering delays, resulting in a poor user experience. Summary of the invention
[0004] The purpose of the present disclosure is to provide a human-computer interaction device, system, method, program product and vehicle to enhance the user experience.
[0005] In order to achieve the above-mentioned object, the first aspect of the present disclosure provides a human-computer interaction device, which is used to obtain driving environment information and / or intelligent driving control information of a vehicle, and construct an intelligent driving scene image according to the driving environment information and / or the intelligent driving control information;
[0006] The human-computer interaction device is also used to send the intelligent driving scene image to a display device so that the intelligent driving scene image is displayed on the display device.
[0007] Optionally, the human-computer interaction device includes:
[0008] A human-computer interaction engine module and a human-computer interaction agent module, wherein the human-computer interaction engine module is connected to the human-computer interaction agent module;
[0009] The human-computer interaction engine module is used to obtain the driving environment information and / or the intelligent driving control information, construct an intelligent driving scene image according to the driving environment information and / or the intelligent driving control information, and send the intelligent driving scene image to the human-computer interaction agent module;
[0010] The human-computer interaction agent module is connected to the display device and is used to send the intelligent driving scene image to the display device so that the intelligent driving scene image is displayed on the display device.
[0011] Optionally, the human-computer interaction agent module is further used to obtain body state information of the vehicle, and send the body state information to the display device, so that the body state information is displayed on the display device.
[0012] Optionally, the human-computer interaction engine module includes:
[0013] A perception information fusion unit and a scene construction unit, wherein the perception information fusion unit is connected to the scene construction unit;
[0014] The perception information fusion unit is used to obtain the driving environment information, process the driving environment information to obtain perception fusion data, and send the perception fusion data to the scene construction unit;
[0015] The scene construction unit is used to obtain the intelligent driving control information, and construct the intelligent driving scene image according to the perception fusion data and the intelligent driving control information, and send the intelligent driving scene image to the human-computer interaction agent module.
[0016] Optionally, the human-computer interaction engine module further includes:
[0017] An intelligent driving state acquisition unit, the intelligent driving state acquisition unit is connected to the scenario construction unit;
[0018] The intelligent driving state acquisition unit is used to acquire current intelligent driving state data of the vehicle and send the current intelligent driving state data to the scenario construction unit;
[0019] The scenario construction unit is used to obtain the intelligent driving control information corresponding to the current intelligent driving state data in the intelligent driving system according to the current intelligent driving state data.
[0020] Optionally, the scene construction unit is also used to send the current intelligent driving state data to the human-computer interaction agent module, and send the current intelligent driving state data to the display device through the human-computer interaction agent module, so that the current intelligent driving state data is displayed on the display device.
[0021] Optionally, the human-computer interaction agent module includes: an information encapsulation unit and a first information transmission interface;
[0022] The information encapsulation unit is connected to the human-computer interaction engine module, and is used to receive the intelligent driving scene image and / or the current intelligent driving status data sent by the human-computer interaction engine module, encapsulate the intelligent driving scene image and / or the current intelligent driving status data to obtain encapsulated information, and send the encapsulated information to the display device via the first information transmission interface.
[0023] Optionally, the human-computer interaction agent module further includes a second information transmission interface;
[0024] The second information transmission interface is connected to the information encapsulation unit, and is used to obtain the body state information of the vehicle and send the body state information to the information encapsulation unit;
[0025] The information encapsulation unit is used to encapsulate the intelligent driving scene image and / or the current intelligent driving state data and / or the vehicle body state information to obtain encapsulated information, and send the encapsulated information to the display device via the first information transmission interface.
[0026] A second aspect of the present disclosure provides a human-computer interaction system, the human-computer interaction system comprising the human-computer interaction device and the display device provided in the first aspect of the present disclosure;
[0027] The human-machine interaction device is used to obtain driving environment information and / or intelligent driving control information of the vehicle, and construct an intelligent driving scene image according to the driving environment information and / or the intelligent driving control information;
[0028] The human-computer interaction device is connected to the display device and is used to send the intelligent driving scene image to the display device so that the intelligent driving scene image is displayed on the display device.
[0029] Optionally, the human-computer interaction system further includes:
[0030] An environment perception device, the human-computer interaction device is connected to the environment perception device, and is used to obtain the driving environment information of the vehicle detected by the environment perception device.
[0031] Optionally, the human-computer interaction system includes:
[0032] Intelligent driving system, the human-computer interaction device is connected to the intelligent driving system and is used to obtain intelligent driving control information from the intelligent driving system.
[0033] Optionally, the intelligent driving system includes a plurality of intelligent driving function modules, each of which corresponds to an intelligent driving state;
[0034] The human-computer interaction device is used to obtain the current intelligent driving state data of the vehicle and the intelligent driving control information in the intelligent driving function module corresponding to the current intelligent driving state data.
[0035] Optionally, the intelligent driving system further includes an intelligent driving state detection module;
[0036] The intelligent driving state detection module is connected to the human-computer interaction device, and is used to detect the current intelligent driving state data of the vehicle and send the current intelligent driving state data to the human-computer interaction device.
[0037] Optionally, the human-computer interaction device is further used to obtain body status information of the vehicle, and send the body status information to the display device, so that the body status information is displayed on the display device.
[0038] A third aspect of the present disclosure provides a human-computer interaction method, which is applied to the human-computer interaction device according to the first aspect of the present disclosure, and the human-computer interaction method includes:
[0039] Acquiring vehicle driving environment information and / or intelligent driving control information;
[0040] Constructing an intelligent driving scene image according to the driving environment information and / or the intelligent driving control information;
[0041] The intelligent driving scene image is sent to a display device so that the intelligent driving scene image is displayed on the display device.
[0042] Optionally, the human-computer interaction method further includes:
[0043] Acquiring body state information of the vehicle;
[0044] The vehicle body state information is sent to the display device so that the vehicle body state information is displayed on the display device.
[0045] Optionally, obtaining intelligent driving control information includes:
[0046] Acquiring current intelligent driving status data of the vehicle;
[0047] According to the current intelligent driving state data, intelligent driving control information corresponding to the current intelligent driving state data is obtained in the intelligent driving system.
[0048] Optionally, the human-computer interaction method further includes:
[0049] The current intelligent driving state data is sent to the display device so that the current intelligent driving state data is displayed on the display device.
[0050] A fourth aspect of the present disclosure provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of any one of the methods described in the third aspect of the present disclosure.
[0051] A fifth aspect of the present disclosure provides a vehicle, comprising a human-computer interaction system as described in any one of the second aspects of the present disclosure.
[0052] By adopting the above technical solution, the human-machine interaction device obtains the driving environment information and / or intelligent driving control information of the vehicle, and constructs an intelligent driving scene image according to the driving environment information and / or intelligent driving control information, and then sends the automatic driving scene image to the display device so that the intelligent driving scene image is displayed on the display device. In this way, the driving environment information and / or intelligent driving control information are processed by the human-machine interaction device to obtain the intelligent driving scene image, and the display device does not need to perform rendering processing, which effectively reduces the delay of rendering the intelligent driving scene image, thereby improving the user experience.
[0053] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] The accompanying drawings are used to provide further understanding of the present disclosure and constitute a part of the specification. They are used to explain the present disclosure together with the following detailed description, but do not constitute a limitation of the present disclosure.
[0055] Figure 1A It is a schematic diagram of a human-computer interaction system in the related technology of this application.
[0056] Figure 1B It is a schematic diagram of the environmental perception information transmission channel in the related technology of this application.
[0057] Figure 2 It is a block diagram of a human-computer interaction system shown in the present application according to an exemplary embodiment.
[0058] Figure 3 It is a block diagram of another human-computer interaction system shown in the present application according to an exemplary embodiment.
[0059] Figure 4 It is a schematic diagram of a human-computer interaction system according to an exemplary embodiment of the present application.
[0060] Figure 5 It is a flow chart of a human-computer interaction method shown in the present application according to an exemplary embodiment. DETAILED DESCRIPTION
[0061] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.
[0062] In related technologies, there are two main types of information transmitted by the human-machine interface (HMI), one is vehicle body status information, and the other is intelligent driving control information. With the development of automobile intelligence, the transmission of environmental perception information has become an indispensable part of the automobile intelligent driving system. At present, the channel for transmitting environmental perception information to the instrument or central control screen is independent of the human-machine interface HMI channel.
[0063] For example, Figure 1A It is a schematic diagram of a human-computer interaction system in the related art. Figure 1A As shown, the human-computer interaction device obtains vehicle body status information and intelligent driving control information, and transmits the vehicle body status information and intelligent driving control information to the instrument or central control screen. Figure 1B Schematic diagram of the environmental perception information transmission channel in the related technology. Figure 1B As shown, the perception fusion module obtains environmental perception information and performs fusion processing, and then transmits the processed environmental perception information to the instrument or central control screen. In this way, on the one hand, the instrument or central control screen needs to set up two sets of transmission interfaces to receive the information transmitted by the human-computer interaction device and the information transmitted by the perception fusion module, which is not conducive to the unified management and iteration of the interface. On the other hand, after receiving the intelligent driving control information transmitted by the human-computer interaction device and the environmental perception information transmitted by the perception fusion module, the instrument or central control screen usually needs to reconstruct the intelligent driving control information and the environmental perception information to obtain the intelligent driving scene image. However, since the instrument or central control screen is usually set in the vehicle domain, and the computing performance of the vehicle domain controller is limited, it will cause delays in rendering the image, resulting in a poor user experience.
[0064] In view of this, the present disclosure provides a human-computer interaction device, system, method, program product and vehicle, which utilize the human-computer interaction device to process driving environment information and / or intelligent driving control information to obtain an intelligent driving scene image, without the need for a display device for rendering processing, effectively reducing the delay in rendering the intelligent driving scene image, thereby improving the user experience.
[0065] For ease of description, the human-computer interaction system provided by the present disclosure is described below.
[0066] Figure 2 FIG. 1 is a block diagram of a human-computer interaction system according to an exemplary embodiment. Figure 2 As shown, the human-computer interaction system 200 may include a human-computer interaction device 201 and a display device 204 .
[0067] The human-machine interaction device 201 is used to obtain the driving environment information and / or intelligent driving control information of the vehicle, and construct an intelligent driving scene image according to the driving environment information and / or intelligent driving control information. The human-machine interaction device 201 is connected to the display device 204 and is used to send the intelligent driving scene image to the display device 204 so that the intelligent driving scene image is displayed on the display device 204.
[0068] By adopting the above technical solution, the human-machine interaction device obtains the driving environment information and / or intelligent driving control information of the vehicle, and constructs an intelligent driving scene image according to the driving environment information and / or intelligent driving control information, and then sends the automatic driving scene image to the display device so that the intelligent driving scene image is displayed on the display device. In this way, the driving environment information and / or intelligent driving control information are processed by the human-machine interaction device to obtain the intelligent driving scene image, and the display device does not need to perform rendering processing, which effectively reduces the delay of rendering the intelligent driving scene image, thereby improving the user experience.
[0069] In one embodiment, the human-computer interaction system 200 may further include an environment sensing device 203 . Figure 3 FIG. 1 is a block diagram of another human-computer interaction system according to an exemplary embodiment. Figure 3 As shown, the human-computer interaction device 201 is connected to the environment perception device 203 to obtain the driving environment information of the vehicle detected by the environment perception device 203.
[0070] The environment sensing device 203 may include but is not limited to a camera, a millimeter wave radar, and a laser radar, etc. Accordingly, the driving environment information of the vehicle may include but is not limited to the original information collected by the camera, the original information collected by the millimeter wave radar, and the original information collected by the laser radar, which can characterize the relationship between the vehicle's surrounding environment and the vehicle, such as the vehicle's position and posture information, road lane information, obstacle information, indicator light and road sign information, parking space information, etc.
[0071] In another embodiment, the human-computer interaction system 200 may further include an intelligent driving system 202. Figure 3As shown, the human-machine interaction device 201 can also be connected to the intelligent driving system 202 to obtain intelligent driving control information. Among them, the intelligent driving system 202 may include multiple intelligent driving function modules, for example, it may include but not limited to: remote parking assist (RPA) function module, automatic parking assist (APA) function module, intelligent navigation (Navigate On Autopilot, NOA) function module and advanced driver assistance system (ADAS) function module. Accordingly, the intelligent driving control information may include intelligent driving control information of the remote parking function module, intelligent driving control information of the automatic parking function module, intelligent driving control information of the intelligent navigation function module and intelligent driving control information of the advanced driver assistance system function module.
[0072] For example, the intelligent driving control information of the automatic parking function module includes but is not limited to: parking path, parking direction, current vehicle speed, vehicle posture and target parking space and other control information. The intelligent driving control information of the intelligent navigation function module includes but is not limited to: lane keeping, lane selection, lane change indication, target vehicle following reminder and other control information.
[0073] It should be understood that among multiple intelligent driving functional modules, only one functional module may be in an effective state at the same time, and the human-computer interaction device can only obtain the intelligent driving control information of the functional module in the effective state.
[0074] For example, the intelligent driving system 202 includes multiple intelligent driving function modules, each of which corresponds to an intelligent driving state. The human-computer interaction device 201 is used to obtain the current intelligent driving state data of the vehicle and the intelligent driving control information in the intelligent driving function module corresponding to the current intelligent driving state data.
[0075] In addition, the intelligent driving system 202 may also include an intelligent driving state detection module, which is connected to the human-computer interaction device 201 and is used to detect the current intelligent driving state data of the vehicle and send the current intelligent driving state data to the human-computer interaction device 201.
[0076] The intelligent driving state may include, but is not limited to, one of a remote control parking state, an automatic parking state, an intelligent navigation state, and a high-level assisted driving state. For example, if the plurality of intelligent driving function modules may include a remote control parking function module, an automatic parking function module, an intelligent navigation function module, and a high-level driving assistance system function module, the intelligent driving state may include one of a remote control parking state, an automatic parking state, an intelligent navigation state, and a high-level assisted driving state.
[0077] For example, if the intelligent driving state is the remote parking state, the human-computer interaction device 201 obtains the intelligent driving control information of the remote parking function module; if the intelligent driving state is the automatic parking state, the human-computer interaction device 201 obtains the intelligent driving control information of the automatic parking function module; if the intelligent driving state is the intelligent navigation state, the human-computer interaction device 201 obtains the intelligent driving control information of the intelligent navigation function module; if the intelligent driving state is the high-order assisted driving state, the human-computer interaction device 201 obtains the intelligent driving control information of the high-order driving assistance system function module.
[0078] In this way, the human-computer interaction device can obtain the current intelligent driving status data of the vehicle, and then obtain the intelligent driving control information in the intelligent driving function module corresponding to the current intelligent driving status data, thereby improving the reliability of the human-computer interaction device in obtaining the intelligent driving control information.
[0079] Among them, the way in which the human-computer interaction device 201 constructs the intelligent driving scene image according to the driving environment information and / or intelligent driving control information can be at least one of the following: the human-computer interaction device 201 constructs the intelligent driving scene image according to the driving environment information; the human-computer interaction device 201 constructs the intelligent driving scene image according to the intelligent driving control information; the human-computer interaction device 201 constructs the intelligent driving scene image according to the driving environment information and the intelligent driving control information.
[0080] The specific implementation method of the human-machine interaction device 201 constructing the intelligent driving scene image according to the driving environment information and the intelligent driving control information is as follows: the human-machine interaction device 201 synchronizes the driving environment information and the intelligent driving control information in time, and then constructs the intelligent driving scene image according to the time-synchronized driving environment information and the intelligent driving control information. The technology of time-synchronizing the driving environment information and the intelligent driving control information, and the technology of constructing the intelligent driving scene image according to the time-synchronized driving environment information and the intelligent driving control information are relatively mature technologies, and the present disclosure does not specifically limit this.
[0081] For example, the human-machine interaction device 201 may be arranged in the intelligent driving domain of the vehicle. The display device 204 is arranged in the vehicle-machine domain of the vehicle. For example, the display device 204 may include an onboard display screen and / or an onboard instrument.
[0082] Since the display device is not required to construct the intelligent driving scene image, the coupling relationship between the vehicle domain where the display device is located and the intelligent driving domain is effectively reduced. The vehicle domain where the display device is located only needs to provide a unified interface for the human-computer interaction device, and there is no need to perform specific processing steps such as information collection, time synchronization, and intelligent driving scene construction within the intelligent driving domain to achieve decoupling. In this way, the problem of vehicle software and hardware iteration updates affecting vehicle rendering is reduced, and the delay in vehicle rendering images is reduced by reducing the amount of calculation in the vehicle domain.
[0083] By adopting the above technical solution, the human-machine interaction device obtains the driving environment information and / or intelligent driving control information of the vehicle, and constructs an intelligent driving scene image according to the driving environment information and / or intelligent driving control information, and then sends the automatic driving scene image to the display device so that the intelligent driving scene image is displayed on the display device. In this way, the driving environment information and / or intelligent driving control information are processed by the human-machine interaction device to obtain the intelligent driving scene image, and the display device does not need to perform rendering processing, which effectively reduces the delay of rendering the intelligent driving scene image, thereby improving the user experience.
[0084] In addition, compared with the prior art, the present disclosure integrates the independent perception environment information channel into the human-computer interaction system, so that the driving environment information and intelligent driving control information can be transmitted to the display device through the human-computer interaction device. In this way, unified interface design and management are achieved, the coupling between the display device and the vehicle's intelligent driving system is reduced, and the safety and stability of interface iteration management are improved.
[0085] The human-computer interaction device 201 is described in detail below with reference to the accompanying drawings.
[0086] Figure 4 FIG. 1 is a schematic diagram of a human-computer interaction system according to an exemplary embodiment. Figure 4 As shown, the human-computer interaction device 201 may include a human-computer interaction engine module 2011 and a human-computer interaction agent module 2012. The human-computer interaction engine module 2011 is connected to the human-computer interaction agent module 2012. The human-computer interaction engine module 2011 is used to obtain driving environment information and / or intelligent driving control information, and construct an intelligent driving scene image according to the driving environment information and / or intelligent driving control information, and send the intelligent driving scene image to the human-computer interaction agent module 2012.
[0087] For example, Figure 4As shown, the human-computer interaction engine module 2011 is respectively connected to the human-computer interaction agent module 2012, the environment perception device 203, and the intelligent driving system 202, and is used to obtain the driving environment information and intelligent driving control information of the vehicle, and construct an intelligent driving scene image according to the driving environment information and the intelligent driving control information, and send the intelligent driving scene image to the human-computer interaction agent module 2012. Among them, the intelligent driving system 202 can be located in the intelligent driving domain.
[0088] The human-computer interaction agent module 2012 is connected to the display device 204 and is used to send the intelligent driving scene image to the display device 204 when receiving the intelligent driving scene image, so that the intelligent driving scene image is displayed on the display device 204.
[0089] That is, in the present disclosure, the human-computer interaction engine module 2011 is used to receive the original driving environment information and / or intelligent driving control information, and to make a complete intelligent driving scene image according to the original driving environment information and / or intelligent driving control information and send it to the human-computer interaction agent module 2012. The human-computer interaction agent module 2012 is used for information collection and forwarding, for example, receiving the intelligent driving scene image from the human-computer interaction engine module 2011, and forwarding the intelligent driving scene image to the display device 204.
[0090] With this solution, the human-computer interaction device 201 adopts a composite module design, decoupling the interface encapsulation function and the information collection and processing function in the human-computer interaction device 201, and respectively designing them into a human-computer interaction engine module 2011 and a human-computer interaction agent module 2012. In this way, local iterative updates of the human-computer interaction device 201 can be achieved, effectively reducing the iterative risk and difficulty of updating the human-computer interaction device, which is beneficial to the stability of the update iteration of the human-computer interaction device 201.
[0091] For example, when the display device is a vehicle display screen and / or a vehicle instrument, when the vehicle-machine communication interface is updated, only the human-machine interaction agent module 2012 needs to be updated to match the update of the vehicle-machine interface. When the interface for obtaining driving environment information or the interface for obtaining intelligent driving control information is modified, only the human-machine interaction engine module 2011 needs to be updated to match the modification of the interface for obtaining driving environment information or the interface for obtaining intelligent driving control information.
[0092] In addition, in the present disclosure, the human-computer interaction device 201 can also send the vehicle body state information to the display device 204, so that the vehicle body state information is displayed on the display device 204, so that the driver can better understand the vehicle body state. Figure 4 As shown, the human-computer interaction agent module 2012 is also used to obtain the vehicle body status information of the vehicle, and send the vehicle body status information to the display device 204 to display the vehicle body status information on the display device 204.
[0093] like Figure 4 As shown, the vehicle body state information may include but is not limited to chassis state control information, cockpit state control information, door state information, driving state control information, etc. Among them, the chassis state information may include power system state information, wheel suspension state information, brake system state information, etc. The cockpit state control information may include state information of whether the user is wearing a seat belt. The door state information may include state information of whether the vehicle body is closed. The driving state control information may include vehicle speed information, driving mode state information, etc.
[0094] For example, in a high-speed driving scenario, the human-computer interaction agent module 2012 can receive vehicle speed information and chassis component fault status information from the chassis, receive status information on whether passengers are wearing seat belts from the cockpit, and receive status information on whether the doors and / or trunk are open from the vehicle body. After the above information is collected, it is packaged in the format of the vehicle-computer communication interface through the controller area network (CAN) protocol and sent to the display device at an agreed frequency for display on the display device.
[0095] With the above technical solution, the human-computer interaction engine module 2011 sends the intelligent driving scene image constructed by it to the human-computer interaction agent module 2012, so that the human-computer interaction agent module 2012 sends it to the display device 204. In addition, the human-computer interaction agent module 2012 can also send the vehicle body state information it obtains to the display device 204. In this way, the intelligent driving scene image and the vehicle body state information are both sent to the display device 204 via the human-computer interaction agent module 2012.
[0096] In this way, on the one hand, the human-machine interaction device can simultaneously receive and process three types of information: vehicle body status information, driving environment information, and intelligent driving control information, and can communicate with the display device set in the vehicle domain controller through a unified encapsulated communication interface, thus realizing the decoupling between the vehicle domain controller and other domain controllers such as chassis domain controller, cockpit domain controller, and intelligent driving domain controller. On the other hand, compared with Figure 1A Compared with the prior art shown in the figure, the present disclosure uses different modules to obtain the vehicle state information and the intelligent driving control information, that is, the vehicle state information processing function and the intelligent driving control information processing function are set on different modules, so that when an abnormality occurs in the human-computer interaction engine module 2011, the normal display of the vehicle state information is not affected, thereby improving the robustness of the human-computer interaction device. In addition, during the iterative update, the human-computer interaction device can realize partial iteration, effectively reducing the difficulty of updating and iterating the human-computer interaction device, which is beneficial to the stability of the version update and iteration of the human-computer interaction device.
[0097] It should be understood that the human-computer interaction device can send the vehicle body status information to the display device alone, or can send the intelligent driving scene image to the display device alone, or can send the vehicle body status information and the intelligent driving scene image to the display device at the same time. The present disclosure does not make specific limitations on this.
[0098] In one embodiment, if Figure 4 As shown, the human-computer interaction engine module 2011 may include a perception information fusion unit 20111 and a scene construction unit 20112. The perception information fusion unit 20111 and the scene construction unit 20112 are connected. The perception information fusion unit 20111 is used to obtain the driving environment information of the vehicle, process the driving environment information to obtain perception fusion data, and send the perception fusion data to the scene construction unit 20112.
[0099] like Figure 4 As shown, the perception information fusion unit 20111 is connected to the environment perception device 203, and is used to obtain the driving environment information of the vehicle from the environment perception device 203. Afterwards, the obtained driving environment information of the vehicle is processed to obtain the perception fusion data, for example, the driving environment information is filtered, compared, fused, etc., and the original driving environment information collected by the environment perception device 203 is converted into driving environment information with a higher confidence, that is, perception fusion data. Among them, the perception fusion data may include but is not limited to: vehicle posture information, obstacle information, road and lane line information, street light and signboard information, parking space information, etc. Finally, the perception information fusion unit 20111 sends the driving environment information with a higher confidence, that is, perception fusion data, to the scene construction unit 20112 for the construction of the intelligent driving scene.
[0100] The scene construction unit 20112 obtains the perception fusion data sent by the perception information fusion unit 20111, and connects with the intelligent driving system 202 to obtain the intelligent driving control information from the intelligent driving system 202. Thereafter, the obtained perception fusion data and intelligent driving control information are subjected to time synchronization processing. After the synchronization processing, a real-time intelligent driving scene image is constructed.
[0101] In another embodiment, the human-computer interaction engine module 2011 may further include an intelligent driving state acquisition unit 20113. The intelligent driving state acquisition unit 20113 may be connected to the intelligent driving system 202. Figure 4As shown, the intelligent driving state acquisition unit 20113 is also connected to the intelligent driving state detection module 2021 in the intelligent driving system 202, and is used to obtain the current intelligent driving state data of the intelligent driving system 202. For example, the intelligent driving state represented by the current intelligent driving state data includes but is not limited to one of the remote control parking state, the automatic parking state, the intelligent navigation state, and the advanced assisted driving state. It should be understood that when the intelligent driving state is the remote control parking state or the automatic parking state, the intelligent driving state may further include the parking state or the parking state.
[0102] The scene construction unit 20112 is used to obtain the intelligent driving control information corresponding to the current intelligent driving state data in the intelligent driving system according to the current intelligent driving state data. Figure 4 As shown, the scene construction unit 20112 is connected to the remote parking function module 2022, the automatic parking function module 2023, the intelligent navigation function module 2024 and the advanced driving assistance system function module 2025 included in the intelligent driving system 202. When the intelligent driving state represented by the current intelligent driving state data is the remote parking state, the scene construction unit 20112 obtains the intelligent driving control information of the remote parking function module 2022. Alternatively, when the intelligent driving state represented by the current intelligent driving state data is the advanced assisted driving state, the scene construction unit 20112 obtains the intelligent driving control information of the advanced driving assistance system function module 2025.
[0103] After acquiring the driving environment information and the intelligent driving control information, the scene construction unit 20112 performs time synchronization processing on the acquired perception fusion data and the intelligent driving control information, and after the synchronization processing, constructs a real-time intelligent driving scene image.
[0104] For example, when the intelligent driving state is the intelligent navigation state, the function flag of the intelligent driving state detection module 2021 is switched to the intelligent navigation state, and the state is synchronized to the intelligent driving state acquisition unit 20113, so that the scene construction unit 20112 can obtain the current intelligent driving state data. According to the intelligent navigation function requirements, the scene construction unit 20112 obtains intelligent navigation state control information such as lane keeping, lane selection, lane change indication, target vehicle following reminder, etc. from the intelligent navigation function module 2024, and synchronizes the intelligent navigation state control information with the lane line information, indicator light information, road sign information, surrounding vehicle information and other obstacle information from the perception information fusion unit 20111, and produces a real-time intelligent navigation scene image, and forwards it to the human-computer interaction agent module 2012.
[0105] As another example, when the intelligent driving state is the automatic parking state or the remote parking state, the scene construction unit 20112 can also render the target parking space as highlighted when acquiring the intelligent driving control information, and number the parking spaces from 1 to 6, and then process it with the acquired perception fusion data to construct a complete parking scene image including the highlighted target parking space, and display it on the display device. In addition, if the user updates the target parking space, the scene construction unit 20112 can also obtain the target parking space updated by the user from the intelligent driving system 202, and render the updated target parking space as highlighted when constructing the parking scene image.
[0106] In addition, in this embodiment, the scene construction unit 20112 can obtain the current intelligent driving state data from the intelligent driving state acquisition unit 20113, so that the scene construction unit 20112 can also send the current intelligent driving state data to the human-computer interaction agent module 2012, so that the human-computer interaction agent module 2012 can send the current intelligent driving state data to the display device 204, so as to display the current intelligent driving state data in the display device 204, so as to facilitate the people in the vehicle to know the intelligent driving state of the vehicle.
[0107] It should be understood that in the present disclosure, according to actual needs, the human-computer interaction device can send at least one of the vehicle body status information, the intelligent driving scene image and the current intelligent driving status data to the display device, and then the display device can display at least one of the vehicle body status information, the intelligent driving scene image and the current intelligent driving status data.
[0108] The structure of the human-computer interaction agent module 2012 is described below.
[0109] In one embodiment, the human-computer interaction agent module 2012 may include an information encapsulation unit 20121 and a first information transmission interface 20122. The information encapsulation unit 20121 is connected to the human-computer interaction engine module 2011, and is used to receive the intelligent driving scene image and / or the current intelligent driving state data sent by the human-computer interaction engine module 2011, encapsulate the intelligent driving scene image and / or the current intelligent driving state data to obtain encapsulated information, and send the encapsulated information to the display device via the first information transmission interface 20122.
[0110] like Figure 4As shown, the information encapsulation unit 20121 can be connected to the scene construction unit 20112 in the human-computer interaction engine module 2011, and is used to receive the intelligent driving scene image sent by the scene construction unit 20112, and perform format conversion and encapsulation processing on the intelligent driving scene image according to the preset communication protocol, and send the encapsulated intelligent driving scene image to the display device 204 via the first information transmission interface 20122. And / or, the information encapsulation unit 20121 is used to receive the current intelligent driving state data sent by the scene construction unit 20112, and perform format conversion and encapsulation processing on the current intelligent driving state data according to the preset communication protocol, and send the encapsulated current intelligent driving state data to the display device 204 via the first information transmission interface 20122.
[0111] In addition, if Figure 4 As shown, the human-computer interaction agent module 2012 may also include a second information transmission interface 20123. The second information transmission interface 20123 is connected to the information packaging unit 20121, and is used to obtain the vehicle body state information, and send the vehicle body state information to the information packaging unit 20121. The information packaging unit 20121 is used to package the intelligent driving scene image and / or the current intelligent driving state data and / or the vehicle body state information to obtain packaged information, and send the packaged information to the display device 204 via the first information transmission interface 20122.
[0112] For example, the information encapsulation unit 20121 performs format conversion and encapsulation processing on at least one of the intelligent driving scene image, the current intelligent driving state data and the vehicle body state information according to the communication protocol, and sends the encapsulated information to the display device 204 via the first information transmission interface 20122.
[0113] In this way, the vehicle body status information, intelligent driving scene images, and current intelligent driving status data can all be transmitted to the display device 204 through the unified first information transmission interface 20122. In this way, the display device does not need to set up two sets of transmission interfaces, which facilitates the unified management and iteration of the interface. In addition, the independent perception environment information channel is integrated into the human-computer interaction system, so that the driving environment information and intelligent driving control information can be transmitted to the display device through a unified interface. In this way, unified interface design and management are achieved, the coupling between the display device and the vehicle's intelligent driving system is reduced, and the safety and stability of interface iteration management are improved.
[0114] Based on the same inventive concept, the present disclosure also provides a human-computer interaction method. Figure 5 FIG. 1 is a flowchart of a human-computer interaction method according to an exemplary embodiment, which method can be applied to the human-computer interaction device provided by the present disclosure. Figure 5 As shown, the human-computer interaction method may include the following steps.
[0115] In step S51, the vehicle's driving environment information and / or intelligent driving control information is obtained.
[0116] In step S52, an intelligent driving scene image is constructed according to the driving environment information and / or the intelligent driving control information.
[0117] In step S53 , the intelligent driving scene image is sent to a display device so that the intelligent driving scene image is displayed on the display device.
[0118] With the above technical solution, the human-machine interaction device obtains the driving environment information and / or intelligent driving control information of the vehicle, and constructs an intelligent driving scene image based on the driving environment information and / or intelligent driving control information, and then sends the intelligent driving scene image to the display device to display the intelligent driving scene image on the display device. In this way, the driving environment information and / or intelligent driving control information are processed by the human-machine interaction device to obtain the intelligent driving scene image, and the display device does not need to perform rendering processing, which effectively reduces the delay in rendering the intelligent driving scene image, thereby improving the user experience.
[0119] Optionally, the human-computer interaction method further includes:
[0120] Acquiring body state information of the vehicle;
[0121] The vehicle body state information is sent to the display device so that the vehicle body state information is displayed on the display device.
[0122] Optionally, obtaining intelligent driving control information includes:
[0123] Acquiring current intelligent driving status data of the vehicle;
[0124] According to the current intelligent driving state data, the intelligent driving control information corresponding to the current intelligent driving state data in the intelligent driving system is obtained.
[0125] Optionally, the human-computer interaction method further includes:
[0126] The current intelligent driving state data is sent to the display device so that the current intelligent driving state data is displayed on the display device.
[0127] Regarding the human-computer interaction method in the above embodiment, the specific manner of each step has been described in detail in the embodiments of the human-computer interaction device and the human-computer interaction system, and will not be elaborated here.
[0128] Based on the same inventive concept, the present disclosure also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the human-computer interaction method provided by the present disclosure are implemented.
[0129] Based on the same inventive concept, the present disclosure also provides a computer program product, including a computer program, which implements the steps of the human-computer interaction method provided by the present disclosure when executed by a processor.
[0130] Based on the same inventive concept, the present disclosure also provides a vehicle, such as the human-computer interaction system provided by the present disclosure.
[0131] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.
[0132] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.
[0133] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.
Claims
1. A human-computer interaction device, characterized in that: The human-machine interaction device is used to obtain driving environment information and / or intelligent driving control information of the vehicle, and construct an intelligent driving scene image according to the driving environment information and / or the intelligent driving control information; The human-computer interaction device is further used to send the intelligent driving scene image to a display device so that the intelligent driving scene image is displayed on the display device; The human-computer interaction device comprises: A human-computer interaction engine module and a human-computer interaction agent module, wherein the human-computer interaction engine module is connected to the human-computer interaction agent module; The human-computer interaction engine module is used to obtain the driving environment information and / or the intelligent driving control information, construct an intelligent driving scene image according to the driving environment information and / or the intelligent driving control information, and send the intelligent driving scene image to the human-computer interaction agent module; The human-computer interaction agent module is connected to the display device, and is used to send the intelligent driving scene image to the display device, so that the intelligent driving scene image is displayed on the display device; The human-computer interaction engine module includes: A perception information fusion unit and a scene construction unit, wherein the perception information fusion unit is connected to the scene construction unit; The perception information fusion unit is used to obtain the driving environment information, process the driving environment information to obtain perception fusion data, and send the perception fusion data to the scene construction unit; The scene construction unit is used to obtain the intelligent driving control information, and construct the intelligent driving scene image according to the perception fusion data and the intelligent driving control information, and send the intelligent driving scene image to the human-computer interaction agent module.
2. The human-computer interaction device according to claim 1, characterized in that: The human-computer interaction agent module is further used to obtain body state information of the vehicle and send the body state information to the display device so that the body state information is displayed on the display device.
3. The human-computer interaction device according to claim 1, characterized in that: The human-computer interaction engine module also includes: An intelligent driving state acquisition unit, the intelligent driving state acquisition unit is connected to the scenario construction unit; The intelligent driving state acquisition unit is used to acquire current intelligent driving state data of the vehicle and send the current intelligent driving state data to the scenario construction unit; The scenario construction unit is used to obtain the intelligent driving control information corresponding to the current intelligent driving state data in the intelligent driving system according to the current intelligent driving state data.
4. The human-computer interaction device according to claim 3, characterized in that: The scene construction unit is also used to send the current intelligent driving state data to the human-computer interaction agent module, and send the current intelligent driving state data to the display device through the human-computer interaction agent module, so that the current intelligent driving state data is displayed on the display device.
5. The human-computer interaction device according to any one of claims 1 to 4, characterized in that: The human-computer interaction agent module includes: an information packaging unit and a first information transmission interface, The information encapsulation unit is connected to the human-computer interaction engine module, and is used to receive the intelligent driving scene image and / or the current intelligent driving status data sent by the human-computer interaction engine module, encapsulate the intelligent driving scene image and / or the current intelligent driving status data to obtain encapsulated information, and send the encapsulated information to the display device via the first information transmission interface.
6. The human-computer interaction device according to claim 5, characterized in that: The human-computer interaction agent module also includes a second information transmission interface; The second information transmission interface is connected to the information encapsulation unit, and is used to obtain the body state information of the vehicle and send the body state information to the information encapsulation unit; The information encapsulation unit is used to encapsulate the intelligent driving scene image and / or the current intelligent driving state data and / or the vehicle body state information to obtain encapsulated information, and send the encapsulated information to the display device via the first information transmission interface.
7. A human-computer interaction system, characterized in that: The human-computer interaction system comprises the human-computer interaction device and the display device according to any one of claims 1 to 6; The human-machine interaction device is used to obtain driving environment information and / or intelligent driving control information of the vehicle, and construct an intelligent driving scene image according to the driving environment information and / or the intelligent driving control information; The human-computer interaction device is connected to the display device, and is used to send the intelligent driving scene image to the display device, so that the intelligent driving scene image is displayed on the display device; Wherein, the human-computer interaction device comprises: a human-computer interaction engine module and a human-computer interaction agent module, and the human-computer interaction engine module is connected to the human-computer interaction agent module; The human-computer interaction engine module is used to obtain the driving environment information and / or the intelligent driving control information, construct an intelligent driving scene image according to the driving environment information and / or the intelligent driving control information, and send the intelligent driving scene image to the human-computer interaction agent module; The human-computer interaction agent module is connected to the display device, and is used to send the intelligent driving scene image to the display device, so that the intelligent driving scene image is displayed on the display device; The human-computer interaction engine module includes: A perception information fusion unit and a scene construction unit, wherein the perception information fusion unit is connected to the scene construction unit; The perception information fusion unit is used to obtain the driving environment information, process the driving environment information to obtain perception fusion data, and send the perception fusion data to the scene construction unit; The scene construction unit is used to obtain the intelligent driving control information, and construct the intelligent driving scene image according to the perception fusion data and the intelligent driving control information, and send the intelligent driving scene image to the human-computer interaction agent module.
8. The human-computer interaction system according to claim 7, characterized in that: The human-computer interaction system further includes: An environment perception device, the human-computer interaction device is connected to the environment perception device, and is used to obtain the driving environment information of the vehicle detected by the environment perception device.
9. The human-computer interaction system according to claim 7, characterized in that: The human-computer interaction system comprises: Intelligent driving system, the human-computer interaction device is connected to the intelligent driving system and is used to obtain intelligent driving control information from the intelligent driving system.
10. The human-computer interaction system according to claim 9, characterized in that: The intelligent driving system includes a plurality of intelligent driving function modules, each of which corresponds to an intelligent driving state; The human-computer interaction device is used to obtain the current intelligent driving state data of the vehicle and the intelligent driving control information in the intelligent driving function module corresponding to the current intelligent driving state data.
11. The human-computer interaction system according to claim 10, characterized in that: The intelligent driving system also includes an intelligent driving state detection module; The intelligent driving state detection module is connected to the human-computer interaction device, and is used to detect the current intelligent driving state data of the vehicle and send the current intelligent driving state data to the human-computer interaction device.
12. The human-computer interaction system according to claim 7, characterized in that: The human-machine interaction device is further used to obtain body state information of the vehicle and send the body state information to the display device so that the body state information is displayed on the display device.
13. A human-computer interaction method, characterized in that: The human-computer interaction device according to any one of claims 1 to 6 comprises a human-computer interaction engine module and a human-computer interaction agent module, wherein the human-computer interaction engine module comprises: a perception information fusion unit and a scene construction unit; and the human-computer interaction method comprises: Acquiring vehicle driving environment information and / or intelligent driving control information; Constructing an intelligent driving scene image according to the driving environment information and / or the intelligent driving control information; Sending the intelligent driving scene image to a display device so that the intelligent driving scene image is displayed on the display device; The obtaining of the vehicle's driving environment information and / or intelligent driving control information includes: The perception information fusion unit acquires driving environment information, processes the driving environment information to obtain perception fusion data, and sends the perception fusion data to the scene construction unit; The scene construction unit obtains intelligent driving control information, constructs the intelligent driving scene image according to the perception fusion data and the intelligent driving control information, and sends the intelligent driving scene image to the human-computer interaction agent module; The sending the intelligent driving scene image to a display device so that the intelligent driving scene image is displayed on the display device includes: The human-computer interaction agent module sends the intelligent driving scene image to a display device so that the intelligent driving scene image is displayed on the display device.
14. The human-computer interaction method according to claim 13, characterized in that: The human-computer interaction method further comprises: Acquiring body state information of the vehicle; The vehicle body state information is sent to the display device so that the vehicle body state information is displayed on the display device.
15. The human-computer interaction method according to claim 13, characterized in that: Obtain intelligent driving control information, including: Acquiring current intelligent driving status data of the vehicle; According to the current intelligent driving state data, intelligent driving control information corresponding to the current intelligent driving state data is obtained in the intelligent driving system.
16. The human-computer interaction method according to claim 15, characterized in that: The human-computer interaction method further comprises: The current intelligent driving state data is sent to the display device so that the current intelligent driving state data is displayed on the display device.
17. A computer program product, characterized in that The method comprises a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 13 to 16.
18. A vehicle, characterized in that: Comprising a human-computer interaction system as described in any one of claims 7-12.
Citation Information
Patent Citations
System configuring a human machine interface on multiple displays
CN106896978A
Auxiliary parking system and method
CN107856667A
Intelligent driving interaction system and vehicle
CN113552566A
Interaction system for automobile parking based on man-machine interaction
CN113830080A
Novel intelligent driving vehicle system
CN114537366A
