A bidirectional human-vehicle interaction device and method for autonomous vehicles

Abstract

The application discloses a bidirectional human-vehicle interaction device and method for an automatic driving vehicle, and the device comprises a light-emitting unit, a control module and a communication unit. The application relates to the field of automatic driving decision planning. The application directly informs pedestrians of the future driving area of the vehicle through the light-emitting unit, and actively determines the cooperation performance of the pedestrians by relying on the vehicle-mounted sensing equipment, and changes the projection color, so that the friendliness and safety of the automatic driving vehicle are improved, and the overall traffic efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and more specifically to a two-way human-vehicle interaction device and method for autonomous vehicles. Background Technology

[0002] When autonomous vehicles encounter large numbers of pedestrians, their onboard decision-making and planning algorithms primarily employ passive avoidance measures for safety reasons. While this improves the safety of autonomous vehicles, the FRP (Frequency-Based Planning) problem arises when dense crowds collide with vehicles. The onboard decision-making and planning algorithms may fail to plan a safe path, leaving the vehicle stuck in the crowd and significantly reducing traffic efficiency. To address this, many autonomous driving algorithms considering human-vehicle interaction have emerged. However, they all share a common problem: one-way interaction. The vehicle observes the pedestrian's position and posture, predicts their future trajectory, and even anticipates their cooperation (whether they actively avoid the vehicle's future driving area) before making interactive decisions and planning the path. This approach fails to consider situations where pedestrians are willing to cooperate but are unsure of the vehicle's future path and how to cooperate. It also neglects to directly inform pedestrians of the vehicle's future driving area and to proactively assess their cooperation through reliable measures. Summary of the Invention

[0003] To address the aforementioned issues, this invention provides a two-way human-vehicle interaction device and method for autonomous vehicles. This device can intuitively inform pedestrians of the vehicle's future driving area and, by identifying whether a pedestrian enters the vehicle's future driving area, determine the pedestrian's cooperation. The information is then input into the vehicle's built-in decision-making and planning algorithm, which outputs a driving path that considers human-vehicle interaction, thereby further improving traffic efficiency and reducing safety risks.

[0004] This invention provides a two-way human-vehicle interaction device for autonomous vehicles, comprising a light-emitting unit, a control unit, and a communication unit. The light-emitting unit projects images onto the ground. The control unit receives information from the communication unit, processes the information, and outputs control information to the light-emitting unit. The communication unit connects to the vehicle's CAN bus and the control module, reads data from the vehicle's CAN bus, and outputs information to the control unit after reading the data. The communication unit also receives feedback information from the control unit and transmits it to the vehicle's CAN bus. Furthermore, the communication unit connects to the light-emitting unit and provides feedback on the status of the light-emitting unit to the control unit.

[0005] As a preferred embodiment of the present invention, the light-emitting unit includes a light source and a lens, wherein the light source, after receiving a signal from the control unit, can emit light, which is then projected onto the ground through the lens, and the shape of the image is adjusted according to the vehicle speed and the steering angle of the vehicle.

[0006] As a preferred embodiment of the present invention, the light source is capable of producing blue, yellow, and red light.

[0007] The present invention also provides a human-vehicle interaction method for a two-way human-vehicle interaction device for autonomous vehicles, comprising the following steps:

[0008] 1) The communication unit reads the vehicle's CAN bus data, obtains the information from the data, and transmits the information to the control unit;

[0009] 2) Based on the information sent by the communication unit, the control unit calculates the position, shape, and color of the image to be projected, and transmits the calculated information to the light-emitting unit;

[0010] 3) The light-emitting unit receives the information transmitted by the control unit and projects the required image according to the information;

[0011] 4) The communication unit reads the real-time perception information from the onboard camera of the autonomous vehicle through the vehicle CAN bus and transmits the perception information to the control unit.

[0012] 5) The control unit judges the pedestrian's cooperative behavior based on the perception information transmitted by the communication unit, and outputs the judgment result to the communication unit;

[0013] 6) The communication unit transmits the judgment information to the vehicle CAN bus, which then feeds it back to the decision-making and planning algorithm built into the autonomous vehicle. The decision-making and planning algorithm built into the autonomous vehicle outputs the further actions of the autonomous vehicle, taking into account human-vehicle interaction.

[0014] As a preferred embodiment of the present invention, the communication chip in step 1) can read the vehicle CAN bus data, specifically by reading the corresponding signal and decrypting it according to the vehicle's DBC, thereby obtaining the information in the data.

[0015] As a preferred embodiment of the present invention, the vehicle CAN bus data in step 1) includes the surrounding perception information of the autonomous vehicle, path planning information, the status information of the autonomous vehicle, and the status information of the surrounding pedestrians.

[0016] As a preferred embodiment of the present invention, the state information of the autonomous vehicle includes the position and orientation of the autonomous vehicle; the state information of the surrounding pedestrians includes the position and orientation of the surrounding pedestrians.

[0017] As a preferred embodiment of the present invention, in step 2), the calculation performed by the control unit based on the information sent by the communication unit specifically involves: equating the front and rear wheels to the center points of the front and rear axles, and establishing a Cartesian coordinate system with the rear axle center point as the origin O1. The projected driving trajectory for the next five seconds is geometrically determined as follows:

[0018] The coordinates of point O2 are:

[0019]

[0020]

[0021] S = V * 5s

[0022] Where L1 is the vehicle length, L2 is the vehicle width, α is the steering angle, V is the vehicle speed, a is the distance from the front axle to the front of the vehicle, b is the distance from the center of gravity to the front axle, c is the distance from the center of gravity to the rear axle, d is the distance from the rear axle to the rear of the vehicle, S is the projected length, R1 is the turning radius of the front of the vehicle, R2 is the turning radius of the rear of the vehicle, and O2 is the center of the circle that the car turns.

[0023] In the O1 coordinate system, circles are drawn with O2 as the center and R1 and R2 as radii to serve as the left and right boundaries of the projection area. The tail of the projection area overlaps with the vehicle outline, and the front of the projection area is truncated by S. The O1 coordinate system is fixed on the vehicle and moves with the vehicle, so the projection area also moves with the vehicle and is updated every 0.1s.

[0024] As a preferred embodiment of the present invention, the control unit in step 5) judges the pedestrian's cooperative performance by receiving the current position of the pedestrian from the perception information transmitted by the communication unit, and judging the pedestrian's cooperative performance based on the distance between the current position of the pedestrian and the position of the image to be projected calculated by the control unit; if the current position of the pedestrian is not within the position of the projected image, the pedestrian's cooperative performance is strong, and if the current position of the pedestrian is within the position of the projected image, the pedestrian's cooperative performance is weak.

[0025] The beneficial effects of this invention are as follows: By installing a light-emitting unit on the vehicle body, this invention directly informs pedestrians of the future driving area of ​​the vehicle, giving willing pedestrians a clear direction, thereby improving traffic efficiency; based on whether pedestrians avoid the vehicle's future driving area, the invention judges the pedestrians' cooperation performance and feeds it back to the vehicle's decision-making and planning algorithm, helping the vehicle's built-in decision-making and planning algorithm to output a driving path that takes into account human-vehicle interaction, thereby further improving traffic efficiency and reducing safety risks.

[0026] It should be noted that the above general description and the following detailed description are merely exemplary and explanatory, and do not limit the present invention. Attached Figure Description

[0027] The above and other objects, features and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings.

[0028] Figure 1 A schematic diagram illustrating the overall principle of a two-way human-vehicle interaction device for autonomous vehicles.

[0029] Figure 2 This is a schematic diagram of the light-emitting unit principle;

[0030] Figure 3 A schematic diagram showing the installation location of the light-emitting unit;

[0031] Figure 4 A schematic diagram showing the installation location of the light-emitting unit;

[0032] Figure 5 This is a schematic diagram of the projection algorithm;

[0033] Figure 6 A schematic diagram of the projection of an autonomous vehicle in motion and the scene of pedestrians crossing outside the vehicle.

[0034] Figure 7 This is a schematic diagram of the projection of an autonomous vehicle in motion and the scene of pedestrians crossing outside the vehicle. Detailed Implementation

[0035] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0036] In existing technologies, autonomous vehicles use turn signals to interact with pedestrians, and then feed back the pedestrians' cooperative behavior to the vehicle's onboard decision-making and planning algorithm. The algorithm then outputs a plan for the autonomous vehicle to take further action, taking into account the human-vehicle interaction. However, turn signals can only provide limited interactive information when turning, and the target is very vague.

[0037] This invention introduces a projection device that can provide clear interactive information in various scenarios such as going straight and turning, informing pedestrians and vehicles of the future travel path and helping pedestrians and vehicles cooperate to pass through.

[0038] like Figure 2 The diagram shows the structure of the light-emitting unit, which projects a blue future driving area in front of the vehicle, and adjusts the color according to the vehicle speed and steering angle. When the vehicle camera detects that a pedestrian has entered the future driving area projected by the light-emitting unit in front of the vehicle, the color changes to yellow. When the vehicle camera detects that the distance between the vehicle and the pedestrian is too close, the color changes to red.

[0039] Figure 3 , 4 The installation location of the projection device is shown. Figure 4 One unit is installed on the roof or above the windshield. Figure 3 There are three, installed at the front and sides of the vehicle. After receiving the control signal output by the control unit, they project the corresponding projection area and color.

[0040] like Figure 1 The diagram illustrates the overall framework of the pedestrian-vehicle interaction module, including a light-emitting unit, a control unit, and a communication unit. The light-emitting unit projects images onto the ground. The control unit receives information from the communication unit, processes the information, and outputs control information to the light-emitting unit. The communication unit connects to the vehicle's CAN bus and the control module, reads data from the CAN bus, and outputs information to the control unit. Furthermore, the communication unit receives feedback from the control unit and transmits it to the CAN bus. The communication unit also connects to the light-emitting unit and provides feedback on its status to the control unit.

[0041] like Figure 5 The diagram illustrates the projection algorithm, where the vehicle length is L1, width is L2, steering angle is α, speed is V, distance from the front axle to the front of the vehicle is a, distance from the center of gravity to the front axle is b, distance from the center of gravity to the rear axle is c, distance from the rear axle to the rear of the vehicle is d, projection length is S, turning radius is R0, turning radius at the front is R1, and turning radius at the rear is R2. Since the interaction between the autonomous vehicle and pedestrians occurs at low speeds, a two-degree-of-freedom bicycle model is used. The front and rear wheels are equated to the center points of the front and rear axles, and a Cartesian coordinate system is established with the rear axle center point as the origin O1. Assuming the projected trajectory for the next five seconds is obtained from the geometric relationships shown in the diagram:

[0042] The coordinates of point O2 are:

[0043]

[0044]

[0045] S = V * 5s

[0046] In the O1 coordinate system, circles are drawn with O2 as the center and R1 and R2 as radii to serve as the left and right boundaries of the projection area. The tail of the projection area overlaps with the vehicle outline, and the front of the projection area is truncated by S. The O1 coordinate system is fixed on the vehicle and moves with the vehicle, so the projection area also moves with the vehicle and is updated every 0.1s.

[0047] When the vehicle approaches a scene where pedestrians are crossing, the communication unit of this device reports the status of the emitting unit to the control unit, which then issues a command to activate the emitting unit. The emitting unit receives the information transmitted by the control unit and projects an image of the area the vehicle will be traveling in for the next 5 seconds onto the ground. Meanwhile, the communication unit reads the real-time perception information from the onboard camera of the autonomous vehicle via the vehicle's CAN bus and transmits this perception information to the control unit. The control unit receives the pedestrian's current position from the perception information transmitted by the communication unit and determines the distance between the pedestrian's current position and the position of the image to be projected calculated by the control unit to assess the pedestrian's cooperation. The control unit then outputs the assessment result to the communication unit. The communication unit transmits the assessment information to the vehicle's CAN bus, which feeds it back to the autonomous vehicle's decision-making and planning algorithm. The autonomous vehicle's decision-making and planning algorithm then outputs a further action for the autonomous vehicle that takes into account human-vehicle interaction.

[0048] like Figure 6 and 7 The image shows two scenarios: an autonomous vehicle and pedestrians crossing the road. When both the pedestrian and the vehicle continue moving forward, the pedestrian is not within the vehicle's current future driving area, meaning they will maintain their current motion without a collision. In this case, the projected image is blue. The pedestrian, observing the projection, can choose to walk normally or accelerate across, indicating strong cooperation. The vehicle can then either slow down to avoid the pedestrian or continue driving normally. Figure 7 As shown, when pedestrians and vehicles continue moving forward, the pedestrian will appear in the vehicle's current future driving area, meaning that if the pedestrian and vehicle continue to maintain their current state of motion, a collision may occur. At this time, the projected image color is yellow. After observing the projection, the pedestrian can choose to stop and wait, walk normally, accelerate and cross, or walk around the rear of the vehicle. This indicates that the pedestrian's cooperation is strong, and the vehicle will slow down to avoid the pedestrian or stop moving based on the pedestrian's actions. However, if the pedestrian is too close to the vehicle or suddenly enters the vehicle's current future driving area, the projected image color is red, indicating that the pedestrian's cooperation is poor. The vehicle will then brake to stop and avoid a collision with the pedestrian. This device allows pedestrians and vehicles to obtain more interactive information, thereby improving overall traffic efficiency and safety.

[0049] The contents not described in detail in this specification are common knowledge to those skilled in the art.

[0050] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A human-vehicle interaction method based on a two-way human-vehicle interaction device for autonomous vehicles, characterized in that, The two-way human-vehicle interaction device includes a light-emitting unit, a control unit, and a communication unit. The light-emitting unit projects images onto the ground. The control unit receives information from the communication unit, processes the information, and outputs control information to the light-emitting unit. The communication unit connects to the vehicle's CAN bus and the control module, reads data from the vehicle's CAN bus, and outputs information to the control unit after reading the data. The communication unit also receives feedback from the control unit and transmits it to the vehicle's CAN bus. Furthermore, the communication unit connects to the light-emitting unit and provides feedback on the status of the light-emitting unit to the control unit. The human-vehicle interaction method includes the following steps: 1) The communication unit reads the vehicle's CAN bus data, obtains the information from the data, and transmits the information to the control unit; The communication unit mentioned in step 1) can read the vehicle CAN bus data, specifically by reading the corresponding signals and decrypting them according to the vehicle's DBC, thereby obtaining the information in the data; the vehicle CAN bus data includes the autonomous vehicle's surrounding perception information, path planning information, autonomous vehicle status information, and surrounding pedestrian status information; the autonomous vehicle status information includes the autonomous vehicle's position and orientation; the surrounding pedestrian status information includes the surrounding pedestrian's position and orientation. 2) Based on the information sent by the communication unit, the control unit calculates the position, shape, and color of the image to be projected, and transmits the calculated information to the light-emitting unit; In step 2), the control unit performs the following calculations based on the information sent by the communication unit: The front and rear wheels are equated to the center points of the front and rear axles, and a Cartesian coordinate system is established with the rear axle center point as the origin O1; the driving trajectory for the next five seconds is projected, and the geometric relationship is as follows: The coordinates of point O2 are: S = V * 5s Where L1 is the vehicle length, L2 is the vehicle width, α is the steering angle, V is the vehicle speed, a is the distance from the front axle to the front of the vehicle, b is the distance from the center of gravity to the front axle, c is the distance from the center of gravity to the rear axle, d is the distance from the rear axle to the rear of the vehicle, S is the projected length, R1 is the turning radius of the front of the vehicle, R2 is the turning radius of the rear of the vehicle, and O2 is the center of the circle that the car turns. In the O1 coordinate system, circles are drawn with O2 as the center and R1 and R2 as radii to serve as the left and right boundaries of the projection area. The tail of the projection area overlaps with the vehicle outline, and the front of the projection area is cut off by S. The O1 coordinate system is fixed on the vehicle and moves with the vehicle, so the projection area also moves with the vehicle and is updated every 0.1s. 3) The light-emitting unit receives the information transmitted by the control unit and projects the required image according to the information; 4) The communication unit reads the real-time perception information from the onboard camera of the autonomous vehicle through the vehicle CAN bus and transmits the perception information to the control unit. 5) The control unit judges the pedestrian's cooperative behavior based on the perception information transmitted by the communication unit, and outputs the judgment result to the communication unit; The control unit receives the pedestrian's current position from the sensing information transmitted by the communication unit, and judges the pedestrian's cooperative performance based on the distance between the pedestrian's current position and the image position to be projected calculated by the control unit; if the pedestrian's current position is not within the image position, the pedestrian's cooperative performance is strong; if the pedestrian's current position is within the image position, the pedestrian's cooperative performance is weak. 6) The communication unit transmits the judgment information to the vehicle CAN bus, which then feeds it back to the decision-making and planning algorithm built into the autonomous vehicle. The decision-making and planning algorithm built into the autonomous vehicle outputs the next action of the autonomous vehicle, taking into account human-vehicle interaction.

2. The human-vehicle interaction method according to claim 1, characterized in that, The light-emitting unit includes a light source and a lens. After receiving a signal from the control unit, the light source can emit light, which is then projected onto the ground through the lens. The shape of the image is adjusted according to the vehicle speed and the steering angle of the car.

3. The human-vehicle interaction method according to claim 2, characterized in that, The light source can produce blue, yellow, and red light.

Images