Automobile exterior human-computer interaction system and method based on vehicle-road cloud cooperation

Abstract

The application relates to the technical field of intelligent driving, in particular to an automatic driving vehicle external man-machine interaction system and method based on vehicle-road cloud cooperation, which comprises an automatic driving vehicle, a roadside device, a wireless communication unit and a cloud control platform; after networking, the automatic driving vehicle sends vehicle motion state and position information to the cloud control platform; after networking, the roadside device sends sensing data, traffic information and road information to the cloud control platform; the cloud control platform processes the data information sent by the automatic driving vehicle and the roadside device, and feeds back decision information and control instructions to the automatic driving vehicle and the roadside device; the wireless communication unit adopts DSRC (Dedicated Short-Range Communication) technology. The application solves the interaction dilemma that the automatic driving vehicle and the pedestrian in the road right fuzzy zone do not know each other's intention, and the problem that the existing vehicle-mounted eHMI (Electronic Human Machine Interface) is affected by environmental light and weather conditions, has low visibility and may cause unclear information transmission.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent driving, and in particular to an automatic driving vehicle external human-machine interaction system and method based on vehicle-road cloud cooperation. BACKGROUND

[0002] Automatic driving vehicles (AVs) effectively reduce road traffic accidents caused by human errors such as driver overspeed, drunk driving, fatigue driving, and distracted driving. For complex urban environments, AVs may need to share space with other road users such as pedestrians, cyclists, and manually driven vehicles. Among them, pedestrians are the most vulnerable road users, and most collisions occur when pedestrians cross the road. In such mixed traffic, in order to ensure the safety of pedestrians and improve the acceptance and trust of the public to AVs, on the one hand, pedestrians need to understand the intentions of AVs, and on the other hand, AVs need to effectively interact with pedestrians in the case of ambiguous road rights and the need to negotiate priority of right-of-way. Under the current circumstances, pedestrians and drivers can communicate through gestures, head movements, and eye contact. Such explicit communication methods can enhance the safety of pedestrians crossing the road. For AVs, drivers can engage in activities unrelated to driving, such as reading, socializing, or even sleeping, and no longer interact with pedestrians. At this time, a new communication strategy needs to be developed to meet the interaction needs of AVs and pedestrians, i.e., external human-machine interface (eHMI). Currently, eHMI is mainly in the form of light strips or display screens installed on the car, but its visibility is highly dependent on environmental light and weather conditions. In addition, in some cases, the interaction objects faced by the automatic driving vehicle may not be just one pedestrian, but multiple pedestrians at different positions, and the meaning of the vehicle-mounted eHMI may not be accurately conveyed to the corresponding pedestrians. ISO and SAE have made some standardized recommendations for the design of eHMI, but there are huge individual differences in pedestrian behavior and perception, and no consensus has been reached on the appropriate form of eHMI.

[0003] Intelligent vehicle-road cooperative system (IVICS) is the latest development direction of intelligent transportation system (ITS), which combines people (travelers), vehicles (transportation tools), roads (road infrastructure), and clouds (traffic control centers) in the transportation system through advanced technologies such as wireless communication and sensing detection, and can realize comprehensive perception and intelligent cooperation of people, vehicles, and roads, while augmented reality (AR) technology can provide targeted and personalized communication, which is expected to solve the problem of many-to-many interaction. SUMMARY

[0004] In order to solve the above technical problems existing in the prior art, the application provides an automatic driving car external man-machine interaction system and method based on vehicle-road cloud cooperation, solves the interaction dilemma that the automatic driving car and the pedestrian in the right-of-way ambiguous zone do not know each other's intentions, and the problems that the existing vehicle-mounted eHMI is affected by environmental light and weather conditions, has low visibility, and information transmission may be unclear, and the specific technical scheme is as follows.

[0005] An automatic driving car external man-machine interaction system based on vehicle-road cloud cooperation comprises an automatic driving car, a roadside device, a wireless communication unit and a cloud control platform, the automatic driving car and the roadside device are network connected with the wireless communication unit respectively, and the wireless communication unit and the cloud control platform perform data interaction, specifically, the automatic driving car sends the vehicle motion state and position information to the cloud control platform after networking, the roadside device sends the sensing data, traffic information and road information to the cloud control platform after networking, the cloud control platform processes the data information sent by the automatic driving car and the roadside device, and feeds back the decision information and control instructions to the automatic driving car and the roadside device, and the wireless communication unit adopts DSRC special short-range communication technology.

[0006] Further, the automatic driving car is provided with a vehicle-mounted positioning unit, a vehicle-mounted computing unit and a motion control unit, the vehicle-mounted positioning unit comprises a global navigation satellite system (GNSS) and an inertial navigation system, and is used for acquiring the real-time position and motion state of the vehicle, the vehicle-mounted computing unit analyzes and processes the data transmitted by the vehicle-mounted positioning unit, obtains the speed, position, attitude and heading information of the automatic driving car, and fuses the decision information or control instructions received by the vehicle to calculate the control information of the vehicle, and the motion control unit comprises a vehicle controller and a body controller, and is used for executing the corresponding control logic according to the received control instructions to realize the control of the execution components.

[0007] Further, the roadside device comprises a roadside sensing unit, a roadside computing unit and an AR projection unit, the roadside sensing unit comprises a binocular camera and a frequency-modulated continuous wave millimeter wave radar, the binocular camera shoots and obtains the video image of the target area, and the frequency-modulated continuous wave millimeter wave radar continuously tracks and measures the distance and speed of the pedestrian, the roadside computing unit identifies the pedestrian in the target area, performs intention recognition and trajectory prediction on the pedestrian, and finally calculates the control information of the roadside device according to the decision information or control instructions received by the roadside device, and the AR projection unit comprises a projector, a mounting vertical rod and corresponding connecting components, the AR projection unit executes the corresponding control logic according to the control instructions transmitted by the cloud control platform, projects a green pedestrian crossing or a pedestrian crossing covered with a red cross indicating prohibition of passing in front of the pedestrian in the target area through the projector, and the transverse width of the projection of the projector is determined according to the number of pedestrians.

[0008] Furthermore, the roadside computing unit identifies pedestrians within the target area by: constructing a pedestrian recognition model training library using video images acquired by a binocular camera, removing invalid image data, and using the deep learning model RetinaNet for pedestrian detection.

[0009] Furthermore, the pedestrian intent recognition and trajectory prediction are specifically performed as follows: a multi-source information fusion recognition network (MIFRN) is used, which employs a lightweight scene semantic understanding network based on E-NET to encode local traffic scenes, an action information encoding network based on prior learnable video prediction to encode future pedestrian action information, and a GRU temporal data processing network based on attention mechanism weighting to encode vehicle speed and pedestrian-vehicle distance. Pedestrian targets that have not undergone significant displacement within several frames are selected as targets of interest, i.e., pedestrians waiting on the roadside. The targets of interest, vehicle speed, and pedestrian-vehicle distance are fed into the multi-source information fusion recognition network (MIFRN). Finally, a bidirectional GRU is introduced to perform deep information fusion, and the fusion result is fed into a multilayer perceptron to obtain the probability of pedestrian crossing or non-crossing.

[0010] Furthermore, the cloud control platform processes the data information sent by the autonomous vehicle and roadside equipment, and feeds back decision information and control commands to the autonomous vehicle and roadside equipment. Specifically, the cloud control platform obtains the real-time position and speed of vehicles and pedestrians transmitted by the autonomous vehicle and roadside equipment, calculates decision information using a collision conflict algorithm, and transmits control commands back to the on-board computing unit and the roadside computing unit through the wireless communication unit.

[0011] Furthermore, the collision conflict algorithm is specifically as follows:

[0012] The cloud control platform calculates the remaining arrival time of vehicles and pedestrians to the collision risk area. Let's assume the remaining arrival time of vehicles to the collision risk area is T. c The remaining arrival time for pedestrians to reach the collision risk area is [T]. p1 ,T p2 ], T p1 and T p2 This represents the time taken for the pedestrian to reach the two boundaries of the collision risk zone. A negative value indicates that the pedestrian has already crossed the boundary; if T... c In [T] p1 ,T p2 If the vehicle passes through the pedestrian's area within the range of T, it is determined that the vehicle has crossed the pedestrian's area, and a collision has occurred between the pedestrian and the vehicle; if T c Not in [T] p1 ,T p2 If the pedestrian and vehicle are within the range of [ ], it is determined that there is no collision or conflict.

[0013] When a pedestrian is crossing the street: If a collision is likely between the pedestrian and a vehicle, the cloud control platform sends a deceleration command to the autonomous vehicle, causing T... c ≥T p2 The cloud control platform sends a command to project a green pedestrian crossing to the roadside equipment. If there is no collision between pedestrians and vehicles, the cloud control platform sends a command to the autonomous vehicle to maintain its speed and sends a command to project a green pedestrian crossing to the roadside equipment.

[0014] When a pedestrian intends to cross the street but has an initial velocity of 0, the average velocity of the pedestrian crossing the street is used to calculate T. c If a collision occurs between pedestrians and vehicles, the cloud control platform sends a deceleration command to the autonomous vehicle, causing T... c ≥T p2 The cloud control platform sends a command to project a green pedestrian crossing to the roadside equipment. If there is no collision between pedestrians and vehicles, the cloud control platform sends a command to the autonomous vehicle to maintain its speed and sends a command to project a green pedestrian crossing to the roadside equipment.

[0015] When a pedestrian has no intention of crossing the street but their initial speed is 0, the average speed of the pedestrian crossing the street is used to calculate T. c If a collision occurs between pedestrians and vehicles, the cloud control platform sends a command to the autonomous vehicle to maintain its speed and a command to project a pedestrian crossing marked with a red cross onto the roadside equipment. If no collision occurs between pedestrians and vehicles, the cloud control platform sends a command to the autonomous vehicle to maintain its speed and a command to project a green pedestrian crossing onto the roadside equipment.

[0016] Wherein, the remaining arrival time T of the vehicle c The calculation method is as follows:

[0017] ;

[0018] The remaining arrival time of the pedestrian [T] p1 ,T p2 The calculation method for ] is:

[0019] .

[0020] in, S It is the distance along the road between the vehicle's center of gravity and the pedestrian's center of gravity. S 安 This represents the minimum safe distance between pedestrians and vehicle heads to avoid collision, preferably set at 5m; the lateral width of the vehicle is... a The longitudinal length is b The vehicle's speed is v c m / s; the pedestrian's speed is v pm / s; the width of a single lane is L; the width of the collision risk zone. d The area centered on the vehicle's center point is 1.4 times the vehicle's width. d =1.4 a The length is the distance along the road between the front of the vehicle and the pedestrian when the entire vehicle exceeds the safe distance, i.e. S 安 - b The distance between a pedestrian and the edge of the road in a direction perpendicular to the road is... x .

[0021] A method for external human-machine interaction of autonomous vehicles based on vehicle-road-cloud collaboration includes the following steps:

[0022] In step S1, the on-board positioning unit of the autonomous vehicle obtains the real-time location of the vehicle, and the roadside sensing unit of the roadside equipment obtains the pedestrian's location, speed and video image information.

[0023] In step S2, the on-board computing unit of the autonomous vehicle and the roadside computing unit of the roadside equipment analyze and process the data information obtained in step S1 to obtain the real-time position, distance and speed of the vehicle and pedestrians. The roadside computing unit also performs intention recognition and trajectory prediction for pedestrians waiting on the roadside.

[0024] Step S3: The cloud control platform acquires and stores the real-time location, distance and speed of vehicles and pedestrians transmitted by the autonomous vehicle and roadside equipment through the wireless communication unit, and calculates the remaining arrival time of vehicles and pedestrians to the collision risk area.

[0025] In step S4, the cloud control platform uses a collision conflict algorithm to calculate decision information and transmits control commands back to the vehicle computing unit and the roadside computing unit via a wireless communication unit.

[0026] In step S5, the on-board computing unit and the roadside computing unit calculate the control information of the autonomous vehicle and the roadside equipment based on the received decision information or control commands and the data information obtained through their own analysis and processing. Finally, they control the motion control unit of the autonomous vehicle and the AR projection unit of the roadside equipment to drive the actuators to execute commands. Beneficial effects

[0027] (1) This invention uses roadside sensing and computing units, rather than vehicle-mounted sensing and computing units, to identify the intentions and predict the behavior of pedestrians in the target area. The behavior of pedestrians can be predicted in advance when the autonomous vehicle is far away from the pedestrian, which can greatly improve traffic efficiency and avoid traffic congestion and the difficulty of autonomous vehicle trajectory planning.

[0028] (2) This invention, based on vehicle-road-cloud collaboration, can obtain the real-time positions and speeds of pedestrians and autonomous vehicles, calculate their remaining arrival times in the cloud, and determine whether a collision will occur. This serves as a new control strategy, improving the interactivity between autonomous vehicles and pedestrians. It avoids the problem that autonomous vehicles are designed to avoid pedestrians upon sight, which could lead to pedestrians jaywalking. This allows autonomous vehicles and pedestrians to cooperate, thereby improving traffic efficiency. Furthermore, through vehicle-road-cloud collaboration, the cloud control platform sends control commands to autonomous vehicles and roadside equipment, potentially enabling interaction between multiple autonomous vehicles and multiple pedestrians in the target area.

[0029] (3) This invention provides a novel external human-computer interaction strategy that uses an AR projection device installed on the roadside to convey the intention of the autonomous vehicle to pedestrians. Compared with the on-board eHMI, it has higher visibility and is less affected by weather conditions, allowing pedestrians to quickly make a decision on whether to cross, thereby ensuring pedestrian safety and increasing their trust and acceptance of AVs. Attached Figure Description

[0030] Figure 1 This invention provides a structural block diagram of an external human-machine interaction system for autonomous vehicles under vehicle-road-cloud collaboration.

[0031] Figure 2 A flowchart of an external human-machine interaction method for autonomous vehicles under vehicle-road-cloud collaboration provided in an embodiment of the present invention;

[0032] Figure 3 A schematic diagram of a roadside device provided in an embodiment of the present invention;

[0033] Figure 4 A schematic diagram of the collision risk area provided in an embodiment of the present invention;

[0034] Figure 5 A flowchart of the collision conflict algorithm adopted by the cloud control platform provided in this embodiment of the invention;

[0035] Figure 6 A schematic diagram of an interactive scenario where an autonomous vehicle and a pedestrian do not collide, provided in an embodiment of the present invention.

[0036] Figure 7 This is a schematic diagram of an interactive scenario provided by an embodiment of the present invention, which predicts a collision between an autonomous vehicle and a pedestrian, and the pedestrian is crossing the street or has the intention to cross the street.

[0037] Figure 8 This is a schematic diagram of an interactive scenario provided by an embodiment of the present invention, in which a collision is predicted between an autonomous vehicle and a pedestrian, but the pedestrian has no intention of crossing the street. Detailed Implementation

[0038] To make the objectives, technical solutions, and technical effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0039] like Figure 1 As shown, the present invention provides an external human-machine interaction system for autonomous vehicles based on vehicle-road-cloud collaboration, comprising: an autonomous vehicle, roadside equipment, a wireless communication unit, and a cloud control platform. The autonomous vehicle and the roadside equipment are respectively network-connected to the wireless communication unit and interact with the cloud control platform through the wireless communication unit.

[0040] The autonomous vehicle possesses perception, decision-making, control, and communication capabilities. After being connected to the network, it can send information such as the vehicle's motion status and location to the cloud control platform. The roadside equipment possesses perception, computing, and communication capabilities. After being connected to the network, it can send perception data, traffic information, and road information to the cloud control platform. The cloud control platform has capabilities such as network device access, distributed computing, and distributed storage. It processes the perception data sent from the vehicle (autonomous vehicle) and roadside equipment, and feeds back decision results and control commands to the autonomous vehicle and roadside equipment. The wireless communication unit can use DSRC (Dedicated Short Range Communication) technology to facilitate the communication and sharing of vehicle status data, perception data, traffic information, and road information between the autonomous vehicle, roadside equipment, and cloud control platform.

[0041] The autonomous vehicle is equipped with an on-board positioning unit, an on-board computing unit, a motion control unit, etc., and each unit communicates via an in-vehicle gateway.

[0042] The vehicle positioning unit includes a Global Navigation Satellite System (GNSS) and an inertial navigation system, used to achieve precise vehicle positioning;

[0043] The on-board computing unit fuses multi-source data from the on-board positioning unit, analyzes and processes it to obtain information such as the speed, position, attitude and heading of the autonomous vehicle, and fuses the decision information or control commands received by the vehicle to finally calculate the vehicle's control information.

[0044] The motion control unit includes a vehicle controller, a body controller, etc., and is mainly used to execute corresponding control logic according to the received instructions to control the actuators.

[0045] like Figure 3As shown, the roadside equipment includes a roadside sensing unit, a roadside computing unit, and an AR projection unit; the roadside sensing unit includes a binocular camera and a frequency-modulated continuous wave millimeter-wave radar. The binocular camera can acquire the posture of pedestrian movement within a large required range, and the millimeter-wave radar can continuously track and measure the distance and speed of pedestrians.

[0046] The roadside computing unit can fuse data from the roadside sensing unit, identify pedestrians in the target area, perform intention recognition and trajectory prediction for pedestrians, and fuse decision information or control commands received by the roadside equipment to finally calculate the control information of the roadside equipment.

[0047] The AR projection unit mainly includes a projector, a mounting pole, and corresponding connecting components. It can execute corresponding control logic according to the control instructions transmitted from the cloud control platform, and project a green pedestrian crossing or a pedestrian crossing marked with a red cross indicating that crossing is prohibited in front of pedestrians in the target area.

[0048] As a preferred embodiment of the present invention, the method of the roadside computing unit to identify pedestrians in the target area is to construct a pedestrian recognition model training library using video images acquired by a binocular camera, remove invalid image data, and use the Retina Net deep learning model, which has high accuracy and detection efficiency, for pedestrian detection.

[0049] The method for pedestrian intent recognition and trajectory prediction employs a Multi-Source Information Fusion Recognition Network (MIFRN). This network utilizes a lightweight scene semantic understanding network based on E-NET to encode local traffic scenes, a motion information encoding network based on prior learnable video prediction to encode future pedestrian actions, and a GRU temporal data processing network based on attention mechanisms to encode vehicle speed and pedestrian-vehicle distance. Assuming that pedestrians without significant displacement within 20 frames are considered as targets of interest (i.e., pedestrians waiting on the roadside), the targets of interest, vehicle speed, and pedestrian-vehicle distance are fed into the MIFRN. Finally, a bidirectional GRU is introduced for deep information fusion, and the fusion result is fed into a multilayer perceptron to obtain the probability of pedestrian crossing or non-crossing.

[0050] like Figures 4-8 As shown in the preferred embodiment of the present invention, the cloud control platform obtains the real-time location, distance, and speed of vehicles and pedestrians from the autonomous vehicle and roadside equipment, and calculates the remaining arrival time of both vehicles and pedestrians. It is assumed that the remaining arrival time of the vehicle to the collision risk area is T. c The remaining arrival time for pedestrians to reach the collision risk area is [T]. p1 ,T p2 ], T p1 and T p2This is the time it takes for the pedestrian to reach the two boundaries of the collision risk area (a negative value indicates that the pedestrian has already crossed the boundary). If T... c In [T] p1 ,T p2 If the vehicle passes through the pedestrian's area within the range of T, it is determined that the vehicle has crossed the pedestrian's area, and a collision has occurred between the pedestrian and the vehicle; if T c Not in [T] p1 ,T p2 If the distance is within the specified range, it is determined that there is no collision between the pedestrian and the vehicle.

[0051] When a pedestrian is crossing the street: If a collision is likely between the pedestrian and a vehicle, the cloud control platform sends a deceleration command to the autonomous vehicle, causing T... c ≥T p2 The cloud control platform sends a command to project a green pedestrian crossing to the roadside equipment. If there is no collision between pedestrians and vehicles, the cloud control platform sends a command to the autonomous vehicle to maintain its speed and sends a command to project a green pedestrian crossing to the roadside equipment.

[0052] When a pedestrian intends to cross the street but has an initial velocity of 0, the average velocity of the pedestrian crossing the street is used to calculate T. c If a collision occurs between pedestrians and vehicles, the cloud control platform sends a deceleration command to the autonomous vehicle, causing T... c ≥T p2 The cloud control platform sends a command to project a green pedestrian crossing to the roadside equipment. If there is no collision between pedestrians and vehicles, the cloud control platform sends a command to the autonomous vehicle to maintain its speed and sends a command to project a green pedestrian crossing to the roadside equipment.

[0053] When a pedestrian has no intention of crossing the street but their initial speed is 0, the average speed of the pedestrian crossing the street is used to calculate T. c If a collision occurs between pedestrians and vehicles, the cloud control platform sends a command to the autonomous vehicle to maintain its speed and sends a command to the roadside equipment to project a pedestrian crossing marked with a red cross. If no collision occurs between pedestrians and vehicles, the cloud control platform sends a command to the autonomous vehicle to maintain its speed and sends a command to the roadside equipment to project a green pedestrian crossing.

[0054] Wherein, the remaining arrival time T of the vehicle c The calculation method is as follows:

[0055] ;

[0056] The remaining arrival time of the pedestrian [T] p1 ,T p2 The calculation method for ] is:

[0057] .

[0058] in, S It is the distance along the road between the vehicle's center of gravity (assuming the vehicle has uniform mass) and the pedestrian's center of gravity. S 安 This represents the minimum safe distance between pedestrians and vehicle heads to avoid collision, preferably set at 5m; the lateral width of the vehicle is... a The longitudinal length is b The vehicle's speed is v c m / s; the pedestrian's speed is v p m / s; the width of a single lane is L; the width of the collision risk zone. d The area centered on the vehicle's center point is 1.4 times the vehicle's width. d =1.4 a The length is the distance along the road between the front of the vehicle and the pedestrian when the entire vehicle exceeds the safe distance, i.e. S 安 - b The distance between a pedestrian and the edge of the road in a direction perpendicular to the road is... x .

[0059] The AR projection unit conveys the intentions of the autonomous vehicle to pedestrians by projecting a green pedestrian crossing or a pedestrian crossing marked with a red cross indicating no crossing onto the road in front of the pedestrian based on the control commands received from the roadside computing unit. The horizontal width of the projection is determined according to the number of pedestrians.

[0060] like Figure 2 As shown, the present invention also provides a method for external human-machine interaction of autonomous vehicles based on vehicle-road-cloud collaboration, comprising the following steps:

[0061] In step S1, the on-board positioning unit of the autonomous vehicle obtains the precise positioning of the vehicle, and the roadside sensing unit of the roadside equipment obtains the pedestrian's position, speed and video image information.

[0062] In step S2, the vehicle-mounted computing unit and the roadside computing unit analyze and process the data information obtained in step S1 to obtain the processing result data, including the real-time position, distance and speed of vehicles and pedestrians. The roadside computing unit can also perform intent recognition and trajectory prediction for pedestrians waiting on the roadside.

[0063] Specifically, the roadside computing unit identifies pedestrians within the target area by: constructing a pedestrian recognition model training library using video images acquired by a binocular camera; removing invalid image data; and employing the Retina Net deep learning model, which boasts high accuracy and detection efficiency, for pedestrian detection. The roadside computing unit performs pedestrian intent recognition and trajectory prediction using a Multi-Source Information Fusion Recognition Network (MIFRN). This network employs a lightweight scene semantic understanding network based on E-NET to encode local traffic scenes, a motion information encoding network based on prior learnable video prediction to encode future pedestrian actions, and a GRU temporal data processing network based on an attention mechanism weighted to encode vehicle speed and pedestrian-vehicle distance. Assuming pedestrians without significant displacement within 20 frames are considered as targets of interest (i.e., pedestrians waiting on the roadside), the targets of interest, vehicle speed, and pedestrian-vehicle distance are fed into the MIFRN. Finally, a bidirectional GRU is introduced for deep information fusion, and the fusion result is fed into a multilayer perceptron to obtain the pedestrian crossing / non-crossing probability.

[0064] Step S3: The cloud control platform acquires and stores the real-time location, distance and speed of vehicles and pedestrians transmitted by the autonomous vehicle and roadside equipment through the wireless communication unit, and calculates the remaining arrival time of vehicles and pedestrians to the collision risk area.

[0065] Wherein, the remaining arrival time T of the vehicle c The calculation method is as follows:

[0066] ;

[0067] The remaining arrival time of the pedestrian [T] p1 ,T p2 The calculation method for ] is: ;

[0068] in, S It is the distance along the road between the vehicle's center of gravity (assuming the vehicle has uniform mass) and the pedestrian's center of gravity. S 安 This indicates the minimum safe distance between pedestrians and vehicles to prevent head-on collisions, and is preferably set at 5 meters; the lateral width of the vehicle is... a The longitudinal length is b The vehicle's speed is v c m / s; the pedestrian's speed is v p m / s; the width of a single lane is L; the width of the collision risk zone. d The area centered on the vehicle's center point is 1.4 times the vehicle's width. d =1.4 aThe length is the distance along the road between the front of the vehicle and the pedestrian when the entire vehicle exceeds the safe distance, i.e. S 安 - b The distance between a pedestrian and the edge of the road in a direction perpendicular to the road is... x .

[0069] Step S4: Calculate and make decisions using a collision conflict algorithm, and transmit control commands back to the onboard computing unit and the roadside computing unit via the wireless communication unit;

[0070] The collision conflict algorithm is as follows:

[0071] If T c In [T] p1 ,T p2 If the vehicle passes through the pedestrian's area within the range of T, it is determined that the vehicle has crossed the pedestrian's area, and a collision has occurred between the pedestrian and the vehicle; if T c Not in [T] p1 ,T p2 Within the specified range, it is determined that there is no collision between the pedestrian and the vehicle. When a pedestrian is crossing the street: if a collision is likely between the pedestrian and the vehicle, the cloud control platform sends a deceleration command to the autonomous vehicle, causing T... c ≥T p2 The cloud control platform sends a command to project a green pedestrian crossing to the roadside equipment. If there is no collision between pedestrians and vehicles, the cloud control platform sends a command to the autonomous vehicle to maintain its speed and also sends a command to project the green pedestrian crossing to the roadside equipment. When a pedestrian intends to cross the street but has an initial speed of 0, the average speed of the pedestrian crossing is used to calculate T. c If a collision occurs between pedestrians and vehicles, the cloud control platform sends a deceleration command to the autonomous vehicle, causing T... c ≥T p2 The cloud control platform sends a command to project a green pedestrian crossing to the roadside equipment. If there is no collision between pedestrians and vehicles, the cloud control platform sends a command to the autonomous vehicle to maintain its speed and sends a command to project a green pedestrian crossing to the roadside equipment. When a pedestrian does not intend to cross the street but has an initial speed of 0, the average speed of the pedestrian crossing is used to calculate T. c If a collision occurs between pedestrians and vehicles, the cloud control platform sends a command to the autonomous vehicle to maintain its speed and sends a command to the roadside equipment to project a pedestrian crossing marked with a red cross. If no collision occurs between pedestrians and vehicles, the cloud control platform sends a command to the autonomous vehicle to maintain its speed and sends a command to the roadside equipment to project a green pedestrian crossing.

[0072] In step S5, the on-board computing unit and the roadside computing unit calculate the control information of the autonomous vehicle and the roadside equipment based on the received decision information or control commands and the data information obtained through their own analysis and processing. Finally, they control the motion control unit of the autonomous vehicle and the AR projection unit of the roadside equipment to drive the actuators to execute commands.

[0073] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Although the implementation process of the present invention has been described in detail above, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An external human-machine interaction system for autonomous vehicles based on vehicle-road-cloud collaboration, characterized in that, The system includes an autonomous vehicle, roadside equipment, a wireless communication unit, and a cloud control platform. The autonomous vehicle and roadside equipment are respectively network-connected to the wireless communication unit and interact with the cloud control platform through the wireless communication unit. Specifically, after the autonomous vehicle is connected to the network, it sends its own motion status and location information to the cloud control platform. After the roadside equipment is connected to the network, it sends perception data, traffic information, and road information to the cloud control platform. The cloud control platform processes the data information sent by the autonomous vehicle and the roadside equipment and feeds back decision information and control commands to the autonomous vehicle and the roadside equipment. The wireless communication unit adopts DSRC dedicated short-range communication technology. The autonomous vehicle is equipped with an onboard positioning unit, an onboard computing unit, and a motion control unit. The onboard positioning unit includes a Global Navigation Satellite System (GNSS) and an inertial navigation system to perform real-time positioning and motion status acquisition of the vehicle. The onboard computing unit analyzes and processes the data transmitted from the onboard positioning unit to obtain information on the autonomous vehicle's speed, position, attitude, and heading, and integrates the decision information or control commands received by the vehicle to calculate the vehicle's control information. The motion control unit includes a vehicle controller and a body controller, which executes corresponding control logic according to the received control commands to control the actuators. The roadside equipment includes a roadside sensing unit, a roadside computing unit, and an AR projection unit; the roadside sensing unit includes a binocular camera and a frequency-modulated continuous wave millimeter-wave radar, the binocular camera captures video images of the target area, and the frequency-modulated continuous wave millimeter-wave radar continuously tracks and measures the distance and speed of pedestrians; The roadside computing unit identifies pedestrians within the target area, performs intent recognition and trajectory prediction, and calculates the control information for the roadside equipment based on decision information or control commands received from the roadside equipment. The AR projection unit includes a projector, a mounting pole, and corresponding connecting components. The AR projection unit executes corresponding control logic based on control commands transmitted from the cloud control platform, projecting a green pedestrian crossing or a pedestrian crossing marked with a red cross indicating no crossing in front of pedestrians within the target area. The horizontal width of the projection is determined by the number of pedestrians. The cloud control platform processes the data information sent by the autonomous vehicle and the roadside equipment, and feeds back the decision information and control commands to the autonomous vehicle and the roadside equipment. Specifically, the cloud control platform obtains the real-time position and speed of the vehicle and pedestrians transmitted by the autonomous vehicle and the roadside equipment, calculates the decision information using a collision conflict algorithm, and transmits the control commands back to the on-board computing unit and the roadside computing unit through the wireless communication unit. The collision conflict algorithm is as follows: The cloud control platform calculates the remaining arrival time of vehicles and pedestrians to the collision risk area. Let's assume the remaining arrival time of vehicles to the collision risk area is T. c The remaining arrival time for pedestrians to reach the collision risk area is [T]. p1 ,T p2 ], T p1 and T p2 This represents the time taken for the pedestrian to reach the two boundaries of the collision risk zone. A negative value indicates that the pedestrian has already crossed the boundary; if T... c In [T] p1 ,T p2 If the vehicle passes through the pedestrian's area within the range of T, it is determined that the vehicle has crossed the pedestrian's area, and a collision has occurred between the pedestrian and the vehicle; if T c Not in [T] p1 ,T p2 If the pedestrian and vehicle are within the range of [ ], it is determined that there is no collision or conflict. When a pedestrian is crossing the street: If a collision is likely between the pedestrian and a vehicle, the cloud control platform sends a deceleration command to the autonomous vehicle, causing T... c ≥T p2 The cloud control platform sends a command to project a green pedestrian crossing to the roadside equipment. If there is no collision between pedestrians and vehicles, the cloud control platform sends a command to the autonomous vehicle to maintain its speed and sends a command to project a green pedestrian crossing to the roadside equipment. When a pedestrian intends to cross the street but has an initial velocity of 0, the average velocity of the pedestrian crossing the street is used to calculate T. c If a collision occurs between pedestrians and vehicles, the cloud control platform sends a deceleration command to the autonomous vehicle, causing T... c ≥T p2 The cloud control platform sends a command to project a green pedestrian crossing to the roadside equipment. If there is no collision between pedestrians and vehicles, the cloud control platform sends a command to the autonomous vehicle to maintain its speed and sends a command to project a green pedestrian crossing to the roadside equipment. When a pedestrian has no intention of crossing the street but their initial speed is 0, the average speed of the pedestrian crossing the street is used to calculate T. c If a collision occurs between pedestrians and vehicles, the cloud control platform sends a command to the autonomous vehicle to maintain its speed and a command to project a pedestrian crossing marked with a red cross onto the roadside equipment. If no collision occurs between pedestrians and vehicles, the cloud control platform sends a command to the autonomous vehicle to maintain its speed and a command to project a green pedestrian crossing onto the roadside equipment. Wherein, the remaining arrival time T of the vehicle c The calculation method is as follows: ； The remaining arrival time of the pedestrian [T] p1 ,T p2 The calculation method for ] is: ， in, S It is the distance along the road between the vehicle's center of gravity and the pedestrian's center of gravity. S 安 This represents the minimum safe distance between pedestrians and vehicle heads to avoid collision, preferably set at 5m; the lateral width of the vehicle is... a The longitudinal length is b The vehicle's speed is v c m / s; the pedestrian's speed is v p m / s; the width of a single lane is L; the width of the collision risk zone. d The area centered on the vehicle's center point is 1.4 times the vehicle's width. d =1.4 a The length is the distance along the road between the front of the vehicle and the pedestrian when the entire vehicle exceeds the safe distance, i.e. S 安 - b The distance between a pedestrian and the edge of the road in a direction perpendicular to the road is... x .

2. The external human-machine interaction system for autonomous vehicles based on vehicle-road-cloud collaboration as described in claim 1, characterized in that, The roadside computing unit identifies pedestrians within the target area by: constructing a pedestrian recognition model training library using video images acquired by a binocular camera, removing invalid image data, and using a deep learning model, Retina Net, for pedestrian detection.

3. The external human-machine interaction system for autonomous vehicles based on vehicle-road-cloud collaboration as described in claim 1, characterized in that, The process of pedestrian intent recognition and trajectory prediction specifically involves: employing a multi-source information fusion recognition network (MIFRN), a lightweight scene semantic understanding network based on E-NET for encoding local traffic scenes, an action information encoding network based on prior learnable video prediction for encoding future pedestrian action information, and an attention-weighted GRU temporal data processing network for encoding vehicle speed and pedestrian-vehicle distance. Pedestrian targets that do not undergo significant displacement within several frames are selected as targets of interest, i.e., pedestrians waiting on the roadside. The targets of interest, vehicle speed, and distance between pedestrians and vehicles are fed into the Multi-Source Information Fusion Recognition Network (MIFRN). Finally, a bidirectional GRU is introduced to perform deep information fusion, and the fusion result is fed into a multilayer perceptron to obtain the probability of pedestrian crossing or non-crossing.

4. A method for external human-machine interaction of an autonomous vehicle using a vehicle-road-cloud cooperative external human-machine interaction system as described in any one of claims 1 to 3, characterized in that, Includes the following steps: In step S1, the on-board positioning unit of the autonomous vehicle obtains the real-time location of the vehicle, and the roadside sensing unit of the roadside equipment obtains the pedestrian's location, speed and video image information. In step S2, the on-board computing unit of the autonomous vehicle and the roadside computing unit of the roadside equipment analyze and process the data information obtained in step S1 to obtain the real-time position, distance and speed of the vehicle and pedestrians. The roadside computing unit also performs intention recognition and trajectory prediction for pedestrians waiting on the roadside. Step S3: The cloud control platform acquires and stores the real-time location, distance and speed of vehicles and pedestrians transmitted by the autonomous vehicle and roadside equipment through the wireless communication unit, and calculates the remaining arrival time of vehicles and pedestrians to the collision risk area. In step S4, the cloud control platform uses a collision conflict algorithm to calculate decision information and transmits control commands back to the vehicle computing unit and the roadside computing unit via a wireless communication unit. In step S5, the on-board computing unit and the roadside computing unit calculate the control information of the autonomous vehicle and the roadside equipment based on the received decision information or control commands and the data information obtained through their own analysis and processing. Finally, they control the motion control unit of the autonomous vehicle and the AR projection unit of the roadside equipment to drive the actuators to execute commands.

Images