Safety control method and system for remote driving of unmanned vehicle in low-speed environment

Abstract

This invention proposes a safety control method for remote driving of unmanned vehicles in low-speed environments. The method includes: the unmanned vehicle's onboard sensing device collecting real-time vehicle data; uploading the vehicle data to a remote control device in real-time; the remote control device being equipped with a display device to show the real-time status of the unmanned vehicle and sending driving control commands to the unmanned vehicle; the remote control device defining a safe zone for the unmanned vehicle, and the unmanned vehicle defining a vehicle safety protection zone based on the safe zone, the vehicle safety protection zone being an area greater than a certain threshold from the boundary of the safe zone; the safety zone being updated in real-time based on the vehicle's driving status and surrounding road conditions; and when the unmanned vehicle detects that it has entered the safety protection zone, the unmanned vehicle will enter an emergency protection state and issue a warning message.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving, specifically to a safety control method and system for remote driving of an autonomous vehicle in a low-speed environment. Background Technology

[0002] With the development of autonomous driving technology, parallel driving has become an important solution. Parallel driving refers to autonomous vehicles using cameras to collect video footage of the road, uploading the footage to remote control equipment, and allowing users to remotely control the vehicle based on the video data.

[0003] In the current parallel driving scheme, the driverless car can establish a communication channel with the remote control device. The collected video is uploaded to the remote control device through the communication channel, and the driver can control the driverless car based on the video transmitted back by the driverless car on the remote control device.

[0004] However, during autonomous vehicle operation, communication failures, network delays, or sudden environmental obstacles can occur when the vehicle enters certain areas, preventing remote control devices from issuing control commands in a timely manner. For parallel driving, where safety requirements are extremely high, the inability to issue timely driving control commands poses a significant safety hazard. Summary of the Invention

[0005] In view of this, the present invention provides a safety control method for remote driving of unmanned vehicles in low-speed environments, comprising: the vehicle-mounted sensing device of the unmanned vehicle collecting the current vehicle data of the unmanned vehicle in real time; and uploading the vehicle data to a remote control device in real time.

[0006] The remote control device is equipped with a display device to show the real-time status of the unmanned vehicle and send driving control commands to the unmanned vehicle; the remote control device defines a safe area for the unmanned vehicle, and the unmanned vehicle defines a vehicle safety protection area for itself based on the safe area. The vehicle safety protection area is the area that is greater than a certain threshold from the boundary of the safe area; the safe area is updated in real time according to the vehicle's driving status and the surrounding road conditions.

[0007] When the unmanned vehicle detects that it has entered the safety protection area in real time, the unmanned vehicle will enter an emergency protection state and issue a warning message.

[0008] Specifically, the vehicle data includes the location of the unmanned vehicle and real-time video streams from the outside; displaying the real-time status of the unmanned vehicle includes displaying the actual location of the unmanned vehicle and a simulated display of the safe area.

[0009] In particular, the vehicle safety protection zone can be flexibly changed according to different road conditions.

[0010] Specifically, when in the emergency protection state, the unmanned vehicle will drive according to control commands from itself, which have higher priority than control commands issued by remote control devices.

[0011] Specifically, the onboard sensing equipment of the autonomous vehicle includes a surround-view fisheye camera, a perspective color camera, an event camera, an infrared camera, and a 32-beam LiDAR.

[0012] This invention also proposes a safety control system for remote driving of unmanned vehicles in low-speed environments. The system includes an unmanned vehicle and onboard sensing equipment.

[0013] The onboard sensing equipment of the unmanned vehicle collects the current vehicle data of the unmanned vehicle in real time; and uploads the vehicle data to the remote control equipment in real time.

[0014] The remote control device is equipped with a display device to show the real-time status of the unmanned vehicle and send driving control commands to the unmanned vehicle; the remote control device defines a safe area for the unmanned vehicle, and the unmanned vehicle defines a vehicle safety protection area for itself based on the safe area. The vehicle safety protection area is the area that is greater than a certain threshold from the boundary of the safe area; the safe area is updated in real time according to the vehicle's driving status and the surrounding road conditions.

[0015] When the unmanned vehicle detects that it has entered the safety protection area in real time, the unmanned vehicle will enter an emergency protection state and issue a warning message.

[0016] Specifically, the vehicle data includes the location of the unmanned vehicle and real-time video streams from the outside; displaying the real-time status of the vehicle includes displaying the actual location of the unmanned vehicle and a simulated display of the safe area.

[0017] In particular, the vehicle safety protection zone can be flexibly changed according to different road conditions.

[0018] Specifically, when in the emergency protection state, the unmanned vehicle will drive according to control commands from itself, which have higher priority than control commands issued by remote control devices.

[0019] Specifically, the onboard sensing equipment of the autonomous vehicle includes a surround-view fisheye camera, a perspective color camera, an event camera, an infrared camera, and a 32-beam LiDAR.

[0020] Beneficial effects:

[0021] 1) The safety control method for remote driving of unmanned vehicles proposed in this invention can ensure that even when the unmanned vehicle cannot receive driving control commands from the remote control device, it will not blindly drive into an unsafe area, causing rear-end collisions or scrapes.

[0022] 2) The safety control method for remote driving of unmanned vehicles proposed in this invention can form different command priorities for controlling unmanned vehicles, thus enriching the control methods of unmanned vehicles;

[0023] 3) The safety control method for remote driving of unmanned vehicles proposed in this invention can display the actual position of the unmanned vehicle and the simulated display of the safe area on the remote control device, and can monitor the safe area of ​​the unmanned vehicle in real time and display the vehicle status intuitively.

[0024] 4) The safety control method for remote driving of unmanned vehicles proposed in this invention can flexibly change the size of the safety protection area according to factors such as actual road conditions, thereby increasing the flexibility of unmanned vehicle control. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the safe zone for unmanned vehicles in this invention. Detailed Implementation

[0026] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0027] This invention provides a safety control method for remote driving of unmanned vehicles in low-speed environments. The remote control device (e.g., a PC or console) can simultaneously communicate with and control multiple unmanned vehicles, enabling their movement. Currently, parallel driving schemes are applicable to unmanned vehicles used in warehousing and logistics, mining trucks, and other similar applications. For example, in a warehousing and logistics scenario, the remote control device can simultaneously control multiple unmanned vehicles connected end-to-end to move within a warehouse.

[0028] Before the remote control device takes control of the autonomous vehicle, the autonomous vehicle and the remote control device establish multiple communication channels simultaneously. These communication channels can be kept alive via a heartbeat mechanism, and the communication quality of each channel can be evaluated in real time (e.g., packet loss rate, round-trip time (RTT) latency, etc.).

[0029] The onboard sensing equipment of the unmanned vehicle collects the current vehicle data of the unmanned vehicle in real time; and uploads the vehicle data to the remote control device in real time; the vehicle data includes the position of the unmanned vehicle and the real-time video stream from the outside; the unmanned vehicle is equipped with video shooting equipment such as surround-view fisheye cameras and perspective color cameras to collect surrounding video data.

[0030] The remote control device is equipped with a display device to show the real-time status of the vehicle and send driving control commands to the unmanned vehicle. The display device may be a large-screen monitor, which displays the real-time status of the unmanned vehicle, including its actual position and a simulated display of the safe zone. The safe zone is an area defined by the remote control device to ensure the safety of the unmanned vehicle. Taking a rectangle as an example, its size can be flexibly changed according to actual road conditions, obstacles, etc. In this embodiment, it can be defined as 2 meters in front and behind, and 1 meter to the left and right. That is, the area 2 meters in front and behind, and 1 meter to the left and right of the unmanned vehicle is defined as the safe zone. This distance is defined based on the distance between the unmanned vehicle and surrounding obstacles such as other vehicles. For example, if the vehicle in front is 4 meters away, it is reasonable to define the area 2 meters in front of the unmanned vehicle as the safe zone.

[0031] As the autonomous vehicle moves forward, the safe zone also moves forward accordingly. The large screen on the remote control device displays the autonomous vehicle's movement within the safe zone, simulating the same scenario. The remote control device continuously issues control commands, sending the latest safe zone range to the autonomous vehicle and instructing it to accelerate, decelerate, and perform other maneuvers. If communication between the remote device and the autonomous vehicle experiences significant delays due to poor network service quality, and the vehicle gradually decelerates after maintaining a constant speed, the remote device should similarly issue commands to decelerate and stop, ensuring the autonomous vehicle remains within the safe zone. However, due to network latency, the autonomous vehicle may not receive the deceleration and stop commands in time and continues forward at its original speed, potentially veering out of the original 2-meter safe zone. To avoid a potential rear-end collision, the autonomous vehicle defines its own safe protection zone, which can be defined as a rectangular area defined by a threshold of 50 centimeters beyond the safe zone. When the autonomous vehicle detects that it has entered the safe protection zone, it enters an emergency protection state and issues a warning message. At this point, the autonomous vehicle will take emergency braking measures to ensure its own safety and the safety of obstacles. The autonomous vehicle may only then receive the remote control device's command for maintaining a constant speed or decelerating, which may be delayed. However, the autonomous vehicle has already left the safe zone and entered the protected safety zone. In this situation, the autonomous vehicle will prioritize its own control commands and disregard the driving commands from the remote control device; that is, the autonomous vehicle will execute the emergency braking command while ignoring the control commands from the remote control device.

[0032] Similarly, in another scenario, while the autonomous vehicle is driving normally, with a distance of 4 meters from the vehicle in front and a safety zone of 2 meters, an obstacle suddenly appears in front. For example, a pedestrian suddenly appears running across the road 1 meter in front of the autonomous vehicle. At this moment, before the remote control device can issue a deceleration command, the autonomous vehicle's safety zone will suddenly shrink due to the appearance of the pedestrian as an obstacle. This is equivalent to the autonomous vehicle moving out of the safety zone and into the safety protection zone. The autonomous vehicle will enter an emergency protection state and issue a warning. At this time, the autonomous vehicle will take emergency braking actions to ensure its own safety and the safety of the obstacle. Likewise, in this situation, the autonomous vehicle will execute its own control commands with higher priority, and will not execute driving commands from the remote control device. That is, the autonomous vehicle will execute the emergency braking command and ignore the control commands from the remote control device.

[0033] Furthermore, the scope of the safe zone and the scope of the safety protection zone can be flexibly changed according to factors such as road conditions, and are not limited to the distances mentioned earlier.

[0034] The present invention also proposes a safety control system for remote driving of unmanned vehicles in low-speed environments. The system includes an unmanned vehicle and on-board sensing devices, which correspond one-to-one with the features of the aforementioned method embodiments, and therefore will not be described in detail.

[0035] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0036] It will be apparent to those skilled in the art that the embodiments of the present invention are not limited to the details of the exemplary embodiments described above, and that the embodiments of the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the embodiments of the present invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the embodiments of the present invention is defined by the appended claims rather than the foregoing description. Therefore, all variations falling within the meaning and scope of equivalents of the claims are intended to be encompassed within the embodiments of the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units, modules, or devices recited in the system, apparatus, or terminal claims may also be implemented by the same unit, module, or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any particular order.

[0037] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention and are not intended to limit them. Although the embodiments of the present invention have been described in detail with reference to the above preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions to the technical solutions of the embodiments of the present invention should not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A safety control method for remote driving of an unmanned vehicle in a low-speed environment, characterized by, include: The onboard sensing equipment of the unmanned vehicle collects the current vehicle data of the unmanned vehicle in real time; and uploads the vehicle data to the remote control equipment in real time. The remote control device is equipped with a display device to show the real-time status of the unmanned vehicle and send driving control commands to the unmanned vehicle; the remote control device defines a safe area for the unmanned vehicle, and the unmanned vehicle defines a vehicle safety protection area for itself based on the safe area. The vehicle safety protection area is the area that is greater than a certain threshold from the boundary of the safe area; the safe area is updated in real time according to the vehicle's driving status and the surrounding road conditions. When the unmanned vehicle detects that it has entered the safety protection area in real time, the unmanned vehicle will enter an emergency protection state and issue a warning message; when in the emergency protection state, the unmanned vehicle will drive according to the control commands from itself, which have higher priority than the control commands issued by the remote control device.

2. The safety control method for remote driving of unmanned vehicles in low-speed environments as described in claim 1, characterized in that, The vehicle data includes the location of the unmanned vehicle and real-time video streams from the outside; the real-time status display of the unmanned vehicle includes a display of the actual location of the unmanned vehicle and a simulated display of the safe area.

3. The safety control method for remote driving of unmanned vehicles in low-speed environments as described in claim 1, characterized in that, The vehicle safety protection zone can be flexibly changed according to different road conditions.

4. The safety control method for remote driving of unmanned vehicles in low-speed environments as described in any one of claims 1-3, characterized in that, The onboard sensing equipment of the driverless car includes a surround-view fisheye camera, a perspective color camera, an event camera, an infrared camera, and a 32-beam lidar.

5. A safety control system for remote driving of an unmanned vehicle in a low-speed environment, the system comprising an unmanned vehicle and onboard sensing equipment, characterized in that: The onboard sensing equipment of the unmanned vehicle collects the current vehicle data of the unmanned vehicle in real time; and uploads the vehicle data to the remote control equipment in real time. The remote control device is equipped with a display device to show the real-time status of the unmanned vehicle and send driving control commands to the unmanned vehicle; the remote control device defines a safe area for the unmanned vehicle, and the unmanned vehicle defines a vehicle safety protection area for itself based on the safe area. The vehicle safety protection area is the area that is greater than a certain threshold from the boundary of the safe area; the safe area is updated in real time according to the vehicle's driving status and the surrounding road conditions. When the unmanned vehicle detects that it has entered the safety protection area in real time, the unmanned vehicle will enter an emergency protection state and issue a warning message; when in the emergency protection state, the unmanned vehicle will drive according to the control commands from itself, which have higher priority than the control commands issued by the remote control device.

6. The safety control system for remote driving of unmanned vehicles in low-speed environments as described in claim 5, characterized in that, The vehicle data includes the location of the unmanned vehicle and real-time video streams from the outside; the real-time status of the vehicle is displayed by showing the actual location of the unmanned vehicle and a simulated display of the safe area.

7. The safety control system for remote driving of unmanned vehicles in low-speed environments as described in claim 5, characterized in that, The vehicle safety protection zone can be flexibly changed according to different road conditions.

8. The safety control system for remote driving of unmanned vehicles in low-speed environments as described in any one of claims 5-7, characterized in that, The onboard sensing equipment of the driverless car includes a surround-view fisheye camera, a perspective color camera, an event camera, an infrared camera, and a 32-beam lidar.

Images