Unmanned vehicle remote driving and early warning system based on sensing, communication and calculation integrated equipment

Abstract

The invention relates to an unmanned vehicle remote driving and early warning system based on sensing, communication and calculation integrated equipment, which belongs to the technical field of automatic driving and comprises a vehicle-mounted remote driving suite, a remote driving cabin, a vehicle-mounted sensor and sensing, communication and calculation integrated intelligent roadside equipment, the vehicle-mounted remote driving suite receives a remote cockpit operation instruction; the sensing, communication and calculation integrated intelligent roadside equipment monitors road and vehicle information; the vehicle-mounted sensor acquires information of traffic targets around a vehicle in real time; and the remote cockpit receives and fuses data uploaded by the sensing, communication and calculation integrated intelligent roadside equipment and the vehicle-mounted sensor. When the automatic driving vehicle is in a remote driving mode, pedestrian / non-motor vehicle data sensed by the sensing, communication and calculation integrated equipment is displayed on a screen of a remote driving cabin, so that a remote safety officer is assisted in finding pedestrians and non-motor vehicles of which vehicle-mounted cameras are shielded and hidden in the middle of the vehicle; and measures are taken in advance to avoid traffic accidents.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, and in particular to an unmanned vehicle remote driving and early warning system based on a sense and algorithm integrated device. BACKGROUND

[0002] When the automatic driving vehicle is equipped with a remote driving cabin, the vehicle will be unmanned for services such as connection; when the automatic driving vehicle encounters a scene that cannot be handled by itself, such as rain causing vehicle sensor data confusion, road construction, or other vehicles invading and unable to pass, a remote takeover request can be triggered, and a remote safety officer in the monitoring hall can control the vehicle through the remote driving cabin to complete detour, parking, or driving the vehicle to the destination.

[0003] When the remote safety officer controls the vehicle using the images transmitted by the vehicle-mounted camera to the remote driving cabin, the images are provided by the vehicle-mounted camera, and if there are pedestrians or two-wheeled vehicles blocking the road side parking, it is easy to cause collision risks due to visual blind area. SUMMARY

[0004] To solve the above technical problems, the present application provides an unmanned vehicle remote driving and early warning system based on a sense and algorithm integrated device.

[0005] An unmanned vehicle remote driving and early warning system based on a sense and algorithm integrated device, comprising: a vehicle-mounted remote driving kit, a remote driving cabin, a vehicle-mounted sensor, and a sense and algorithm integrated intelligent roadside device; The vehicle-mounted remote driving kit is installed on the automatic driving vehicle, used to receive the remote driving cabin operation instruction, and control the automatic driving vehicle according to the operation instruction; The sense and algorithm integrated intelligent roadside device is installed on the roadside rod, used to monitor the road and vehicle information, and return the roadside data set obtained by calculation and analysis to the remote driving cabin; The vehicle-mounted sensor is used to obtain the traffic target information around the vehicle in real time, generate the vehicle end data set from the target information, and return the vehicle end data set to the server of the remote driving cabin in real time; The remote driving cabin is installed in the monitoring hall, receives and fuses the data uploaded by the sense and algorithm integrated intelligent roadside device and the vehicle-mounted sensor, and provides the remote safety officer with all-around coverage of road information for remote operation of the automatic driving vehicle through the remote driving cabin.

[0006] Further, the remote driving cabin includes a screen, a control panel, a steering wheel, a gear controller, a brake pedal, and an accelerator pedal for operation of the automatic driving vehicle, and a server for data receiving, sending, and fusion processing.

[0007] Further, the fusion processing of the server specifically includes the following steps: Data fusion: Spatiotemporally aligning roadside datasets and vehicle-side datasets; Differential detection: Based on ID and location comparison, it detects and identifies blind spot targets in the roadside dataset that are not present in the vehicle-side dataset.

[0008] Furthermore, the blind spot targets are displayed as icons on the remote cockpit screen to assist the remote safety operator in detecting things that the onboard sensors fail to identify.

[0009] Furthermore, the in-vehicle remote driving kit includes an in-vehicle controller, an in-vehicle camera, and a 5G smart gateway.

[0010] Furthermore, the integrated intelligent roadside device includes a sensing module, a communication module, and an edge computing module.

[0011] The beneficial effects of this invention are as follows: When driving an autonomous vehicle in remote driving mode, the invention displays pedestrian / non-motorized vehicle data sensed by the integrated sensing, communication, and computing device on the screen of the remote cockpit, assisting the remote safety operator in discovering pedestrians and non-motorized vehicles whose onboard cameras are obstructed and hidden in the middle of the vehicle, and taking measures in advance to avoid traffic accidents. Attached Figure Description

[0012] Figure 1 This is a system architecture diagram of the present invention. Detailed Implementation

[0013] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0014] In this embodiment: Refer to Figure 1 As shown in the figure, an embodiment of the present invention provides a remote driving and early warning system for unmanned vehicles based on an integrated sensing, communication and computing device, which includes: an on-board remote driving kit, a remote cockpit, on-board sensors, and an integrated sensing, communication and computing intelligent roadside device. The vehicle-mounted remote driving kit is installed on autonomous vehicles and includes an on-board controller, on-board camera, and 5GCPE. It is used to receive remote cockpit operation commands and control the autonomous vehicle according to the operation commands. The integrated intelligent roadside device is installed on roadside poles (such as on approximately 6 poles) and includes a sensing module, a communication module, and an edge computing module; it is used to monitor road and vehicle information and transmit the calculated and analyzed roadside dataset back to the remote cockpit. Vehicle-mounted sensors are used to acquire real-time information on traffic objects around the vehicle, generate vehicle-side datasets from the object information, and transmit the vehicle-side datasets back to the server in the remote cockpit in real time. The remote cockpit is installed in the monitoring hall and includes a screen, control panel, steering wheel, gear shift controller, brake pedal, accelerator pedal, and server. It is used to receive and integrate data uploaded by the integrated intelligent roadside equipment and vehicle sensors, providing comprehensive road information for remote safety operators to remotely operate autonomous vehicles through the remote cockpit.

[0015] In this embodiment, the integrated sensing, communication, and computing intelligent roadside device senses traffic targets within its range through the sensing module; it obtains a roadside target dataset (specifically including: time, ID, type, location, speed, and heading angle, where the type includes vehicles, non-motorized vehicles, pedestrians, and other traffic participants) through the edge computing module, and then transmits the roadside target data back to the server in the remote cockpit.

[0016] The vehicle-mounted sensors acquire real-time information about traffic objects around the vehicle and generate a vehicle-side dataset containing the object's time, ID, type, location, speed, and heading angle. This dataset is then transmitted back to the remote cockpit server in real time.

[0017] In this embodiment, the remote cockpit server merges the roadside dataset and the vehicle-side dataset. The purpose is to extract the target object C (i.e., the blind spot target object) that is identified by the roadside dataset but not by the vehicle-side dataset. By using the position coordinates of the target object C and the position coordinates of the autonomous vehicle itself, the relative position of the target object C and the autonomous vehicle is calculated. For example, C is 20 meters away at a 30-degree angle to the left front of the autonomous vehicle. The server displays the target object C as an icon on the screen of the remote safety operator to help the remote safety operator discover pedestrians, non-motorized vehicles, etc. that are hidden in the middle of the vehicle and cannot be identified by the vehicle-mounted camera, so as to take measures in advance to avoid traffic accidents.

[0018] The specific process of data fusion to extract target object C is as follows: Assume the roadside dataset is... The vehicle-side dataset is Each data entry contains attributes such as location and time; 1. Data fusion (spatiotemporal alignment); , The Align function aligns data based on time and location. For example, it matches targets with the same timestamp and those that are geographically close.

[0019] , Δt: Time tolerance threshold (e.g., 0.1s), the document implicitly requires real-time performance.

[0020] Δd: Location distance threshold (e.g., 2m), based on device coverage accuracy.

[0021] When a match is successful, merge the entry: d f =(t,ID,Type,Position,…), roadside datasets are preferred (because roadside datasets have wider coverage).

[0022] Differential detection: , Based on ID and location comparison, target object C (an entry that exists in the roadside dataset but is missing in the vehicle-side dataset) is detected and identified, and the server extracts it for subsequent warnings.

[0023] , Output: The set of target objects C, C={d c} Data fusion ensures data consistency, and difference detection utilizes set operations (difference sets) to identify targets in blind spots. The algorithm optimizes computational efficiency, achieving O(1) complexity matching through a hash table.

[0024] Explanation of parameters in the above formula: Time (t): Data entry timestamp, in seconds (s). Source: Roadside or vehicle-mounted sensor.

[0025] ID (ID): Unique ID of the target, an integer. Source: Algorithm-assigned, used for matching.

[0026] Type: Target category, an enumeration value. Source: Detection output.

[0027] Position: Three-dimensional coordinates (x, y, z), unit: meters (m). Source: Calculated from the roadside or vehicle.

[0028] Velocity: Velocity vector (v_x, v_y, v_z), unit: meters per second (m / s). Source: Same as above.

[0029] Heading Angle: The angle of direction, measured in radians (rad). Source: Same as above.

[0030] Threshold parameters: Δt (time threshold, default 0.1s), Δd (position threshold, default 2m), based on system calibration.

[0031] In the description of embodiments of the present invention, the terms "first," "second," "third," and "fourth" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first," "second," "third," or "fourth" may explicitly or implicitly include one or more of that feature. In the description of the present invention, unless otherwise stated, "a plurality of" means two or more.

[0032] In the description of embodiments of the present invention, the term "and / or" is used only to describe the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Additionally, the character " / " generally indicates that the preceding and following associated objects are in an "or" relationship.

[0033] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An unmanned vehicle remote driving and early warning system based on a sense-through-computing integrated device, characterized in that, The application relates to a vehicle-mounted remote driving kit, a remote driving cabin, a vehicle-mounted sensor and a sensing-computing integrated intelligent roadside equipment. The vehicle-mounted remote driving kit is installed on an automatic driving vehicle and used for receiving an operation instruction of the remote driving cabin and controlling the automatic driving vehicle according to the operation instruction. The sensing-computing integrated intelligent roadside equipment is installed on a roadside rod and used for monitoring road and vehicle information and returning a roadside data set obtained through calculation and analysis to the remote driving cabin. The vehicle-mounted sensor is used for acquiring traffic target information around the vehicle in real time, generating a vehicle-end data set from the target information and returning the vehicle-end data set to a server of the remote driving cabin in real time. The remote driving cabin is installed in a monitoring hall, receives and fuses data uploaded by the sensing-computing integrated intelligent roadside equipment and the vehicle-mounted sensor and provides omnibearing road information for a remote safety officer to remotely operate the automatic driving vehicle through the remote driving cabin. The remote driving cabin comprises a screen, a control panel, a steering wheel, a gear controller, a brake pedal and an accelerator pedal for operating the automatic driving vehicle and a server for data receiving, sending and fusion processing.

2. The unmanned vehicle remote driving and early warning system based on the integrated device of the sense channel according to claim 1, characterized in that, The fusion processing of the server comprises the following steps: 3.The remote driving and early warning system based on the integrated device of the sense channel and the algorithm of the unmanned vehicle according to claim 2, characterized in that, Data fusion: time and space alignment of the roadside data set and the vehicle-end data set; Differential detection: based on ID and position comparison, a blind spot target object in the roadside data set which is not in the vehicle-end data set is detected and identified. The blind spot target object is displayed on the screen of the remote driving cabin in the form of an icon and used for assisting the remote safety officer to find things which cannot be recognized by the vehicle-mounted sensor.

4. The unmanned vehicle remote driving and early warning system based on the integrated device of the sense channel according to claim 3, characterized in that, The vehicle-mounted remote driving kit comprises a vehicle-mounted controller, a vehicle-mounted camera and a 5G intelligent gateway.

5. The unmanned vehicle remote driving and early warning system based on the integration of the sense channel and the algorithm device according to claim 1, characterized in that, The sensing-computing integrated intelligent roadside equipment comprises a sensing module, a communication module and an edge computing module.

6. The unmanned vehicle remote driving and early warning system based on the integrated device of the sense channel according to claim 1, characterized in that, ​

Images