Automatic driving equipment monitoring system on expressway

Abstract

The invention relates to an automatic driving equipment monitoring system on an expressway, and belongs to the technical field of intelligent traffic. The sensor collects road data; the anti-interference module is used for performing space-time alignment on data acquired by the sensors and dynamically adjusting the fusion weight of the sensors according to weather conditions; the edge calculation layer is used for performing real-time processing and dynamic task scheduling on the data and balancing calculation load and response speed; and the cloud decision-making layer is used for carrying out global integration and decision-making optimization on the data, triggering a multi-level early warning protocol when an emergency event or a road condition is detected, and broadcasting early warning information and vehicle passing suggestions through road test equipment. Manual work is replaced by automatic inspection, the disease identification accuracy is greatly improved, the inspection efficiency is improved, the emergency response time is shortened through real-time data uploading and global scheduling, inspection can be completed through a common vehicle-mounted terminal, special equipment does not need to be leased, and the daily cost is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent transportation, and in particular to an automatic driving device monitoring system on a highway. BACKGROUND

[0002] Traditional inspection technology on highways mainly relies on manual or special vehicles, which is low in efficiency and high in cost, and is difficult to cover the entire section of the highway. Fixed monitoring devices (such as video monitoring and weather stations) have blind spots and cannot respond to sudden events (such as fog and road collapse). Multi-source heterogeneous data (such as sensor data and traffic flow information) lack efficient fusion and dynamic analysis capabilities. SUMMARY

[0003] To solve the above technical problems, the present application provides an automatic driving device monitoring system on a highway.

[0004] In a first aspect, the present application provides an automatic driving device monitoring system on a highway, comprising

[0005] A vehicle-mounted perception layer includes sensors and an anti-interference module; the sensors collect road data; the anti-interference module performs spatio-temporal alignment on the sensor data and dynamically adjusts the weight of the sensor fusion according to the weather conditions;

[0006] An edge computing layer performs real-time processing and dynamic task scheduling on the data, balances the computing load and response speed;

[0007] A cloud decision layer integrates and optimizes the data globally, and triggers a multi-level early warning protocol when an emergency event or road condition is detected, and broadcasts warning information and vehicle passage suggestions through road testing equipment.

[0008] Further, the sensors include a laser radar, a millimeter wave radar, a visual camera, and a GNSS / IMU combined positioning module.

[0009] Further, the anti-interference module performs spatio-temporal alignment on the sensor data, specifically including:

[0010] The global synchronization timestamp is obtained through the PTP protocol, with an accuracy of microseconds (±1us), so that the laser radar, millimeter wave radar, visual camera, and GNSS / IMU combined positioning module are aligned in the time dimension, avoiding time misalignment caused by differences in sensor sampling frequency;

[0011] The coordinate systems of different sensors are unified into the same reference coordinate system through a transformation matrix, solving the problem of spatial misalignment caused by differences in sensor installation positions;

[0012] According to the synchronization timestamp, the point cloud data of the laser radar is acquired, the point cloud left side is converted to a unified coordinate system through a transformation matrix, and the spatio-temporally aligned laser radar point cloud is obtained.

[0013] According to the synchronization timestamp, the tracking target data of the millimeter wave radar is acquired, and the local coordinates of the radar data are projected into a reference coordinate system in combination with the vehicle pose such as position and orientation, so as to obtain the radar target trajectory in the reference coordinate system.

[0014] Finally, a fusion data structure containing the spatio-temporally aligned laser radar point cloud, radar target trajectory and camera image is output.

[0015] Further, the camera image in the fusion data structure further includes a distortion removal process for the visual camera before fusion, eliminating lens distortion and ensuring accurate spatial alignment of the image with the laser radar / radar data.

[0016] Further, the dynamic adjustment of the recognition weight ratio of each sensor according to the weather condition specifically includes:

[0017] The visual camera image is identified in real time to determine the current weather type.

[0018] Based on the weather type, the sensor weight is dynamically allocated.

[0019] Further, the edge computing layer includes a bandwidth detection module, a dynamic offloading module, a cloud processing module and a local processing module; the network detection module is used to detect the network state of data transmission, including the bandwidth and delay of the network; the dynamic offloading module judges whether to offload the edge computing task to the cloud processing module or the local processing module for processing according to the detection result of the network detection module; the cloud processing module receives the uploaded data through the cloud server, analyzes the data using an AI model, generates a decision instruction and issues it to the vehicle-mounted device and the roadside device; the local processing module analyzes and detects the data through a local lightweight model, generates an abnormal detection result, and broadcasts it through the roadside unit.

[0020] Further, the global integration and decision optimization of the cloud decision layer on data specifically includes:

[0021] The edge data is summarized;

[0022] Using an LSTM prediction model, combining 7 days of historical data, predicting the road degradation probability to ensure that the prediction error is less than 5%;

[0023] Through a genetic algorithm optimizer, the coverage rate and time cost of the patrol route are comprehensively evaluated, the patrol path is optimized, a multi-objective optimization result is obtained, and the optimal patrol path is generated.

[0024] Further, the anti-interference module further comprises a physical protection unit, and the physical protection unit comprises a hydrophobic coating and a high-pressure air blowing device.

[0025] The beneficial effects of the present application are embodied in:

[0026] Hardware level: develop an anti-interference sensor array with self-cleaning function to solve the problem of performance degradation in rain and fog weather detection;

[0027] Algorithm level: build a dynamic weight fusion model to realize multi-source data confidence quantitative evaluation;

[0028] System level: establish a "end-edge-cloud" three-level collaborative architecture, and realize sub-second emergency response through V2X communication. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 The system structure diagram of the present application.

[0030] Figure 2 The data processing flowchart of the vehicle-mounted perception layer.

[0031] Figure 3 The data processing flowchart of the edge computing layer. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0033] Embodiment 1: refer to Figure 1 The automatic driving device monitoring system on a highway according to the present application comprises

[0034] 1) Vehicle-mounted perception layer: comprising sensors and an anti-interference module; the sensors collect road data; the anti-interference module performs spatio-temporal alignment on the data collected by the sensors and dynamically adjusts the weight of the sensor fusion according to the weather condition to improve the recognition effect;

[0035] As shown in Figure 2 The sensors comprise a laser radar, a millimeter wave radar, a visual camera, and a GNSS / IMU combined positioning module. The spatio-temporal alignment is performed on the data collected by the sensors, specifically: the global synchronization timestamp is obtained through the PTP protocol, with a precision of microsecond level (±1 μs), so that the laser radar, the millimeter wave radar, the visual camera, and the GNSS / IMU combined positioning module are aligned in the time dimension, avoiding the time misalignment caused by the difference in sampling frequency of the sensors;

[0036] The coordinate systems of different sensors are unified into the same reference coordinate system through a transformation matrix, solving the spatial misalignment problem caused by the difference in sensor installation positions;

[0037] The point cloud data of the laser radar is obtained according to the synchronization timestamp, and the point cloud left is converted to a unified coordinate system through a transformation matrix, to obtain the spatio-temporally aligned laser radar point cloud;

[0038] The tracking target data of the millimeter wave radar is obtained according to the synchronization timestamp, and the local coordinates of the radar data are projected into the reference coordinate system in combination with the vehicle pose such as position and orientation, to obtain the radar target trajectory in the reference coordinate system;

[0039] Finally, a fusion data structure containing the spatio-temporally aligned laser radar point cloud, radar target trajectory and camera image is output.

[0040] Among them, the camera image in the fusion data structure also includes the distortion removal processing (such as correcting radial distortion and tangential distortion) of the visual camera before fusion, eliminating lens distortion and ensuring that the image is accurately aligned in space with the laser radar / radar data.

[0041] Among them, dynamically adjusting the recognition weight ratio of each sensor according to the weather conditions specifically includes: real-time recognition of the visual camera image to determine the current weather type; based on the weather type, dynamically allocating sensor weights, and the weight allocation logic is as follows: rain / snowy weather: weaken the weight of the visual sensor (low visibility), and increase the weight of the laser radar and millimeter wave radar (strong penetration). Snowy weather: increase the weight of the laser radar (stable geometric information), and reduce the weight of the camera (reflection interference). Table 1 is an example of weight allocation.

[0042] Table 1

[0043]

[0044]

[0045] In poor weather conditions, such as in rainy weather, the weight of the laser radar is increased to 50%, and the laser radar (Lidar) is less affected by rain, which can provide more reliable geometric information (such as obstacle distance); the weight of the camera is reduced to 20%, and rain can cause image blur, reflection or reduced visibility, reducing the dependence on visual data.

[0046] 2) Edge computing layer: real-time processing and dynamic task scheduling of data, balancing computing load and response speed; in different network bandwidth conditions, different processing modes are used to enhance system processing capacity and accuracy, as shown in Figure 3 .

[0047] The edge computing layer includes a bandwidth detection module, a dynamic offloading module, a cloud processing module and a local processing module; the network detection module is used for detecting the network state of data transmission, including the bandwidth and delay of the network; the dynamic offloading module judges whether to offload the edge computing task to the cloud processing module or the local processing module for processing according to the detection result of the network detection module; the cloud processing module receives uploaded data through a cloud server, analyzes the data by using an AI model, generates a decision instruction and issues the decision instruction to a vehicle-mounted device and a roadside device; the local processing module analyzes and detects data by using a local lightweight model, generates an abnormality detection result, and broadcasts the abnormality detection result through a roadside unit.

[0048] 3) Cloud decision layer: globally integrates and optimizes data, and triggers a multi-level early warning protocol when detecting an emergency event or a road condition, and broadcasts early warning information and vehicle passing suggestions through a road testing device.

[0049] The cloud decision layer globally integrates and optimizes data, and specifically:

[0050] The edge data is summarized;

[0051] An LSTM prediction model is used to predict the road degradation probability by combining 7-day historical data, so as to ensure that the prediction error is less than 5%;

[0052] A genetic algorithm optimizer is used to comprehensively evaluate the coverage rate and time cost of a patrol route, to optimize the patrol path, to obtain a multi-objective optimization result, and to generate an optimal patrol path.

[0053] The system establishes a "end-edge-cloud" three-level collaborative architecture, and realizes sub-second emergency response through V2X communication.

[0054] In the description of the embodiments of the present application, it should be understood that the orientations or positional relationships indicated by the terms "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "center", "top", "bottom", "top", "bottom", "inner", "outer", "inner side", "outer side" and the like are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. Among them, "inboard" refers to the interior or enclosed area or space. "Periphery" refers to the area around a particular component or a particular area.

[0055] In the description of the embodiments of the present application, the terms "first", "second", "third", "fourth" are used only to describe purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second", "third", "fourth" can explicitly or implicitly include one or more of the features. In the description of the present application, unless otherwise specified, the meaning of "a plurality of" is two or more.

[0056] In the description of the embodiments of the present application, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0057] In the description of the embodiments of the present application, the term "and / or" herein is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the three cases of A alone, A and B together, and B alone. In addition, the character " / " herein generally represents that the front and rear associated objects are in an "or" relationship.

[0058] Although the embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made to the embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. An automatic driving device monitoring system on an expressway, characterized by, The system comprises a vehicle-mounted perception layer, an edge computing layer, and a cloud decision layer. The vehicle-mounted perception layer comprises sensors and an anti-interference module. The sensors collect road data.

2. The automatic driving device monitoring system on a high-speed road according to claim 1, characterized by, The anti-interference module performs spatio-temporal alignment on the data collected by the sensors and dynamically adjusts the weight of sensor fusion according to the weather conditions.

3. The automatic driving device monitoring system on a high-speed road according to claim 2, characterized by, The edge computing layer performs real-time processing and dynamic task scheduling on the data, balances the computing load and response speed, and triggers a multi-level early warning protocol when an emergency event or road condition is detected. The cloud decision layer integrates and optimizes the data globally, triggers a multi-level early warning protocol when an emergency event or road condition is detected, and broadcasts warning information and vehicle passage suggestions through road testing equipment. The sensors include laser radar, millimeter wave radar, visual camera, and GNSS / IMU combined positioning module. The anti-interference module performs spatio-temporal alignment on the data collected by the sensors, which includes the following steps: Obtain global synchronization timestamps through PTP protocol to align the laser radar, millimeter wave radar, visual camera, and GNSS / IMU combined positioning module in the time dimension. Transform the coordinate systems of different sensors into the same reference coordinate system through a transformation matrix.

4. The automatic driving device monitoring system on a high-speed road according to claim 3, characterized by, Obtain the point cloud data of the laser radar according to the synchronization timestamps, and convert the point cloud to the unified coordinate system through the transformation matrix to obtain the spatio-temporally aligned laser radar point cloud.

5. The automatic driving device monitoring system on a high-speed road according to claim 1, characterized by, Obtain the tracking target data of the millimeter wave radar according to the synchronization timestamps, project the local coordinates of the radar data to the reference coordinate system based on the vehicle pose, and obtain the radar target trajectory in the reference coordinate system. Finally, output a fusion data structure containing the spatio-temporally aligned laser radar point cloud, radar target trajectory, and camera image. The camera image in the fusion data structure also includes distortion correction of the visual camera before fusion to eliminate lens distortion and ensure accurate spatial alignment of the image with the laser radar / radar data.

6. The automatic driving device monitoring system on a high-speed road according to claim 1, characterized by, The dynamic adjustment of the recognition weight ratio of each sensor according to the weather conditions includes the following steps:

7. The automatic driving device monitoring system on a high-speed road according to claim 1, characterized by, Perform real-time recognition on the visual camera image to determine the current weather type. Dynamically allocate sensor weights based on the weather type. The edge computing layer comprises a bandwidth detection module, a dynamic offloading module, a cloud processing module, and a local processing module. The network detection module detects the network state of data transmission, including the bandwidth and delay of the network. The dynamic offloading module determines whether to offload the edge computing task to the cloud processing module or the local processing module for processing based on the detection results of the network detection module. The cloud processing module receives and uploads data through the cloud server, analyzes the data using an AI model, generates decision instructions, and sends them to the vehicle-mounted equipment and roadside equipment. The local processing module analyzes and detects the data using a local lightweight model, generates abnormal detection results, and broadcasts them through the roadside unit. The cloud decision layer integrates and optimizes the data globally, which includes the following steps: Aggregate the edge data. Use an LSTM prediction model to predict the road degradation probability based on 7 days of historical data. Optimize the patrol route using a genetic algorithm optimizer to evaluate the coverage rate and time cost of the patrol route, obtain multi-objective optimization results, and generate the optimal patrol route.

8. The automatic driving device monitoring system on a high-speed road according to claim 1, characterized by, The anti-interference module further comprises a physical protection unit, which comprises a hydrophobic coating and a high-pressure air blowing device.

Images