An L4 level automatic driving monitoring system based on a ROS system

Abstract

The application relates to an L4-level automatic driving monitoring system based on a ROS system, which comprises a monitoring data integration module, a process state monitoring module, a data frequency monitoring module, a state discrimination module and a system resource monitoring module, receives and integrates information of the process state monitoring module, the data frequency monitoring module, the state discrimination module and the system resource monitoring module, then sends the integrated information to a data reporting module, simultaneously performs a function degradation, a request takeover, a safe parking or an emergency parking execution strategy according to the fault classification of the automatic driving system when a fault occurs in a hardware module and a software module of the automatic driving system; the data reporting module receives the integrated information sent by the monitoring data integration module, then analyzes and encodes the integrated information, reports the analyzed and encoded information to a cloud platform monitoring module through a data gateway; the cloud platform monitoring module receives the analyzed and encoded information reported by the data reporting module, displays in real time through a monitoring page, generates an analysis report, and realizes a full life cycle service and quality safety guarantee of a vehicle automatic driving function.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to an L4-level autonomous driving monitoring system based on the ROS system. Background Technology

[0002] In recent years, autonomous driving heavy-duty truck technology has been continuously developed, and its technological and industrial applications are expanding. The application of autonomous driving technology can effectively address the shortage of heavy-duty truck drivers, reduce their workload, improve safety, reduce costs and increase efficiency, and meet the growing demand for transportation capacity. With the gradual application of autonomous driving systems, autonomous driving heavy-duty truck technology has become a development trend for new heavy-duty trucks; therefore, the safety and stability of the system are increasingly valued within the industry.

[0003] Currently, most autonomous driving systems operate based on the ROS system. There are several methods for monitoring the running modules within the ROS system, with two commonly used ones. One is to use a script to call the "ps" or "top" command to view the process status of the running module, and the other is to use an algorithm to read the system file " / proc" to monitor the process status of the running module. However, the first method involves frequent calls, which can consume a lot of system resources. In addition, the command calls load a number of .so files. If some .so files are missing, the command calls cannot be executed. If an error occurs after execution, it can create zombie processes. The second method, while able to monitor the process status of modules, cannot monitor the data frequency of each autonomous driving module. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention proposes an L4-level autonomous driving monitoring system based on the ROS system.

[0005] To address the problems mentioned above in the background section, the present invention is achieved through the following technical solution:

[0006] A Level 4 autonomous driving monitoring system based on ROS includes:

[0007] The process status monitoring module obtains the process names of the hardware and software modules of the autonomous driving system from the configuration file, obtains the process status, then obtains the process start location and command execution parameters based on the process name, and sends the obtained process status to the monitoring data integration module for storage through the status recognition module.

[0008] Data frequency monitoring module: Employs a timing device to count the number of data releases from the hardware and software modules within one second, thereby determining whether the data release frequency of the hardware and software modules is normal. The corresponding data release frequency of the hardware and software modules is then sent to the monitoring data integration module for storage via the status recognition module. Status judgment module: Determines the operating status of the hardware and software modules based on their process status and data release frequency, and sends the final operating status of the hardware and software modules to the monitoring data integration module.

[0009] The system resource monitoring module includes:

[0010] The CPU monitoring module is used to monitor CPU utilization.

[0011] The memory monitoring module is used to monitor memory usage.

[0012] The disk memory monitoring module is used to monitor disk memory usage and send the stored values ​​of CPU utilization, memory usage, and disk memory usage to the monitoring data integration module.

[0013] The monitoring data integration module receives and integrates information from the process status monitoring module, data frequency monitoring module, status discrimination module, and system resource monitoring module. Then, it sends the integrated information to the data reporting module. At the same time, based on the fault classification of the autonomous driving system, when the hardware and software modules of the autonomous driving system fail, it implements strategies such as function degradation, requesting takeover, safe parking, or emergency parking.

[0014] The data reporting module receives the integrated information sent by the monitoring data integration module, then parses and encodes the integrated information, and reports the parsed and encoded information to the cloud platform monitoring module through the data gateway.

[0015] The cloud platform monitoring module receives the parsed encoding information reported by the data reporting module, displays it in real time on the monitoring page, and generates analysis reports.

[0016] Furthermore, the hardware module includes a camera sensor, a lidar sensor, a millimeter-wave sensor, and a combined navigation sensor; the software module includes a perception software module, a prediction software module, a map software module, a path planning software module, a decision planning software module, a control software module, and a chassis analysis software module.

[0017] The process status monitoring module obtains the process names of the hardware and software modules from the configuration file, obtains the process status based on the process name, and reads the start position and command execution parameters of all processes of the hardware and software modules from proc / * / cmdline. If the process exists and the process status is S or R, the process status is determined to be Normal; otherwise, it is determined to be Abnormal. The process name and process status are then sent to the monitoring data integration module for storage through the status recognition module.

[0018] Furthermore, the data frequency monitoring module uses the TimeWall function of the ROS system to collect the number of times N data is published by the hardware module and software module within 1 second, and then determines whether N is less than the set frequency threshold. If so, the frequency status is determined to be Abnormal; otherwise, the frequency status is determined to be Normal. Then, the data publication frequency corresponding to the hardware module and software module is sent to the monitoring data integration module for storage.

[0019] Furthermore, the state discrimination module simultaneously receives hardware module and software module information output by the process state monitoring module and the data frequency monitoring module, searches and matches by process name, if the process state is Normal and the data release frequency state is also Normal, then the state of the hardware module or software module is determined to be Normal; otherwise, the state of the hardware module or software module is determined to be Abnormal, and the process name and final state of the hardware module or software module are stored.

[0020] Furthermore, the monitoring data integration module receives information from the process status monitoring module, data frequency monitoring module, status discrimination module, and system resource monitoring module, integrates it through the message_filters of the ROS system, and then sends the integrated information to the data reporting module.

[0021] The monitoring data integration module receives information from the hardware and software modules of the autonomous driving system. Based on the fault classification of the autonomous driving system, when the hardware and software modules of the autonomous driving system fail, it implements strategies such as function degradation, requesting takeover, safe parking, or emergency parking.

[0022] Furthermore, the fault classification of the autonomous driving system is as follows:

[0023] Level 1: Causes damage to important components of the vehicle, rendering it unusable, endangering personal safety, and potentially leading to death;

[0024] Level 2: Causes damage to major vehicle components, resulting in serious personal injury, significant economic loss, or failure of autonomous driving;

[0025] Level 3 causes partial failure of the autonomous driving system, and the failed part cannot be repaired in a short time, which may cause minor personal injury and some economic loss.

[0026] Level 4 has a slight impact on the autonomous driving function, and the autonomous driving system can repair the fault in a short time.

[0027] Level 5: Does not affect the autonomous driving function; can drive normally.

[0028] Furthermore, the execution strategies for function degradation, request takeover, safe shutdown, or emergency shutdown are as follows:

[0029] The autonomous driving system requests manual intervention.

[0030] Functional degradation: Autonomous driving function is downgraded, warning lights illuminate on the vehicle's infotainment screen, and voice prompts indicate a malfunction.

[0031] Safe parking: The automatic driving system enables the vehicle to stop quickly and safely and terminate its operation.

[0032] Emergency stop: The automatic driving system brings the vehicle to an emergency stop and terminates operation.

[0033] Compared with the prior art, the present invention has the following beneficial technical effects:

[0034] Compared to existing methods for monitoring the operating modules of the ROS system, this method can better achieve real-time monitoring of the operating status and system resource usage of the hardware and software modules of the autonomous driving system. It does not require calling system commands, consumes fewer system resources, and is less likely to generate zombie processes. Furthermore, it can monitor the data frequency of the autonomous driving hardware and software modules through a data frequency monitoring module. The integrated information is reported to the cloud platform monitoring module through a gateway to generate monitoring reports. This addresses the difficulties in troubleshooting autonomous driving system failures caused by memory overflows and freezes when autonomous driving heavy trucks have too many autonomous driving function applications installed. Additionally, by classifying autonomous driving faults, appropriate policy controls are implemented when the autonomous driving system malfunctions, greatly improving the stability and safety of the autonomous driving heavy truck system, thereby achieving full lifecycle service and quality and safety assurance for the vehicle's autonomous driving function. Attached Figure Description

[0035] Figure 1 This is a diagram of the architecture of the present invention;

[0036] Figure 2 This is a flowchart of the process status monitoring module of the present invention;

[0037] Figure 3 This is a flowchart of the data frequency monitoring module of the present invention;

[0038] Figure 4 This is a flowchart of the system resource monitoring module of the present invention. Detailed Implementation

[0039] like Figures 1-4 As shown, an L4-level autonomous driving monitoring system based on the ROS system includes:

[0040] The process status monitoring module obtains the process names of the hardware and software modules of the autonomous driving system from the configuration file, obtains the process status, then obtains the process start location and command execution parameters based on the process name, and sends the obtained process status to the monitoring data integration module for storage through the status recognition module.

[0041] Data frequency monitoring module: It uses a timing device to count the number of times the hardware module and software module publish data within 1 second, in order to determine whether the data publication frequency of the hardware module and software module is normal, and sends the corresponding data publication frequency of the hardware module and software module to the monitoring data integration module for storage through the status recognition module.

[0042] The status determination module determines the operating status of the hardware and software modules by analyzing their process status and data release frequency, and then sends the final operating status of the hardware and software modules to the monitoring data integration module.

[0043] The system resource monitoring module includes:

[0044] The CPU monitoring module is used to monitor CPU utilization.

[0045] The memory monitoring module is used to monitor memory usage.

[0046] The disk memory monitoring module is used to monitor disk memory usage and send the stored values ​​of CPU utilization, memory usage, and disk memory usage to the monitoring data integration module.

[0047] CPU utilization is calculated as follows:

[0048] First, collect CPU metrics for two sufficiently short time intervals ΔT, and calculate the CPUTime within ΔT using the following formula.

[0049] CPUTime=s2-s1=(user2+nice2+system2+idle2+iowait2+irq2+softirq2+steal2+guest2+guest_nice2)-(user1+nice1+system1+idle1+iowait1+irq1+softirq1+steal1+guest1+guest_nice1)

[0050] Then calculate the idle time ΔT.

[0051] idle = idle2 - idle1

[0052] Total CPU utilization of the computing system (CPU Usage)

[0053] CPUUsage=(CPUTime-idle) / CPUTime.

[0054] The calculation method for memory usage is as follows:

[0055] MemoryUsage=(MemTotal–MemFree–Buffers–Cached) / MemTotal

[0056] MemTotal is the total memory size, MemFree is the free memory size, and Buffer and Cached are the cache sizes. MemTotal, MemFree, Buffer, and Cached are all obtained through proc / meminfo.

[0057] The formula for calculating disk usage (DiskUsage) is as follows:

[0058] DiskUsage=(Capacity-Available) / Capacity

[0059] Here, Capacity is the disk capacity, and Available is the available disk space. Both Capacity and Available can be obtained through proc / meminfo.

[0060] The monitoring data integration module receives and integrates information from the process status monitoring module, data frequency monitoring module, status discrimination module, and system resource monitoring module. Then, it sends the integrated information to the data reporting module. At the same time, based on the fault classification of the autonomous driving system, when the hardware and software modules of the autonomous driving system fail, it implements strategies such as function degradation, requesting takeover, safe parking, or emergency parking.

[0061] The data reporting module receives the integrated information sent by the monitoring data integration module, then parses and encodes the integrated information, and reports the parsed and encoded information to the cloud platform monitoring module through the data gateway.

[0062] The cloud platform monitoring module receives the parsed encoding information reported by the data reporting module, displays it in real time on the monitoring page, and generates analysis reports.

[0063] The hardware modules include camera sensors, LiDAR sensors, millimeter-wave sensors, and integrated navigation sensors; the software modules include perception software modules, prediction software modules, mapping software modules, path planning software modules, decision planning software modules, control software modules, and chassis analysis software modules.

[0064] The process status monitoring module obtains the process names of the hardware and software modules from the configuration file, and obtains the process status based on the process name. It reads the start position and command execution parameters of all processes of the hardware and software modules from proc / * / cmdline. If the process exists and the process status is S or R, the process status is determined to be Normal; otherwise, it is determined to be Abnormal. The process name and process status are then sent to the monitoring data integration module for storage through the status recognition module.

[0065] The data frequency monitoring module uses the TimeWall function in the ROS system to collect the number of times N data is published by the hardware and software modules within 1 second. Then it determines whether N is less than the set frequency threshold. If so, the frequency status is determined to be Abnormal; otherwise, the frequency status is determined to be Normal. Then, the data publication frequency corresponding to the hardware and software modules is sent to the monitoring data integration module for storage.

[0066] The status discrimination module simultaneously receives hardware and software module information output by the process status monitoring module and the data frequency monitoring module. It searches and matches by process name. If the process status is Normal and the data release frequency status is also Normal, the hardware or software module status is determined to be Normal. Otherwise, the hardware or software module status is determined to be Abnormal, and the process name and final status of the hardware or software module are stored.

[0067] The monitoring data integration module receives information from the process status monitoring module, data frequency monitoring module, status discrimination module and system resource monitoring module, integrates it through the message_filters of the ROS system, and then sends the integrated information to the data reporting module.

[0068] The monitoring data integration module receives information from the hardware and software modules of the autonomous driving system. Based on the fault classification of the autonomous driving system, when the hardware and software modules of the autonomous driving system fail, it implements strategies such as function degradation, requesting takeover, safe parking, or emergency parking.

[0069] The fault classification of autonomous driving systems is as follows:

[0070] Level 1: Causes damage to important components of the vehicle, rendering it unusable, endangering personal safety, and potentially leading to death;

[0071] Level 2: Causes damage to major vehicle components, resulting in serious personal injury, significant economic loss, or failure of autonomous driving;

[0072] Level 3 causes partial failure of the autonomous driving system, and the failed part cannot be repaired in a short time, which may cause minor personal injury and some economic loss.

[0073] Level 4 has a slight impact on the autonomous driving function, and the autonomous driving system can repair the fault in a short time.

[0074] Level 5: Does not affect the autonomous driving function; can drive normally.

[0075] The execution strategies for function degradation, requesting takeover, safe shutdown, or emergency shutdown are as follows:

[0076] The autonomous driving system requests manual intervention.

[0077] Functional degradation: Autonomous driving function is downgraded, warning lights illuminate on the vehicle's infotainment screen, and voice prompts indicate a malfunction.

[0078] Safe parking: The automatic driving system enables the vehicle to stop quickly and safely and terminate its operation.

[0079] Emergency stop: The automatic driving system brings the vehicle to an emergency stop and terminates operation.

[0080] For example, when the vehicle is running in autonomous driving mode, if the lidar sensor in the hardware module malfunctions, the warning light on the vehicle interface will light up, the voice prompt will indicate the fault, the vehicle will stop safely, or the autonomous driving mode will be downgraded to ACC adaptive loop mode.

[0081] When the vehicle is running in autonomous driving mode, if the perception software module in the software module malfunctions, the warning light on the vehicle's infotainment screen will illuminate, a voice prompt will indicate the malfunction, and the vehicle will come to a safe stop.

[0082] When the vehicle is in autonomous driving mode, if the control software module in the software module malfunctions, the warning light on the vehicle's infotainment screen will illuminate, a voice prompt will indicate the malfunction, and manual takeover of the vehicle will be required.

Claims

1. A Level 4 autonomous driving monitoring system based on ROS, characterized in that, include: The process status monitoring module obtains the process names of the hardware and software modules of the autonomous driving system from the configuration file, obtains the process status, then obtains the process start location and command execution parameters based on the process name, and sends the obtained process status to the monitoring data integration module for storage through the status recognition module. Data frequency monitoring module: It uses a timing device to count the number of times the hardware module and software module publish data within 1 second, in order to determine whether the data publication frequency of the hardware module and software module is normal, and sends the corresponding data publication frequency of the hardware module and software module to the monitoring data integration module for storage through the status recognition module. The status determination module determines the operating status of the hardware and software modules by analyzing their process status and data release frequency, and then sends the final operating status of the hardware and software modules to the monitoring data integration module. The system resource monitoring module includes: The CPU monitoring module is used to monitor CPU utilization. The memory monitoring module is used to monitor memory usage. The disk memory monitoring module is used to monitor disk memory usage and send the stored values ​​of CPU utilization, memory usage, and disk memory usage to the monitoring data integration module. The monitoring data integration module receives and integrates information from the process status monitoring module, data frequency monitoring module, status discrimination module, and system resource monitoring module. Then, it sends the integrated information to the data reporting module. At the same time, based on the fault classification of the autonomous driving system, when the hardware and software modules of the autonomous driving system fail, it implements strategies such as function degradation, requesting takeover, safe parking, or emergency parking. The data reporting module receives the integrated information sent by the monitoring data integration module, then parses and encodes the integrated information, and reports the parsed and encoded information to the cloud platform monitoring module through the data gateway. The cloud platform monitoring module receives the parsed encoding information reported by the data reporting module, displays it in real time on the monitoring page, and generates analysis reports.

2. The Level 4 autonomous driving monitoring system based on the ROS system according to claim 1, characterized in that, The hardware module includes a camera sensor, a lidar sensor, a millimeter-wave sensor, and a combined navigation sensor; the software module includes a perception software module, a prediction software module, a map software module, a path planning software module, a decision planning software module, a control software module, and a chassis analysis software module. The process status monitoring module obtains the process names of the hardware and software modules from the configuration file, obtains the process status based on the process name, and reads the start position and command execution parameters of all processes of the hardware and software modules from proc / * / cmdline. If the process exists and the process status is S or R, the process status is determined to be Normal; otherwise, it is determined to be Abnormal. The process name and process status are then sent to the monitoring data integration module for storage through the status recognition module.

3. The Level 4 autonomous driving monitoring system based on the ROS system according to claim 2, characterized in that, The data frequency monitoring module uses the TimeWall function in the ROS system to collect the number of times N data is published by the hardware and software modules within 1 second. Then it determines whether N is less than the set frequency threshold. If so, the frequency status is determined to be Abnormal; otherwise, the frequency status is determined to be Normal. Then, the data publication frequency corresponding to the hardware and software modules is sent to the monitoring data integration module for storage.

4. The Level 4 autonomous driving monitoring system based on the ROS system according to claim 3, characterized in that, The state discrimination module simultaneously receives hardware and software module information output by the process state monitoring module and the data frequency monitoring module. It searches and matches by process name. If the process state is Normal and the data release frequency state is also Normal, the state of the hardware module or software module is determined to be Normal. Otherwise, the state of the hardware module or software module is determined to be Abnormal, and the process name and final state of the hardware module or software module are stored.

5. The Level 4 autonomous driving monitoring system based on the ROS system according to claim 4, characterized in that, The monitoring data integration module receives information from the process status monitoring module, data frequency monitoring module, status discrimination module and system resource monitoring module, integrates it through the message_filters of the ROS system, and then sends the integrated information to the data reporting module. The monitoring data integration module receives information from the hardware and software modules of the autonomous driving system. Based on the fault classification of the autonomous driving system, when the hardware and software modules of the autonomous driving system fail, it implements strategies such as function degradation, requesting takeover, safe parking, or emergency parking.

6. The Level 4 autonomous driving monitoring system based on the ROS system according to claim 5, characterized in that, The fault classification of the autonomous driving system is as follows: Level 1: Causes damage to important components of the vehicle, rendering it unusable, endangering personal safety, and potentially leading to death; Level 2: Causes damage to major vehicle components, resulting in serious personal injury, significant economic loss, or failure of autonomous driving; Level 3 causes partial failure of the autonomous driving system, and the failed part cannot be repaired in a short time, which may cause minor personal injury and some economic loss. Level 4 has a slight impact on the autonomous driving function, and the autonomous driving system can repair the fault in a short time. Level 5: Does not affect the autonomous driving function; can drive normally.

7. The Level 4 autonomous driving monitoring system based on the ROS system according to claim 5, characterized in that, The execution strategies for function degradation, takeover request, safe shutdown, or emergency shutdown are as follows: The autonomous driving system requests manual intervention. Functional degradation: Autonomous driving function is downgraded, warning lights illuminate on the vehicle's infotainment screen, and voice prompts indicate a malfunction. Safe parking: The automatic driving system enables the vehicle to stop quickly and safely and terminate its operation. Emergency stop: The automatic driving system brings the vehicle to an emergency stop and terminates operation.

Images