Vehicle automatic driving state monitoring method and device, vehicle machine system and storage medium

Abstract

This application discloses a method, device, vehicle infotainment system, and storage medium for monitoring the state of autonomous driving in vehicles, belonging to the field of autonomous driving technology. The method includes acquiring vehicle state information; wherein the state information includes sensor state information, domain controller monitoring state information, vehicle component state information, vehicle infotainment system process operation information, and vehicle infotainment system functional module state information; based on the state information, determining whether the vehicle's autonomous driving operation state is abnormal. This application can comprehensively monitor the state of vehicles by integrating multiple aspects of state information during the autonomous driving process, improving the comprehensiveness of autonomous driving state monitoring.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a method, device, vehicle system and storage medium for monitoring the autonomous driving status of a vehicle. Background Technology

[0002] In related technologies, in order to achieve the safe operation of autonomous driving systems, autonomous driving systems generally have state monitoring functions. However, such state monitoring functions are often single state monitoring such as sensor abnormal state monitoring, vehicle-side abnormal monitoring, or domain controller abnormal monitoring, lacking a state monitoring method with more comprehensive coverage. Summary of the Invention

[0003] The main purpose of this application is to provide a method, device, vehicle system and storage medium for monitoring the autonomous driving status of a vehicle, with the aim of achieving comprehensive monitoring of the autonomous driving status of the vehicle.

[0004] To achieve the above objectives, this application provides a method for monitoring the state of autonomous driving in a vehicle, the method comprising the following steps:

[0005] Acquire vehicle status information; including sensor status information, domain controller monitoring status information, vehicle component status information, vehicle system process operation information, and vehicle system functional module status information;

[0006] Based on the status information, determine whether the vehicle's autonomous driving operation status is abnormal.

[0007] Optionally, obtain the vehicle's infotainment system process running information, including:

[0008] Call the vehicle infotainment system's process monitoring command to obtain the current running process information of the vehicle's infotainment system;

[0009] Based on the pre-configured process configuration information to be monitored, the information of the currently running process is detected to obtain the process running information of the vehicle system.

[0010] Optionally, obtain the vehicle's sensor status information, including:

[0011] The sensor status information of each sensor is read through the preset status interface of each sensor; wherein the sensor includes at least one of GNSS, camera, lidar, millimeter wave and ultrasonic radar, and the sensor status information includes sensor malfunction, inability to open sensor or inability to read sensor data.

[0012] Optionally, the domain controller monitors status information including resource usage status information and communication status information;

[0013] Obtain the domain controller monitoring status information of the vehicle, including:

[0014] Obtain resource usage status information of the domain controller through operating system instructions; wherein the resource usage status information includes at least one of CPU usage status, memory usage status, disk usage status, and temperature status; and / or

[0015] The communication status between the domain controller and the vehicle controller is determined by monitoring whether the domain controller receives heartbeat feedback information from the vehicle controller.

[0016] Optionally, obtain vehicle component status information, including:

[0017] Obtain vehicle component status information forwarded by the vehicle controller; wherein, the vehicle component status information includes at least one of braking information, steering information, ignition information, door information, truck bed status information, and fault code information.

[0018] Optionally, obtain the status information of the vehicle's in-vehicle infotainment module, including:

[0019] Monitor the module topic information of each subscribed in-vehicle infotainment system function module;

[0020] Based on the module topic information, obtain the status information of the vehicle infotainment function modules.

[0021] Optionally, after obtaining the vehicle's status information, the method further includes:

[0022] Based on the status information, determine the level of abnormality of the vehicle's current abnormal state;

[0023] Based on the anomaly level, generate the corresponding vehicle control command.

[0024] Furthermore, to achieve the above objectives, this application also proposes a vehicle autonomous driving state monitoring device, which includes:

[0025] The status acquisition module is used to acquire the vehicle's status information;

[0026] The anomaly detection module is used to determine whether the autonomous driving operation status of the vehicle is abnormal based on the status information.

[0027] In addition, to achieve the above objectives, this application also proposes a vehicle-mounted system, which includes a vehicle and a cloud control terminal.

[0028] In addition, to achieve the above objectives, this application also proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the vehicle autonomous driving state monitoring method described above.

[0029] This application provides a method for monitoring the state of autonomous driving in vehicles, which can acquire vehicle state information. This state information includes sensor state information, domain controller monitoring state information, vehicle component state information, vehicle system process operation information, and vehicle system functional module state information. Based on this state information, it determines whether the vehicle's autonomous driving operation state is abnormal. This allows for comprehensive monitoring of multiple aspects of the vehicle's autonomous driving process, improving the overall comprehensiveness of autonomous driving state monitoring. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the vehicle infotainment system involved in the embodiments of this application;

[0031] Figure 2 This is a logical architecture diagram of the vehicle infotainment system involved in the embodiments of this application;

[0032] Figure 3 This is a flowchart illustrating the first embodiment of the vehicle autonomous driving state monitoring method provided in this application.

[0033] Figure 4 This is a flowchart illustrating the second embodiment of the vehicle autonomous driving state monitoring method provided in this application.

[0034] Figure 5 This is a flowchart illustrating the third embodiment of the vehicle autonomous driving state monitoring method provided in this application.

[0035] Figure 6 This is a schematic diagram of the functional modules of the vehicle autonomous driving status monitoring device of this application.

[0036] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0037] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0038] Autonomous driving systems cannot guarantee 100% stable operation. Failures can occur in areas such as sensors, algorithms, and communications. Under such circumstances, it cannot be guaranteed that the vehicle can continue to operate stably or stop safely in the event of a serious failure. Therefore, it is necessary to perform anomaly diagnosis on autonomous vehicles to improve operational safety.

[0039] Currently, in order to ensure the safe operation of autonomous driving systems, most autonomous driving systems have some state monitoring functions. However, these state monitoring functions are often single state monitoring functions such as sensor abnormal state monitoring, vehicle-side abnormal monitoring, or domain controller abnormal monitoring, lacking a more comprehensive state monitoring method.

[0040] To address this issue, this application proposes a vehicle autonomous driving state monitoring method that can acquire vehicle state information. This state information includes sensor state information, domain controller monitoring state information, vehicle component state information, vehicle system process operation information, and vehicle system functional module state information. Based on this state information, it can determine whether the vehicle's autonomous driving operation state is abnormal. This method can comprehensively monitor the state by integrating multiple aspects of the vehicle's autonomous driving process, thereby improving the comprehensiveness of autonomous driving state monitoring.

[0041] The following will be explained and described through several examples.

[0042] Reference Figure 1 , Figure 1 This is a schematic diagram of the vehicle infotainment system involved in the embodiments of this application.

[0043] like Figure 1 As shown, the vehicle system includes the vehicle and the cloud control terminal.

[0044] The vehicle is equipped with autonomous driving capabilities and can communicate with a cloud control terminal. Communication between the vehicle and the cloud control terminal can be via wireless network, satellite, or local area network. The cloud control terminal, which can be a smart terminal device such as a computer or mobile phone, is used to remotely monitor and dispatch the vehicle through this communication connection.

[0045] Reference Figure 2 , Figure 2 This is a logical architecture diagram of the vehicle infotainment system involved in the embodiments of this application. Figure 2 The middle hardware layer consists of the vehicle's hardware components, such as sensors like lasers, cameras, millimeter-wave sensors, and ultrasonic sensors, as well as the vehicle-end components and domain controller components; the module layer is the vehicle's software functional layer, which includes functional modules such as vehicle positioning, perception, task, prediction, planning, and control, as well as safety monitoring modules and the vehicle control system (VCS); the cloud control layer is the cloud control terminal of the vehicle's infotainment system.

[0046] Based on, but not limited to, the above-described vehicle infotainment system, this application provides a first embodiment of a vehicle autonomous driving state monitoring method. (Refer to...) Figure 3 , Figure 3 A flowchart illustrating a first embodiment of the vehicle autonomous driving state monitoring method of this application is shown.

[0047] It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0048] In this embodiment, the method for monitoring the state of autonomous driving of a vehicle includes the following steps:

[0049] Step S100: Obtain vehicle status information.

[0050] The status information includes sensor status information, domain controller monitoring status information, vehicle component status information, vehicle system process operation information, and vehicle function module status information.

[0051] Specifically, vehicle status information plays a crucial role in the safe, reliable, and efficient operation of autonomous driving systems. By acquiring different types of vehicle status information during autonomous driving, real-time decision support can be provided to the autonomous driving system. Vehicle status information includes sensor status information, domain controller monitoring status information, vehicle component status information, vehicle infotainment system process operation information, and vehicle infotainment functional module status information.

[0052] Specifically, the sensor status information can be obtained by reading the sensor status information of each sensor through the preset status interface of each sensor. The sensors include at least one of GNSS (Global Navigation Satellite System), cameras, LiDAR, millimeter-wave radar, and ultrasonic radar. Sensor status information includes sensor malfunction, inability to open the sensor, or inability to read sensor data.

[0053] In one example, the preset state interface settings for commonly used sensors are shown in Table 1.

[0054] Table 1

[0055]

[0056] The real-time operating status of each corresponding sensor can be read through the preset status interface in Table 1. For abnormal status information such as detected sensor malfunctions, inability to open the sensor, or inability to read sensor data, it can be named in the form of "module_error" and reported. The template for the "module_error" information content is as follows:

[0057]

[0058] For vehicle component status information, it can be obtained by acquiring the vehicle component status information forwarded by the vehicle controller.

[0059] The vehicle component status information includes at least one of the following: braking information, steering information, ignition information, door information, cargo bed status information, and fault code information. Braking information may include brake light signals, brake pedal status, etc.; steering information may include steering wheel angle, turn signal information, etc.; ignition information includes ignition switch status, accelerator pedal status, etc.; door information includes door open / closed status, window status, and door lock status, etc.; cargo bed status information includes trunk status, roof status, etc.; and fault code information can be used to record fault codes, fault types, and fault levels for various parts of the vehicle. In one example, communication with the vehicle control unit (VCU) can be used to focus on monitoring whether the core signals of each component on the vehicle are received and executed, thereby obtaining the vehicle component status information. This vehicle component status information can be read and fed back via CAN signals.

[0060] Domain controller monitoring status information typically includes the domain controller's resource usage status information and the domain controller's communication status information.

[0061] Specifically, the resource usage status information of the domain controller can be obtained through operating system commands; the resource usage status information includes at least one of CPU usage status, memory usage status, disk usage status, and temperature status.

[0062] For example, you can obtain the CPU resource usage of the current domain controller by calling the Linux system's built-in command " / proc / stat"; you can obtain the memory usage of the current domain controller by calling the Linux system's built-in command " / proc / memeinfo"; you can obtain the disk usage of the current domain controller by calling the Linux system's built-in command " / proc / diskstats"; and you can also obtain the temperature status information of the current domain controller by calling the Linux system's built-in command " / sys / class / thermal / thermal_zone0 / temp".

[0063] The communication status between the domain controller and the vehicle controller can also be determined by monitoring whether the domain controller receives heartbeat feedback information from the vehicle controller.

[0064] The domain controller and the vehicle control unit (VCU) can establish a communication connection through a heartbeat mechanism. The domain controller can send heartbeat information to the VCU. If the domain controller does not receive heartbeat feedback information from the VCU, it can determine the current abnormal communication status between the domain controller and the VCU. By combining the resource usage status information and communication connection status information of the domain controller, the monitoring status information of the domain controller can be obtained.

[0065] The vehicle's infotainment system process operation information includes the process status and running conditions of various software programs. This information can be determined by calling relevant process monitoring commands. Obtaining this information allows for the timely detection and resolution of software issues, ensuring the stability of autonomous driving.

[0066] In addition, the system can monitor the module topic information of each vehicle infotainment function module it subscribes to; and obtain the status information of the vehicle infotainment function modules based on the module topic information.

[0067] Specifically, operators can set up configuration files in advance before autonomous driving begins to determine the functional modules that need to be monitored, such as perception, localization, task, planning, prediction, and control. Each functional module can generate and send module topic information containing names such as module_error and module_topic. Anomalies of functional modules can be monitored through module_error information, and the completeness of the topic message content and the message delay status of each functional module can be determined by monitoring the valid information in the form of module_topic generated by each functional module.

[0068] Step S200: Based on the status information, determine whether the autonomous driving operation status of the vehicle is abnormal.

[0069] Specifically, by acquiring various types of status information, such as sensor status information, domain controller monitoring status information, vehicle component status information, vehicle infotainment system process operation information, and vehicle infotainment function module status information, the autonomous driving status of the vehicle can be comprehensively monitored and analyzed, and the presence and cause of any abnormalities in vehicle operation can be determined in a timely manner. For example, if the status information includes communication anomaly information such as the domain controller not receiving VCU heartbeat feedback, it can be determined that the vehicle has lost connection and the autonomous driving status is abnormal.

[0070] Furthermore, the acquired vehicle status information can be fused and sent uniformly to the vehicle's Vehicle Control System (VCS). The VCS then uploads it to the cloud control terminal. The cloud control terminal can output the vehicle status information in text format or, based on the vehicle's rendered model, directly display the monitored vehicle anomaly information at the corresponding location on the vehicle. For example, in one instance, the vehicle status message can be packaged and named "Monitorlnfo" and sent to the cloud control terminal. The message content template is as follows:

[0071]

[0072]

[0073] In addition, vehicle status information can be stored as a log file in a standard log format. A new table entry and a new error checkpoint can be added to the error code encoding specification of the system file to record and store abnormal vehicle status information, so that testers can directly search the log file for anomaly analysis and optimization later.

[0074] It is worth mentioning that the cloud control terminal can also establish a connection with the domain controller through a heartbeat mechanism. The cloud control terminal can directly send heartbeat information to the domain controller. If the cloud control terminal does not receive heartbeat feedback information from the domain controller, the operator can confirm that the vehicle is in an abnormal state of being out of contact.

[0075] It is easy to understand that in this embodiment, the vehicle's status information can be obtained; the status information includes sensor status information, domain controller monitoring status information, vehicle component status information, vehicle system process operation information, and vehicle function module status information; thus, based on the status information, it can be determined whether the vehicle's autonomous driving operation status is abnormal. This allows for comprehensive status monitoring of multiple aspects of the vehicle's autonomous driving process, thereby improving the comprehensiveness of autonomous driving status monitoring.

[0076] Furthermore, based on the above embodiments, a second embodiment of this application is proposed. In this embodiment, step S100 includes steps S101 and S102, used to obtain the vehicle's in-vehicle system process operation information. (Refer to...) Figure 4 , Figure 4 A flowchart illustrating a second embodiment of a vehicle autonomous driving status monitoring method is shown.

[0077] In this embodiment, step S100 includes:

[0078] Step S101: Call the process monitoring command of the vehicle infotainment system to obtain the current running process information of the vehicle infotainment system.

[0079] Step S102: Based on the pre-configured process configuration information to be monitored, detect the information of the currently running process to obtain the process running information of the vehicle system.

[0080] Specifically, information about currently running processes in the vehicle infotainment system can be obtained by calling the system's built-in process monitoring commands or by using pre-compiled commands tailored to the system's specific needs. This information can include the current process status and running conditions of various vehicle software programs. Based on the configured process information, the system can then monitor the currently running processes to determine if they match expectations and obtain process status information. This facilitates the detection of abnormal processes and timely intervention. For example, in one scenario, the vehicle infotainment system configuration file can be pre-configured to specify the process names to be monitored. The Linux system's built-in command ` / proc / pid / cmdline` can then be used to obtain information about currently running processes. Further checks can be performed to ensure the process names match the names specified in the configuration file. If they do not match, abnormal process status information can be generated. If the monitored process is in a crashed state, a process restart command can be invoked to revive it.

[0081] It's easy to understand that by invoking the vehicle's infotainment system's process monitoring commands, the current running process information of the vehicle's infotainment system can be obtained. Based on the pre-configured process settings, the current running process information is detected, allowing for real-time monitoring of the infotainment system's process status. This enables timely detection and handling of abnormal process issues, ensuring the system's stability.

[0082] Furthermore, based on the above embodiments, a third embodiment of this application is proposed. In this embodiment, step A100 and step A200 are included after step S100 for generating vehicle control commands. (Refer to...) Figure 5 , Figure 5 A flowchart illustrating a third embodiment of a vehicle autonomous driving state monitoring method is shown.

[0083] In this embodiment, the following steps are included after step S100:

[0084] Step A100: Determine the abnormality level of the vehicle's current abnormal state based on the status information.

[0085] Step A200: Generate the corresponding vehicle control command based on the anomaly level.

[0086] Specifically, potential abnormal vehicle states can be predefined, and different abnormalities can be categorized into levels such as minor, moderate, or severe. This helps to quantify and classify abnormal vehicle conditions, providing a reference for subsequent emergency handling. Once the vehicle's status information is obtained, the predefined abnormality levels can be used to determine the current abnormal state level of the vehicle. Different control and handling methods can then be implemented based on the abnormality level, such as emergency stop, gradual stop, or forced restart. Necessary alarms can also be issued. For example, a warning command can be issued for minor abnormalities, while an emergency stop or other emergency control commands may be required for severe abnormalities. Furthermore, vehicle control commands can be named MonitorCommand, and the template for the control command is as follows:

[0087]

[0088] It is easy to understand that by determining the level of abnormality of the vehicle's current abnormal state based on the status information, corresponding vehicle control commands can be generated according to predefined processing strategies and rules to respond to the vehicle's current abnormal situation in a timely manner, thereby improving the safety of the vehicle's autonomous driving system.

[0089] Corresponding to the vehicle autonomous driving state monitoring method, this application proposes a vehicle autonomous driving state monitoring device, such as... Figure 6 As shown, the device includes:

[0090] The status acquisition module is used to acquire the vehicle's status information;

[0091] The anomaly detection module is used to determine whether the autonomous driving operation status of the vehicle is abnormal based on the status information.

[0092] It should be noted that the various implementation methods of the vehicle autonomous driving state monitoring device in this embodiment and the technical effects they achieve can be referred to the various implementation methods of the vehicle autonomous driving state monitoring method in the foregoing embodiments, and will not be repeated here.

[0093] Furthermore, embodiments of this application also propose a computer storage medium storing a vehicle autonomous driving state monitoring program. When executed by a processor, the vehicle autonomous driving state monitoring program implements the steps of the vehicle autonomous driving state monitoring method described above. Therefore, it will not be repeated here. Additionally, the beneficial effects of using the same method will not be repeated here either. For technical details not disclosed in the computer-readable storage medium embodiments of this application, please refer to the description of the method embodiments of this application. As an example, program instructions can be deployed to execute on a single computing device, or on multiple computing devices located at one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0094] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0095] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided in this application, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0096] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0097] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for monitoring the state of autonomous driving in a vehicle, characterized in that, The method includes: Acquire vehicle status information; wherein, the status information includes sensor status information, domain controller monitoring status information, vehicle component status information, vehicle system process operation information, and vehicle system functional module status information; Based on the status information, determine whether the autonomous driving operation status of the vehicle is abnormal; If the autonomous driving operation status of the vehicle is abnormal, then the abnormality level of the current abnormal status of the vehicle is determined based on the status information. Based on the anomaly level, generate the corresponding vehicle control command; The step of obtaining the vehicle's infotainment system process running information includes: calling the process monitoring instruction of the infotainment system to obtain the current running process information of the vehicle's infotainment system; and detecting the current running process information according to the pre-configured process configuration information to be monitored to obtain the infotainment system process running information. The acquisition of vehicle sensor status information includes: reading sensor status information of each sensor through a preset status interface of each sensor; wherein, the sensor includes at least one of Global Navigation Satellite System (GNSS), camera, lidar, millimeter wave, and ultrasonic radar, and the sensor status information includes sensor malfunction, inability to open the sensor, or inability to read sensor data; The domain controller monitoring status information includes resource usage status information and communication status information; obtaining the vehicle's domain controller monitoring status information includes: The resource usage status information of the domain controller is obtained through operating system instructions; wherein the resource usage status information includes at least one of CPU usage status, memory usage status, disk usage status, and temperature status; and / or The communication status information between the domain controller and the vehicle controller is determined by monitoring whether the domain controller receives heartbeat feedback information from the vehicle controller.

2. The vehicle autonomous driving state monitoring method according to claim 1, characterized in that, The acquisition of vehicle component status information includes: Obtain vehicle component status information forwarded by the vehicle controller; wherein, the vehicle component status information includes at least one of braking information, steering information, ignition information, door information, truck bed status information, and fault code information.

3. The vehicle autonomous driving status monitoring method according to claim 1, characterized in that, The acquisition of the vehicle's in-vehicle infotainment module status information includes: Monitor the module topic information of each subscribed vehicle infotainment system module; the module topic information includes anomaly and validity information of perception, positioning, task, planning, prediction and control function modules; Based on the module topic information, the status information of the vehicle infotainment function module is obtained.

4. A vehicle autonomous driving state monitoring device, used to implement the vehicle autonomous driving state monitoring method according to any one of claims 1-3, characterized in that, The device includes: The status acquisition module is used to acquire the vehicle's status information; The anomaly determination module is used to determine whether the autonomous driving operation status of the vehicle is abnormal based on the status information.

5. A vehicle infotainment system for implementing the vehicle autonomous driving state monitoring method according to any one of claims 1-3, characterized in that, The system includes vehicles and cloud control terminals.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the vehicle autonomous driving state monitoring method as described in any one of claims 1 to 3.

Images