A multi-sensor state management method and system for autonomous driving

Abstract

This invention provides a multi-sensor state management method and system for autonomous driving. The method includes: registering a monitor communication thread; registering a CAN monitor thread and a GPS monitor thread based on the monitor communication thread; registering a module / hardware monitor thread and a state monitor thread; periodically looping the CAN monitor thread, GPS monitor thread, module / hardware monitor thread, and state monitor thread, and statistically analyzing the current system state and safety mode state; and sending the current system state and safety mode state to the user through a visual interface via autonomous driving platform middleware. This invention can run on monitors of different autonomous driving platforms, has strong compatibility, can uniformly manage the states of multiple sensors, and is compatible with various visual interfaces as needed.

Description

Technical Field

[0001] This invention belongs to the technical field of autonomous driving, and specifically relates to a multi-sensor state management method and system for autonomous driving. Background Technology

[0002] In the field of autonomous driving, vehicles are equipped with a wide variety of sensors, and even the same type of sensor may have different brands and drivers. The messages transmitted between different modules in an autonomous driving platform are different, and the middleware for transmitting messages on different platforms is also different. The status of these hardware and modules needs to be displayed through a visual interface, and different visual interfaces provide different interfaces. However, the existing technology has poor compatibility with multiple sensors, which leads to the inability to achieve unified management of the status of multiple sensors, thus affecting the autonomous driving of the car. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention provides a multi-sensor state management method and system for autonomous driving. This system solves the technical issues existing in the prior art, simultaneously managing the hardware states of multiple different sensors and monitoring the state of autonomous driving platform modules. The data is then parsed, statistically analyzed, and output to a visual interface. Furthermore, based on user configuration, different monitoring threads are created to monitor the corresponding hardware and software states. Different data are monitored according to the monitored content, and finally, the different data are parsed, statistically analyzed, and output to the corresponding visual interface for user viewing and operation.

[0004] In a first aspect, the invention provides the following technical solution: a multi-sensor state management method for autonomous driving, the method comprising:

[0005] Read the user's vehicle configuration to obtain user configuration information, and activate the corresponding autonomous driving platform middleware based on user needs;

[0006] Register a monitor communication thread on the middleware of the autonomous driving platform, and register a CAN monitor thread and a GPS monitor thread based on the monitor communication thread;

[0007] Register the module / hardware monitor thread and the status monitor thread according to the user configuration information;

[0008] The CAN monitor thread, the GPS monitor thread, the module / hardware monitor thread, and the status monitor thread are periodically looped, and the current system status and security mode status are statistically analyzed.

[0009] The current system status and the safety mode status are sent to the user through a visual interface via the autonomous driving platform middleware.

[0010] Compared with existing technologies, the beneficial effects of this application are as follows: This application first obtains user configuration information by reading the user's vehicle configuration, and then mobilizes the corresponding autonomous driving platform middleware based on user needs to meet the subsequent registration, message communication, and data storage between various hardware and modules. Then, a monitor communication thread is registered on the autonomous driving platform middleware, and a CAN monitor thread and a GPS monitor thread are registered based on the monitor communication thread. Module / hardware monitor threads and status monitor threads are registered according to the user configuration information. By registering the above-mentioned different monitors, the status of multiple different sensor hardware can be managed simultaneously, and the status of autonomous driving platform modules can be monitored at the same time. Afterwards, the current system status and the safety mode status are sent to the user through a visual interface via the autonomous driving platform middleware. This allows the present invention to create different monitoring threads to monitor the corresponding hardware and software status according to the user configuration, perform different data monitoring according to the monitored content, and finally parse and statistically analyze the different data, outputting it to the corresponding visual interface for the user to view and operate.

[0011] Preferably, the step of registering the CAN monitor thread and the GPS monitor thread based on the monitor communication thread includes:

[0012] Read CAN configuration information, obtain CAN card signals based on CAN configuration information and select the corresponding function, read CAN messages and set CAN summary status and msg messages;

[0013] Obtain GPS communication messages and determine whether the autonomous driving platform middleware has received the GPS communication messages;

[0014] If the autonomous driving platform middleware receives the GPS communication message, it parses and configures the GPS summary state. If the autonomous driving platform middleware does not receive the GPS communication message, it configures the GPS summary state to UNKNOMN.

[0015] Preferably, the step of registering the module / hardware monitor thread based on the user configuration information includes:

[0016] Read the module software module category in the user configuration information, declare the module_state class of the first module to be monitored corresponding to the module / hardware monitor, and determine whether the configuration information of the first module to be monitored contains the first communication thread ID;

[0017] If the configuration information of the module to be monitored contains the first communication thread ID, then the first Topic monitor thread and the Process monitor thread are registered in sequence.

[0018] If the configuration information of the module to be monitored does not contain the first communication thread ID, then register the Process monitor thread;

[0019] Determine whether there is a module category in the user configuration information. If there is no module category in the user configuration information, read the hardware category in the user configuration information, declare the hw_state class of the second module to be monitored corresponding to the module / hardware monitor, and determine whether the configuration information of the second module to be monitored contains the second communication thread ID.

[0020] If the configuration information of the second module to be monitored contains the second communication thread ID, then register the second Topic monitor thread, and determine whether the configuration information of the second module to be monitored contains the hardware mount path. If the configuration information of the second module to be monitored contains the hardware mount path, then register the Path monitor thread.

[0021] If the configuration information of the second module to be monitored does not contain the second communication thread ID, then it is determined whether the configuration information of the second module to be monitored contains the hardware mount path. If the configuration information of the second module to be monitored contains the hardware mount path, then the Path monitor thread is registered.

[0022] Preferably, the step of registering the first Topic monitor thread includes:

[0023] Obtain the Topic message and determine whether the autonomous driving platform middleware has received the Topic message;

[0024] If the autonomous driving platform middleware receives the Topic message, it obtains the Topic message delay and determines whether the Topic message delay is greater than the preset configuration delay.

[0025] If the Topic message delay is greater than the preset configuration delay, then the message_delay of Topic_status is configured as the delay data; if the Topic message delay is not greater than the preset configuration delay, then the delay data of message_delay of Topic_status is cleared.

[0026] Preferably, the step of registering the Process monitor thread includes:

[0027] Get the user thread ID in the system directory, and get the process name from the cmdline in the directory corresponding to the user thread ID and save it as a process list;

[0028] Determine if the process list contains a name corresponding to module_state;

[0029] If the process list contains a name corresponding to module_state, then configure the running value in Process_state to true;

[0030] If there is no name corresponding to module_state in the process list, then check if the running value in Process_state is true. If the running value in Process_state is true, then check if the vehicle status is autonomous driving. If the vehicle status is autonomous driving, then print the ERROR log, configure the msg message, and configure the running value in Process_state to false.

[0031] Preferably, the step of registering the Path monitor thread includes:

[0032] Read the hardware configuration of the second module to be monitored and obtain the hardware mount path;

[0033] Determine if the hardware mount path exists in the system;

[0034] If the hardware mount path exists in the system, the summary status is set to OK; otherwise, the summary status is set to UNKNOWN.

[0035] Preferably, the step of registering a status monitor thread based on the user configuration information includes:

[0036] Obtain the module_state information and determine whether the running value in Process_state is true;

[0037] If the running value in Process_state is true, then the summary status of the corresponding module is counted, and it is determined whether the module_state information contains the first Topic_state information. If the module_state information contains the first Topic_state information, then it is determined whether the message_delay in the first Topic_state information is less than a preset value.

[0038] If the message_delay in the first Topic_state information is not less than a preset value, then the message_delay of the corresponding module is calculated, and it is determined whether there is still module_state information in the user configuration information.

[0039] If the user configuration information does not contain module_state information, then hw_state information is obtained and it is determined whether the user configuration information contains summary state information. If the user configuration information contains summary state information, then the summary state of the corresponding module is counted and it is determined whether the hw_state information contains second Topic_state information. If the hw_state information contains second Topic_state information, then it is determined whether message_delay in the second Topic_state information is less than a preset value.

[0040] If the message_delay in the second Topic_state information is not less than the preset value, then the message_delay of the corresponding hardware is counted, and it is determined whether there is still hw_state information in the user configuration information.

[0041] If the user configuration information does not contain hw_state information, then check the status monitor, obtain the current timestamp, and publish system status messages, module and hardware summary status messages, and message_delay delay messages.

[0042] Preferably, after the step of sending the current system state and the safety mode state to the user through a visual interface via the autonomous driving platform middleware, the method further includes:

[0043] The CAN monitor thread, GPS monitor thread, module / hardware monitor thread, and status monitor thread perform system self-tests at preset time intervals.

[0044] The step of performing system self-tests on the CAN monitor thread, GPS monitor thread, module / hardware monitor thread, and status monitor thread at preset time intervals includes:

[0045] Check the operating status of each monitor and determine whether the visualization interface has received a stop command from the user;

[0046] If the visualization interface does not receive a stop command from the user, the current program timestamp is obtained and module_state and hw_state are initialized;

[0047] Read the CAN monitor thread, GPS monitor thread, module / hardware monitor thread, and status monitor thread, and execute the corresponding monitor's periodic function to determine if there are any registered monitor threads.

[0048] If there are still registered monitor threads, the steps to check the running status of each monitor will be returned at preset time intervals.

[0049] Secondly, the invention provides the following technical solution: a multi-sensor state management system for autonomous driving, the system comprising:

[0050] The configuration reading module is used to read the user's vehicle configuration to obtain the user's configuration information and to invoke the corresponding autonomous driving platform middleware based on the user's needs.

[0051] The first registration module is used to register a monitor communication thread on the middleware of the autonomous driving platform, and to register a CAN monitor thread and a GPS monitor thread based on the monitor communication thread.

[0052] The second registration module is used to register the module / hardware monitor thread and the status monitor thread according to the user configuration information.

[0053] The status statistics module is used to periodically cycle through the CAN monitor thread, the GPS monitor thread, the module / hardware monitor thread, and the status monitor thread, and to collect statistics on the current system status and security mode status.

[0054] The user operation module is used to send the current system status and the safety mode status to the user through a visual interface via the autonomous driving platform middleware.

[0055] Preferably, the system further includes:

[0056] The self-test module is used to perform system self-tests on the CAN monitor thread, GPS monitor thread, module / hardware monitor thread, and status monitor thread at preset time intervals.

[0057] The self-test module includes:

[0058] The status detection submodule is used to check the operating status of each monitor and determine whether the visualization interface has received a stop command issued by the user.

[0059] The instruction receiving submodule is used to obtain the current program timestamp and initialize module_state and hw_state if the visualization interface does not receive a stop instruction from the user.

[0060] The reading submodule is used to read the CAN monitor thread, GPS monitor thread, module / hardware monitor thread, and status monitor thread, and execute the periodic function of the corresponding monitor to determine whether there are any registered monitor threads.

[0061] The loop submodule is used to control the status detection submodule to check the running status of each monitor at preset time intervals if there are still registered monitor threads. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 A flowchart of a multi-sensor state management method for autonomous driving provided in the first embodiment of the present invention;

[0064] Figure 2 A detailed flowchart of step S2 in the multi-sensor state management method for autonomous driving provided in the first embodiment of the present invention;

[0065] Figure 3 This is a detailed flowchart of step S31 in the multi-sensor state management method for autonomous driving provided in the first embodiment of the present invention;

[0066] Figure 4 Detailed flowchart of step S312 in the multi-sensor state management method for autonomous driving provided in the first embodiment of the present invention Figure 1 ;

[0067] Figure 5 Detailed flowchart of step S312 in the multi-sensor state management method for autonomous driving provided in the first embodiment of the present invention Figure 2 ;

[0068] Figure 6 This is a detailed flowchart of step S315 in the multi-sensor state management method for autonomous driving provided in the first embodiment of the present invention;

[0069] Figure 7 Detailed flowchart of step S32 in the multi-sensor state management method for autonomous driving provided in the first embodiment of the present invention Figure 1 ;

[0070] Figure 8 A flowchart of a multi-sensor state management method for autonomous driving provided in the second embodiment of the present invention;

[0071] Figure 9 This is a detailed flowchart of step S60 in the multi-sensor state management method for autonomous driving provided in the second embodiment of the present invention;

[0072] Figure 10 The structural framework of the multi-sensor state management system for autonomous driving provided in the third embodiment of the present invention Figure 1 ;

[0073] Figure 11 The structural framework of the multi-sensor state management system for autonomous driving provided in the third embodiment of the present invention Figure 2 ;

[0074] Figure 12 This is a hardware structure block diagram of a computer provided for another embodiment of the present invention.

[0075] The embodiments of the present invention will be further described below with reference to the accompanying drawings. Detailed Implementation

[0076] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0077] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0078] Example 1

[0079] like Figure 1 As shown, in the first embodiment of the present invention, the invention provides the following technical solution: a multi-sensor state management method for autonomous driving, the method comprising:

[0080] S1. Read the user's vehicle configuration to obtain the user's configuration information, and activate the corresponding autonomous driving platform middleware based on the user's needs.

[0081] Specifically, the user vehicle configuration refers to the configuration of each hardware and module in the autonomous vehicle driven by the user. The user vehicle configuration also includes the cycle and software and hardware configuration path. Based on the user's configuration information, the corresponding autonomous driving platform middleware can be invoked. The autonomous driving platform middleware refers to the software or service program in the autonomous driving platform framework that provides the connection between system software and application software and is responsible for functions such as module registration, message communication, and data storage.

[0082] S2. Register a monitor communication thread on the middleware of the autonomous driving platform, and register a CAN monitor thread and a GPS monitor thread based on the monitor communication thread;

[0083] Specifically, since vehicle status information needs to be read via the CAN bus, a CAN monitor needs to be registered, while GPS information contains multiple statuses, so a separate GPS monitor needs to be registered.

[0084] Meanwhile, before registering the CAN monitor thread and GPS monitor thread, since vehicle status information is obtained through several monitors, the corresponding vehicle status information is sent to the autonomous driving platform through the monitors. In the process of sending vehicle status information, it is necessary to establish a monitor communication thread so that the monitor can send the obtained vehicle status information to the autonomous driving platform middleware, so as to realize the information transmission and reception between the sensors when managing multiple sensors in a unified manner.

[0085] like Figure 2 As shown, step S2 includes:

[0086] S21. Read CAN configuration information, obtain CAN card signals according to CAN configuration information and select the corresponding function, read CAN messages and set CAN summary status and msg messages;

[0087] S22. Obtain GPS communication messages and determine whether the autonomous driving platform middleware has received the GPS communication messages;

[0088] S23. If the autonomous driving platform middleware receives the GPS communication message, it parses and configures the GPS summary state; if the autonomous driving platform middleware does not receive the GPS communication message, it configures the GPS summary state to UNKNOMN.

[0089] The GPS summary status includes ERR, NOT_READY, GOOD, INVALID, etc.

[0090] S3. Register the module / hardware monitor thread and the status monitor thread according to the user configuration information;

[0091] Specifically, the user configuration information includes module name / hardware name, configuration path, communication thread ID, and communication message type. Configurations can be added and corresponding monitors can be added according to actual needs. The configuration path, communication thread ID, and communication message type in the user configuration information are all optional configuration information.

[0092] It is worth noting that the module / hardware monitor thread includes a first topic monitor thread, a process monitor thread, a second topic monitor thread, and a path monitor thread. The first topic monitor thread and the second topic monitor thread are communication message type monitor threads, the process monitor thread is a thread name monitor thread, and the path monitor thread is a module / hardware mounting path monitor thread in the autonomous driving platform.

[0093] The status monitor thread is specifically the summary monitor thread, which includes 4. the registration system status monitor thread, the vehicle status monitor thread, the safe mode thread, etc.

[0094] Step S3 includes steps S31 and S32, wherein step S31 is the step of registering the module / hardware monitor thread, and step S32 is the step of registering the status monitor thread.

[0095] like Figure 3 As shown, step S31 includes:

[0096] S311. Read the module software module classification in the user configuration information, declare the module_state class of the first module to be monitored corresponding to the module / hardware monitor, and determine whether the configuration information of the first module to be monitored contains the first communication thread id.

[0097] The module_state class contains summary status, msg messages, process_status thread status, topic_status communication status, etc., which can be adjusted according to actual needs;

[0098] S312. If the configuration information of the module to be monitored contains the first communication thread ID, then register the first Topic monitor thread and the Process monitor thread in sequence.

[0099] like Figure 4 As shown, the steps for registering the first Topic monitor thread include:

[0100] S312A. Obtain the Topic message and determine whether the autonomous driving platform middleware has received the Topic message;

[0101] S312B. If the autonomous driving platform middleware receives the Topic message, it obtains the Topic message delay and determines whether the Topic message delay is greater than the preset configuration delay.

[0102] Specifically, if the autonomous driving platform middleware does not receive the Topic message, the message_delay of Topic_status is configured to be -1;

[0103] S312C. If the Topic message delay is greater than the preset configuration delay, then configure the message_delay of Topic_status as the delay data; if the Topic message delay is not greater than the preset configuration delay, then clear the delay data of message_delay of Topic_status.

[0104] The preset configuration delay is the maximum delay that the autonomous driving platform middleware can accept.

[0105] like Figure 5 As shown, the steps for registering a Process monitor thread include:

[0106] S312a. Obtain the user thread ID in the system directory, and obtain the process name from the cmdline in the directory corresponding to the user thread ID and save it as a process list;

[0107] The system directory is the / proc directory;

[0108] S312b: Determine whether there is a name corresponding to module_state in the process list;

[0109] S312c. If the process list contains a name corresponding to module_state, then configure the running value in Process_state to true;

[0110] S312d. If there is no name corresponding to module_state in the process list, determine whether the running value in Process_state is true. If the running value in Process_state is true, determine whether the vehicle status is autonomous driving. If the vehicle status is autonomous driving, print the ERROR log and configure the msg message, and configure the running value in Process_state to false.

[0111] Specifically, if the running value in Process_state is not true, then the running value in Process_state is configured to false. If the vehicle state is not in autonomous driving state, then an ERROR log is printed and a msg message is configured, and the running value in Process_state is configured to false.

[0112] S313. If the configuration information of the module to be monitored does not contain the first communication thread ID, then register the Process monitor thread;

[0113] S314. Determine whether there is a module category in the user configuration information. If there is no module category in the user configuration information, read the hardware category in the user configuration information, declare the hw_state class of the second module to be monitored corresponding to the module / hardware monitor, and determine whether the configuration information of the second module to be monitored contains the second communication thread id.

[0114] Specifically, if the user configuration information also contains a module category, then return to the step of reading the module software module category in the user configuration information, declaring the module_state class of the first module to be monitored corresponding to the module / hardware monitor, and determining whether the configuration information of the first module to be monitored contains the first communication thread ID, that is, return to the step of executing step S311.

[0115] The hw_state class contains summary status, msg messages, topic_status communication status, etc., which can be adjusted according to actual needs;

[0116] S315. If the configuration information of the second module to be monitored contains the second communication thread ID, then register the second Topic monitor thread and determine whether the configuration information of the second module to be monitored contains the hardware mount path. If the configuration information of the second module to be monitored contains the hardware mount path, then register the Path monitor thread.

[0117] Specifically, in step S315, the process of registering the second Topic monitor thread is exactly the same as that of registering the first Topic monitor thread;

[0118] like Figure 6 As shown, step S315 includes:

[0119] S315A: Read the hardware configuration of the second module to be monitored and obtain the hardware mount path;

[0120] S315B: Determine whether the hardware mounting path exists in the system;

[0121] S315C: If the hardware mount path exists in the system, the summary status is configured as OK; if the hardware mount path does not exist in the system, the summary status is configured as UNKNOWN.

[0122] S316. If the configuration information of the second module to be monitored does not contain the second communication thread ID, then determine whether the configuration information of the second module to be monitored contains the hardware mount path. If the configuration information of the second module to be monitored contains the hardware mount path, then register the Path monitor thread.

[0123] like Figure 7 As shown, step S32 includes:

[0124] S321. Obtain module_state information and determine whether the running value in Process_state is true;

[0125] S322. If the running value in Process_state is true, then the summary state of the corresponding module is counted, and it is determined whether the module_state information contains the first Topic_state information. If the module_state information contains the first Topic_state information, then it is determined whether the message_delay in the first Topic_state information is less than a preset value.

[0126] Specifically, if the running value in Process_state is not true, an ERR log is published accordingly. If the module_state information does not contain the first Topic_state information, the step of determining whether there is still module_state information in the user configuration information is executed.

[0127] S323. If the message_delay in the first Topic_state information is not less than a preset value, then the message_delay of the corresponding module is counted, and it is determined whether there is still module_state information in the user configuration information.

[0128] S324. If the user configuration information does not contain module_state information, then obtain hw_state information and determine whether the user configuration information contains summary state information. If the user configuration information contains summary state information, then count the summary state of the corresponding module and determine whether the hw_state information contains second Topic_state information. If the hw_state information contains second Topic_state information, then determine whether message_delay in the second Topic_state information is less than a preset value.

[0129] S325. If the message_delay in the second Topic_state information is not less than a preset value, then the message_delay delay of the corresponding hardware is calculated, and it is determined whether there is still hw_state information in the user configuration information.

[0130] Specifically, the preset value is 0. If the message_delay in the second Topic_state information is less than the preset value, an ERR log will be published accordingly.

[0131] S326. If the user configuration information does not contain hw_state information, check the status monitor, obtain the current timestamp, and publish system status messages, module and hardware summary status messages, and message_delay delay messages.

[0132] Specifically, if the user configuration information contains module_state information, then return to execute step S321.

[0133] S4. Periodically cycle through the CAN monitor thread, the GPS monitor thread, the module / hardware monitor thread, and the status monitor thread, and count the current system status and security mode status;

[0134] S5. The current system status and the safety mode status are sent to the user through a visual interface via the autonomous driving platform middleware.

[0135] Specifically, the system status is obtained by calling the status interface of the computing platform where the monitoring program is running. For example, Linux uses shell commands, and Windows uses DOS commands. After processing, the information is transmitted to the visualization interface. Information such as safe mode status and vehicle status is obtained by filtering the messages transmitted by the monitored CAN module.

[0136] The advantages of this first embodiment are as follows: First, by reading the user's vehicle configuration, user configuration information is obtained. Based on user needs, the corresponding autonomous driving platform middleware is activated to meet the subsequent registration, message communication, and data storage between various hardware and modules. Then, a monitor communication thread is registered on the autonomous driving platform middleware. Based on the monitor communication thread, a CAN monitor thread and a GPS monitor thread are registered. Module / hardware monitor threads and status monitor threads are registered according to the user configuration information. By registering the above-mentioned different monitors, the status of multiple different sensor hardware can be managed simultaneously, and the status of autonomous driving platform modules can be monitored at the same time. Afterwards, the current system status and the safety mode status are sent to the user through a visual interface via the autonomous driving platform middleware. This invention creates different monitoring threads to monitor the corresponding hardware and software status according to the user configuration, performs different data monitoring according to the monitored content, and finally parses and statistically analyzes the different data, outputting it to the corresponding visual interface for the user to view and operate.

[0137] Example 2

[0138] like Figure 8 As shown, in a second embodiment of the present invention, a multi-sensor state management method for autonomous driving is provided, the system comprising:

[0139] S10. Read the user's vehicle configuration to obtain the user's configuration information, and activate the corresponding autonomous driving platform middleware based on the user's needs.

[0140] S20. Register a monitor communication thread on the middleware of the autonomous driving platform, and register a CAN monitor thread and a GPS monitor thread based on the monitor communication thread;

[0141] S30. Register the module / hardware monitor thread and the status monitor thread according to the user configuration information;

[0142] S40. Periodically cycle through the CAN monitor thread, the GPS monitor thread, the module / hardware monitor thread, and the status monitor thread, and count the current system status and security mode status;

[0143] S50. The current system state and the safety mode state are sent to the user through a visual interface via the autonomous driving platform middleware.

[0144] S60. Perform system self-tests on the CAN monitor thread, GPS monitor thread, module / hardware monitor thread, and status monitor thread at preset time intervals.

[0145] The steps S10, S20, S30, S40, and S50 described above are the same as steps S1, S2, S3, S4, and S5 in Example 1.

[0146] like Figure 9 As shown, step S60 includes:

[0147] S61. Check the operating status of each monitor and determine whether the visualization interface has received a stop command issued by the user.

[0148] S62. If the visualization interface does not receive a stop command from the user, then obtain the current program timestamp and initialize module_state and hw_state;

[0149] S63. Read the CAN monitor thread, GPS monitor thread, module / hardware monitor thread and status monitor thread, and execute the periodic function of the corresponding monitor to determine whether there are any registered monitor threads.

[0150] S64. If there are still registered monitor threads, return to the steps of checking the running status of each monitor at preset time intervals.

[0151] Specifically, the user can operate in the visual interface. If the user does not issue a stop command in the visual interface, then step S61 is executed. The current program timestamp is obtained and module_state and hw_state are initialized. Then, the periodic functions of the corresponding monitors are executed by reading the registered monitor threads and it is determined whether there are still registered monitors. The system self-check process is implemented in sequence. If a stop command is received from the user, the monitor threads are deregistered and the process ends.

[0152] Compared with Embodiment 1, the advantages of Embodiment 2 are as follows: This embodiment adds a system self-test process, which can sequentially determine the registration status of the monitor thread, the motion status of the vehicle's sensors, and the vehicle's status. At the same time, by determining whether a stop command issued by the user has been received, the system self-test process is controlled to end and move through the stop command, so that the user can control and manage multiple sensors.

[0153] Example 3

[0154] like Figure 10 As shown, in a third embodiment of the present invention, a multi-sensor state management system for autonomous driving is provided, the system comprising:

[0155] Configuration reading module 1 is used to read the user's vehicle configuration to obtain user configuration information and to invoke the corresponding autonomous driving platform middleware based on user needs.

[0156] The first registration module 2 is used to register a monitor communication thread on the middleware of the autonomous driving platform, and to register a CAN monitor thread and a GPS monitor thread based on the monitor communication thread.

[0157] The second registration module 3 is used to register the module / hardware monitor thread and the status monitor thread according to the user configuration information.

[0158] The status statistics module 4 is used to periodically cycle through the CAN monitor thread, the GPS monitor thread, the module / hardware monitor thread, and the status monitor thread, and to count the current system status and security mode status.

[0159] User operation module 5 is used to send the current system status and the safety mode status to the user through a visual interface via the autonomous driving platform middleware.

[0160] The self-test module 6 is used to perform system self-tests on the CAN monitor thread, GPS monitor thread, module / hardware monitor thread, and status monitor thread at preset time intervals.

[0161] Specifically, the multi-sensor state management system applied to autonomous driving includes a vehicle platform, an autonomous driving platform, a monitor, and an HMI display;

[0162] like Figure 11 As shown, the vehicle platform is equipped with a variety of sensors to acquire vehicle status, a monitor to monitor the vehicle and sensor status, and an autonomous driving platform to control the vehicle's autonomous driving and send the vehicle and sensor status to the HMI display. The HMI display is a visual interface used to display vehicle information, monitoring switches, monitoring status, and other information.

[0163] The first registration module 2 includes:

[0164] The CAN configuration submodule is used to read CAN configuration information, obtain CAN card signals based on the CAN configuration information and select the corresponding functions, read CAN messages and set the CAN summary status and msg messages;

[0165] The GPS communication module is used to acquire GPS communication messages and determine whether the autonomous driving platform middleware has received the GPS communication messages.

[0166] The status configuration submodule is used to parse and configure the GPS summary status if the autonomous driving platform middleware receives the GPS communication message, and to configure the GPS summary status to UNKNOMN if the autonomous driving platform middleware does not receive the GPS communication message.

[0167] The second registration module 3 includes:

[0168] The reading submodule is used to read the module software module classification in the user configuration information, declare the module_state class of the first module to be monitored corresponding to the module / hardware monitor, and determine whether the configuration information of the first module to be monitored contains the first communication thread ID.

[0169] The first registration submodule is used to register the first Topic monitor thread and the Process monitor thread sequentially if the configuration information of the module to be monitored contains the first communication thread ID.

[0170] The second registration submodule is used to register a Process monitor thread if the configuration information of the module to be monitored does not contain the first communication thread ID.

[0171] The module category submodule is used to determine whether there is a module category in the user configuration information. If there is no module category in the user configuration information, the hardware category in the user configuration information is read, the hw_state class of the second module to be monitored corresponding to the module / hardware monitor is declared, and it is determined whether the configuration information of the second module to be monitored contains the second communication thread ID.

[0172] The third registration submodule is used to register a second Topic monitor thread if the configuration information of the second module to be monitored contains a second communication thread ID, and to determine whether the configuration information of the second module to be monitored contains a hardware mount path. If the configuration information of the second module to be monitored contains a hardware mount path, then register a Path monitor thread.

[0173] The fourth registration submodule is used to determine whether the configuration information of the second module to be monitored contains a hardware mount path if the configuration information of the second module to be monitored does not contain a second communication thread ID. If the configuration information of the second module to be monitored contains a hardware mount path, then the Path monitor thread is registered.

[0174] The first registration submodule includes:

[0175] The Topic message unit is used to obtain Topic messages and determine whether the autonomous driving platform middleware has received the Topic message.

[0176] The message receiving unit is used to obtain the Topic message delay and determine whether the Topic message delay is greater than the preset configuration delay if the autonomous driving platform middleware receives the Topic message.

[0177] The delay unit is used to configure the message_delay of Topic_status as delayed data if the Topic message delay is greater than the preset configuration delay, and to clear the delayed data of message_delay of Topic_status if the Topic message delay is not greater than the preset configuration delay.

[0178] The first registration submodule also includes:

[0179] The user thread ID acquisition unit is used to acquire the user thread ID in the system directory, and to obtain the process name from the cmdline in the directory corresponding to the user thread ID and save it as a process list;

[0180] The name determination unit is used to determine whether there is a name corresponding to module_state in the process list;

[0181] The first configuration unit is used to configure the running value in Process_state to true if there is a name corresponding to module_state in the process list;

[0182] The second configuration unit is used to determine whether the running value in Process_state is true if there is no name corresponding to module_state in the process list. If the running value in Process_state is true, it determines whether the vehicle status is autonomous driving. If the vehicle status is autonomous driving, it prints an ERROR log and configures a msg message, and configures the running value in Process_state to false.

[0183] The third registration submodule includes:

[0184] The path reading unit is used to read the hardware configuration of the second module to be monitored and obtain the hardware mount path;

[0185] The path determination unit is used to determine whether the hardware mounting path exists in the system.

[0186] The third configuration unit is used to configure the summary status as OK if the hardware mount path exists in the system, and to configure the summary status as UNKNOWN if the hardware mount path does not exist in the system.

[0187] The second registration module 3 includes:

[0188] The module_state information submodule is used to obtain module_state information and determine whether the running value in Process_state is true;

[0189] The status statistics submodule is used to count the summary status of the corresponding module if the running value in Process_state is true, and to determine whether the module_state information contains the first Topic_state information. If the module_state information contains the first Topic_state information, it is determined whether the message_delay in the first Topic_state information is less than a preset value.

[0190] The delay statistics submodule is used to calculate the message_delay delay of the corresponding module if the message_delay in the first Topic_state information is not less than a preset value, and to determine whether there is still module_state information in the user configuration information.

[0191] The hw_state information acquisition submodule is used to acquire hw_state information and determine whether there is summary state information in the user configuration information if there is no module_state information in the user configuration information. If there is summary state information in the user configuration information, the summary state of the corresponding module is counted and it is determined whether the hw_state information contains second Topic_state information. If the hw_state information contains second Topic_state information, it is determined whether message_delay in the second Topic_state information is less than a preset value.

[0192] The hardware latency statistics submodule is used to calculate the corresponding hardware's message_delay latency if the message_delay in the second Topic_state information is not less than a preset value, and to determine whether there is still hw_state information in the user configuration information.

[0193] The status check submodule is used to check the status monitor if the user configuration information does not contain hw_state information, obtain the current timestamp, and publish system status messages, module and hardware summary status messages, and message_delay delay messages.

[0194] The self-test module 6 includes:

[0195] The status detection submodule is used to check the operating status of each monitor and determine whether the visualization interface has received a stop command issued by the user.

[0196] The instruction receiving submodule is used to obtain the current program timestamp and initialize module_state and hw_state if the visualization interface does not receive a stop instruction from the user.

[0197] The reading submodule is used to read the CAN monitor thread, GPS monitor thread, module / hardware monitor thread, and status monitor thread, and execute the periodic function of the corresponding monitor to determine whether there are any registered monitor threads.

[0198] The loop submodule is used to control the status detection submodule to check the running status of each monitor at preset time intervals if there are still registered monitor threads.

[0199] In other embodiments of the present invention, the present invention provides the following technical solution: a computer device, including a memory 102, a processor 101, and a computer program stored in the memory 102 and executable on the processor 101, wherein the processor 101 executes the computer program to implement the multi-sensor state management method for autonomous driving described above.

[0200] Specifically, the processor 101 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0201] The memory 102 may include a large-capacity memory for data or instructions. For example, and not limitingly, the memory 102 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 102 may include removable or non-removable (or fixed) media. Where appropriate, the memory 102 may be internal or external to a data processing device. In a particular embodiment, the memory 102 is non-volatile memory. In a particular embodiment, the memory 102 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only memory (PROM), an erasable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), an electrically alterable read-only memory (EAROM), or flash memory, or a combination of two or more of these. Where appropriate, the RAM can be Static Random-Access Memory (SRAM) or Dynamic Random-Access Memory (DRAM). DRAM can be Fast Page Mode Dynamic Random Access Memory (FPMDRAM), Extended Data Out Dynamic Random Access Memory (EDODRAM), Synchronous Dynamic Random-Access Memory (SDRAM), etc.

[0202] The memory 102 can be used to store or cache various data files that need to be processed and / or used for communication, as well as possible computer program instructions executed by the processor 101.

[0203] The processor 101 reads and executes the computer program instructions stored in the memory 102 to implement the multi-sensor state management method applied to autonomous driving.

[0204] In some embodiments, the computer device may further include a communication interface 103 and a bus 100. For example, Figure 12 As shown, the processor 101, memory 102, and communication interface 103 are connected through bus 100 and complete communication with each other.

[0205] The communication interface 103 is used to enable communication between the various modules, devices, units, and / or equipment in the embodiments of this application. The communication interface 103 can also enable data communication with other components such as external devices, image / data acquisition devices, databases, external storage, and image / data processing workstations.

[0206] Bus 100 includes hardware, software, or both, that couples components of a computer device together. Bus 100 includes, but is not limited to, at least one of the following: data bus, address bus, control bus, expansion bus, and local bus. For example, and not as a limitation, bus 100 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 100 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this application, this application considers any suitable bus or interconnection.

[0207] The computer can acquire the multi-sensor state management system for autonomous driving and execute the multi-sensor state management method for autonomous driving of this application, thereby realizing the management of the state of multiple sensors.

[0208] In some further embodiments of the present invention, in conjunction with the above-described multi-sensor state management method for autonomous driving, the present invention provides the following technical solution: a readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the above-described multi-sensor state management method for autonomous driving.

[0209] Those skilled in the art will understand that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can mean any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0210] More specific examples of readable media (a non-exhaustive list) include: electrical connections (electronic devices) with one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0211] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0212] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0213] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A multi-sensor state management method for autonomous driving, characterized in that, The method includes: Read the user's vehicle configuration to obtain user configuration information, and activate the corresponding autonomous driving platform middleware based on user needs; Register a monitor communication thread on the middleware of the autonomous driving platform, and register a CAN monitor thread and a GPS monitor thread based on the monitor communication thread; Register the module / hardware monitor thread and the status monitor thread according to the user configuration information; The CAN monitor thread, the GPS monitor thread, the module / hardware monitor thread, and the status monitor thread are periodically looped, and the current system status and security mode status are statistically analyzed. The current system status and the safety mode status are sent to the user through a visual interface via the autonomous driving platform middleware.

2. The multi-sensor state management method for autonomous driving according to claim 1, characterized in that, The steps of registering the CAN monitor thread and the GPS monitor thread based on the monitor communication thread include: Read CAN configuration information, obtain CAN card signals based on CAN configuration information and select the corresponding function, read CAN messages and set CAN summary status and msg messages; Obtain GPS communication messages and determine whether the autonomous driving platform middleware has received the GPS communication messages; If the autonomous driving platform middleware receives the GPS communication message, it parses and configures the GPS summary state. If the autonomous driving platform middleware does not receive the GPS communication message, it configures the GPS summary state to UNKNOMN.

3. The multi-sensor state management method for autonomous driving according to claim 1, characterized in that, The step of registering the module / hardware monitor thread based on the user configuration information includes: Read the module software module category in the user configuration information, declare the module_state class of the first module to be monitored corresponding to the module / hardware monitor, and determine whether the configuration information of the first module to be monitored contains the first communication thread ID; If the configuration information of the module to be monitored contains the first communication thread ID, then the first Topic monitor thread and the Process monitor thread are registered in sequence. If the configuration information of the module to be monitored does not contain the first communication thread ID, then register the Process monitor thread; Determine whether there is a module category in the user configuration information. If there is no module category in the user configuration information, read the hardware category in the user configuration information, declare the hw_state class of the second module to be monitored corresponding to the module / hardware monitor, and determine whether the configuration information of the second module to be monitored contains the second communication thread ID. If the configuration information of the second module to be monitored contains the second communication thread ID, then register the second Topic monitor thread, and determine whether the configuration information of the second module to be monitored contains the hardware mount path. If the configuration information of the second module to be monitored contains the hardware mount path, then register the Path monitor thread. If the configuration information of the second module to be monitored does not contain the second communication thread ID, then it is determined whether the configuration information of the second module to be monitored contains the hardware mount path. If the configuration information of the second module to be monitored contains the hardware mount path, then the Path monitor thread is registered.

4. The multi-sensor state management method for autonomous driving according to claim 3, characterized in that, The steps for registering the first Topic monitor thread include: Obtain the Topic message and determine whether the autonomous driving platform middleware has received the Topic message; If the autonomous driving platform middleware receives the Topic message, it obtains the Topic message delay and determines whether the Topic message delay is greater than the preset configuration delay. If the Topic message delay is greater than the preset configuration delay, then the message_delay of Topic_status is configured as the delay data; if the Topic message delay is not greater than the preset configuration delay, then the delay data of message_delay of Topic_status is cleared.

5. The multi-sensor state management method for autonomous driving according to claim 3, characterized in that, The steps for registering the Process monitor thread include: Get the user thread ID in the system directory, and get the process name from the cmdline in the directory corresponding to the user thread ID and save it as a process list; Determine if the process list contains a name corresponding to module_state; If the process list contains a name corresponding to module_state, then configure the running value in Process_state to true; If there is no name corresponding to module_state in the process list, then check if the running value in Process_state is true. If the running value in Process_state is true, then check if the vehicle status is autonomous driving. If the vehicle status is autonomous driving, then print the ERROR log, configure the msg message, and configure the running value in Process_state to false.

6. The multi-sensor state management method for autonomous driving according to claim 3, characterized in that, The steps for registering the Path monitor thread include: Read the hardware configuration of the second module to be monitored and obtain the hardware mount path; Determine if the hardware mount path exists in the system; If the hardware mount path exists in the system, the summary status is set to OK; otherwise, the summary status is set to UNKNOWN.

7. The multi-sensor state management method for autonomous driving according to claim 1, characterized in that, The step of registering a status monitor thread based on the user configuration information includes: Obtain the module_state information and determine whether the running value in Process_state is true; If the running value in Process_state is true, then the summary status of the corresponding module is counted, and it is determined whether the module_state information contains the first Topic_state information. If the module_state information contains the first Topic_state information, then it is determined whether the message_delay in the first Topic_state information is less than a preset value. If the message_delay in the first Topic_state information is not less than a preset value, then the message_delay of the corresponding module is calculated, and it is determined whether there is still module_state information in the user configuration information. If the user configuration information does not contain module_state information, then hw_state information is obtained and it is determined whether the user configuration information contains summary state information. If the user configuration information contains summary state information, then the summary state of the corresponding module is counted and it is determined whether the hw_state information contains second Topic_state information. If the hw_state information contains second Topic_state information, then it is determined whether message_delay in the second Topic_state information is less than a preset value. If the message_delay in the second Topic_state information is not less than the preset value, then the message_delay of the corresponding hardware is counted, and it is determined whether there is still hw_state information in the user configuration information. If the user configuration information does not contain hw_state information, then check the status monitor, obtain the current timestamp, and publish system status messages, module and hardware summary status messages, and message_delay delay messages.

8. The multi-sensor state management method for autonomous driving according to claim 1, characterized in that, After the step of sending the current system state and the safety mode state to the user through a visual interface via the autonomous driving platform middleware, the method further includes: The CAN monitor thread, GPS monitor thread, module / hardware monitor thread, and status monitor thread perform system self-tests at preset time intervals. The step of performing system self-tests on the CAN monitor thread, GPS monitor thread, module / hardware monitor thread, and status monitor thread at preset time intervals includes: Check the operating status of each monitor and determine whether the visualization interface has received a stop command from the user; If the visualization interface does not receive a stop command from the user, the current program timestamp is obtained and module_state and hw_state are initialized; Read the CAN monitor thread, GPS monitor thread, module / hardware monitor thread, and status monitor thread, and execute the corresponding monitor's periodic function to determine if there are any registered monitor threads. If there are still registered monitor threads, the steps to check the running status of each monitor will be returned at preset time intervals.

9. A multi-sensor state management system for autonomous driving, characterized in that, The system includes: The configuration reading module is used to read the user's vehicle configuration to obtain the user's configuration information and to invoke the corresponding autonomous driving platform middleware based on the user's needs. The first registration module is used to register a monitor communication thread on the middleware of the autonomous driving platform, and to register a CAN monitor thread and a GPS monitor thread based on the monitor communication thread. The second registration module is used to register the module / hardware monitor thread and the status monitor thread according to the user configuration information. The status statistics module is used to periodically cycle through the CAN monitor thread, the GPS monitor thread, the module / hardware monitor thread, and the status monitor thread, and to collect statistics on the current system status and security mode status. The user operation module is used to send the current system status and the safety mode status to the user through a visual interface via the autonomous driving platform middleware.

10. The multi-sensor state management system for autonomous driving according to claim 9, characterized in that, The system also includes: The self-test module is used to perform system self-tests on the CAN monitor thread, GPS monitor thread, module / hardware monitor thread, and status monitor thread at preset time intervals. The self-test module includes: The status detection submodule is used to check the operating status of each monitor and determine whether the visualization interface has received a stop command issued by the user. The instruction receiving submodule is used to obtain the current program timestamp and initialize module_state and hw_state if the visualization interface does not receive a stop instruction from the user. The reading submodule is used to read the CAN monitor thread, GPS monitor thread, module / hardware monitor thread, and status monitor thread, and execute the periodic function of the corresponding monitor to determine whether there are any registered monitor threads. The loop submodule is used to control the status detection submodule to check the running status of each monitor at preset time intervals if there are still registered monitor threads.

Images