A method for intelligent driving data analysis and synchronization based on ROS
Patent Information
- Application Number
- CN202210140958.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-02-16
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2042-02-16
AI Technical Summary
其他领域时间同步方法也都是在采集前就做好了硬件同步方法,至少保持两个系统输出的数据是同时间源的;没有一种后处理方法,对已有的两种不同时间源不同频率的数据进行同步
[0025] 1) The extraction plug-in of the rosbag (ROS data packet) package of the present invention integrates the functions of accessing and starting the MDC (intelligent driving controller), recording, and batch conversion, quickly realizing the connection between the Linux collection computer and the MDC, and completing the collection efficiently From a series of actions to generating raw data, the two plug-ins act as a complete collection tool. They are simple to operate, efficient and accurate, providing a strong guarantee for subsequent data analysis.
Smart Images

Figure CN114548058B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of intelligent driving, and in particular relates to an intelligent driving data analysis and synchronization method based on ROS. Background Art
[0002] Data extraction and analysis of intelligent driving systems developed based on the ROS environment is the most important part of integrated testing. The current common practice in the industry is to output the algorithm products of the perception, regulation and control modules to the CAN protocol stack, and then provide a CAN interface on the hardware for collection and analysis with CAN analysis tools. There is no method to directly extract rosbag (ROS data packet) data and analyze it through visualization tools.
[0003] For the synchronous comparison of the algorithm output data of the intelligent driving system and the vehicle status data, the industry is based on a collection device, the same time source, and the same frequency for post-collection analysis. Time synchronization methods in other fields also use hardware synchronization methods before collection, at least to keep the data output by the two systems from the same time source; there is no post-processing method to synchronize the existing data from two different time sources and different frequencies. Summary of the invention
[0004] The purpose of the present invention is to solve the deficiencies of the above-mentioned background technology and provide an intelligent driving data analysis and synchronization method based on ROS.
[0005] The technical solution adopted by the present invention is: a ROS-based intelligent driving data analysis and synchronization method to obtain vehicle-mounted CAN data from the vehicle-mounted CAN network;
[0006] Get rosbag data from the intelligent driving controller and convert it into a csv file;
[0007] Filter each csv file to obtain all the data corresponding to the activation of the autonomous driving mode and store them as xls files;
[0008] Get vehicle CAN data from the vehicle CAN network;
[0009] Perform time synchronization on the vehicle-mounted CAN data and the rosbag data that forms the xls file;
[0010] Filter the time-synchronized data to obtain the data required for the test and form a merged xls file;
[0011] Select the required data from the merged xls file to make a comparison chart and display it to achieve synchronization and analysis of intelligent driving data.
[0012] Furthermore, the process of filtering each csv file to obtain all the data corresponding to the activation of the autonomous driving mode and storing them as an xls file is as follows: each csv file contains an indicator, and each indicator contains the status value of the autonomous driving mode; each time one or more csv files are filtered, the initial time point and the end time point corresponding to the valid value of the status value are found, and the data between the initial time point and the end time point are stored as an xls file.
[0013] Furthermore, the indicator is any one of a positioning signal, a decision and planning output, a braking control signal, and a steering control signal.
[0014] Furthermore, the process of time synchronization is as follows:
[0015] Select the time point corresponding to the initial moment of the same characteristic signal from the rosbag data and the vehicle-mounted CAN data respectively, and organize the data corresponding to this time point and the subsequent time points into two columns of time axis data;
[0016] Compare the lengths of the two time axis data columns, find the time point in the longer time axis data that is closest to each time point in the shorter time axis data, and mark it;
[0017] Replace the time points in the longer timeline data with the marked time points, retain the data corresponding to the marked time points, and achieve time synchronization between the rosbag data and the vehicle-mounted CAN data.
[0018] Furthermore, the characteristic signal is any one of a brake pedal opening signal, a steering wheel angle signal, and a throttle opening signal.
[0019] Furthermore, before marking the time points, the time units of the two columns of time axis data need to be converted into the same unit.
[0020] Furthermore, before marking the time points, the time magnitudes of the two columns of time axis data need to be converted to the same magnitude.
[0021] Furthermore, the process of converting the time magnitude of two columns of timeline data to the same magnitude is: overlaying the first time point in the shorter timeline data with the first time point in the longer timeline data, and then adding the difference between the two columns of timelines to each of the remaining time points in the longer timeline data, so that the two columns of timeline data have the same magnitude.
[0022] Furthermore, the process of screening the time-synchronized data is as follows: selecting one or more data required for testing for different indicators in the xls file, respectively, and merging the corresponding data of the selected multiple indicators into the same table to form a merged xls file.
[0023] Furthermore, rosbag data is obtained from the intelligent driving controller through the rosbag package on the Linux host computer, and the rosbag package includes a collection plug-in and a conversion plug-in.
[0024] The beneficial effects of the present invention are:
[0025] 1) The extraction plug-in of the rosbag (ROS data packet) package of the present invention integrates the functions of accessing and starting MDC (intelligent driving controller), recording, and batch conversion, quickly realizes the connection between the Linux acquisition computer and MDC, and efficiently completes a series of actions from acquisition to raw data generation. The two plug-ins play the role of a complete acquisition tool, which is simple to operate, efficient and accurate, and provides a strong guarantee for subsequent data analysis.
[0026] 2) The data analysis and synchronization process of the present invention completes the integration of all information in rosbag, effective length screening, and comparison of upstream and downstream process variables, and supports various comparison methods, which can help developers locate problems in different algorithm modules, support upstream and downstream data analysis, and can quickly optimize algorithms and support iterations; different plot modes can be selected to adapt to different analysis requirements; it has strong scalability, and the code provides a wealth of open interfaces, which can be quickly customized and developed according to needs.
[0027] 3) The data synchronization method of the present invention breaks the inherent synchronization method based on hardware protocols. It establishes time relationships at different time points through common signals, and synchronizes the time axis using a post-processing method to complete data synchronization. It is not limited by the type and protocol of acquisition tools, has a wide range of applications, and is low in cost. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1 This is a schematic diagram of the test environment collection equipment of the present invention.
[0029] Figure 2 This is a flowchart of converting csv to xls and screening according to the present invention.
[0030] Figure 3 The present invention is a flowchart of post-time frequency synchronization of vehicle-mounted CAN data and rosbag data.
[0031] Figure 4 Flow chart of Rosbag data merging of the present invention.
[0032] Figure 5 The present invention is a flow chart for outputting the merged xls file.
[0033] Figure 6 This is an example diagram of the output of the present invention. DETAILED DESCRIPTION
[0034] The specific embodiments of the present invention are further described below in conjunction with the accompanying drawings. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
[0035] The present invention provides a ROS-based intelligent driving data analysis and synchronization method to obtain vehicle-mounted CAN data from the vehicle-mounted CAN network;
[0036] Get rosbag data from the intelligent driving controller and convert it into a csv file;
[0037] Filter each csv file to obtain all the data corresponding to the activation of the autonomous driving mode and store them as xls files;
[0038] Perform time synchronization on the vehicle-mounted CAN data and the rosbag data that forms the xls file; filter the time-synchronized data to obtain the data required for the test to form a merged rosbag data;
[0039] Select required data from the merged rosbag data to make a comparison chart and display it, realizing the synchronization and analysis of intelligent driving data.
[0040] In the above scheme, the process of filtering each csv file to obtain all the data corresponding to the activation of the autonomous driving mode and storing them as xls files is as follows: each csv file contains an indicator, and each indicator contains the state value of the autonomous driving mode; each time one or more csv files are filtered and processed, the initial time point and the end time point corresponding to the valid value of the state value are found, and the data between the initial time point and the end time point are stored as xls files. The indicator is any one of the positioning signal, decision and planning output, braking control signal, and steering control signal.
[0041] In the above scheme, the process of time synchronization is as follows:
[0042] The time points corresponding to the initial moment of the same characteristic signal are selected from the rosbag data and the vehicle-mounted CAN data respectively, and the data corresponding to the time point and the subsequent time points are sorted into two columns of time axis data; the characteristic signal is any one of the brake pedal opening signal, the steering wheel angle signal, and the throttle opening signal.
[0043] Convert the time units of the two columns of time axis data to the same unit, and convert the time magnitudes of the two columns of time axis data to the same magnitude;
[0044] Compare the lengths of the two time axis data columns, find the time point in the longer time axis data that is closest to each time point in the shorter time axis data, and mark it;
[0045] Replace the time points in the longer timeline data with the marked time points, retain the data corresponding to the marked time points, and achieve time synchronization between the rosbag data and the vehicle-mounted CAN data.
[0046] In the above scheme, the process of converting the time magnitude of two columns of timeline data to the same magnitude is: overlaying the first time point in the timeline data with a shorter length onto the first time point in the timeline data with a longer length, and calculating the difference between the two first time points (i.e., the difference between the first time point in the timeline data with a longer length and the first time point in the timeline data with a shorter length, which can be a positive value or a negative value), and then subtracting the difference from each of the remaining time points in the timeline data with a longer length to obtain new timeline data with a longer length, so that the two columns of timeline data have the same magnitude.
[0047] In the above scheme, the process of screening the data after time synchronization is: select one or more data required for the test for different indicators in the xls file, and merge the corresponding data of the selected multiple indicators into the same table to form the merged rosbag data.
[0048] Example 1
[0049] Before data analysis begins, prepare the collection environment and equipment, such as Figure 1 , Figure 2 As shown, the specific collection steps are:
[0050] 1. The intelligent driving controller is connected to the Linux (operating system) host computer via Ethernet;
[0051] 2. Run the built-in plug-in 1 of the Linux host computer, connect to MDC (intelligent driving controller), and start MDC;
[0052] 3. After starting MDC, delay 10 seconds and the built-in plug-in 1 starts recording the rosbag package; after the collection is completed, stop running the plug-in;
[0053] 4. After starting Ros collection, use the CANoe tool to start vehicle CAN network data collection through the vehicle OBD on the window host computer. After the collection is completed, stop CANoe collection;
[0054] 4. Built-in plug-in 2 of the Linux host computer converts the recorded rosbag packages into csv files in batches;
[0055] 5. After the conversion is completed, plug-in 2 will copy the remote end to the local end of Linux (operating system).
[0056] 6. Copy the converted CSV file on the local side to the window host computer;
[0057] 7. Analysis: The host computer has obtained ROS data and vehicle data, and is ready to start analysis. The collection is completed.
[0058] The data analysis process includes csv to xls conversion and screening, synchronization, data (rosbag and vehicle CAN data) merging, and exporting. Figure 4 The configuration of the four functions is shown in Table 1:
[0059] Table 1 Configuration table of four functions
[0060]
[0061] The above four functions are in a progressive relationship. After obtaining the original data, it starts with converting to xls and filtering the data, synchronizing rosbag data and vehicle data, and then merging these data into a sheet, and finally automatically generating a graph based on the data in a sheet.
[0062] Data processing and analysis process:
[0063] 1. Convert csv to xls and filter: Figure 2 As shown, the file copied from the MDC remote to the local is a csv file converted by the rosbag package. Each csv file has a topic (indicator). Specific topics include location_topic (positioning signal), dp_topic (decision-making and planning output), ebs_topic (brake control signal), steer_topic (steering control signal), etc.; each topic has multiple signal values belonging to this category, and must include the state of the autonomous driving mode (aducstate, 0 is not activated, 1 is preparing, 2 is prepared, 3 is activated). You can process a single or multiple csv files. First open the csv, filter and process, find the message whose status bit changes from ready-active, and the message from active (valid) to inactive (invalid), find two active time points, delete other messages, and leave the messages between the two active; then save the remaining data messages as xls files, all of which are valid data.
[0064] 2. Perform time and frequency synchronization on the vehicle CAN network data and rosbag data: Figure 3 As shown, the synchronization process is as follows:
[0065] A. The acquisition system starts to collect and record.
[0066] B. When the system is in automatic driving mode, the control module will receive the system instruction "send 100% brake pedal request" and there will be a voice reminder "send brake request". Then the control module sends instructions to the bus to control the braking system.
[0067] C. At this time, the acquisition system will record the control instructions issued by rosbag and the brake pedal position feedback request received on the bus at the same time, but the former is the MDC system time and the latter is the industrial computer system time (relative time).
[0068] D. The Windows host computer converts the two into Excel format files, and then uses Python (a programming language) to process the XLS file, find the specific time point when the brake pedal is 100 at the initial moment of VehDa_rBrkPedl_mp (brake pedal signal defined by ROS) and Brake_Padel_Position (brake pedal signal defined by the vehicle network), and start synchronization from this point. The specific principle process is shown in Table 2:
[0069] Table 2 Time axis synchronization data record table
[0070]
[0071] a. Arrange the timeline of the two types of data;
[0072] b. Since the data in the rosbag package is long, the decimal point is processed and converted into the format of seconds + milliseconds. Canoe itself uses seconds as the unit, so the units of the two columns of data remain consistent.
[0073] c. The first point is the specific time point when the brake pedal is 100. It is directly overlaid on the canoe time axis. Then, at each time point in the canoe, the difference between the two columns of time axes when the pedal is 100 is subtracted to make the two columns of data of the same magnitude. This step of time axis coverage ensures that the two columns of data are on the same time axis.
[0074] d. Data length processing: First, calculate the length of the two columns of data, find the point closest in time to the time point corresponding to each value with a shorter data length on the other set of data, and mark this point.
[0075] e. Timeline replacement: directly replace the closest time point marked in the previous step with the time of rosbag, and keep the corresponding numerical value of the time; delete other time points and values, so as to ensure that the time points are exactly the same, laying the foundation for subsequent drawing.
[0076] The brake pedal is not the only signal used as the synchronization start time. You can also use signals with representative characteristics such as steering wheel angle and throttle opening. It only needs to be recorded in rosbag and the vehicle network at the same time.
[0077] Rosbag contains many signals, so in order to reduce the amount of calculation, the vehicle CAN message time is changed based on the rosbag package timeline
[0078] 3. Rosbag merging: For each topic, the included signals are the full set of this indicator. Many data are useful for developers to monitor but not for testing, and some are just reserved, so it is necessary to select the signals needed for testing; for example, for the location-topic, for a certain analysis, only the x / y coordinate positioning information is needed, so only these two columns of information are filtered and retained, as shown in Table 3
[0079] Table 3 x / y coordinate positioning information record table
[0080] field.header.seq field.header.stamp field.header.frame_id field.X field.Y field.yaw 737 1.56E+18 df-truck 507877.2474 3369768.96 -165.6823 738 1.56E+18 df-truck 507878.2474 3369769.96 -165.6823 739 1.56E+18 df-truck 507879.2474 3369770.96 -165.6823 740 1.56E+18 df-truck 507880.2474 3369771.96 -165.6823 741 1.56E+18 df-truck 507881.2474 3369772.96 -165.6823 742 1.56E+18 df-truck 507882.2474 3369773.96 -165.6823 743 1.56E+18 df-truck 507883.2474 3369774.96 -165.6823 744 1.56E+18 df-truck 507884.2474 3369775.96 -165.6823 745 1.56E+18 df-truck 507885.2474 3369776.96 -165.6823 746 1.56E+18 df-truck 507886.2474 3369777.96 -165.6823 747 1.56E+18 df-truck 507887.2474 3369778.96 -165.6823 748 1.56E+18 df-truck 507888.2474 3369779.96 -165.6823
[0081] Secondly, for the comparative analysis of different topics, we first need to merge the information of different topics into a table to provide a basis for subsequent drawing. Figure 4 As shown, through the above process, all the data that need to be analyzed after a single collection can be merged into an xls file, which paves the way for the next step of analysis and comparison.
[0082] 4. Finally, the merged rosbag package is plotted and analyzed, and several plotting methods are provided. Users can observe and analyze the output of each part (corresponding to the corresponding algorithm) from the three dimensions of perception-decision-control bus signals. These three dimensions correspond to the parameter selection detection_outputs (perception output), planning_outputs (decision output), and control_outputs (control output). Each part can plot the corresponding signal separately or merge it. It is also possible to observe the relationship between upstream and downstream, quickly locate the failure point, and thus locate which algorithm failure caused the abnormal data, helping to quickly locate the rapid regression test. Whether it is the analysis of a single algorithm product or the analysis of upstream and downstream, the processing logic is the same, such as Figure 5 shown.
[0083] For example, if you want to monitor the relationship between the accelerator pedal output and the actual vehicle speed feedback for a certain test, select the parameter "control_CanFeedback" (control and vehicle actual feedback CAN message), select the number of pictures in method 3, set other related interface selection parts, and the actual effect is as follows: Figure 6 Shown
[0084] Figure 6 The top row is the output of the longitudinal control, that is, the output of the control algorithm, and the bottom row is the actual status of the vehicle feedback. Date5 / date6 is the reserved data output by the longitudinal control algorithm, AccPedl is the output throttle opening information, and can_vEgoSpd is the actual speed of the vehicle. The comparison of the two columns of data can detect the control and vehicle response, helping developers to quickly adjust the throttle opening output under corresponding conditions according to the actual response process of the vehicle. It is also possible to monitor the current positioning point output by the positioning module and the lateral offset calculated by the planning module, the lateral offset calculated by the fire certificate planning module and the control angle output relationship and other upstream and downstream data to help quickly locate the problem.
[0085] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of protection of the present disclosure. The attached method claims present the elements of the various steps in an exemplary order and are not intended to be limited to the specific order or hierarchy described.
[0086] In order to make the description of the present disclosure more detailed and complete, the above description is provided for the implementation and specific embodiments of the present invention; however, this is not the only form of implementing or using the specific embodiments of the present invention. The implementation covers the features of multiple specific embodiments and the method steps and sequences used to construct and operate these specific embodiments. However, other specific embodiments can also be used to achieve the same or equivalent functions and step sequences.
[0087] In the above detailed description, various features are grouped together in a single embodiment to simplify the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the embodiments of the claimed subject matter require more features than are clearly stated in each claim. On the contrary, as reflected in the appended claims, the invention is in a state of having less than all the features of the disclosed individual embodiments. Therefore, the appended claims are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.
[0088] The disclosed embodiments are described above to enable any person skilled in the art to implement or use the present invention. Various modifications of these embodiments are obvious to those skilled in the art, and the general principles defined herein may also be applied to other embodiments without departing from the spirit and scope of the present disclosure. Therefore, the present disclosure is not limited to the embodiments given herein, but is consistent with the broadest scope of the principles and novel features disclosed in this application.
[0089] The above description includes examples of one or more embodiments. Of course, it is impossible to describe all possible combinations of components or methods for the purpose of describing the above embodiments, but it should be recognized by those skilled in the art that the various embodiments may be further combined and arranged. Therefore, the embodiments described herein are intended to cover all such changes, modifications and variations that fall within the scope of protection of the appended claims. In addition, with respect to the term "comprising" used in the specification or claims, the word is covered in a manner similar to the term "including", just as "including," is explained as a transitional word in the claims. In addition, any term "or" used in the specification of the claims is intended to mean "non-exclusive or".
[0090] Those skilled in the art may also understand that the various illustrative logical blocks, units, and steps listed in the embodiments of the present invention may be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly demonstrate the interchangeability of hardware and software, the various illustrative components, units, and steps described above have generally described their functions. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement the described functions for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present invention.
[0091] The various illustrative logic blocks or units described in the embodiments of the present invention can be implemented or operated by a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or any combination of the above. The general-purpose processor can be a microprocessor, and optionally, the general-purpose processor can also be any conventional processor, controller, microcontroller or state machine. The processor can also be implemented by a combination of computing devices, such as a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.
[0092] The above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, combinations and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims
1. A ROS-based intelligent driving data analysis and synchronization method, characterized in that: Get rosbag data from the intelligent driving controller and convert it into a csv file; Filter each csv file to obtain all the data corresponding to the activation of the autonomous driving mode and store them as xls files; Get vehicle CAN data from the vehicle CAN network; Perform time synchronization on the vehicle-mounted CAN data and the rosbag data that forms the xls file; Filter the time-synchronized data to obtain the data required for the test and form a merged xls file; Select the required data from the merged xls file to make a comparison chart and display it to achieve synchronization and analysis of intelligent driving data; The process of time synchronization is as follows: Select the time point corresponding to the initial moment of the same characteristic signal from the rosbag data and the vehicle-mounted CAN data respectively, and organize the data corresponding to this time point and the subsequent time points into two columns of time axis data; Compare the lengths of the two time axis data columns, find the time point in the longer time axis data that is closest to each time point in the shorter time axis data, and mark it; Replace the time points in the longer time axis data with the marked time points, retain the data corresponding to the marked time points, and achieve time synchronization between rosbag data and vehicle-mounted CAN data; Before marking the time points, the time units of the two columns of time axis data need to be converted to the same unit, and the time magnitudes of the two columns of time axis data need to be converted to the same magnitude; The process of converting the time magnitude of two columns of timeline data to the same magnitude is as follows: overwrite the first time point in the shorter timeline data with the first time point in the longer timeline data, and then subtract the difference between the two first time points from each of the remaining time points in the longer timeline data to make the two columns of timeline data the same magnitude.
2. The ROS-based intelligent driving data analysis and synchronization method according to claim 1, characterized in that: The process of filtering each csv file to obtain all the data corresponding to the activation of the autonomous driving mode and storing them as xls files is as follows: each csv file contains an indicator, and each indicator contains the status value of the autonomous driving mode; each time one or more csv files are filtered, the initial time point and the end time point corresponding to the valid value of the status value are found, and the data between the initial time point and the end time point are stored as xls files.
3. The ROS-based intelligent driving data analysis and synchronization method according to claim 2, characterized in that: The indicator is any one of a positioning signal, a decision and planning output, a braking control signal, and a steering control signal.
4. The ROS-based intelligent driving data analysis and synchronization method according to claim 1, characterized in that: The characteristic signal is any one of a brake pedal opening signal, a steering wheel angle signal, and a throttle opening signal.
5. The ROS-based intelligent driving data analysis and synchronization method according to claim 1, characterized in that: The process of screening the time-synchronized data is as follows: selecting one or more data required for the test for different indicators in the xls file, and merging the corresponding data of the selected multiple indicators into the same table to form a merged xls file.
6. The ROS-based intelligent driving data analysis and synchronization method according to claim 1, characterized in that: The rosbag data is obtained from the intelligent driving controller through the extraction plug-in on the Linux host computer, and the extraction plug-in includes a collection plug-in and a conversion plug-in.
Citation Information
Patent Citations
Method of synchronizing time, device and server
CN109116357A
Test data synchronization method, device and system based on combined data acquisition instrument
CN112181906A
Method for automatically calibrating robot sensor parameters, calibration room, equipment and computer medium
CN112606000A
Unified management method for rosbag file information
CN112860660A
Optical flow estimation method and system based on asynchronous event flow and grayscale image fusion
CN113269699A
