Performance Evaluation Method, Device, Equipment and Storage Medium of Integrated Navigation Device

Abstract

The present invention relates to the field of autonomous driving control, and discloses a method, device, equipment and storage medium for performance evaluation of a combined navigation device. The method includes: when the combined navigation system meets a preset evaluation condition, obtaining positioning data output by at least two combined navigation devices, wherein at least two of the at least two combined navigation devices include a device under test; detecting the positioning accuracy of the device under test based on the positioning data output by different combined navigation devices to obtain a detection result; if the detection result is inaccurate, obtaining an attitude data message corresponding to the positioning data output by the device under test, and performing transmission monitoring on the attitude data message to obtain a monitoring result; and performing performance evaluation on the device under test based on the monitoring result to obtain a first evaluation result. The present invention realizes locating specific device problems or software algorithm problems when the combined navigation system has an abnormality, and comprehensively evaluates the performance of the combined navigation system.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving control, and particularly to a method, device, equipment and storage medium for evaluating the performance of a combined navigation device. Background Art

[0002] In driverless driving, a combined navigation device (hereinafter referred to as P-BOX, Positioning Box) is one of the essential components, such as CPT6 and CPT7 of Novatel. The Novatel protocol is one of the main protocols for data transmission between the P-BOX and the outside, and RAWIMUS (Raw IMU short data) is an important and high-frequency data message among them. The RAWIMUS message mainly contains the attitude data of the IMU of the combined navigation device, and the transmission frequency is usually 200HZ or 100HZ. The attitude data of the IMU plays an important role in the position calculation and positioning of driverless driving. Due to the high frequency of RWAIMUS data and the fact that the link often uses the RS232 bus for data transmission, once there is a link delay or link interruption, it will have a certain impact on the positioning accuracy of the driverless vehicle.

[0003] Since the RAWIMUS data transmission frequency is too high, it is impossible to accurately count the link delay. When an abnormality occurs, it is impossible to locate whether the abnormality is caused by the combined navigation device or the software algorithm. At the same time, it is impossible to calculate the frequency stability of the combined navigation RAWIMUS message and evaluate the performance of the combined navigation device. In short, the existing methods for evaluating the abnormality of the combined navigation device are not comprehensive enough. Summary of the Invention

[0004] The main object of the present invention is to solve the technical problem that the existing methods for evaluating the abnormality of the combined navigation device are not comprehensive enough.

[0005] The first aspect of the present invention provides a method for evaluating the performance of a combined navigation device, including: when the combined navigation system meets the preset evaluation conditions, obtaining the positioning data output by at least two combined navigation devices, where at least two of the at least two combined navigation devices include the device under test; detecting the positioning accuracy of the device under test based on the positioning data output by different combined navigation devices to obtain a detection result; if the detection result is inaccurate, obtaining the attitude data message corresponding to the positioning data output by the device under test and performing transmission monitoring on the attitude data message to obtain a monitoring result; and performing performance evaluation on the device under test based on the monitoring result to obtain a first evaluation result.

[0006] Optionally, in the first implementation manner of the first aspect of the present invention, the at least two integrated navigation devices further include a reference device at least. The method for detecting the positioning accuracy of the device under test based on the positioning data output by different integrated navigation devices and obtaining a detection result includes: determining a first positioning trajectory of the autonomous vehicle based on the positioning data output by the reference device, and determining a second positioning trajectory of the autonomous vehicle based on the positioning data output by the device under test; comparing the first positioning trajectory and the second positioning trajectory, and calculating a positioning error between the reference device and the device under test based on the comparison result; judging whether the positioning accuracy of the device under test is accurate according to the positioning error, and obtaining a detection result based on the discrimination result.

[0007] Optionally, in the second implementation manner of the first aspect of the present invention, before obtaining the positioning data output by at least two integrated navigation devices, it further includes: monitoring the integrated navigation device currently responding in the integrated navigation system of the autonomous vehicle; when it is monitored that the integrated navigation device currently responding in the integrated navigation system switches from the device under test to the reference device, or switches from the reference device to the device under test, it is determined that the integrated navigation system meets a preset evaluation condition.

[0008] Optionally, in the third implementation manner of the first aspect of the present invention, after detecting the positioning accuracy of the device under test based on the positioning data output by different integrated navigation devices and obtaining a detection result, it further includes: if the detection result is accurate, obtaining high-precision map data corresponding to the positioning data, extracting lane data of at least one lane line in the high-precision map data, and storing the lane data.

[0009] Optionally, in the fourth implementation manner of the first aspect of the present invention, after performing a performance evaluation on the device under test based on the monitoring result and obtaining a first evaluation result, it further includes: determining a third positioning trajectory of the autonomous vehicle based on the lane data stored when detecting the positioning accuracy of the device under test in the previous round; performing an accuracy detection and a performance evaluation on the reference device based on the first positioning trajectory and the third positioning trajectory, and obtaining a second evaluation result.

[0010] Optionally, in the fifth implementation manner of the first aspect of the present invention, the method for performing transmission monitoring on the attitude data message and obtaining a monitoring result includes: parsing the attitude data message to obtain the device sending time and the system parsing time corresponding to the device under test; calculating the log storage time of the attitude data message based on the device sending time and the system parsing time, and recording the log storage time as the system receiving time; obtaining a monitoring result of the attitude data message based on the device sending time, the system parsing time, and the system receiving time.

[0011] Optionally, in the sixth implementation manner of the first aspect of the present invention, the performance evaluation of the device under test based on the monitoring result to obtain a first evaluation result includes: calculating the device sending interval of the device under test for the attitude data message based on the device sending time, and calculating the system parsing interval for the attitude data message based on the system parsing time; comparing the device sending interval, the system parsing interval, and the system receiving time with corresponding preset time ranges to obtain a comparison result; and determining the first evaluation result of the performance evaluation of the device under test with reference to the comparison result.

[0012] The second aspect of the present invention provides a performance evaluation device for a combined navigation device, including: an acquisition module, configured to acquire positioning data output by at least two combined navigation devices when the combined navigation system meets a preset evaluation condition, where at least two of the at least two combined navigation devices include a device under test; an accuracy detection module, configured to detect the positioning accuracy of the device under test based on the positioning data output by different combined navigation devices to obtain a detection result; a transmission monitoring module, configured to, if the detection result is inaccurate, acquire an attitude data message corresponding to the positioning data output by the device under test and perform transmission monitoring on the attitude data message to obtain a monitoring result; and a performance evaluation module, configured to perform performance evaluation on the device under test based on the monitoring result to obtain a first evaluation result.

[0013] Optionally, in the first implementation manner of the second aspect of the present invention, at least two of the at least two combined navigation devices further include a reference device, and the accuracy detection module includes: a determination unit, configured to determine a first positioning trajectory of an autonomous driving vehicle based on the positioning data output by the reference device, and determine a second positioning trajectory of the autonomous driving vehicle based on the positioning data output by the device under test; an error calculation unit, configured to compare the first positioning trajectory and the second positioning trajectory and calculate a positioning error between the reference device and the device under test based on the comparison result; and a discrimination unit, configured to determine whether the positioning accuracy of the device under test is accurate according to the positioning error and obtain a detection result based on the discrimination result.

[0014] Optionally, in the second implementation manner of the second aspect of the present invention, the performance evaluation device for the combined navigation device further includes a condition monitoring module, configured to: monitor the combined navigation device currently responding in the combined navigation system of the autonomous driving vehicle; and determine that the combined navigation system meets the preset evaluation condition when it is monitored that the combined navigation device currently responding in the combined navigation system switches from the device under test to the reference device or from the reference device to the device under test.

[0015] Optionally, in the third implementation manner of the second aspect of the present invention, the performance evaluation device of the integrated navigation device further includes a lane extraction module, which is configured to: if the detection result is accurate, obtain the high-precision map data corresponding to the positioning data, extract the lane data of at least one lane line in the high-precision map data, and store the lane data.

[0016] Optionally, in the fourth implementation manner of the second aspect of the present invention, the performance evaluation device of the integrated navigation device further includes a detection and evaluation module, which is configured to: determine the third positioning trajectory of the autonomous vehicle based on the lane data stored when detecting the positioning accuracy of the device under test in the previous round; perform accuracy detection and performance evaluation on the reference device based on the first positioning trajectory and the third positioning trajectory to obtain a second evaluation result.

[0017] Optionally, in the fifth implementation manner of the second aspect of the present invention, the transmission monitoring module includes: an analysis unit, configured to analyze the attitude data message to obtain the device sending time and the system analysis time corresponding to the device under test; a recording unit, configured to calculate the log storage time of the attitude data message based on the device sending time and the system analysis time, and record the log storage time as the system reception time; a generation unit, configured to obtain the monitoring result of the attitude data message based on the device sending time, the system analysis time, and the system reception time.

[0018] Optionally, in the sixth implementation manner of the second aspect of the present invention, the performance evaluation module includes: an interval calculation unit, configured to calculate the device sending interval of the device under test for the attitude data message based on the device sending time, and calculate the system analysis interval for the attitude data message based on the system analysis time; a comparison unit, configured to compare the device sending interval, the system analysis interval, and the system reception time with the corresponding preset time ranges to obtain a comparison result; an evaluation unit, configured to determine the first evaluation result of the performance evaluation of the device under test with reference to the comparison result.

[0019] The third aspect of the present invention provides a performance evaluation device for an integrated navigation device, including: a memory and at least one processor, where instructions are stored in the memory; the at least one processor invokes the instructions in the memory to enable the performance evaluation device of the integrated navigation device to execute the above-mentioned performance evaluation method for the integrated navigation device.

[0020] The fourth aspect of the present invention provides a computer-readable storage medium, in which instructions are stored, and when the instructions are run on a computer, the computer is enabled to execute the above-mentioned performance evaluation method for the integrated navigation device.

[0021] In the technical solution provided by the present invention, when the integrated navigation system meets the preset evaluation conditions, positioning data output by at least two integrated navigation devices is acquired, where at least two of the at least two integrated navigation devices include the device under test; based on the positioning data output by different integrated navigation devices, the positioning accuracy of the device under test is detected to obtain a detection result; if the detection result is inaccurate, an attitude data message corresponding to the positioning data output by the device under test is acquired, and transmission monitoring is performed on the attitude data message to obtain a monitoring result; based on the monitoring result, performance evaluation of the device under test is performed to obtain a first evaluation result. The present invention realizes locating specific device problems or software algorithm problems when the integrated navigation system has an abnormality, and comprehensively evaluates the performance of the integrated navigation system. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 Schematic diagram of the first embodiment of the performance evaluation method for the integrated navigation device in the present invention;

[0023] Figure 2 Schematic diagram of the second embodiment of the performance evaluation method for the integrated navigation device in the present invention;

[0024] Figure 3 Schematic diagram of the third embodiment of the performance evaluation method for the integrated navigation device in the present invention;

[0025] Figure 4 Schematic diagram of an embodiment of the performance evaluation device for the integrated navigation device in the present invention;

[0026] Figure 5 Schematic diagram of another embodiment of the performance evaluation device for the integrated navigation device in the present invention;

[0027] Figure 6 Schematic diagram of an embodiment of the performance evaluation device for the integrated navigation device in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] An embodiment of the present invention provides a method, device, equipment and storage medium for evaluating the performance of a combined navigation device. When the combined navigation system meets the preset evaluation conditions, positioning data output by at least two combined navigation devices is obtained, wherein at least two of the at least two combined navigation devices include the device under test; based on the positioning data output by different combined navigation devices, the positioning accuracy of the device under test is detected to obtain a detection result; if the detection result is inaccurate, an attitude data message corresponding to the positioning data output by the device under test is obtained, and transmission monitoring is performed on the attitude data message to obtain a monitoring result; based on the monitoring result, performance evaluation of the device under test is performed to obtain a first evaluation result. The present invention realizes positioning specific device problems or software algorithm problems when the combined navigation system has an abnormality, and comprehensively evaluates the performance of the combined navigation system.

[0029] The terms "first", "second", "third", "fourth", etc. (if any) in the description and claims of the present invention and the above drawings are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments described herein can be implemented in an order different from that shown or described herein. In addition, the term "comprising" or "having" and any variation thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or equipment comprising a series of steps or units does not have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or equipment.

[0030] For ease of understanding, the specific process of the embodiment of the present invention is described below. Please refer to Figure 1 , the first embodiment of the method for evaluating the performance of a combined navigation device in the embodiment of the present invention includes:

[0031] 101. When the combined navigation system meets the preset evaluation conditions, obtain the positioning data output by at least two combined navigation devices, wherein at least two of the at least two combined navigation devices include the device under test;

[0032] It can be understood that the execution subject of the present invention can be a device for evaluating the performance of a combined navigation device, or a terminal or a server. Specifically, no limitation is made here. The embodiment of the present invention is described by taking the server as the execution subject as an example.

[0033] In this embodiment, preset evaluation conditions are set. When the integrated navigation system meets the preset evaluation conditions, the performance evaluation of the device under test is triggered. The integrated navigation system includes at least two integrated navigation devices. Among them, the at least two integrated navigation devices can be a combination of a high-precision integrated navigation device and the device under test, or a combination of the device under test with the same precision and redundant integrated navigation devices, where at least one device under test is included. The performance of the device under test is evaluated with at least one high-precision integrated navigation device or redundant integrated navigation device as a reference.

[0034] In this embodiment, it is not necessary to perform real-time performance evaluation on the device under test. Instead, when the integrated navigation system meets the preset evaluation conditions, the performance evaluation of the device under test will be triggered. After starting the performance evaluation of the device under test, first obtain the positioning data of all integrated navigation devices in the integrated navigation system. The positioning data includes the current longitude, latitude, speed, and motion direction angle of the autonomous vehicle, etc.

[0035] 102. Detect the positioning accuracy of the device under test based on the positioning data output by different integrated navigation devices to obtain a detection result;

[0036] In this embodiment, in the integrated navigation system, different integrated navigation devices will output their own positioning data for the autonomous driving system to respond to the positioning data of one of the integrated navigation devices for driving control. When the integrated navigation device responded by the autonomous driving system does not malfunction, the driving control will be performed based on the positioning data output by this integrated navigation device. This integrated navigation device is preferably the device under test.

[0037] Specifically, as described above, the previous evaluation condition can be preset as an abnormality of the device under test, then the performance evaluation of the device under test is triggered. Here, first detect the positioning accuracy of the device under test to determine whether the abnormality "considered" by the autonomous driving system for this device under test is actually a lack of detection accuracy, so as to decide whether to perform further performance evaluation and locate the specific problem area.

[0038] Specifically, here, the positioning data output by different integrated navigation devices can be compared to calculate the positioning error between the device under test and other integrated navigation devices, initially determine whether the device under test is abnormal, and obtain a detection result of accurate or inaccurate positioning accuracy.

[0039] 103. If the detection result is inaccurate, obtain the attitude data packet corresponding to the positioning data output by the device under test, and perform transmission monitoring on the attitude data packet to obtain a monitoring result;

[0040] In this embodiment, when it is determined that the detection result of the device under test is indeed inaccurate in positioning accuracy, further transmission monitoring of its attitude data message (i.e., RWAIMUS data) is performed to determine which link has a problem in the transmission and processing process of the attitude data message between the device under test and the system software algorithm, so as to evaluate the performance of the device under test.

[0041] Specifically, in the process of transmitting the attitude data message, it at least includes three processes: the output of the attitude data message by the device under test, the parsing of the attitude data message by the system, and the reception of the attitude data message by the system. Here, the detection result can be obtained by monitoring the output of the attitude data message by the device under test, the parsing of the attitude data message by the system, and the reception of the attitude data message by the system, and further calculating the processing time of these three processes.

[0042] 104. Based on the monitoring result, perform a performance evaluation on the device under test to obtain a first evaluation result.

[0043] In this embodiment, the processing times of processes such as the output of the attitude data message by the device under test, the parsing of the attitude data message by the system, and the reception of the attitude data message by the system are obtained through the previous monitoring. Since the output of the attitude data message by the device under test represents the performance of the device under test, and the parsing and reception of the attitude data message by the system represent the load performance of the system, the performance of the device under test and the corresponding system is evaluated by calculating the processing times of multiple processes, so as to locate whether the problem of the device under test is the device itself or the corresponding software algorithm.

[0044] In the embodiment of the present invention, when the integrated navigation system meets the preset evaluation conditions, obtain the positioning data output by at least two integrated navigation devices, where at least one of the at least two integrated navigation devices includes the device under test; based on the positioning data output by different integrated navigation devices, detect the positioning accuracy of the device under test to obtain a detection result; if the detection result is inaccurate, obtain the attitude data message corresponding to the positioning data output by the device under test, and perform transmission monitoring on the attitude data message to obtain a monitoring result; based on the monitoring result, perform a performance evaluation on the device under test to obtain a first evaluation result. The present invention realizes locating specific device problems or software algorithm problems when the integrated navigation system has an abnormality, and comprehensively evaluates the performance of the integrated navigation system.

[0045] Please refer to Figure 2 , the second embodiment of the performance evaluation method of the integrated navigation device in the embodiment of the present invention includes:

[0046] 201. Monitor the currently responsive integrated navigation device in the integrated navigation system of an autonomous vehicle; when it is monitored that the currently responsive integrated navigation device in the integrated navigation system switches from the device under test to the reference device, or from the reference device to the device under test, determine that the integrated navigation system meets the preset evaluation conditions;

[0047] In this embodiment, the reference device here can be a redundant integrated navigation device or a high-precision integrated navigation device to be used as a reference to evaluate the positioning accuracy of the device under test. In the integrated navigation system, by default, the device under test is used to perform the positioning task of the autonomous vehicle. When the device under test has an abnormality, it will be switched to the reference device to perform the positioning task. After the abnormality problem of the device under test is solved, it will be switched back from the reference device to the device under test to perform the positioning task. Based on these two situations, whether the device under test is replaced or resumes the positioning task, the evaluation conditions will be triggered to perform the performance evaluation of the device under test.

[0048] Specifically, when the variable p-box switch flag can be used to monitor whether the currently responsive integrated navigation device in the integrated navigation system has switched, set p-box switch flag = 0 for the device under test and p-box switch flag = 1 for the reference device. Then, when it changes from p-box switch flag = 0 to p-box switch flag = 1 or from p-box switch flag = 1 to p-box switch flag = 0, it is confirmed that the currently responsive integrated navigation device has switched, and it is determined that the integrated navigation system meets the preset evaluation conditions.

[0049] 202. When the integrated navigation system meets the preset evaluation conditions, obtain the positioning data output by at least two integrated navigation devices, where at least two of the at least two integrated navigation devices include the device under test;

[0050] 203. Based on the positioning data output by the reference device, determine the first positioning trajectory of the autonomous vehicle, and based on the positioning data output by the device under test, determine the second positioning trajectory of the autonomous vehicle;

[0051] 204. Compare the first positioning trajectory and the second positioning trajectory, and based on the comparison result, calculate the positioning error between the reference device and the device under test;

[0052] 205. According to the positioning error, determine whether the positioning accuracy of the device under test is accurate, and based on the determination result, obtain the detection result;

[0053] In this embodiment, based on the positioning data output by the reference device, the current longitude, latitude, speed, and motion direction angle of the autonomous vehicle can be determined. In a preset time period in the past, based on multiple sets of positioning data, a first positioning trajectory of the autonomous vehicle's displacement over time can be constructed. According to this principle, based on the positioning data output by the device under test, a second positioning trajectory of the autonomous vehicle's displacement over time is constructed.

[0054] Based on the above, when the positioning accuracies of the device under test and the reference device are both accurate, the first positioning trajectory and the second positioning trajectory constructed by them should be the same. However, if there is a transmission link delay or link interruption in one of the integrated navigation devices, it will cause positioning errors in the first positioning trajectory and the second positioning trajectory. Since the object being processed here is the device under test and the integrated navigation system met the evaluation conditions earlier, when a positioning error occurs here, it is defaulted that the positioning accuracy of the device under test is not accurate enough. By presetting a positioning error threshold, when the positioning error exceeds the positioning error threshold, it is determined that the positioning accuracy of the device under test is accurate; otherwise, it is determined that the positioning accuracy is inaccurate.

[0055] 206. If the detection result is accurate, obtain the high-precision map data corresponding to the positioning data, extract the lane data of at least one lane line in the high-precision map data, and store the lane data.

[0056] In this embodiment, when the detection result of the device under test is accurate, the high-precision map data corresponding to the positioning data is obtained. Here, the high-precision map data has been pre-downloaded and can be directly obtained. Among them, for the high-precision map data, high-precision map can also be generated by obtaining the three-dimensional point cloud data collected by the radar and the positioning data collected by the integrated navigation system, and processing the three-dimensional point cloud data with the positioning data.

[0057] In this embodiment, the high-precision map contains the initial lane data of each lane line. The lane data extracted here can be the lane data of all lanes or partial lane data of a specified lane, which can be set and extracted according to requirements, and is quickly stored in the database for later use in performing performance evaluation on the reference device.

[0058] 207. If the detection result is inaccurate, obtain the attitude data message corresponding to the positioning data output by the device under test, and perform transmission monitoring on the attitude data message to obtain a monitoring result.

[0059] 208. Based on the monitoring result, perform a performance evaluation on the device under test to obtain a first evaluation result.

[0060] 209. Determine the third positioning trajectory of the autonomous vehicle based on the lane data stored when detecting the positioning accuracy of the device under test in the previous round;

[0061] 210. Based on the first positioning trajectory and the third positioning trajectory, perform accuracy detection and performance evaluation on the reference device to obtain a second evaluation result.

[0062] In this embodiment, based on the lane data and combined with the positioning data, the third positioning trajectory of the autonomous vehicle driving within the lane lines can be further constructed. This third positioning trajectory is more accurate and can be used to further perform performance evaluation and calibration on the reference device. Since the monitoring and evaluation processes are the same as those of the first and second positioning trajectories before, they will not be elaborated here. Refer to the foregoing steps 201-206.

[0063] Please refer to Figure 3 , the third embodiment of the performance evaluation method of the integrated navigation device in the embodiment of the present invention includes:

[0064] 301. When the integrated navigation system meets the preset evaluation conditions, obtain the positioning data output by at least two integrated navigation devices, where at least two of the at least two integrated navigation devices include the device under test;

[0065] 302. Based on the positioning data output by different integrated navigation devices, detect the positioning accuracy of the device under test to obtain a detection result;

[0066] 303. If the detection result is inaccurate, obtain the attitude data message corresponding to the positioning data output by the device under test, and parse the attitude data message to obtain the device sending time and system parsing time corresponding to the device under test;

[0067] 304. Calculate the log storage time of the attitude data message based on the device sending time and system parsing time, and record the log storage time as the system receiving time;

[0068] 305. Based on the device sending time, system parsing time, and system receiving time, obtain the monitoring result of the attitude data message;

[0069] In this embodiment, after parsing the attitude data packet (RAWIMUS data packet), the present invention records the GPS time (device sending time) carried in the RAWIMUS data packet at this moment, and the system time (system parsing time) at this time; then calculates the difference between the system parsing time and the device sending time as the relative link delay from the device under test to the system transmission; finally, stores all the calculated times in the log, and records the log storage time as the system receiving time. It should be noted that this relative link delay includes the deviation between the host and the GPS time, as well as the absolute link delay. Since this deviation is relatively stable, this value can represent the relative link delay.

[0070] 306. Calculate the device sending interval of the attitude data packet of the device under test based on the device sending time, and calculate the system parsing interval of the attitude data packet based on the system parsing time.

[0071] 307. Compare the device sending interval, the system parsing interval, and the system receiving time with the corresponding preset time ranges to obtain a comparison result.

[0072] 308. Determine the first evaluation result of the performance evaluation of the device under test with reference to the comparison result.

[0073] In this embodiment, according to the device sending time and the system parsing time recorded between every two frames of positioning data, calculate the sending interval of the attitude data packet of the device under test, and calculate the parsing interval of the system receiving the attitude data packet, as shown below:

[0074] Device sending interval = GPS time in the attitude data packet parsed this time - GPS time in the attitude data packet parsed last time;

[0075] System parsing interval = System parsing time of the attitude data packet parsed this time - System parsing time of the attitude data packet parsed last time;

[0076] In this embodiment, then according to the accuracy requirements of the positioning system, the IMU data update frequency should be 100 hz (i.e., the time interval is 10 ms). According to the empirical value, the update should not exceed 20 ms at the longest. Therefore, the preset time range is 10 ± 10 ms.

[0077] Specifically, the internal architecture of the P-BOX device usually includes a mid- to low-performance Soc + IMU + baseband chip + radio frequency chip + FPGA. Attitude data packets and other Novatel protocol packets are packetized and sent by the Soc. Therefore, the accuracy of the update frequency of the attitude data packet depends to a large extent on the performance of the Soc and the algorithm load. Therefore, it can be evaluated by monitoring the update interval of the attitude data packet. Among them, when the RAWIMUS packet of the P-BOX is set to output at a frequency of 100hz, that is, the initial transmission interval is 10ms, the preset time range is 10ms ± 10ms. When the device transmission interval or the system parsing time is within 10 ± 10ms, the time reliability is relatively high (the performance is reliable). Otherwise, it does not meet the performance requirements and there are performance problems. From the system reception time, it can be seen that when the time-consuming from reception, parsing, storage, etc. of the attitude data packet is within the preset time range of 10ms ± 4ms, it is determined that the system reception time is reasonable. Therefore, the abnormal problem at this time should be that the system load is relatively high, which is caused by the untimely reception of the attitude data packet by the system kernel.

[0078] In addition, the device transmission time, system parsing time, system reception time, device transmission interval, and system parsing interval calculated above are saved in a file in binary format to reduce the storage capacity, reduce the writing time, and avoid the impact of this performance evaluation method on the system; the relative delay time and frequency stability of the device are calculated through the parsed device transmission time to evaluate the performance of the P-BOX; in addition, there is a Profiler tool to perform performance analysis on the system, and evaluate the total time consumed from reading, parsing, and monitoring the attitude data packet. When the data is abnormal, it can be parsed from this Profiler tool whether the data reading process is abnormal (P-BOX device abnormal or hardwired connection abnormal), or the parsing process is abnormal (system load abnormal), etc.

[0079] The performance evaluation method of the integrated navigation device in the embodiment of the present invention is described above. Next, the performance evaluation device of the integrated navigation device in the embodiment of the present invention will be described. Please refer to Figure 4 , an embodiment of the performance evaluation device of the integrated navigation device in the embodiment of the present invention includes:

[0080] An acquisition module 401, configured to obtain positioning data output by at least two integrated navigation devices when the integrated navigation system meets a preset evaluation condition, where at least one of the at least two integrated navigation devices includes a device to be measured;

[0081] An accuracy detection module 402, configured to detect the positioning accuracy of the device to be measured based on the positioning data output by different integrated navigation devices, and obtain a detection result;

[0082] A transmission monitoring module 403, configured to, if the detection result is inaccurate, obtain an attitude data message corresponding to the positioning data output by the device under test, and perform transmission monitoring on the attitude data message to obtain a monitoring result;

[0083] A performance evaluation module 404, configured to perform performance evaluation on the device under test based on the monitoring result to obtain a first evaluation result.

[0084] In an embodiment of the present invention, when the integrated navigation system meets a preset evaluation condition, positioning data output by at least two integrated navigation devices is obtained, where at least two of the at least two integrated navigation devices include the device under test; based on the positioning data output by different integrated navigation devices, the positioning accuracy of the device under test is detected to obtain a detection result; if the detection result is inaccurate, an attitude data message corresponding to the positioning data output by the device under test is obtained, and transmission monitoring is performed on the attitude data message to obtain a monitoring result; based on the monitoring result, performance evaluation is performed on the device under test to obtain a first evaluation result. The present invention realizes locating specific device problems or software algorithm problems when the integrated navigation system is abnormal, and comprehensively evaluates the performance of the integrated navigation system.

[0085] Please refer to Figure 5 , another embodiment of the performance evaluation device for an integrated navigation device in an embodiment of the present invention includes:

[0086] An acquisition module 401, configured to, when the integrated navigation system meets a preset evaluation condition, obtain positioning data output by at least two integrated navigation devices, where at least two of the at least two integrated navigation devices include the device under test;

[0087] An accuracy detection module 402, configured to detect the positioning accuracy of the device under test based on the positioning data output by different integrated navigation devices to obtain a detection result;

[0088] A transmission monitoring module 403, configured to, if the detection result is inaccurate, obtain an attitude data message corresponding to the positioning data output by the device under test, and perform transmission monitoring on the attitude data message to obtain a monitoring result;

[0089] A performance evaluation module 404, configured to perform performance evaluation on the device under test based on the monitoring result to obtain a first evaluation result.

[0090] Specifically, at least two of the at least two integrated navigation devices further include a reference device, and the accuracy detection module 402 includes:

[0091] A determination unit 4021, configured to determine a first positioning trajectory of the autonomous vehicle based on the positioning data output by the reference device, and determine a second positioning trajectory of the autonomous vehicle based on the positioning data output by the device under test;

[0092] An error calculation unit 4022, configured to compare the first positioning trajectory and the second positioning trajectory, and calculate a positioning error between the reference device and the device under test based on the comparison result;

[0093] A discrimination unit 4023, configured to determine whether the positioning accuracy of the device under test is accurate according to the positioning error, and obtain a detection result based on the discrimination result.

[0094] Specifically, the performance evaluation device of the integrated navigation device further includes a condition monitoring module 405, configured to:

[0095] Monitor the integrated navigation device currently responding in the integrated navigation system of the autonomous vehicle;

[0096] When it is monitored that the integrated navigation device currently responding in the integrated navigation system switches from the device under test to the reference device, or from the reference device to the device under test, it is determined that the integrated navigation system meets the preset evaluation conditions.

[0097] Specifically, the performance evaluation device of the integrated navigation device further includes a lane extraction module 406, configured to:

[0098] If the detection result is accurate, obtain the high-precision map data corresponding to the positioning data, extract the lane data of at least one lane line in the high-precision map data, and store the lane data.

[0099] Specifically, the performance evaluation device of the integrated navigation device further includes a detection and evaluation module 407, configured to:

[0100] Determine a third positioning trajectory of the autonomous vehicle based on the lane data stored when the positioning accuracy of the device under test was detected in the previous round;

[0101] Perform accuracy detection and performance evaluation on the reference device based on the first positioning trajectory and the third positioning trajectory, and obtain a second evaluation result.

[0102] Specifically, the transmission monitoring module 403 includes:

[0103] A parsing unit 4031, configured to parse the attitude data message to obtain the device sending time and the system parsing time corresponding to the device under test;

[0104] A recording unit 4032, configured to calculate the log storage time of the attitude data message based on the device sending time and the system parsing time, and record the log storage time as the system receiving time;

[0105] A generating unit 4033, configured to obtain a monitoring result of the attitude data message based on the device sending time, the system parsing time, and the system receiving time.

[0106] Specifically, the performance evaluation module 404 includes:

[0107] An interval calculation unit 4041, configured to calculate the device sending interval of the attitude data message for the device under test based on the device sending time, and calculate the system parsing interval of the attitude data message based on the system parsing time;

[0108] A comparison unit 4042, configured to compare the device sending interval, the system parsing interval, and the system receiving time with corresponding preset time ranges to obtain a comparison result;

[0109] An evaluation unit 4043, configured to determine a first evaluation result of the performance evaluation of the device under test with reference to the comparison result.

[0110] Above Figure 4 And Figure 5 The performance evaluation device of the integrated navigation device in the embodiment of the present invention is described in detail from the perspective of modular functional entities. Next, the performance evaluation device of the integrated navigation device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0111] Figure 6 FIG. is a schematic structural diagram of a performance evaluation device of an integrated navigation device provided by an embodiment of the present invention. The performance evaluation device 600 of the integrated navigation device may vary greatly due to configuration or performance, and may include one or more processors (central processing units, CPUs) 610 (for example, one or more processors) and a memory 620, and one or more storage media 630 (for example, one or more mass storage devices) for storing application programs 633 or data 632. Among them, the memory 620 and the storage media 630 may be transient storage or persistent storage. The program stored in the storage media 630 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the performance evaluation device 600 of the integrated navigation device. Further, the processor 610 may be configured to communicate with the storage media 630 and execute a series of instruction operations in the storage media 630 on the performance evaluation device 600 of the integrated navigation device.

[0112] The performance evaluation device 600 of the integrated navigation device may further include one or more power supplies 640, one or more wired or wireless network interfaces 650, one or more input / output interfaces 660, and / or one or more operating systems 631, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art can understand that Figure 6 The structure of the performance evaluation device of the illustrated integrated navigation device does not constitute a limitation on the performance evaluation device of the integrated navigation device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0113] The present invention also provides a performance evaluation device of an integrated navigation device. The computer device includes a memory and a processor. When the computer-readable instructions stored in the memory are executed by the processor, the processor is caused to execute the steps of the performance evaluation method of the integrated navigation device in the above-mentioned various embodiments.

[0114] The present invention also provides a computer-readable storage medium. The computer-readable storage medium may be a non-volatile computer-readable storage medium, or may also be a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions run on a computer, the computer is caused to execute the steps of the performance evaluation method of the integrated navigation device.

[0115] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0116] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The foregoing storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs, etc., which can store program codes.

[0117] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A performance evaluation method for a combined navigation device, characterized in that, The performance evaluation method of the combined navigation device includes: When the combined navigation system meets the preset evaluation conditions, obtain the positioning data output by at least two combined navigation devices, where at least two of the at least two combined navigation devices include the device under test; Based on the positioning data output by different combined navigation devices, detect the positioning accuracy of the device under test to obtain a detection result; If the detection result is inaccurate, obtain the attitude data message corresponding to the positioning data output by the device under test, and perform transmission monitoring on the attitude data message to obtain a monitoring result; Based on the monitoring result, perform performance evaluation on the device under test to obtain a first evaluation result.

2. The performance evaluation method of the integrated navigation device according to claim 1, wherein At least two of the at least two combined navigation devices further include a reference device. The detecting the positioning accuracy of the device under test based on the positioning data output by different combined navigation devices to obtain a detection result includes: Based on the positioning data output by the reference device, determine the first positioning trajectory of the autonomous vehicle, and based on the positioning data output by the device under test, determine the second positioning trajectory of the autonomous vehicle; Compare the first positioning trajectory and the second positioning trajectory, and based on the comparison result, calculate the positioning error between the reference device and the device under test; According to the positioning error, determine whether the positioning accuracy of the device under test is accurate, and based on the determination result, obtain a detection result.

3. The performance evaluation method of the integrated navigation device according to claim 2, wherein Before obtaining the positioning data output by at least two combined navigation devices, it further includes: Monitor the combined navigation device currently responding in the combined navigation system of the autonomous vehicle; When it is monitored that the combined navigation device currently responding in the combined navigation system switches from the device under test to the reference device, or from the reference device to the device under test, it is determined that the combined navigation system meets the preset evaluation conditions.

4. The performance evaluation method of the integrated navigation device according to claim 2, wherein After detecting the positioning accuracy of the device under test based on the positioning data output by different combined navigation devices to obtain a detection result, it further includes: If the detection result is accurate, obtain the high-precision map data corresponding to the positioning data, extract the lane data of at least one lane line in the high-precision map data, and store the lane data.

5. The performance evaluation method of the integrated navigation device according to claim 4, wherein After performing performance evaluation on the device under test based on the monitoring result to obtain a first evaluation result, it further includes: Based on the lane data stored when detecting the positioning accuracy of the device under test in the previous round, determine the third positioning trajectory of the autonomous vehicle; Based on the first positioning trajectory and the third positioning trajectory, perform accuracy detection and performance evaluation on the reference device to obtain a second evaluation result.

6. The performance evaluation method of the integrated navigation device according to any one of claims 1-5, characterized in that The performing transmission monitoring on the attitude data message to obtain a monitoring result includes: Parse the attitude data message to obtain the device sending time and system parsing time corresponding to the device under test; Calculate the log storage time of the attitude data message based on the device sending time and system parsing time, and record the log storage time as the system receiving time; Based on the device sending time, system parsing time, and system receiving time, obtain the monitoring result of the attitude data message.

7. The performance evaluation method of the integrated navigation device according to claim 6, wherein Based on the monitoring results, perform performance evaluation on the device under test to obtain the first evaluation result, including: Based on the device sending time, calculate the device sending interval of the attitude data message for the device under test, and based on the system parsing time, calculate the system parsing interval for the attitude data message; Compare the device sending interval, the system parsing interval, and the system receiving time with the corresponding preset time ranges to obtain a comparison result; With reference to the comparison result, determine the first evaluation result of the performance evaluation of the device under test.

8. A performance evaluation device for a combined navigation device, characterized in that The performance evaluation device of the integrated navigation device includes: An acquisition module, configured to acquire positioning data output by at least two integrated navigation devices when the integrated navigation system meets a preset evaluation condition, where at least two of the at least two integrated navigation devices include the device under test; An accuracy detection module, configured to detect the positioning accuracy of the device under test based on the positioning data output by different integrated navigation devices to obtain a detection result; A transmission monitoring module, configured to, if the detection result is inaccurate, acquire the attitude data message corresponding to the positioning data output by the device under test and perform transmission monitoring on the attitude data message to obtain a monitoring result; A performance evaluation module, configured to perform performance evaluation on the device under test based on the monitoring result to obtain a first evaluation result.

9. A performance evaluation device for a combined navigation device, characterized in that, The performance evaluation device of the integrated navigation device includes: a memory and at least one processor, and instructions are stored in the memory; The at least one processor invokes the instructions in the memory so that the performance evaluation device of the integrated navigation device executes the steps of the performance evaluation method of the integrated navigation device according to any one of claims 1-7.

10. A computer-readable storage medium, on which instructions are stored, characterized in that, When the instructions are executed by the processor, the steps of the performance evaluation method of the integrated navigation device according to any one of claims 1-7 are implemented.

Images