A method and system for evaluating the performance of a millimeter-wave radar

Abstract

The present invention discloses a method and system for evaluating the performance of a millimeter-wave radar. A millimeter-wave radar performance evaluation system involved therein includes: an integrity calculation module for calculating the integrity of fields, content, and detection distance corresponding to the millimeter-wave radar to obtain a first calculation result; an accuracy calculation module for calculating the accuracy of targets, traffic flow, speed, longitude and latitude positions, mileage stake numbers, lane positions, and vehicle types corresponding to the millimeter-wave radar to obtain a second calculation result; a timeliness calculation module for calculating the adjustability of the acquisition frequency and the timeliness of data upload corresponding to the millimeter-wave radar to obtain a third calculation result; and a statistics module for statistically processing the first calculation result, the second calculation result, and the third calculation result to obtain a millimeter-wave radar performance evaluation result.

Description

Technical Field

[0001] The present invention relates to the technical field of performance evaluation, and in particular, to a method and system for evaluating the performance of a millimeter-wave radar. Background Art

[0002] With the increasing complexity of modern radar systems, radar system performance evaluation simulation technology has been widely used. The simulation of radar target and background echo signals plays a very important role in the design, debugging, and acceptance processes of radar systems, and is a key link in radar system performance evaluation. With the development of technology and the continuous improvement of requirements, the complexity of radar systems is getting higher and higher, the types are increasing, and the working environment is complex. The performance evaluation of radar systems has become increasingly difficult. Under such conditions, adopting a performance evaluation simulation system has become an effective means to solve this problem. Radar systems are mainly used to detect the presence of targets and measure the radial distance, radial velocity, and relative angle information of targets. At present, radar performance evaluation often uses the target echo simulation method. Using the simulated target echo signal, the speed measurement or ranging performance of the radar system can be verified. However, to simultaneously verify and evaluate the speed measurement, ranging, and angle measurement performance of the radar system, it is not feasible to use the conventional target echo simulation method.

[0003] Millimeter-wave antennas are an important part of millimeter-wave radar systems for transmitting and receiving electromagnetic wave signals in the millimeter-wave band. In practical applications, when a millimeter-wave radar detects a target, a millimeter-wave antenna with a certain antenna beam width emits an electromagnetic wave signal. All targets located within the radar's effective range and within the antenna beam range often reflect the electromagnetic wave signal in the millimeter-wave band. In this way, the millimeter-wave radar will simultaneously receive the reflected signals from all targets within the antenna beam range. Especially for multiple targets within the radar range resolution, the radar will not be able to distinguish them and will consider them as the reflected signals of the same target. For multiple targets outside the radar range resolution, although the radar can distinguish them, the complexity of the radar system's target processing will increase greatly. Under such complex working environment conditions, it is very difficult to accurately evaluate the performance indicators of the radar system.

[0004] Therefore, it is an urgent technical problem to be solved to design a millimeter-wave radar performance evaluation system, especially for the evaluation of data quality in millimeter-wave radars. Summary of the Invention

[0005] The object of the present invention is to provide a method and system for evaluating the performance of a millimeter-wave radar in view of the defects of the prior art.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A millimeter-wave radar performance evaluation system includes:

[0008] An integrity calculation module, which is used to calculate the integrity of the fields, content, and detection distance corresponding to the millimeter-wave radar to obtain a first calculation result;

[0009] An accuracy calculation module, which is used to calculate the accuracy of the target, traffic flow, speed, longitude and latitude position, mileage stake number, lane position, and vehicle type corresponding to the millimeter-wave radar to obtain a second calculation result;

[0010] A timeliness calculation module, which is used to calculate the adjustability of the acquisition frequency and the timeliness of data upload corresponding to the millimeter-wave radar to obtain a third calculation result;

[0011] A statistics module, which is used to statistically analyze the first calculation result, the second calculation result, and the third calculation result to obtain a performance evaluation result of the millimeter-wave radar.

[0012] Further, the fields in the integrity calculation module include target ID, timestamp, longitude and latitude, mileage stake number, the lane where it is located, the relative position of the lane where it is located, vehicle speed, and vehicle type.

[0013] Further, the calculation of the integrity of the fields corresponding to the millimeter-wave radar in the integrity calculation module is specifically to calculate whether the fields in the data meet the data usage requirements, expressed as:

[0014] X = A / B

[0015] Among them, X represents the calculation result of the integrity of the field; A represents the number of fields that meet the usage requirements in the data; B represents the total number of required fields.

[0016] Further, the calculation of the integrity of the content corresponding to the millimeter-wave radar in the integrity calculation module is specifically to calculate the data content repair quality and missing degree of the millimeter-wave radar, expressed as:

[0017]

[0018] Among them, case 1 means that the vehicle ID in the data repaired by the millimeter-wave radar is correct; case 2 means that the vehicle ID in the data repaired by the millimeter-wave radar is incorrect;

[0019]

[0020] Among them, j represents the field serial number; m represents the maximum value of the field serial number; i represents the serial number of the data frame repaired by the millimeter-wave radar; n represents the maximum value of the serial number of the data frame repaired by the millimeter-wave radar; A ij represents the frame coefficient; X represents the index value of the data content repair quality and missing degree of the millimeter-wave radar; X 1 represents the content integrity weighting coefficient.

[0021] Furthermore, the calculation of the integrity of the detection distance corresponding to the millimeter-wave radar in the integrity calculation module is specifically the calculation of the ability of the millimeter-wave radar to detect and continuously track targets, which is expressed as:

[0022]

[0023] where o represents the number of trajectories; L p represents the detection length of trajectory p; L represents the detection distance requirement; X ave represents the detection ability of the millimeter-wave radar;

[0024] L max = C

[0025] where C represents the maximum distance that the detection range of the millimeter-wave radar can reach; L max represents the maximum distance that the detection range of the millimeter-wave radar can reach.

[0026] Furthermore, the calculation of the accuracy of the target corresponding to the millimeter-wave radar in the accuracy calculation module is specifically the calculation of the ability of the millimeter-wave radar to lock on to targets, which is expressed as:

[0027] Y = S / T

[0028] where S represents the number of vehicles that are the same vehicle but not represented by different IDs in the data; T represents the number of actual passing vehicles in the data; Y represents the calculation result of the accuracy of the target;

[0029] The calculation of the accuracy of the traffic flow corresponding to the millimeter-wave radar in the accuracy calculation module is specifically the calculation of the ability of the millimeter-wave radar to detect the number of multiple targets, which is expressed as:

[0030]

[0031] where d represents the section number, b represents the maximum value of the section number; c represents the lane number, a represents the maximum value of the lane number; S dc represents the number of detected vehicles in lane c and section d; T dc represents the actual number of passing vehicles in lane c and section d;

[0032] The calculation of the accuracy of the speed corresponding to the millimeter-wave radar in the accuracy calculation module is specifically the calculation of the ability of the millimeter-wave radar to detect the moving speed of targets, where the speed includes longitudinal speed and lateral speed;

[0033] The calculation of the longitudinal speed is expressed as:

[0034]

[0035] Among them, w represents the number of times; t represents the number of data frames generated each time; s represents the maximum value of t; D vwt represents the longitudinal speed detected by the vehicle base station at the w-th data frame t; E vwt represents the longitudinal speed synchronously detected by the millimeter-wave radar at the w-th data frame t; Y vw represents the longitudinal moving speed index of the target detected by the millimeter-wave radar;

[0036] The calculation of the lateral speed is expressed as:

[0037]

[0038] Among them, G hwt represents the longitudinal speed detected by the vehicle base station at the w-th data frame t; H hwt represents the longitudinal speed synchronously detected by the millimeter-wave radar at the w-th data frame t; Yh w represents the lateral moving speed index of the target detected by the millimeter-wave radar.

[0039] Furthermore, the calculation of the accuracy of the longitude and latitude position corresponding to the millimeter-wave radar in the accuracy calculation module is specifically the calculation of the accuracy ability of the millimeter-wave radar to detect the target position, expressed as:

[0040]

[0041] Among them, g represents the data frame serial number detected by the millimeter-wave radar; f represents the maximum value of g; lon kg 、lon lg respectively represent the longitudes collected by the base station and the millimeter-wave radar at the data frame g; lat kg 、lat lg respectively represent the latitudes collected by the base station and the millimeter-wave radar at the data frame g;

[0042] The calculation of the accuracy of the mileage stake corresponding to the millimeter-wave radar in the accuracy calculation module is specifically the calculation of converting the target position detected by the millimeter-wave radar into the road stake number, expressed as:

[0043]

[0044] Among them, I g represents the mileage stake number of the vehicle location collected by the base station; J g represents the mileage stake number of the vehicle location detected by the millimeter-wave radar; Z 1 represents the accuracy calculation result of the mileage stake number;

[0045] The calculation of the accuracy of the vehicle type corresponding to the millimeter-wave radar in the accuracy calculation module is specifically the calculation of the target contour recognition ability of the millimeter-wave radar, expressed as:

[0046]

[0047] Among them, r represents the serial number of the vehicle type category; q represents the maximum value of r; M r represents that the vehicle type is r and the number of trajectories recognized as vehicle type r; N represents the number of trajectories of vehicle type r; Z 2 represents the calculation result of the accuracy of the vehicle type.

[0048] Furthermore, the calculation of the accuracy of the lane position corresponding to the millimeter-wave radar in the accuracy calculation module is specifically to calculate the lane position of the vehicle detected by the millimeter-wave radar and the relative position of the lane where it is located, which is expressed as:

[0049]

[0050]

[0051] Z5 = 0.3Z3 + 0.7Z4

[0052] Among them, x represents the serial number of the trajectory where the millimeter-wave radar detects no lane change and it is verified by synchronous video comparison that there is actually no lane change; u represents the maximum value of x; O x represents the number of data frames where the lane number of trajectory x is correct and the relative lane position detection is correct; P x represents the number of frames of trajectory x; z represents the serial number of the trajectory where the millimeter-wave radar detects a lane change and it is verified by synchronous video comparison that there is actually a lane change; y represents the maximum value of z; O z represents the number of data frames where the lane number of trajectory z is correct and the relative lane position detection is correct; P z represents the number of frames of trajectory z; Z 5 represents the calculation result of the relative lane position; Z 3 represents the index value when the vehicle does not change lanes; Z 4 represents the index value when the vehicle changes lanes.

[0053] Furthermore, the calculation of the timeliness of the adjustable acquisition frequency corresponding to the millimeter-wave radar in the timeliness calculation module is specifically to calculate the adjustable acquisition frequency of the millimeter-wave radar, which is expressed as:

[0054] U = V / W

[0055] Among them, V represents the number of acquisition frequencies that the millimeter-wave radar can achieve; W represents the acquisition frequency required by data usage; U represents the calculation result of the timeliness of the adjustable acquisition frequency;

[0056] The calculation of the timeliness of data upload corresponding to the millimeter-wave radar in the timeliness calculation module is specifically to calculate the time delay of data acquisition and recording of the millimeter-wave radar, which is expressed as:

[0057]

[0058] Wherein, w represents the number of times, e represents the maximum value of w; g represents the frame number of the data, f represents the maximum value of g; K wg represents the recording time of the base station equipment of the data frame g in the w-th test; Q wg represents the synchronous recording time of the millimeter-wave radar of the data frame g in the w-th test.

[0059] Correspondingly, a method for evaluating the performance of a millimeter-wave radar is also provided, including:

[0060] S1. Calculate the integrity of the fields, content, and detection distance corresponding to the millimeter-wave radar to obtain a first calculation result;

[0061] S2. Calculate the accuracy of the targets, traffic flow, speed, longitude and latitude positions, mileage stake numbers, lane positions, and vehicle types corresponding to the millimeter-wave radar to obtain a second calculation result;

[0062] S3. Calculate the adjustable acquisition frequency and timeliness of data upload corresponding to the millimeter-wave radar to obtain a third calculation result;

[0063] S4. Statistically process the first calculation result, the second calculation result, and the third calculation result to obtain the performance evaluation result of the millimeter-wave radar.

[0064] Compared with the prior art, according to the requirements of the data rules, the present invention evaluates the integrity of the roadside millimeter-wave radar by the degree to which the detection target is assigned a numerical value, evaluates the accuracy of the roadside millimeter-wave radar by the degree to which the data describes the true value of the actual object, and evaluates the timeliness of the roadside millimeter-wave radar by the degree between the data upload time and the actual acquisition time, improving the performance of the roadside millimeter-wave radar in use. Description of the Drawings

[0065] Figure 1 is a structural diagram of a millimeter-wave radar performance evaluation system provided in Embodiment 1;

[0066] Figure 2 is a flowchart for obtaining the mileage stake number of the test vehicle provided in Embodiment 1. Detailed Embodiments

[0067] The following describes the implementation manners of the present invention through specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation manners. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0068] The object of the present invention is to provide a millimeter-wave radar performance evaluation method and system for the defect of the prior art, which is applicable to the evaluation of the data quality of roadside millimeter-wave radars.

[0069] Embodiment 1

[0070] This embodiment provides a millimeter-wave radar performance evaluation system, including:

[0071] An integrity calculation module 11, configured to calculate the integrity of fields, content, and detection distance corresponding to the millimeter-wave radar, and obtain a first calculation result;

[0072] An accuracy calculation module 12, configured to calculate the accuracy of targets, traffic flow, speed, longitude and latitude positions, mileage stake numbers, lane positions, and vehicle types corresponding to the millimeter-wave radar, and obtain a second calculation result;

[0073] A timeliness calculation module 13, configured to calculate the adjustability of the acquisition frequency and the timeliness of data upload corresponding to the millimeter-wave radar, and obtain a third calculation result;

[0074] A statistics module 14, configured to perform statistics on the first calculation result, the second calculation result, and the third calculation result to obtain a millimeter-wave radar performance evaluation result.

[0075] In the integrity calculation module 11, the integrity of fields, content, and detection distance corresponding to the millimeter-wave radar is calculated to obtain a first calculation result.

[0076] Integrity refers to the degree to which a detection target is assigned a value according to data rule requirements.

[0077] Field integrity refers to the measurement of whether the fields in the data meet the data usage requirements. The fields evaluated in this embodiment include but are not limited to the following fields: target ID, timestamp, longitude and latitude, mileage stake number, lane where located, relative position in the lane, vehicle speed (km / h), and vehicle type.

[0078] The calculation method of field integrity is:

[0079] Retrieve the millimeter-wave radar data for peak and off-peak periods, with the continuous data duration being no less than 30 minutes for each. Calculate the metrics in the following manner. The calculation results for peak and off-peak periods are averaged equally and added together, expressed as:

[0080] X = A / B

[0081] Among them, X represents the calculation result of the field integrity; A represents the number of fields that meet the usage requirements in the data; B represents the total number of all required fields. The fields for this assessment include: vehicle ID, timestamp, longitude and latitude, mileage stake number, lane where the vehicle is located and relative position, vehicle speed, vehicle type, and data generation method, a total of 8 fields.

[0082] The scoring criteria for field integrity are shown in Table 1 below:

[0083] Table 1

[0084] Index value Score 1 5 0.8～1 2 ＜0.8 0

[0085] Content integrity refers to the measurement of the data content repair quality and the degree of missing data by the millimeter-wave radar.

[0086] The calculation method for content integrity is as follows:

[0087] During the period when the proportion of large vehicles is relatively large, test vehicles equipped with RTK (RTK refers to the technology that obtains information through a base station to meet the requirements of high-precision positioning) devices are driven in the area completely blocked by large vehicles (the area where the millimeter-wave radar cannot detect), and the content integrity of the data is evaluated by comparing the differences between the data collected by the test vehicles and the data repaired by the millimeter-wave radar. This assessment includes the evaluation of 6 fields: vehicle ID, timestamp, longitude and latitude, lane where the vehicle is located, vehicle speed, and vehicle type. The calculation method is as follows. The number of tests is not less than 5 times, and the calculation results of multiple tests are added equally, expressed as:

[0088]

[0089] Among them, case 1 means that the vehicle ID in the data repaired by the millimeter-wave radar is correct; case 2 means that the vehicle ID in the data repaired by the millimeter-wave radar is incorrect; X 1 represents the content integrity and safety factor;

[0090]

[0091] Among them, j represents the test field serial number; m represents the maximum value of j. In this assessment, m = 5, corresponding to the timestamp, longitude and latitude, lane where the vehicle is located, vehicle speed, and vehicle type from 1 to 5 respectively; i represents the serial number of the data frame repaired by the millimeter-wave radar; n represents the maximum value of i; A ij represents the frame coefficient; X represents the index value of the data content repair quality and the degree of missing data by the millimeter-wave radar; X 1Indicates the content integrity weighting coefficient.

[0092] The data repair requirements for content integrity are as follows:

[0093] 1) The timestamp error is less than 1 s;

[0094] 2) The longitude and latitude error is less than 0.0025";

[0095] 3) The lane of the millimeter-wave radar repair data is the same as the vehicle position of the actual vehicle;

[0096] 4) The speed error is less than 1 km / h;

[0097] 5) The millimeter-wave radar repair vehicle type is the same as the actual vehicle type.

[0098] The scoring criteria for content integrity are shown in Table 2 below:

[0099] Table 2

[0100] Index value Score 1 10 0.80～1 8 0.70～0.80 4 ＜0.70 1

[0101] It should be noted that before the test, the RTK device needs to be synchronized with the roadside millimeter-wave radar device clock, and the RTK data acquisition frequency and the millimeter-wave radar frequency need to be adjusted to be the same.

[0102] The detection distance integrity refers to the measurement of the ability of the millimeter-wave radar to detect targets and continuously track them.

[0103] The calculation method of the detection distance integrity is as follows:

[0104] Take the millimeter-wave radar data for peak and flat peak respectively, and the continuous data duration is not less than 30 minutes. Calculate the index in the following way. The calculation results for peak and flat peak are averaged equally and added together, expressed as:

[0105]

[0106] Among them, o represents the number of test trajectories; L p represents the detection length of trajectory p; L represents the detection distance requirement, and in this evaluation, L = 250 m; X ave represents the detection ability of the millimeter-wave radar;

[0107] L max = C

[0108] Among them, C represents the maximum distance that the detection range of the millimeter-wave radar can reach; L max represents the maximum distance that the detection range of the millimeter-wave radar can reach.

[0109] In this embodiment, the longitudinal direction refers to the direction consistent with the vehicle driving direction, with the same direction being positive and the opposite direction being negative; the transverse direction refers to the direction perpendicular to the vehicle driving direction, with the left side perpendicular to the driving direction being positive and the right side being negative. The driving direction distance refers to the distance difference between the vehicle position coordinate values of the same vehicle at the next moment and the previous moment.

[0110] The scoring criteria for detecting distance integrity are shown in Table 3 below:

[0111] Table 3

[0112]

[0113] In the accuracy calculation module 12, the accuracy of the target, traffic flow, speed, longitude and latitude position, mileage stake number, lane position, and vehicle type corresponding to the millimeter-wave radar is calculated to obtain the second calculation result.

[0114] Accuracy indicates the degree to which data describes the true value of the actual object.

[0115] The target ID accuracy refers to the measurement of the ability of the millimeter-wave radar to lock the target.

[0116] The calculation method of the target ID accuracy is as follows:

[0117] Take the millimeter-wave radar data for peak and off-peak periods respectively, with the continuous data duration not less than 30 minutes. Calculate the indicators in the following way, and the calculation results for peak and off-peak periods are averaged with equal weights and added together, expressed as:

[0118] Y = S / T

[0119] Wherein, S represents the number of vehicles that are not represented by different IDs for the same vehicle in the data, which is manually checked through synchronous video; T represents the actual number of vehicles passing through in the data; Y represents the calculation result of the accuracy of the target.

[0120] The scoring criteria for the target ID accuracy are shown in Table 4 below:

[0121] Table 4

[0122] Index value Score 0.95～1 15 0.85～0.95 10 0.65～0.85 5 ＜0.65 1

[0123] The traffic flow accuracy refers to the measurement of the ability of the millimeter-wave radar to detect the number of multiple targets. This evaluation refers to measuring the accuracy of detecting the number of vehicles passing through the road section.

[0124] The calculation method of the traffic flow accuracy is as follows:

[0125] Take the millimeter-wave radar data for peak and off-peak periods respectively, with the continuous data duration not less than 30 minutes. Calculate the indicators respectively in the following way, and the calculation results for peak and off-peak periods are averaged with equal weights and added together, expressed as:

[0126]

[0127] Among them, d represents the cross-section number, b represents the maximum value of d, and in this evaluation, b = 3, including the initial stage, intermediate stage, and end stage of the radar detection range; c represents the lane number, a represents the maximum value of c, and in this evaluation, a = 3; S dc represents the number of detected vehicles in lane c and cross-section d; T dc represents the actual number of vehicles passing through in lane c and cross-section d.

[0128] The scoring criteria for traffic flow accuracy are shown in Table 5 below:

[0129] Table 5

[0130] Index value Score 0.90 - 1 or 1 - 1.10 5 0.85 - 0.90 or 1.10 - 1.15 3 0.75 - 0.85 or 1.15 - 1.25 1 <0.75 or >1.25 0

[0131] Speed accuracy refers to the measure of the ability of a millimeter-wave radar to detect the moving speed of a target, including lateral speed and longitudinal speed.

[0132] The calculation method of speed accuracy is as follows:

[0133] It is evaluated by calculating the difference between the driving speed of the test vehicle equipped with RTK equipment and the speed detected by the millimeter-wave radar. During the evaluation process, the test vehicle travels at different speeds, calculates the speed accuracy, and the calculation results are averaged and weighted and added. In this evaluation, the test vehicle travels at a constant speed in three types of speeds: low speed (30 km / h), medium speed (50 km / h), and high speed (70 km / h), and each type of speed is tested no less than 5 times, and the driving distance each time is no less than 150 m.

[0134] The calculation of longitudinal speed is expressed as:

[0135]

[0136] Y v = Y′ v / e

[0137] Among them, w represents the number of tests; e represents the maximum value of w; t represents the number of data frames generated during each test; s represents the maximum value of t; D vwt represents the longitudinal speed detected by the vehicle RTK at the data frame t of the wth test; E vwt represents the longitudinal speed synchronously detected by the millimeter-wave radar at the data frame t of the wth test; Y′ v represents the number of times when the error is less than 0.1 km / h; v has no specific meaning; Y vw represents the longitudinal moving speed index of the target detected by the millimeter-wave radar;

[0138] The calculation of lateral speed is expressed as:

[0139]

[0140] Y h = Y' h / e

[0141] where h has no specific meaning; G hwt represents the longitudinal speed detected by the vehicle RTK at the w-th test data frame t; H hwt represents the longitudinal speed synchronously detected by the millimeter-wave radar at the w-th test data frame t; Y' h represents the number of times when the error is less than 0.1 km / h; Y hw represents the index of the lateral moving speed of the target detected by the millimeter-wave radar.

[0142] During the current speed evaluation test in this embodiment, the RTK frequency is adjusted to 50 hz.

[0143] The scoring criteria for speed accuracy are shown in Table 6 below:

[0144] Table 6

[0145]

[0146]

[0147] The accuracy of the longitude and latitude position refers to the measurement of the ability of the millimeter-wave radar to detect the target position accurately. The accuracy of the millimeter-wave radar in detecting the position of dynamic targets is evaluated this time.

[0148] The calculation method of the longitude and latitude position accuracy is as follows:

[0149] Drive the test vehicle equipped with the RTK device into the detection range of the millimeter-wave radar, and compare the longitude and latitude of the position collected by the RTK with the longitude and latitude of the vehicle detected by the millimeter-wave radar to measure the accuracy of the longitude and latitude position of the millimeter-wave radar. During the evaluation process, the number of driving times is not less than 5 times, and the driving distance each time is not less than 200 m. The calculation results of the indicators are averaged and weighted and added, which is expressed as:

[0150]

[0151] where g represents the data frame number detected by the millimeter-wave radar during the test process; f represents the maximum value of g; lon kg 、lon lg represent the longitudes collected by the RTK and the millimeter-wave radar at data frame g respectively; lat kg 、lat lg represent the latitudes collected by the RTK and the millimeter-wave radar at data frame g respectively.

[0152] The scoring criteria for the longitude and latitude position accuracy are shown in Table 7:

[0153] Table 7

[0154]

[0155]

[0156] The accuracy of the mileage stake number refers to the measurement of the accuracy of converting the position of the target detected by the millimeter-wave radar into the road stake number.

[0157] The calculation method of the accuracy of the mileage stake number is as follows:

[0158] Evaluate by driving a vehicle equipped with an RTK device within the detection range of the millimeter-wave radar and comparing the mileage stake number of the test vehicle at different times with the mileage stake number of the test vehicle detected by the millimeter-wave radar synchronously, which is expressed as:

[0159]

[0160] Among them, I g represents the mileage stake number of the vehicle's location collected by RTK; J g represents the mileage stake number of the vehicle's location detected by the millimeter-wave radar; Z 1 represents the calculation result of the accuracy of the mileage stake number.

[0161] In this embodiment, the process of obtaining the mileage stake number of the test vehicle is as Figure 2 shown. First, select the mileage stake number to be tested, then hold the RTK to measure the longitude and latitude of the location of the stake number, then record the time when the RTK test vehicle with the installation head passes through this stake number, and finally convert the mileage stake number of different time positions according to the driving distance after the test vehicle passes through this stake number. Among them, the conversion of the mileage stake number to the calculation value method: K1381 + 520 = 1381520.

[0162] The scoring standard for the accuracy of the mileage stake number is shown in Table 8 below:

[0163] Table 8

[0164] Index value Score ＜1 5 1～5 3 5～10 1 ＞10 0

[0165] The lane position accuracy refers to the measurement of the accuracy of the lane position of the vehicle detected by the millimeter-wave radar and the relative position of the lane where the vehicle is located. In this evaluation, the accuracy evaluation includes the process of vehicle lane change and non-lane change; the accuracy of the relative position of the lane is only evaluated for vehicles that have not changed lanes.

[0166] The calculation method of the lane position accuracy is as follows:

[0167] Calculate the index values of the vehicle when it changes lanes and does not change lanes respectively, and the calculation results are averaged and weighted and added, which is expressed as:

[0168]

[0169]

[0170] Z 5 = 0.3Z 3 + 0.7Z 4

[0171] Where x represents the serial number of the trajectory where the millimeter-wave radar detects no lane change and it is verified by synchronous video comparison that there is actually no lane change; u represents the maximum value of x; O x represents the number of data frames where the lane number of trajectory x is correct and the lane relative position detection is all correct; P x represents the number of frames of trajectory x; z represents the serial number of the trajectory where the millimeter-wave radar detects a lane change and it is verified by synchronous video comparison that there is actually a lane change; y represents the maximum value of z; O z represents the number of data frames where the lane number of trajectory z is correct and the lane relative position detection is all correct; P z represents the number of frames of trajectory z; Z 5 represents the calculation result of the lane relative position; Z 3 represents the index value when the vehicle does not change lanes; Z 4 represents the index value when the vehicle changes lanes.

[0172] The scoring criteria for lane position accuracy are shown in Table 9 below:

[0173] Table 9

[0174] Index value Score Greater than 0.90 10 0.85～0.90 6 0.8～0.85 4 ＜0.8 2

[0175] Vehicle type accuracy refers to the measurement of the ability of the millimeter-wave radar to recognize the target contour.

[0176] The calculation method of vehicle type accuracy is as follows:

[0177] Through video manual comparison, select the millimeter-wave radar detection data corresponding to different vehicle types. The number of trajectories for each vehicle type is not less than 200. Calculate the indicators in the following ways respectively, and the calculation results are averaged and added with equal weights, expressed as:

[0178]

[0179] Where r represents the serial number of the vehicle type category; q represents the maximum value of r, and in this evaluation q = 10; M r represents the number of trajectories where the vehicle type is r and it is recognized as vehicle type r; N represents the number of trajectories of vehicle type r; Z 2 represents the calculation result of the vehicle type accuracy.

[0180] The scoring criteria for vehicle type accuracy are shown in Table 10 below:

[0181] Table 10

[0182] Index value Score 0.90～1 10 0.80～0.90 6 0.70～0.80 2 ＜0.70 0

[0183] In the timeliness calculation module 13, the adjustability of the acquisition frequency corresponding to the millimeter-wave radar and the timeliness of data upload are calculated to obtain a third calculation result.

[0184] Timeliness represents the degree of describing the data upload time and the actual acquisition time.

[0185] The adjustability of the acquisition frequency refers to the measurement of the adjustability of the acquisition frequency of the millimeter-wave radar.

[0186] The calculation method of the adjustability of the acquisition frequency is as follows:

[0187] U = V / W

[0188] Wherein, V represents the number of acquisition frequencies that meet the data usage requirements and can be achieved by the millimeter-wave radar; W represents the number of acquisition frequencies required for data usage. The acquisition frequencies required for this evaluation include but are not limited to the following frequencies: 0.1Hz, 1Hz, 2Hz, 5Hz, 10Hz; U represents the timeliness calculation result of the adjustability of the acquisition frequency.

[0189] The scoring criteria for the adjustability of the acquisition frequency are shown in Table 11 below:

[0190] Table 11

[0191] Index value Score 0.8～1 5 0.6～0.8 2 ＜0.6 0

[0192] The timeliness of data upload refers to the measurement of the delay in data acquisition and recording of the millimeter-wave radar.

[0193] The calculation method of the timeliness of data upload is as follows:

[0194] It is evaluated by calculating the difference between the data acquisition time of the test vehicle equipped with the RTK device and the synchronous detection and recording time of the millimeter-wave radar, which is expressed as:

[0195]

[0196] Wherein, w represents the number of tests, and e represents the maximum value of w; g represents the frame number of each test data, and f represents the maximum value of g; K wg represents the RTK device recording time of the data frame g in the w-th test; Q wg represents the synchronous recording time of the millimeter-wave radar of the data frame g in the w-th test.

[0197] In this embodiment, before the RTK test, it is necessary to synchronize the clocks of the RTK device and the roadside millimeter-wave radar device.

[0198] The scoring criteria for the timeliness of data upload are shown in Table 12 below:

[0199] Table 12

[0200] Index value Score ＜1s 5 1s - 3s 2 ＞3s 0

[0201] In the statistical module 14, the first calculation result, the second calculation result, and the third calculation result are statistically analyzed to obtain the performance evaluation result of the millimeter-wave radar.

[0202] In this embodiment, each data is evaluated according to the above method, and then the performance evaluation result of the millimeter-wave radar is obtained according to the evaluation detail list; the evaluation detail list is shown in Table 13 below:

[0203] Table 13

[0204]

[0205]

[0206]

[0207] In this embodiment, according to the requirements of the data rules, the integrity of the roadside millimeter-wave radar is evaluated by detecting the degree to which the target is assigned a value, the accuracy of the roadside millimeter-wave radar is evaluated by the degree to which the data describes the true value of the actual object, and the timeliness of the roadside millimeter-wave radar is evaluated by the degree between the data upload time and the actual acquisition time, thereby improving the performance of the roadside millimeter-wave radar.

[0208] Embodiment 2

[0209] This embodiment provides a method for evaluating the performance of a millimeter-wave radar, including:

[0210] S1. Calculate the integrity of the fields, content, and detection distance corresponding to the millimeter-wave radar to obtain a first calculation result;

[0211] S2. Calculate the accuracy of the target, traffic flow, speed, longitude and latitude position, mileage stake number, lane position, and vehicle type corresponding to the millimeter-wave radar to obtain a second calculation result;

[0212] S3. Calculate the adjustable acquisition frequency and timeliness of data upload corresponding to the millimeter-wave radar to obtain a third calculation result;

[0213] S4. Statistically analyze the first calculation result, the second calculation result, and the third calculation result to obtain the performance evaluation result of the millimeter-wave radar.

[0214] It should be noted that the method for evaluating the performance of a millimeter-wave radar provided in this embodiment is similar to that in Embodiment 1, and will not be elaborated here.

[0215] In this embodiment, according to the requirements of data rules, the integrity of the roadside millimeter-wave radar is evaluated by detecting the degree to which the target is assigned a numerical value, the accuracy of the roadside millimeter-wave radar is evaluated by the degree to which the data describes the true value of the actual object, and the timeliness of the roadside millimeter-wave radar is evaluated by the degree between the data upload time and the actual acquisition time, thereby improving the performance evaluation of the roadside millimeter-wave radar.

[0216] Note that the above is only a preferred embodiment of the present invention and the applied technical principles. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein. Various obvious changes, re-adjustments, and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments. Without departing from the concept of the present invention, more other equivalent embodiments can be included, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A millimeter-wave radar performance evaluation system, characterized in that, it includes: An integrity calculation module, which is used to calculate the integrity of the fields, content, and detection distance corresponding to the millimeter-wave radar to obtain a first calculation result; An accuracy calculation module, which is used to calculate the accuracy of the targets, traffic flow, speed, longitude and latitude position, mileage stake number, lane position, and vehicle type corresponding to the millimeter-wave radar to obtain a second calculation result; A timeliness calculation module, which is used to calculate the adjustable acquisition frequency and data upload timeliness corresponding to the millimeter-wave radar to obtain a third calculation result; A statistics module, which is used to statistically analyze the first calculation result, the second calculation result, and the third calculation result to obtain a millimeter-wave radar performance evaluation result; In the integrity calculation module, the calculation of the integrity of the content corresponding to the millimeter-wave radar is specifically to calculate the data content repair quality and missing degree of the millimeter-wave radar, which is expressed as: Among them, case 1 means that the vehicle ID in the millimeter-wave radar repair data is correct; case 2 means that the vehicle ID in the millimeter-wave radar repair data is incorrect; Among them, j represents the field serial number; m represents the maximum value of the field serial number; i represents the serial number of the millimeter-wave radar repair data frame; n represents the maximum value of the serial number of the millimeter-wave radar repair data frame; A ij represents the frame coefficient; X represents the index value of the repair quality and missing degree of the data content by the millimeter-wave radar; X 1 represents the content integrity weighting coefficient.

2. A millimeter-wave radar performance evaluation system according to claim 1, characterized in that, The fields in the integrity calculation module include target ID, timestamp, longitude and latitude, mileage stake number, lane where it is located, relative position of the lane where it is located, vehicle speed, and vehicle type.

3. A millimeter-wave radar performance evaluation system according to claim 2, characterized in that, In the integrity calculation module, the calculation of the integrity of the fields corresponding to the millimeter-wave radar is specifically to calculate whether the fields in the data meet the data usage requirements, which is expressed as: X = A / B Among them, X represents the calculation result of the integrity of the field; A represents the number of fields that meet the usage requirements in the data; B represents the total number of required fields.

4. A millimeter-wave radar performance evaluation system according to claim 2, characterized in that, In the integrity calculation module, the calculation of the integrity of the detection distance corresponding to the millimeter-wave radar is specifically to calculate the millimeter-wave radar's ability to detect targets and continuously track them, which is expressed as: Among them, o represents the number of trajectories; L p represents the detection length of trajectory p; L represents the detection distance requirement; X ave represents the detection ability of the millimeter-wave radar; L max = C Among them, C represents the maximum distance that the detection range of the millimeter-wave radar can reach; L max represents the maximum distance that the detection range of the millimeter-wave radar can reach.

5. A millimeter-wave radar performance evaluation system according to claim 1, characterized in that, In the accuracy calculation module, the calculation of the accuracy of the targets corresponding to the millimeter-wave radar is specifically to calculate the millimeter-wave radar's ability to lock targets, which is expressed as: Y = S / T Among them, S represents the number of vehicles in the data that are not represented by different IDs for the same vehicle; T represents the number of actual passing vehicles in the data; Y represents the calculation result of the accuracy of the target; In the accuracy calculation module, the calculation of the accuracy of the traffic flow corresponding to the millimeter-wave radar is specifically to calculate the millimeter-wave radar's ability to detect the number of multiple targets, which is expressed as: Among them, d represents the cross-section number, and b represents the maximum value of the cross-section number; c represents the lane number, and a represents the maximum value of the lane number; S dc represents the number of detected vehicles in cross-section d of lane c; T dc represents the actual number of vehicles passing through in cross-section d of lane c; In the accuracy calculation module, the calculation of the accuracy of the speed corresponding to the millimeter-wave radar is specifically to calculate the millimeter-wave radar's ability to detect the moving speed of the target, where the speed includes longitudinal speed and lateral speed; The longitudinal speed calculation is expressed as: Among them, w represents the number of times; t represents the number of data frames generated each time; s represents the maximum value of t; D vwt represents the longitudinal speed detected by the vehicle base station at the w-th data frame t; E vwt represents the longitudinal speed synchronously detected by the millimeter-wave radar at the w-th data frame t; Y vw represents the longitudinal moving speed index of the target detected by the millimeter-wave radar; The lateral speed calculation is expressed as: Among them, G hwt represents the longitudinal speed detected by the vehicle base station at the w-th data frame t; H hwt represents the longitudinal speed synchronously detected by the millimeter-wave radar at the w-th data frame t; Y hw represents the target lateral movement speed index detected by the millimeter-wave radar.

6. A millimeter-wave radar performance evaluation system according to claim 5, characterized in that, In the accuracy calculation module, the calculation of the accuracy of the longitude and latitude position corresponding to the millimeter-wave radar is specifically the calculation of the ability of the millimeter-wave radar to detect the target position accuracy, which is expressed as: where g represents the data frame sequence number detected by the millimeter-wave radar; f represents the maximum value of g; lon kg and lon lg represent the longitudes collected by the base station and the millimeter-wave radar at data frame g, respectively; lat kg and lat lg represent the latitudes collected by the base station and the millimeter-wave radar at data frame g, respectively. In the accuracy calculation module, the calculation of the accuracy of the mileage stake number corresponding to the millimeter-wave radar is specifically the calculation of converting the target position detected by the millimeter-wave radar into the road stake number, which is expressed as: Among them, I g represents the mileage stake number of the vehicle's location collected by the base station; J g represents the mileage stake number of the vehicle's location detected by the millimeter-wave radar; Z 1 represents the accuracy calculation result of the mileage stake number; In the accuracy calculation module, the calculation of the accuracy of the vehicle type corresponding to the millimeter-wave radar is specifically the calculation of the ability of the millimeter-wave radar to recognize the target contour, which is expressed as: Among them, r represents the serial number of the vehicle model category; q represents the maximum value of r; M r represents that the vehicle model is r and the number of trajectories recognized as vehicle model r; N r represents the number of trajectories of vehicle model r; Z 2 represents the calculation result of the accuracy of the vehicle model.

7. A millimeter-wave radar performance evaluation system according to claim 6, characterized in that In the accuracy calculation module, the calculation of the accuracy of the lane position corresponding to the millimeter-wave radar is specifically the calculation of the vehicle lane position detected by the millimeter-wave radar and the relative position of the lane where it is located, which is expressed as: Z 5 = 0.3Z 3 + 0.7Z 4 Among them, x represents the serial number of the trajectory number where the millimeter-wave radar detects no lane change and it is verified by synchronous video comparison that there is actually no lane change; u represents the maximum value of x; O x represents the number of data frames where the lane number of trajectory x is correct and the lane relative position detection is all correct; P x represents the number of frames of trajectory x; z represents the serial number of the trajectory number where the millimeter-wave radar detects a lane change and it is verified by synchronous video comparison that there is actually a lane change; y represents the maximum value of z; O z represents the number of data frames where the lane number of trajectory z is correct and the lane relative position detection is all correct; P z represents the number of frames of trajectory z; Z 5 represents the calculation result of the lane relative position; Z 3 represents the index value when the vehicle does not change lanes; Z 4 represents the index value when the vehicle changes lanes.

8. A millimeter-wave radar performance evaluation system according to claim 1, characterized in that In the timeliness calculation module, the calculation of the timeliness of the adjustable acquisition frequency corresponding to the millimeter-wave radar is specifically the calculation of the adjustable acquisition frequency of the millimeter-wave radar, which is expressed as: U = V / W Wherein, V represents the number of acquisition frequencies that the millimeter-wave radar can achieve; W represents the acquisition frequency required by the data usage; U represents the calculation result of the timeliness of the adjustable acquisition frequency; In the timeliness calculation module, the calculation of the timeliness of data upload corresponding to the millimeter-wave radar is specifically the calculation of the time delay of data acquisition and recording by the millimeter-wave radar, which is expressed as: Among them, w represents the number of times, e represents the maximum value of w; g represents the frame number of the data, f represents the maximum value of g; K wg represents the recording time of the base station equipment of the data frame g in the w-th test; Q wg represents the synchronous recording time of the millimeter-wave radar of the data frame g in the w-th test.

9. A millimeter-wave radar performance evaluation method, characterized in that A millimeter-wave radar performance evaluation system according to any one of claims 1-8 is adopted, and the method steps include: S1. Calculate the integrity of the fields, content, and detection distance corresponding to the millimeter-wave radar to obtain a first calculation result; S2. Calculate the accuracy of the target, traffic flow, speed, longitude and latitude position, mileage stake number, lane position, and vehicle type corresponding to the millimeter-wave radar to obtain a second calculation result; S3. Calculate the adjustable acquisition frequency and the timeliness of data upload corresponding to the millimeter-wave radar to obtain a third calculation result; S4. Statistically analyze the first calculation result, the second calculation result, and the third calculation result to obtain the millimeter-wave radar performance evaluation result.

Images