Detection method, device and equipment of roadside perception system and computer storage medium

Abstract

This application discloses a method, apparatus, device, and computer storage medium for testing a roadside sensing system. The method includes: acquiring first detection data and first cost data of the roadside sensing system under test, and acquiring second detection data for calibrating the roadside sensing system; calculating the first and second detection data to obtain a first index value corresponding to a preset first evaluation index, wherein the first evaluation index includes at least one of a data detection quality index of the roadside sensing system under test and a system performance test index; determining a second index value corresponding to a preset system cost index based on the first cost data and pre-acquired reference cost data; and determining the test result of the roadside sensing system under test based on the first and second index values. Based on this, the roadside sensing system can be tested from multiple aspects, obtaining comprehensive and accurate test results.

Description

Technical Field

[0001] This application belongs to the field of roadside sensing equipment technology, and particularly relates to a detection method, device, equipment and computer storage medium for a roadside sensing system. Background Technology

[0002] Roadside perception systems utilize various sensors, such as visual sensors, millimeter-wave radar, and lidar, combined with edge computing devices to monitor current road conditions in real time. Furthermore, based on this data, roadside perception systems can provide drivers with real-time information about the road environment and travel safety warnings, as well as provide relevant traffic departments with monitoring and prediction of road traffic conditions.

[0003] Currently, roadside perception systems are being applied in various scenarios such as intelligent transportation, traffic monitoring, and traffic management. Different application scenarios have different requirements for roadside perception systems. Existing methods for detecting the quality of roadside perception systems yield results that are rather one-sided, incomplete, and inaccurate, with relatively low reference value. Summary of the Invention

[0004] This application provides a detection method, apparatus, device, and computer storage medium for a roadside sensing system. It detects the roadside sensing system from multiple aspects, obtains comprehensive and accurate detection results, and can provide users with detection results of high reference value.

[0005] In a first aspect, embodiments of this application provide a detection method for a roadside sensing system, characterized in that the method includes:

[0006] Acquire first detection data and first cost data of the roadside perception system under test, and acquire second detection data of the roadside perception system under calibration, wherein the first detection data and the second detection data are respectively the detection data of the roadside perception system under test and the roadside perception system under calibration for the target detection objects on the road, and / or the performance test data of the target system performance of the roadside perception system under test and the roadside perception system under calibration, respectively.

[0007] The first detection data and the second detection data are calculated to obtain a first index value corresponding to a preset first evaluation index. The first evaluation index includes at least one of the data detection quality index of the roadside perception system under test and the performance test index of the roadside perception system under test.

[0008] Based on the first cost data and the pre-acquired reference cost data, determine the second indicator value corresponding to the preset system cost indicator;

[0009] The detection result of the roadside sensing system under test is determined based on the first index value and the second index value.

[0010] Secondly, embodiments of this application provide a detection device for a roadside sensing system, characterized in that the device comprises:

[0011] The data acquisition module is used to acquire first detection data and first cost data of the roadside perception system under test, and to acquire second detection data of the roadside perception system under calibration. The first detection data and the second detection data are respectively the detection data of the roadside perception system under test and the roadside perception system under calibration for the target detection objects on the road, and / or the performance test data of the target system performance of the roadside perception system under test and the roadside perception system under calibration.

[0012] The first calculation module is used to calculate the first detection data and the second detection data to obtain the first index value corresponding to the preset first evaluation index. The first evaluation index includes at least one of the data detection quality index of the roadside perception system under test and the performance test index of the roadside perception system under test.

[0013] The second calculation module is used to determine the second indicator value corresponding to the preset system cost indicator based on the first cost data and the pre-acquired reference cost data.

[0014] The detection result determination module is used to determine the detection result of the roadside sensing system under test based on the first index value and the second index value.

[0015] Thirdly, embodiments of this application provide a detection device for a roadside sensing system, characterized in that the device includes: a processor and a memory storing computer program instructions;

[0016] When the processor executes the computer program instructions, it implements the detection method of the roadside perception system as described in any of the above embodiments.

[0017] Fourthly, embodiments of this application provide a computer storage medium storing computer program instructions, which, when executed by a processor, implement the detection method of the roadside perception system described in any of the above embodiments.

[0018] The detection method, apparatus, device, and computer storage medium of the roadside perception system in this application embodiment calculate a first index value corresponding to the data detection quality index and / or performance test index of the roadside perception system under test using first detection data of the roadside perception system under test and second detection data of the calibrated roadside perception system. It also calculates a second index value corresponding to the system cost index using first cost data of the roadside perception system under test and pre-acquired reference cost data. Finally, based on the first and second index values, the detection result of the roadside perception system under test is determined. Thus, through the method of this application embodiment, the first detection data includes detection data of the roadside perception system on the target detection object and / or performance test data for testing the performance of the roadside perception system. Combined with the acquired system cost data, the system is tested not only from the perspective of system performance or detection data quality but also from an economic perspective, considering the system cost of the roadside perception system. This ensures that the obtained detection results are more comprehensive and accurate, providing higher reference value for users. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic flowchart of a detection method for a roadside sensing system provided in one embodiment of this application;

[0021] Figure 2 This is a schematic diagram of the detection device of a roadside sensing system provided in another embodiment of this application;

[0022] Figure 3 This is a schematic diagram of the structure of the detection device of the roadside sensing system provided in another embodiment of this application. Detailed Implementation

[0023] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0024] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0025] To facilitate a detailed explanation of the technical solution of the detection method of the roadside sensing system of this application, a brief introduction to the roadside sensing system will be given below.

[0026] Roadside perception systems utilize a variety of sensors, such as visual sensors, millimeter-wave radar, and lidar, combined with multi-access edge computing (MEC) devices, to detect the location, movement status, and road conditions of current road traffic participants in real time. Through vehicle-to-everything (V2X) technology, information exchange and command control between vehicles, people, roads, and the cloud are achieved according to agreed communication protocols and data interaction standards.

[0027] Moreover, roadside perception systems require comprehensive, high-quality, and stable traffic data from roadside perception devices. These devices mainly include radar, cameras, traffic lights and signs, smart cones, and environmental sensors. Among these, the information from traffic lights and signs, smart cones, and environmental sensors is relatively standardized. Cameras have become part of the roadside infrastructure, radar and video fusion (radar-visual) is increasingly becoming standard roadside perception units, and lidar is also being applied to roadside perception systems.

[0028] Therefore, roadside perception systems can provide drivers with real-time information on the road environment and warnings related to travel safety, such as road construction ahead, congestion alerts ahead, pedestrian / motor vehicle collision warnings, and traffic accident alerts; they can also provide relevant traffic departments with monitoring and prediction of the road traffic environment, such as traffic flow statistics, vehicle illegal parking detection, and section speed measurement.

[0029] Based on the aforementioned functions of roadside sensing systems, various roadside sensing systems exist and are gradually being applied in real-world scenarios, including intelligent transportation, traffic monitoring, and traffic management. However, there is no comprehensive testing and evaluation method for these roadside sensing systems. This prevents users from selecting the appropriate roadside sensing system based on their needs. Furthermore, the test results are often incomplete, hindering technical engineers from understanding the shortcomings of existing roadside sensing systems and making targeted improvements.

[0030] Therefore, in view of the above-mentioned technical problems, the embodiments of this application provide a detection method for a roadside perception system, which can detect the roadside perception system from multiple aspects, obtain comprehensive and accurate detection results, and thus provide users with detection results with high reference value.

[0031] Figure 1 A flowchart illustrating a detection method for a roadside sensing system is shown. The detection method may include the following steps:

[0032] Step 101: Obtain first detection data and first cost data of the roadside perception system under test, and obtain second detection data of the calibrated roadside perception system, wherein the first detection data and the second detection data are respectively the detection data of the roadside perception system under test and the calibrated roadside perception system for the target detection object on the road, and / or the performance test data of the target system performance of the roadside perception system under test and the calibrated roadside perception system, respectively.

[0033] Step 102: Calculate the first detection data and the second detection data to obtain the first index value corresponding to the preset first evaluation index. The first evaluation index includes at least one of the data detection quality index of the roadside perception system under test and the performance test index of the roadside perception system under test.

[0034] Step 103: Based on the first cost data and the pre-acquired reference cost data, determine the second indicator value corresponding to the preset system cost indicator;

[0035] Step 104: Determine the detection result of the roadside sensing system under test based on the first index value and the second index value.

[0036] Based on this, using the first detection data of the roadside perception system under test and the second detection data of the calibrated roadside perception system, a first index value corresponding to the data detection quality index and / or performance test index of the roadside perception system under test is calculated. Similarly, using the first cost data of the roadside perception system under test and pre-acquired reference cost data, a second index value corresponding to the system cost index is calculated. Then, based on the first and second index values, the detection result of the roadside perception system under test is determined. Thus, through the method of this application embodiment, the first detection data includes the detection data of the roadside perception system on the target detection object and / or the performance test data of the roadside perception system. Combined with the acquired system cost data, the system is tested not only from the perspective of system performance or the quality of the detected data but also from an economic perspective, considering the system cost of the roadside perception system. This ensures that the obtained detection results are more comprehensive and accurate, and have higher reference value for users.

[0037] In step 101 above, the first detection data and the first cost data of the roadside sensing system under test are obtained, as well as the second detection data of the roadside sensing system are calibrated.

[0038] The first and second detection data mentioned above can be understood as detection data obtained by the roadside sensing system under test and the calibrated roadside sensing system simultaneously detecting the same detection object on the road, and / or data obtained by simultaneously detecting the same performance of the roadside sensing system under test and the calibrated roadside sensing system. That is, the first and second detection data have a one-to-one correspondence.

[0039] In this embodiment of the application, the target detection object can be a motor vehicle, non-motor vehicle, or pedestrian traveling on the road, etc.; the target system performance can be a certain performance of the roadside perception system itself, such as the detection range, response time, data frequency, maximum detection quantity index, minimum identifiable size index, or environmental adaptability of the roadside perception system, etc.

[0040] Based on the above description, the first detection data and the second detection data may include data such as the size, position, speed, and heading angle of the target detection object; or may include data such as the category, number, and number of successfully tracked target detection objects; or may include performance data such as the detection range, response time, data frequency, maximum detection quantity, minimum identifiable size, or environmental adaptability of the roadside perception system.

[0041] For example, the roadside perception system under test and the roadside perception system under calibration can simultaneously detect the speed of car A (i.e., the target detection object) on the road, and obtain speed data V1 (i.e., the first detection data) and speed data V2 (i.e., the second detection data); or, the roadside perception system under test and the roadside perception system under calibration can simultaneously test the range they can radiate (i.e., the target system performance), and obtain range R1 (i.e., the first detection data) and range R2 (i.e., the second detection data).

[0042] It should be noted that the aforementioned roadside perception system calibration is a pre-built roadside perception system. To ensure the accuracy and reliability of the detection data from the calibration roadside perception system, high-precision sensors such as cameras / AI cameras, fisheye cameras, millimeter-wave radar, LiDAR, or integrated radar-visual sensors can be selected when building the calibration roadside perception system. Moreover, when fusion algorithms are involved, high-precision intelligent algorithms and high-performance edge computing units should also be selected. Since the selection of sensors and the construction of the calibration roadside perception system can be found in existing technologies, this application will not elaborate further.

[0043] In step 102 above, the first detection data and the second detection data obtained in step 101 are calculated to obtain the first indicator value corresponding to the pre-set first evaluation indicator.

[0044] The aforementioned first evaluation index may include at least one of the data detection quality index of the roadside perception system under test and the system performance test index. Specifically, the index value of the data detection quality index of the roadside perception system under test can be calculated based on the detection data of the target detection objects on the road by the roadside perception system under test and the calibration roadside perception system; or, the index value of the performance test index of the roadside perception system under test can be calculated based on the performance test data of the target system performance of the roadside perception system under test and the calibration roadside perception system.

[0045] In addition, in order to detect and evaluate the roadside sensing system under test more accurately and comprehensively, the above-mentioned data detection quality indicators can be further subdivided, that is, the above-mentioned data detection quality indicators can also include at least one of data precision and data accuracy.

[0046] The accuracy of the aforementioned data can be reflected in indicators such as the size accuracy of the target object, the positioning accuracy of the target object, the speed accuracy of the target object, and the heading angle accuracy of the target object; the accuracy of the aforementioned data can be reflected in indicators such as the classification accuracy rate of the target object, the detection accuracy rate of the target object, and the tracking success rate of the target object.

[0047] Similarly, in order to more accurately and comprehensively detect and evaluate the roadside sensing system under test, the performance test indicators of the roadside sensing system under test may include the detection range, response time, data frequency, maximum number of detections, minimum identifiable size, and environmental adaptability of the roadside sensing system under test.

[0048] It should be noted that the above calculation of the first indicator value corresponding to the first evaluation indicator can be based on multiple preset formulas. The corresponding first and second detection data are substituted into the corresponding formulas to calculate the score rate corresponding to the first evaluation indicator. Then, the preset full score (e.g., 100 points) and the corresponding score rate are multiplied to obtain the first indicator value corresponding to the first evaluation indicator.

[0049] For example, the score rate corresponding to the first evaluation indicator can be calculated using the following formula.

[0050] (1) Data accuracy

[0051] A1: Dimensional accuracy, which can be used to indicate the accuracy of the detection results for the length, width, and height of the target object.

[0052] Taking length as an example, the target detection object size accuracy, i.e., the score rate, can be calculated using the following formula 1:

[0053]

[0054] Where L1 is the length data detected by the roadside sensing system under test, and L is the length data detected by the calibrated roadside sensing system.

[0055] B1: Positioning accuracy, which can be used to indicate the accuracy of the detection results in the three dimensions of the target object: horizontal, vertical, and height.

[0056] Taking the horizontal direction as an example, the target detection object localization accuracy, i.e., the score rate, can be calculated using the following formula 2:

[0057]

[0058] Where X1 is the lateral position data detected by the roadside sensing system under test, and X is the lateral position data detected by the calibrated roadside sensing system.

[0059] C1: Speed ​​accuracy, which can be used to indicate the accuracy of the detection results for the speed of the moving object being detected.

[0060] Specifically, the target detection speed accuracy, i.e., the score rate, can be calculated using the following formula 3:

[0061]

[0062] Where V1 is the speed data detected by the roadside sensing system under test, and V is the speed data detected by the calibrated roadside sensing system.

[0063] D1: Heading angle accuracy, which can be used to indicate the accuracy of the detection result of the heading angle of the target object.

[0064] Specifically, the target detection speed accuracy, i.e., the score rate, can be calculated using the following formula 4:

[0065]

[0066] Wherein, H1 is the heading angle data detected by the roadside sensing system under test, and H is the heading angle data detected by the calibrated roadside sensing system.

[0067] (2) Data accuracy

[0068] A2: Classification accuracy, which indicates the accuracy with which the roadside sensing system under test classifies the target objects. These target objects include, but are not limited to, motor vehicles (such as cars, trucks, buses, emergency vehicles, special vehicles, or motorcycles), non-motorized vehicles, pedestrians, or people in wheelchairs.

[0069] Specifically, the classification accuracy, or score rate, of the target detection object can be calculated using the following formula 5:

[0070]

[0071] Where detaC represents the number of discrepancies between the classification results of the roadside perception system under test and the classification results of the roadside perception system under calibration, and C represents the total number of target objects detected by the roadside perception system under calibration.

[0072] B2: Detection accuracy, which can include the detection rate, false negative rate, and false positive rate of the target object.

[0073] Taking pedestrian detection rate as an example, the detection rate of the target object, i.e., the score rate, can be calculated using the following formula 6:

[0074]

[0075] Where N1 is the total number of pedestrians detected by the roadside sensing system under test, and N is the total number of pedestrians detected by the calibrated roadside sensing system.

[0076] Taking the pedestrian missed detection rate as an example, if N1 < N, that is, the total number of pedestrians detected by the to-be-tested roadside perception system is less than the total number of pedestrians detected by the calibrated roadside perception system, the missed detection rate of the target detection object can be calculated by the following formula 7:

[0077]

[0078] Its scoring rate can be:

[0079] Taking the pedestrian false detection rate as an example, the false detection rate of the target detection object can be calculated by the following formula 8:

[0080]

[0081] Where N2 is the number of pedestrians detected as other target detection objects by the to-be-tested roadside perception system or the number of other target detection objects detected as pedestrians, and N is the total number of pedestrians detected by the calibrated roadside perception system.

[0082] Its scoring rate can be:

[0083] C2: Tracking success rate, which can be used to indicate the probability that the to-be-tested roadside perception system can correctly track the target detection object and obtain the required data when tracking the target detection object.

[0084] Specifically, the tracking success rate of the target detection object, that is, the scoring rate, can be calculated by the following formula 9:

[0085]

[0086] Where N3 is the number of target detection objects successfully tracked by the to-be-tested roadside perception system, and N is the number of target detection objects successfully tracked by the calibrated roadside perception system.

[0087] (3) System performance test indicators

[0088] A3: Detection range indicator, which can be used to indicate the range that each sensor of the roadside perception system and the V2X communication device can detect on the premise of ensuring data quality.

[0089] Specifically, the scoring rate of the detection range indicator of the to-be-tested roadside perception system can be calculated by the following formula 10:

[0090]

[0091] Where R1 is the range data that the to-be-tested roadside perception system can radiate, and R is the range data that the calibrated roadside perception system can radiate.

[0092] B3: Response time metric, which can be used to indicate the time it takes for a roadside sensing system to respond to an input or request, and must also ensure that the roadside sensing system under test can respond in a timely manner even when the number of requests reaches a certain value.

[0093] Specifically, the response time index of the roadside sensing system under test can be determined in the following way:

[0094] When the response time of the roadside sensing system under test is the same as that of the calibrated roadside sensing system, the response time index can be recorded as full marks (e.g., 100 points); when the response time of the roadside sensing system under test is more than twice the response time of the calibrated roadside sensing system, the response time index can be recorded as 0 points; when the response time of the roadside sensing system under test is less than twice the response time of the calibrated roadside sensing system, the score rate of the response time index can be calculated according to the following formula 11:

[0095]

[0096] Where T1 is the response time of the roadside sensing system under test, and T is the response time of the calibrated roadside sensing system. Therefore, if the response time of the roadside sensing system under test is less than twice the response time of the calibrated roadside sensing system, the response time index can be obtained by multiplying 100 points by the score rate of the minimum identifiable size index.

[0097] C3: Data frequency index, which can be used to indicate the number of times per second that the roadside sensing system generates and transmits data.

[0098] Specifically, the score rate of the data frequency index of the roadside sensing system under test can be calculated using the following formula 12 or formula 13:

[0099]

[0100] or

[0101] Wherein, n1 is the number of times the roadside sensing system under test generates and transmits data per second, n is the number of times the calibrated roadside sensing system generates and transmits data per second; f1 is the data frequency of the roadside sensing system under test per second, and f is the data frequency of the calibrated roadside sensing system per second.

[0102] D3: Maximum Detection Count Index, which can be used to indicate the maximum number of target objects that a roadside sensing system can detect simultaneously.

[0103] Specifically, the score rate of the maximum detection quantity index of the roadside sensing system under test can be calculated using the following formula 14:

[0104]

[0105] Wherein, MN1 is the maximum number of detections of the roadside sensing system under test, and MN is the maximum number of detections of the calibrated roadside sensing system.

[0106] E3: Minimum identifiable size index, which can be used to indicate the smallest size of a detected object that a roadside sensing system can identify.

[0107] Specifically, the minimum identifiable size index of the roadside sensing system under test can be determined in the following way:

[0108] When the minimum identifiable size of the roadside sensing system under test is the same as that of the calibrated roadside sensing system, the minimum identifiable size index can be scored as full marks (e.g., 100 points); when the minimum identifiable size of the roadside sensing system under test is greater than or equal to twice the minimum identifiable size of the calibrated roadside sensing system, the minimum identifiable size index can be scored as 0 points; when the minimum identifiable size of the roadside sensing system under test is less than twice the minimum identifiable size of the calibrated roadside sensing system, the score rate of the minimum identifiable size index can be calculated according to the following formula 15:

[0109]

[0110] Where S1 is the minimum identifiable size of the roadside sensing system under test, and S is the minimum identifiable size of the calibrated roadside sensing system. Then, under the condition that the minimum identifiable size of the roadside sensing system under test is less than twice the minimum identifiable size of the calibrated roadside sensing system, the score for the minimum identifiable size index is the product of 100 points and the score rate of the minimum identifiable size index.

[0111] F3: Environmental adaptability, which can be used to indicate that the roadside sensing system can still maintain good detection accuracy in adverse environments such as fog, haze, refraction, poor light, and night.

[0112] Taking foggy conditions as an example, environmental adaptability can be determined by calculating the average accuracy of the data detected by the roadside sensing system under test for the target object. Specifically, environmental adaptability can be calculated using the aforementioned method for calculating data accuracy.

[0113] In step 103 above, based on the first cost data of the roadside sensing system to be tested obtained in step 101 above and the reference cost data obtained in advance, the second index value corresponding to the preset system cost index can be determined.

[0114] The aforementioned pre-acquired reference cost data may include a reference average value and its corresponding first score, a reference maximum value and its corresponding second score, and a reference minimum value and its corresponding third score.

[0115] In this embodiment of the application, determining the second indicator value corresponding to the preset system cost indicator based on the first cost data and the pre-acquired reference cost data may include:

[0116] If the first cost data and the reference average value are the same, the second indicator value is determined to be the first score;

[0117] If the first cost data and the highest reference value are the same, the second indicator value is determined to be the second score;

[0118] If the first cost data and the reference minimum value are the same, the second indicator value is determined to be the third score;

[0119] When the first cost data is different from the reference average value, the reference maximum value, and the reference minimum value, the second indicator value is determined to be the fourth score according to a preset ratio.

[0120] Based on this, using the average, highest, and lowest values ​​of the reference cost as benchmarks, the first cost data of the roadside perception system under test can be more reasonably detected and evaluated, making the second indicator value corresponding to the determined system cost index more accurate.

[0121] It should be noted that the third score corresponding to the lowest reference value is greater than the first score corresponding to the average reference value, and the first score corresponding to the average reference value is greater than the second score corresponding to the highest reference value.

[0122] In addition, the above-mentioned preset ratio relationship can be the ratio relationship between the first cost data, the reference average value and its corresponding first score, or the ratio relationship between the first cost data, the reference maximum value and its corresponding second score, or the first cost data, the reference minimum value and its corresponding third score, or the ratio relationship can be determined according to the specific application scenario, without any limitation here.

[0123] For example, the reference average value and its corresponding first score can be set to 1000 yuan and 60 points; the reference highest value and its corresponding second score can be set to 1500 yuan and 30 points; and the reference lowest value and its corresponding third score can be set to 600 yuan and 100 points (i.e., full marks). Then, when the first cost data of the roadside sensing system under test is 600 yuan, the second indicator value can be determined to be 100 points; or, when the first cost data of the roadside sensing system under test is 1500 yuan, the second indicator value can be determined to be 30 points; or, when the first cost data of the roadside sensing system under test is 1000 yuan, the second indicator value can be determined to be 60 points; or, when the first cost data of the roadside sensing system under test is 800 yuan, since the cost data and its corresponding score are inversely proportional, the second indicator value can be calculated as 75 points based on the ratio of the reference average value and its corresponding first score.

[0124] In step 104 above, the detection result of the roadside perception system to be tested is determined based on the first index value obtained in step 102 and the second index value obtained in step 103. This can be achieved by weighting the first index value and the second index value according to the weights of the pre-set evaluation indexes to obtain the detection result of the roadside perception system to be tested.

[0125] It should be noted that the weights of the above evaluation indicators can be set according to the actual situation, and this application does not impose any restrictions on them.

[0126] For example, the weights of the above evaluation indicators can be set to the following values:

[0127] When the first evaluation metric includes data accuracy, data precision, and performance test metrics, the weight values ​​can be set as shown in Table 1.

[0128] Table 1. Weights of Evaluation Indicators (a)

[0129] Evaluation indicators Data accuracy Data accuracy Performance test metrics System cost indicators Weight 3 3 3 1

[0130] After determining the weight values ​​of each evaluation indicator, the next step is to determine the weights of each sub-evaluation indicator contained in each evaluation indicator, as shown in Tables 2-4 below.

[0131] Table 2. Weights of Sub-Evaluation Indicators for Data Accuracy

[0132] Sub-evaluation indicators Dimensional accuracy Positioning accuracy Heading angle accuracy Speed ​​accuracy Weight 1 / 4 1 / 4 1 / 4 1 / 4

[0133] Table 3. Weights of Sub-evaluation Indicators for Data Accuracy

[0134] Sub-evaluation indicators Classification accuracy Detection accuracy Tracking success rate Weight 1 / 3 1 / 3 1 / 3

[0135] Table 4. Weights of Sub-evaluation Indicators for Performance Testing Indicators

[0136]

[0137] In summary, the weights of the sub-evaluation indicators under each evaluation indicator are projected onto the overall detection weights, as shown in Table 5 below.

[0138] Table 5. Weights of Evaluation Indicators (b)

[0139] Evaluation indicators Weight Dimensional accuracy 0.075 Positioning accuracy 0.075 Heading angle accuracy 0.075 Speed ​​accuracy 0.075 Classification accuracy 0.1 Detection accuracy 0.1 Tracking success rate 0.1 Detection range 0.05 Response time 0.05 Data frequency 0.05 Maximum number of detections 0.05 Minimum recognizable size 0.05 Environmental adaptability 0.05 System cost 0.1 sum 1

[0140] Furthermore, as described above, dimensional accuracy refers to the accuracy of the detection results in the length, width, and height of the target object. Therefore, the dimensional accuracy index can be further refined, as shown in Table 6 below:

[0141] Table 6 Dimensional Weights for Dimensional Accuracy

[0142] Dimension long Width high Weight 1 / 3 1 / 3 1 / 3

[0143] Positioning accuracy is used to indicate the precision of the detection results in the three dimensions of the target object: horizontal, vertical, and height. Therefore, the positioning accuracy index can be further refined, as shown in Table 7 below:

[0144] Table 7. Dimensional Weights for Positioning Accuracy

[0145] Dimension Horizontal Vertical high Weight 1 / 3 1 / 3 1 / 3

[0146] Detection accuracy can include three dimensions: detection rate, false negative rate, and false positive rate of the target object. Therefore, the detection accuracy indicators can be further refined, as shown in Table 8 below:

[0147] Table 8. Dimensional Weights for Detection Accuracy

[0148] Dimension Detection rate False negative rate False positive rate Weight 1 / 3 1 / 3 1 / 3

[0149] Environmental adaptability is used to indicate whether the roadside sensing system can maintain good detection accuracy in adverse environments such as fog, haze, refraction, poor light, and night. Therefore, the environmental adaptability indicators can be further refined, as shown in Table 9 below:

[0150] Table 9 Weights of Dimensions of Environmental Adaptability

[0151] Environmental conditions fog haze refraction night Weight 1 / 4 1 / 4 1 / 4 1 / 4

[0152] In summary, after refining each sub-evaluation indicator, its weight is projected onto the overall detection weight, as shown in Table 10 below.

[0153] Table 10 Evaluation Index Weights (c)

[0154] Evaluation indicators Weight long 0.025 Width 0.025 high 0.025 Horizontal 0.025 Vertical 0.025 high 0.025 Heading angle accuracy 0.075 Speed ​​accuracy 0.075 Classification accuracy 0.1 Detection rate 1 / 30 False negative rate 1 / 30 False positive rate 1 / 30 Tracking success rate 0.1 Detection range 0.05 Response time 0.05 Data frequency 0.05 Maximum number of detections 0.05 Minimum recognizable size 0.05 fog 0.0125 haze 0.0125 refraction 0.0125 night 0.0125 System cost 0.1 sum 1

[0155] In summary, the calculated values ​​of the first and second indicators can be weighted according to the weights of each evaluation indicator in Table 10 to obtain the detection results of the roadside sensing system under test.

[0156] In addition, in this embodiment of the application, the function of the roadside sensing system under test can be detected, and it can be scored based on preset detection rules to obtain the index value corresponding to the functional index of the roadside sensing system under test.

[0157] Specifically, before determining the second indicator value corresponding to the preset system cost indicator based on the first cost data and the pre-acquired reference cost data, the method may further include:

[0158] Based on the preset detection rules, the third index value corresponding to the functional index of the roadside sensing system under test is determined.

[0159] It should be noted that the above-mentioned functional indicators can be used as additional scoring items for the roadside perception system under test. Under the condition that they have the ability to add points, the detection score obtained by weighting the first and second indicator values ​​can be added to the third indicator value corresponding to the functional indicator to obtain the final detection result.

[0160] Specifically, before determining the second indicator value corresponding to the preset system cost indicator based on the first cost data and the pre-acquired reference cost data, the method further includes:

[0161] According to the preset detection rules, the third index value corresponding to the functional index of the roadside sensing system under test is determined;

[0162] The step of determining the detection result of the roadside sensing system under test based on the first index value and the second index value specifically includes:

[0163] The first detection score is obtained by weighting the first indicator value and the second indicator value.

[0164] The first detection score and the third index value are added together to obtain the detection result of the roadside perception system under test.

[0165] Based on this, the roadside perception system under test can realize additional functions that help it better perceive road traffic conditions. According to the preset detection rules, the third index value corresponding to the functional index is calculated to add points to the roadside perception system under test, which can make the detection results of the roadside perception system under test more accurate and have reference value.

[0166] It should be noted that the above-mentioned preset detection rules can be set according to actual needs. For example, the maximum value of the functional indicators can be set to 10 points, and then the score can be given based on the degree of improvement of the roadside perception system under test based on the additional functions of the system.

[0167] Figure 2 A schematic diagram of the detection device of the roadside sensing system provided in an embodiment of this application is shown.

[0168] like Figure 2 The detection device 200 of the roadside sensing system shown is characterized in that the device comprises:

[0169] The data acquisition module 201 is used to acquire first detection data and first cost data of the roadside perception system under test, and to acquire second detection data of the roadside perception system under calibration, wherein the first detection data and the second detection data are respectively the detection data of the roadside perception system under test and the roadside perception system under calibration for the target detection object on the road, and / or the performance test data of the target system performance of the roadside perception system under test and the roadside perception system under calibration, respectively.

[0170] The first calculation module 202 is used to calculate the first detection data and the second detection data to obtain a first index value corresponding to a preset first evaluation index. The first evaluation index includes at least one of the data detection quality index of the roadside perception system under test and the performance test index of the roadside perception system under test.

[0171] The second calculation module 203 is used to determine the second index value corresponding to the preset system cost index based on the first cost data and the pre-acquired reference cost data.

[0172] The detection result determination module 204 is used to determine the detection result of the roadside sensing system under test based on the first index value and the second index value.

[0173] Based on this, the detection device for the roadside perception system in this application calculates the first index value corresponding to the data detection quality index and / or performance test index of the roadside perception system under test using the first detection data of the roadside perception system under test and the second detection data of the calibrated roadside perception system. It also calculates the second index value corresponding to the system cost index using the first cost data of the roadside perception system under test and pre-acquired reference cost data. Then, based on the first and second index values, the detection result of the roadside perception system under test is determined. Thus, through the method of this application, the first detection data includes the detection data of the roadside perception system on the target detection object and / or the performance test data of the roadside perception system. Combined with the acquired system cost data, the system is tested not only from the perspective of system performance or detection data quality but also from an economic perspective, considering the system cost of the roadside perception system. This ensures that the obtained detection results are more comprehensive and accurate, and have higher reference value for users.

[0174] Optionally, the data detection quality index of the roadside sensing system under test includes at least one of data precision and data accuracy.

[0175] Optionally, the data accuracy includes at least one of the following: the dimensional accuracy of the target object, the positioning accuracy of the target object, the velocity accuracy of the target object, and the heading angle accuracy of the target object.

[0176] Optionally, the data accuracy includes at least one of the classification accuracy of the target detection object, the detection accuracy of the target detection object, and the tracking success rate of the target detection object.

[0177] Optionally, the performance test indicators of the roadside sensing system under test include at least one of the following: the detection range indicator of the roadside sensing system under test, the response time indicator of the roadside sensing system under test, the data frequency indicator of the roadside sensing system under test, the maximum number of detections of the roadside sensing system under test, the minimum identifiable size indicator of the roadside sensing system under test, and the environmental adaptability of the roadside sensing system under test.

[0178] Optionally, the reference cost data includes a reference average value and its corresponding first score, a reference maximum value and its corresponding second score, and a reference minimum value and its corresponding third score;

[0179] The second calculation module 203 is specifically used for:

[0180] If the first cost data and the reference average value are the same, the second indicator value is determined to be the first score;

[0181] If the first cost data and the highest reference value are the same, the second indicator value is determined to be the second score;

[0182] If the first cost data and the reference minimum value are the same, the second indicator value is determined to be the third score;

[0183] When the first cost data is different from the reference average value, the reference maximum value, and the reference minimum value, the second indicator value is determined to be the fourth score according to a preset ratio.

[0184] Based on this, using the average, highest, and lowest values ​​of the reference cost as benchmarks, the first cost data of the roadside perception system under test can be more reasonably detected and evaluated, making the second indicator value corresponding to the determined system cost index more accurate.

[0185] Optionally, the device 200 further includes:

[0186] According to the preset detection rules, the third index value corresponding to the functional index of the roadside sensing system under test is determined;

[0187] The detection result determination module 204 is specifically used for:

[0188] The first detection score is obtained by weighting the first indicator value and the second indicator value.

[0189] The first detection score and the third index value are added together to obtain the detection result of the roadside perception system under test.

[0190] Based on this, the roadside perception system under test can realize additional functions that help it better perceive road traffic conditions. According to the preset detection rules, the third index value corresponding to the functional index is calculated to add points to the roadside perception system under test, which can make the detection results of the roadside perception system under test more accurate and have reference value.

[0191] The detection device 200 of the roadside sensing system provided in this application embodiment can achieve Figure 1 To avoid repetition, the various processes of the method will not be described in detail here.

[0192] Figure 3 A schematic diagram of the hardware structure of the detection device of the roadside sensing system provided in an embodiment of this application is shown.

[0193] The detection equipment of the roadside sensing system may include a processor 301 and a memory 302 storing computer program instructions.

[0194] Specifically, the processor 301 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0195] Memory 302 may include mass storage for data or instructions. For example, and not limitingly, memory 302 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 302 may include removable or non-removable (or fixed) media. Where appropriate, memory 302 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 302 is non-volatile solid-state memory.

[0196] Memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of this application.

[0197] The processor 301 reads and executes computer program instructions stored in the memory 302 to implement any of the detection methods of the roadside perception system in the above embodiments.

[0198] In one example, the detection equipment of the roadside sensing system may further include a communication interface 303 and a bus 310. For example, Figure 3 As shown, the processor 301, memory 302, and communication interface 303 are connected through bus 310 and complete communication with each other.

[0199] The communication interface 303 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0200] Bus 310 includes hardware, software, or both, that couples components of the detection devices of the roadside sensing system together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 310 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0201] The detection equipment of this roadside sensing system can, based on the detection data and cost data of the roadside sensing system under test, execute the detection method of the roadside sensing system in this application embodiment, thereby achieving a combination of... Figure 1 and Figure 2 The detection method and apparatus of the roadside sensing system are described.

[0202] Furthermore, in conjunction with the detection method of the roadside perception system in the above embodiments, this application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the detection methods of the roadside perception system in the above embodiments.

[0203] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0204] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0205] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0206] The aspects of this application have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0207] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A detection method for a roadside sensing system, characterized in that, The method includes: Acquire first detection data and first cost data of the roadside perception system under test, and acquire second detection data of the roadside perception system under calibration, wherein the first detection data and the second detection data are respectively the detection data of the roadside perception system under test and the roadside perception system under calibration for the target detection objects on the road, and / or the performance test data of the target system performance of the roadside perception system under test and the roadside perception system under calibration, respectively. The first detection data and the second detection data are calculated to obtain a first index value corresponding to a preset first evaluation index. The first evaluation index includes at least one of the data detection quality index of the roadside perception system under test and the performance test index of the roadside perception system under test. The data detection quality indicators of the roadside sensing system under test include at least one of data precision and data accuracy; The performance test indicators of the roadside sensing system under test include at least one of the following: the detection range indicator, the response time indicator, the data frequency indicator, the maximum number of detections indicator, the minimum identifiable size indicator, and the environmental adaptability indicator. Based on the first cost data and the pre-acquired reference cost data, determine the second indicator value corresponding to the preset system cost indicator; The detection result of the roadside sensing system under test is determined based on the first index value and the second index value.

2. The detection method of the roadside sensing system according to claim 1, characterized in that, The data accuracy includes at least one of the following: the dimensional accuracy of the target object, the positioning accuracy of the target object, the velocity accuracy of the target object, and the heading angle accuracy of the target object.

3. The detection method of the roadside sensing system according to claim 1, characterized in that, The data accuracy includes at least one of the classification accuracy of the target detection object, the detection accuracy of the target detection object, and the tracking success rate of the target detection object.

4. The detection method of the roadside sensing system according to claim 1, characterized in that, The reference cost data includes a reference average value and its corresponding first score, a reference maximum value and its corresponding second score, and a reference minimum value and its corresponding third score; The step of determining the second indicator value corresponding to the preset system cost indicator based on the first cost data and the pre-acquired reference cost data specifically includes: If the first cost data and the reference average value are the same, the second indicator value is determined to be the first score; If the first cost data and the highest reference value are the same, the second indicator value is determined to be the second score; If the first cost data and the reference minimum value are the same, the second indicator value is determined to be the third score; When the first cost data is different from the reference average value, the reference maximum value, and the reference minimum value, the second indicator value is determined to be the fourth score according to a preset ratio.

5. The detection method of the roadside sensing system according to claim 1, characterized in that, Before determining the second indicator value corresponding to the preset system cost indicator based on the first cost data and the pre-acquired reference cost data, the method further includes: According to the preset detection rules, the third index value corresponding to the functional index of the roadside sensing system under test is determined; The step of determining the detection result of the roadside sensing system under test based on the first index value and the second index value specifically includes: The first detection score is obtained by weighting the first indicator value and the second indicator value. The first detection score and the third index value are added together to obtain the detection result of the roadside perception system under test.

6. A detection device for a roadside sensing system, characterized in that, The device includes: The data acquisition module is used to acquire first detection data and first cost data of the roadside perception system under test, and to acquire second detection data of the roadside perception system under calibration. The first detection data and the second detection data are respectively the detection data of the roadside perception system under test and the roadside perception system under calibration for the target detection objects on the road, and / or the performance test data of the target system performance of the roadside perception system under test and the roadside perception system under calibration. The first calculation module is used to calculate the first detection data and the second detection data to obtain the first index value corresponding to the preset first evaluation index. The first evaluation index includes at least one of the data detection quality index of the roadside perception system under test and the performance test index of the roadside perception system under test. The data detection quality indicators of the roadside sensing system under test include at least one of data precision and data accuracy; The performance test indicators of the roadside sensing system under test include at least one of the following: the detection range indicator, the response time indicator, the data frequency indicator, the maximum number of detections indicator, the minimum identifiable size indicator, and the environmental adaptability indicator. The second calculation module is used to determine the second indicator value corresponding to the preset system cost indicator based on the first cost data and the pre-acquired reference cost data. The detection result determination module is used to determine the detection result of the roadside sensing system under test based on the first index value and the second index value.

7. A detection device for a roadside sensing system, characterized in that, The device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the detection method of the roadside perception system as described in any one of claims 1-5.

8. A computer storage medium, characterized in that, The computer storage medium stores computer program instructions, which, when executed by a processor, implement the detection method of the roadside perception system as described in any one of claims 1-5.

Images