Intelligent perception system credibility multidimensional test evaluation method

Abstract

The present application relates to the technical field of intelligent driving car perception system evaluation, and relates to a kind of intelligent perception system credibility multidimensional test evaluation method.First, each basic test scene data is acquired, and each interference scene factor is determined based on each extended test scene data.Vehicle automatic driving test is carried out, the perception data of intelligent perception system is determined, and based on the perception data, according to the evaluation method, the evaluation result of intelligent perception system is determined.Based on the prior experience of experts, an optimization model is determined.Genetic algorithm is used to solve the optimization model to determine the weight of the evaluation result.According to the evaluation result weight and the evaluation result, the credibility of the intelligent perception system is determined.This scheme combines the prior experience of experts and genetic algorithm optimization, which not only utilizes the domain knowledge, but also avoids subjective bias through objective optimization, so that the index weight distribution is more in line with the actual demand, and the accuracy of the credibility evaluation is improved.

Description

Technical Field

[0001] This specification relates to the field of intelligent driving vehicle perception system evaluation technology, and in particular to a multi-dimensional test and evaluation method for the reliability of intelligent perception systems. Background Technology

[0002] Against the backdrop of the continuous deepening of the automotive industry's intelligentization process, intelligent perception systems have become the core technological foundation for intelligent driving vehicles to achieve environmental understanding and decision-making. By integrating data from multiple sensors such as cameras, LiDAR, and millimeter-wave radar, this system enables real-time detection, identification, and tracking of surrounding vehicles, pedestrians, traffic signs, and road structures. It provides crucial environmental perception information to downstream decision-making and control modules, and its performance directly affects the driving safety and system reliability of intelligent driving vehicles.

[0003] To ensure that intelligent perception systems meet the safety requirements of autonomous vehicles, their reliability is crucial. Current research on reliability testing and evaluation in the field of autonomous driving largely focuses on independent levels such as algorithm models, single sensors, or perception data, lacking a systematic evaluation scheme that covers the entire intelligent perception system and considers multi-dimensional reliability attributes. Compared to reliability evaluation at the algorithm or sensor level, the reliability of intelligent perception systems differs fundamentally in terms of evaluation dimensions and the level of interaction with the environment, making it difficult to directly apply existing component-level evaluation methods. Furthermore, existing evaluation schemes largely rely on expert experience and subjective judgment, affecting the objectivity of the evaluation and failing to deeply reveal the intrinsic relationships between evaluation indicators and the system's behavioral logic, resulting in insufficient scientific rigor and rationality in the evaluation results.

[0004] Therefore, this specification provides a multi-dimensional testing and evaluation method for the reliability of intelligent sensing systems. Summary of the Invention

[0005] This specification provides a multi-dimensional testing and evaluation method for the reliability of intelligent sensing systems, in order to partially solve the aforementioned problems existing in the prior art.

[0006] The following technical solution is adopted in this specification:

[0007] This specification provides a multi-dimensional testing and evaluation method for the reliability of intelligent sensing systems, including:

[0008] S1. Obtain the preset basic test scenario data;

[0009] S2. Based on the basic test scenario data, and based on the preset interference scenario factors, determine the extended test scenario data corresponding to each basic test scenario data;

[0010] S3. Based on the data from each basic test scenario and each extended test scenario, conduct autonomous driving tests on the vehicle to determine the perception data of the intelligent perception system of the autonomous vehicle. Based on the perception data of the intelligent perception system, determine the evaluation result of the intelligent perception system according to the preset evaluation method. The evaluation result includes the mean average accuracy, the mean intersection-over-union ratio, the ranging accuracy, the speed measurement accuracy, the perception delay, the attenuation rate score, the coefficient of variation, the consistency of multiple sensors, the stability of a single sensor, and the perception uncertainty.

[0011] S4. Determine the optimization model based on pre-set expert prior experience;

[0012] S5. Use a genetic algorithm to solve the optimization model and determine the weights of the evaluation results;

[0013] S6. Determine the credibility of the intelligent sensing system based on the evaluation result weight and the evaluation result.

[0014] Based on the aforementioned technical means, this solution generates extended scenarios by superimposing various interference factors onto a basic scenario, simulating the complex and variable conditions in real driving. This enhances the diversity and realism of the tests, making the evaluation results more representative. The evaluation results cover perception accuracy (such as mean accuracy rate, mean intersection-over-union ratio, and ranging / speed accuracy), as well as perception real-time performance (perception latency), environmental adaptability (attenuation rate score, coefficient of variation), consistency (multi-sensor consistency, single-sensor stability), perception uncertainty, and other performance aspects, comprehensively reflecting the reliability of the intelligent perception system. By combining expert prior experience with genetic algorithm optimization, this approach utilizes domain knowledge while avoiding subjective bias through objective optimization, making the weight allocation of each indicator more aligned with actual needs and improving the accuracy of the reliability assessment.

[0015] Furthermore, the expert's prior experience is a set of judgment matrices pre-constructed by the expert;

[0016] S4 specifically includes:

[0017] Based on the set of judgment matrices, determine the aggregated fuzzy number judgment matrix;

[0018] The aggregated fuzzy number judgment matrix is ​​defuzzified to determine the initial judgment matrix;

[0019] Based on the initial judgment matrix, determine the consistency ratio of the initial judgment matrix;

[0020] If the consistency ratio of the initial judgment matrix meets a preset standard, a preset correction factor is introduced into the initial judgment matrix to determine the corrected judgment matrix.

[0021] Based on the modified judgment matrix, determine the consistency ratio of the modified judgment matrix and the modified weight of the evaluation result;

[0022] Based on the consistency ratio of the modified judgment matrix, a consistency maximization objective function is constructed; and based on the initial judgment matrix and the modified judgment matrix, an expert judgment retention function is constructed.

[0023] An optimization model is determined based on the consensus maximization objective function, the expert judgment retention function, the corrected judgment matrix, and the corrected weights of the evaluation results.

[0024] Based on the aforementioned technical means, by deeply integrating the prior knowledge of multiple domain experts and undergoing rigorous mathematical logic consistency correction and optimization, more reasonable and credible weights are output, thereby optimizing the weights and effectively avoiding the bias caused by a single subjective judgment. Consistency verification ensures that the weight allocation conforms to logical rules, thus making the obtained credible results more scientific, objective and persuasive.

[0025] Furthermore, the expression for the set of judgment matrices is:

[0026]

[0027]

[0028]

[0029]

[0030] in, The set of judgment matrices includes A fuzzy number judgment matrix; For the first A fuzzy number judgment matrix pre-constructed by experts , It is a positive integer; For the first The evaluation results were predetermined by the experts. Compared to the evaluation results Importance assessment results; For the first The evaluation results were predetermined by the experts. Compared to the evaluation results The minimum value of the importance assessment result; For the first The evaluation results were predetermined by the experts. Compared to the evaluation results Reference values ​​for the importance assessment results; For the first The evaluation results were predetermined by the experts. Compared to the evaluation results The maximum value of the importance assessment result; , , It is a positive integer. For positive integers, when hour, ,when hour, ; For the first One evaluation result, For the first One evaluation result.

[0031] Furthermore, the expression for the aggregated fuzzy number judgment matrix is:

[0032]

[0033]

[0034] in, The aggregated fuzzy number judgment matrix; To The result after aggregation; To The result after aggregation processing , Indicates to The result after aggregation processing , Indicates to The result after aggregation processing .

[0035] Furthermore, the expression for the initial judgment matrix is:

[0036]

[0037]

[0038] in, This is the initial judgment matrix; for The result after deblurring.

[0039] Furthermore, the expression for determining the consistency ratio of the initial judgment matrix is ​​as follows:

[0040]

[0041] in, CR The consistency ratio of the initial judgment matrix; The largest eigenvalue of the initial judgment matrix; RI This is the preset average random consistency constant.

[0042] Furthermore, the expression for the modified judgment matrix is:

[0043]

[0044]

[0045] in, The correction judgment matrix is ​​as follows; For the first in the modified judgment matrix i Line number j Column elements; In the initial judgment matrix The preset correction factor, .

[0046] Furthermore, the expression for determining the corrected weights of the evaluation results is as follows:

[0047]

[0048]

[0049]

[0050] in, This represents the geometric mean of the elements in each row of the modified judgment matrix; Indicates the evaluation result Corrected weights; This indicates the corrected weight of the evaluation result.

[0051] Furthermore, the expressions for the consistency maximization objective function and the expert judgment retention function are as follows:

[0052]

[0053]

[0054] in, This represents the consistency maximization objective function; This represents the preset constant attenuation coefficient; This represents the consistency ratio of the modified judgment matrix; This represents the expert's decision retention function.

[0055] Furthermore, the expression for the optimization model is:

[0056] .

[0057] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects:

[0058] This solution generates extended scenarios by overlaying various interference factors onto a basic scenario, simulating the complex and variable conditions of real-world driving. This enhances the diversity and realism of the tests, making the evaluation results more representative. The evaluation results cover perception accuracy (such as mean accuracy rate, mean intersection-over-union ratio, and ranging / speed accuracy), as well as perception real-time performance (perception latency), environmental adaptability (attenuation rate score, coefficient of variation), consistency (multi-sensor consistency, single-sensor stability), and perception uncertainty, comprehensively reflecting the reliability of the intelligent perception system. By combining expert prior experience with genetic algorithm optimization, it utilizes domain knowledge while avoiding subjective bias through objective optimization, making the weight allocation of each indicator more aligned with actual needs and improving the accuracy of the reliability assessment. Attached Figure Description

[0059] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings:

[0060] Figure 1 A flowchart illustrating a multi-dimensional reliability testing and evaluation method for an intelligent sensing system provided in the embodiments of this specification.

[0061] Figure 2 This manual provides a schematic diagram of a crossroads traffic scenario.

[0062] Figure 3 This is a schematic diagram of a scenario where the near end of a target traverses, as provided in this specification.

[0063] Figure 4 This is a schematic diagram illustrating a scenario for passage through a long tunnel, as provided in this specification.

[0064] Figure 5 This is a schematic diagram illustrating a scenario where the vehicle in front cuts out, as provided in this manual.

[0065] Figure 6 This specification provides a corresponding Figure 1 A schematic diagram of the structure of an electronic device. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this specification without creative effort are within the scope of protection of this application.

[0067] In embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0068] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0069] Figure 1 A flowchart illustrating a multi-dimensional reliability testing and evaluation method for an intelligent sensing system, provided in the embodiments of this specification, includes the following steps:

[0070] S1: Obtain data for each preset basic test scenario.

[0071] S2: Based on the basic test scenario data, and based on the preset interference scenario factors, determine the extended test scenario data corresponding to each basic test scenario data.

[0072] This specification describes the process of conducting multi-dimensional reliability testing and evaluation of an intelligent sensing system. In the embodiments described herein, this process can be executed by a server. However, this specification does not limit the type of device or platform used to perform the multi-dimensional reliability testing and evaluation of the intelligent sensing system; for example, a personal computer, mobile terminal, or other devices or platforms can also be used. For ease of description, the following explanation uses a server as the executing entity.

[0073] In one or more embodiments of this specification, referencing the investigation and analysis of traffic accident scenarios by organizations such as the National Automobile Accident In-Depth Investigation System (NAIS), four typical scenarios are identified as preset basic test scenarios: crossing an intersection, crossing a target at near end, crossing a long tunnel, and a vehicle cutting out from in front.

[0074] The scenario for crossing a crossroads is as follows: At an unsignalized crossroads, the test vehicle travels at a constant speed of 60 km / h in the straight lane. The target vehicle (i.e., the object) enters from the right side of the intersection at a speed of 30 km / h and performs a left turn. If neither the test vehicle nor the target vehicle takes any active vehicle control measures such as steering, braking, or acceleration, the two vehicles will collide at a predetermined collision point. Figure 2 This is a schematic diagram illustrating a traffic flow scenario at a crossroads, as provided in this manual. Figure 2 As shown, at an unsignalized intersection, the test vehicle travels in the straight lane. The target vehicle enters from the right side of the intersection and is expected to make a left turn. If neither the test vehicle nor the target vehicle takes any active vehicle control measures such as steering, braking, or acceleration, the two vehicles will collide at the predetermined collision point.

[0075] The scenario for the near-end crossing of the target is as follows: The test vehicle is traveling at a constant speed of 40 km / h, with a series of stationary vehicles forming a visual obstruction on its right. The pedestrian target (i.e., the object) crosses the road from one side of the visual obstruction area at a speed of 5 km / h. If the test vehicle and the pedestrian target do not change their direction of travel and speed, they will collide at the predetermined collision point. Figure 3 This is a schematic diagram illustrating a scenario where the near end of a target traverses the path, as provided in this specification. Figure 3 As shown, the test vehicle travels in a straight line, with a series of stationary vehicles on its right creating a visual obstruction. A pedestrian crosses the road from the side of the visual obstruction area. If the test vehicle and the pedestrian do not change their direction and speed, they will collide at the predetermined collision point.

[0076] The scenario for traveling through a long tunnel is as follows: The test vehicle enters the long tunnel at a constant speed of 60 km / h. 50 m from the tunnel entrance, a target vehicle (i.e., the target object) is traveling at a low speed of 30 km / h in the lane in which the test vehicle is traveling. If the test vehicle and the target vehicle do not change their direction and speed, they will collide at a predetermined collision point. Figure 4 This is a schematic diagram illustrating a scenario for passage through a long tunnel, as provided in this specification. Figure 4As shown, the test vehicle travels in a straight line into a long tunnel. A target vehicle is present in the lane where the test vehicle is traveling, not far from the tunnel entrance. If the target vehicle's speed is lower than the test vehicle's, a collision may occur in this scenario.

[0077] The scenario for the test vehicle cutting out is as follows: While the test vehicle is traveling, target vehicle 1 and target vehicle 2 are in front of it in its lane. Initially, both the test vehicle and target vehicle 1 are traveling in the same direction at a constant speed of 60 km / h, maintaining a distance of 20 m between them. When target vehicle 1 and target vehicle 2 are 20 m apart, target vehicle 1 suddenly cuts out into the adjacent lane. Target vehicle 2 is traveling at a low speed of 30 km / h. If the test vehicle and target vehicle 2 do not change their direction and speed, they will collide at the predetermined collision point. Figure 5 This is a schematic diagram illustrating a scenario where the preceding vehicle cuts through, as provided in this specification. Figure 5 As shown, during the test vehicle's operation, there are target vehicles 1 and 2 located not far ahead in its lane. If target vehicle 1 suddenly cuts into the adjacent lane, and the test vehicle and target vehicle 2 do not change their direction and speed, a collision may occur between them in this scenario.

[0078] For the four basic test scenarios mentioned above, corresponding basic test simulation scenario data can be set up. This allows for the simulation of the basic test scenarios within the simulation software after the data is imported. Alternatively, the four basic test scenarios can be set up at a real test site to conduct real-vehicle autonomous driving tests.

[0079] Therefore, the server can obtain the preset basic test scenario data (including the basic test simulation scenario data corresponding to the four basic test scenarios mentioned above). Then, based on the preset interference factors of each scenario, the server can construct each extended test scenario corresponding to each basic test scenario based on the basic test scenario data, determine the extended test scenario data corresponding to each basic test scenario data, and thus determine the extended test simulation scenario data used for simulation.

[0080] Among them, for the preset interference scenario factors, in this specification, based on dynamic and static scenario elements such as road vibration excitation, sudden changes in illumination, rain and fog weather conditions, and complex traffic participants, the specific interference scenario factors are shown in Table 1.

[0081] Table 1. Factors affecting the scene

[0082]

[0083] It is worth noting that, in the context of sudden lighting changes, for scenarios involving tunnel elements, this refers to an instantaneous and rapid change in lighting conditions at the tunnel entrance. Specifically, this manifests as a significant difference in lighting intensity between the inside and outside of the tunnel, with normal lighting conditions outside the tunnel and weak lighting conditions inside.

[0084] Based on the preset interference factors for each scenario, and on the basis of the data of each basic test scenario, the combination relationship of scenario interference factors in each extended test scenario corresponding to each basic test scenario is constructed as shown in Table 2.

[0085] Table 2. Combination Relationship of Scene Interference Factors in the Test Scenario

[0086]

[0087] S3: Based on the data from each basic test scenario and each extended test scenario, conduct autonomous driving tests on the vehicle to determine the perception data of the intelligent perception system of the autonomous vehicle. Based on the perception data of the intelligent perception system, determine the evaluation result of the intelligent perception system according to the preset evaluation method. The evaluation result includes the mean average accuracy, the mean intersection-over-union ratio, the ranging accuracy, the speed measurement accuracy, the perception delay, the attenuation rate score, the coefficient of variation, the consistency of multiple sensors, the stability of a single sensor, and the perception uncertainty.

[0088] In one or more embodiments of this specification, the server can perform autonomous driving tests on vehicles based on basic test scenario data and extended test scenario data to determine the perception data of the intelligent perception system of the autonomous vehicle.

[0089] For example, based on the basic and extended test simulation scenario data, corresponding test simulation scenarios can be simulated in pre-set simulation software to conduct vehicle autonomous driving tests and determine the perception data of the intelligent perception system in the simulation software. Alternatively, real test scenarios corresponding to the basic and extended test simulation scenario data can be arranged at a real test site to conduct real vehicle autonomous driving tests and determine the perception data of the intelligent perception system.

[0090] Based on the sensing data from the intelligent sensing system, and according to a preset evaluation method, the evaluation results of the intelligent sensing system are determined. The evaluation results of the intelligent sensing system include the mean average accuracy, the mean intersection-over-union ratio, the ranging accuracy, the velocity accuracy, the sensing delay, the attenuation rate score, the coefficient of variation, the consistency of multiple sensors, the stability of a single sensor, and the sensing uncertainty.

[0091] In this specification, the evaluation method for determining the evaluation results of the intelligent sensing system can be calculated using the following formula.

[0092] Specifically, for perception tasks such as target detection, target segmentation, perception ranging, and perception velocity measurement, the system can determine the average accuracy, average intersection-over-union ratio, ranging accuracy, and velocity measurement accuracy of the intelligent perception system across the perception accuracy dimension.

[0093] Specifically, the expression for determining the mean of the average precision is:

[0094]

[0095]

[0096]

[0097] In the formula, each scenario (one of the basic test scenarios and one of the extended test scenarios) is considered. This represents the average precision rate. This indicates the number of target categories to be tested, that is, the number of target object categories in the scene perceived by the intelligent sensing system. r This indicates the recall rate. Indicates for the first i Class of targets to be tested (i.e., the first) i (Target class), recall rate r Precision value at time P .

[0098] It's worth noting that for each frame of an image sensed, the intelligent perception system detects objects within that frame and marks them with bounding boxes. The accuracy of the system's perception is assessed by comparing the marked objects with pre-defined true bounding boxes. Therefore, TP This can represent the number of detection boxes that satisfy the condition that the intersection-union ratio (IoU) with the ground truth bounding box is ≥0.7 and that the two boxes are completely identical in class. FP This indicates the number of detection boxes that do not meet the condition that the intersection-union ratio (IU) between the detection box and the ground truth bounding box is ≥0.7, or that the two boxes are of different categories, or that there is no corresponding ground truth bounding box. FN This represents the difference between the actual number of target boxes and the total number of boxes (TP).

[0099] The expression for determining the average crossover ratio is:

[0100]

[0101] In the formula, This represents the average intersection-union ratio (IUGR). The intelligent sensing system also performs target segmentation on each frame of the image it senses, dividing the area containing the target object. To determine the accuracy of the segmentation, the segmented target object area is compared with a pre-labeled real target area. Therefore, It can represent the firsti Within the actual labeled area of ​​the target object and determined by the intelligent perception system to be the first i The number of pixels of the target object, Indicates non i Within the true labeled area of ​​the target object, and misidentified by the intelligent perception system as the first i The number of pixels of the target object, Indicates the first i The number of pixels in the actual labeled region of the object class and The difference.

[0102] The expression for determining the ranging accuracy is:

[0103]

[0104] In the formula, Indicates the ranging accuracy. This represents the total number of valid image data frames continuously acquired by the intelligent sensing system during the sensing and ranging task. This represents the maximum predicted distance between the test vehicle and the target object from the valid image data frames collected by the intelligent sensing system. This represents the minimum predicted distance between the test vehicle and the target object, as determined by the intelligent sensing system within the collected valid image data frames. This indicates that the intelligent sensing system has acquired the first valid image data frame in the first frame. j The predicted distance between the test vehicle and the target object in the frame prediction. This indicates that the intelligent sensing system pre-labels the first image data frame in the acquired valid image data frame. j The true distance between the test vehicle and the target object in the frame.

[0105] The expression for determining the speed measurement accuracy is:

[0106]

[0107] In the formula, Indicates the speed measurement accuracy. In the process of sensing and measuring speed, the total number of predicted speed values ​​of the target object continuously predicted by the intelligent sensing system is consistent with the total number of valid image data frames continuously acquired in the process of sensing and measuring distance. This represents the maximum value of the predicted speed. This represents the minimum value of the predicted speed. Indicates the first j The predicted velocity value (i.e., the first effective image data acquired) j (Frame velocity prediction value). Indicates the pre-marked first jThe true speed value. It is worth noting that in the above calculations of ranging accuracy and velocity accuracy, and in subsequent calculations, when extended test simulation scenarios involving multiple targets are involved, the evaluation results for the relevant scenarios involving multiple targets can be calculated separately for each (i.e., each type) of target in each test scenario (including each basic test scenario and each extended test scenario). Then, the evaluation result is determined by combining the multiple calculated results. For example, for velocity accuracy, the velocity accuracy can be calculated separately for each target in each extended test scenario, and then the average of the velocity accuracy values ​​is calculated to determine the final velocity accuracy. The above expression for velocity accuracy is only to provide a feasible calculation method; the specific calculation settings can be set according to the actual situation.

[0108] For the cumulative delay caused by data acquisition, transmission, and algorithm processing in intelligent sensing systems, the sensing delay in the real-time dimension can be calculated, and the calculation expression can be:

[0109]

[0110] In the formula, This indicates a perceived delay. This represents the maximum time difference between the completion of data acquisition by each sensor and the output result of the intelligent sensing system. When calculating the sensing delay, since the data acquisition and output results of multiple sensors may be involved, the maximum value of the time difference between the completion of data acquisition and the output result of each sensor is taken as the sensing delay in order to better characterize the sensing delay.

[0111] In this specification, to effectively quantify the degree of interference caused by environmental disturbances to sensing performance, the server can calculate evaluation results for two environmental adaptability dimensions, including attenuation rate score and coefficient of variation.

[0112] The attenuation rate score can be used to evaluate the overall decline in the four perceptual accuracy evaluation results of the extended test scenario relative to its corresponding base scenario. The expression for determining the attenuation rate score is as follows:

[0113]

[0114] In the formula, I 6 represents the attenuation rate score. In the base-th basic test scenario, the first... k The evaluation results of perception accuracy include one of the following: mean accuracy rate, mean intersection-to-union ratio, distance measurement accuracy, and speed measurement accuracy, where base represents passing through an intersection, crossing a target at near end, passing through a long tunnel, and cutting out from the front vehicle, respectively. , respectively represent evaluation indicators , , , . This indicates the extended test scenario of the base test scenario it corresponds to. k The results of a perception accuracy index.

[0115] The coefficient of variation is used to quantitatively evaluate the performance stability of a sensing system under environmental variations. The expression for determining the coefficient of variation is:

[0116]

[0117]

[0118]

[0119] In the formula, This represents the coefficient of variation. For each basic test scenario, it is known that each basic test scenario corresponds to four extended test scenarios. This indicates the first of the four extended test scenarios corresponding to the basic test scenario. k The standard deviation of the evaluation results for the accuracy of perception. This indicates the first of the four extended test scenarios corresponding to the basic test scenario. k The average value of the evaluation results of the perception accuracy. This represents the perception accuracy evaluation result of the m-th item and k-th term corresponding to the basic test scenario, where m ∈ [1, 4]. When m is 1... This is the k-th perceptual accuracy evaluation result of the first extended test scenario corresponding to the basic test scenario, and so on.

[0120] In this specification, to quantify the consistency and reliability of the spatial perception results of the intelligent sensing system, the server can calculate two evaluation indicators for spatial consistency, including multi-sensor consistency and single-sensor stability.

[0121] Multi-sensor consistency is used to evaluate the differences in perception of the same target object by different sensors such as cameras, LiDAR, and millimeter-wave radar in an intelligent sensing system. It represents the degree of consistency of the perception results of multiple sensors. The calculation expression for multi-sensor consistency is:

[0122]

[0123] In the formula, This indicates consistency among multiple sensors. This indicates the number of sensors in the intelligent sensing system. Indicates the first i The distance sensing results of the same target object by the sensors. This represents the average of all perceived distance results.

[0124] Single-sensor stability refers to the degree of fluctuation in the perception of the same target object by the same sensor in consecutive data frames. For example, if the consecutive data frames acquired by a camera are video that can be decomposed into consecutive video frames, the expression for calculating single-sensor stability is:

[0125]

[0126] In the formula, This indicates the stability of a single sensor. B This indicates the number of consecutive data frames collected. Indicates the first b The target prediction box of the frame. Indicates the first k -3 frames to k -1 frame is the average value of the detection bounding boxes of the target object in three consecutive frames. Indicates the first b The intersection-over-union ratio (IoU) of the detection bounding box of the target object in the first frame with the average value of the detection bounding boxes of the target objects in the previous three frames. Represents a conditional function, when Established, Otherwise, it is 0, as stated in this manual. The possible value is 0.7.

[0127] In this specification, perceived uncertainty is used to quantify the degree of matching between the detection confidence level output by the sensing system and the actual detection performance. The expression for calculating perceived uncertainty is as follows:

[0128]

[0129] In the formula, Z represents perceived uncertainty. Z represents the number of confidence sub-intervals. In this specification, the confidence interval [0,1] is divided into 10 sub-intervals, namely [0,0.1), [0.1,0.2), [0.2,0.3), [0.3,0.4), [0.4,0.5), [0.5,0.6), [0.6,0.7), [0.7,0.8), [0.8,0.9), and [0.9,1]. Indicates the first The average confidence level of the perception system output within each sub-interval. Indicates the first The proportion of the number of correctly classified targets within a sub-interval to the total number of confidence scores for that interval. Indicates the first The ratio of the total number of built-in confidence scores in each sub-interval to the total number of built-in confidence scores in the [0,1] interval. For example, in a series of data frames collected by an intelligent sensing system, target object detection is performed and confidence scores are output in each frame. If the sensing system can detect V categories of target objects, it can output V confidence scores for each target object. The v-th confidence score (v∈[1,V]) represents the probability that the sensing system believes the target object belongs to the v-th category of target objects, and the sum of the V confidence scores is 1. The classification result of the target object is determined based on the maximum confidence score value (the category of the target object is determined based on the maximum confidence score value among the V confidence scores of the target object detection). Then, the maximum confidence score value is assigned to the corresponding sub-interval. The remaining confidence scores do not participate in the calculation of sensing uncertainty. Subsequently, the correct classification can be judged based on the pre-labeled target object category.

[0130] S4: Determine the optimization model based on the pre-set expert prior experience.

[0131] In one or more embodiments of this specification, the server may determine the optimization model based on preset expert prior experience.

[0132] Specifically, expert prior experience refers to a set of judgment matrices pre-constructed by the experts. The server can determine this set of pre-constructed judgment matrices. N experts pre-construct fuzzy number judgment matrices, forming a set of judgment matrices. The expression for this set of judgment matrices is:

[0133]

[0134]

[0135]

[0136]

[0137] in, To determine the set of matrices, including A fuzzy number judgment matrix. For the first A fuzzy number judgment matrix pre-constructed by experts , It is a positive integer. Given a ternary fuzzy array, representing the first... The evaluation results were predetermined by the experts. Compared to the evaluation results The importance assessment results. For the first The evaluation results were predetermined by the experts. Compared to the evaluation results The minimum value of the importance assessment result. For the first The evaluation results were predetermined by the experts. Compared to the evaluation results Reference values ​​for the importance assessment results. For the first The evaluation results were predetermined by the experts. Compared to the evaluation results The maximum value of the importance assessment results. , , respectively corresponding to the above There are a total of 10 evaluation results. It is a positive integer. For positive integers, when hour, .when hour, .and The upper and lower limits are determined by referring to the 1-9 scale method, and will not be elaborated here. For the first One evaluation result, For the first One evaluation result.

[0138] Then, the server can aggregate the N fuzzy number judgment matrices in the judgment matrix set to determine the aggregated fuzzy number judgment matrix.

[0139] The expression for the aggregated fuzzy number judgment matrix is:

[0140]

[0141]

[0142] in, This is the aggregated fuzzy number judgment matrix. To The result after aggregation. To The result after aggregation processing , Indicates to The result after aggregation processing , Indicates to The result after aggregation processing .

[0143] Therefore, the server further defuzzifies the aggregated fuzzy number judgment matrix to determine the initial judgment matrix.

[0144] The expression for the initial judgment matrix is:

[0145]

[0146]

[0147] in, This is the initial judgment matrix. for The result after deblurring.

[0148] Then, the server determines the consistency ratio of the initial judgment matrix based on the initial judgment matrix.

[0149] The expression for determining the consistency ratio of the initial judgment matrix is:

[0150]

[0151] in, CR This represents the consistency ratio of the initial judgment matrix. It is the largest eigenvalue of the initial judgment matrix. RI This is the preset average random consistency constant.

[0152] Afterwards, the server can determine whether the consistency ratio of the initial judgment matrix meets the preset standard, such as whether the consistency ratio is not greater than 0.2. If the consistency ratio is greater than 0.2, the server needs to reselect a preset set of N experts to pre-construct the judgment matrix set, and repeat the above process of determining the judgment matrix set until the consistency ratio of the initial judgment matrix is ​​determined, until the consistency ratio of the initial judgment matrix meets the preset standard.

[0153] Therefore, if the consistency ratio of the initial judgment matrix meets the preset standard, the server can introduce a preset correction factor into the initial judgment matrix to determine the corrected judgment matrix. Based on the corrected judgment matrix, the consistency ratio of the corrected judgment matrix and the correction weight of the evaluation results are determined.

[0154] The expression for the corrected judgment matrix is ​​as follows:

[0155]

[0156]

[0157] in, To correct the judgment matrix. To correct the judgment matrix of the first... i Line number j The elements of the column. In the initial judgment matrix The preset correction factor, .

[0158] The expression for determining the corrected weights of the evaluation results is:

[0159]

[0160]

[0161]

[0162] in, This represents the geometric mean of the elements in each row of the correction judgment matrix. Indicates the evaluation result Corrected weights. This indicates the adjusted weight for the evaluation result.

[0163] Next, the server constructs a consistency maximization objective function based on the consistency ratio of the revised judgment matrix. It then constructs an expert judgment retention function based on the initial and revised judgment matrices.

[0164] The expressions for the consensus maximization objective function and the expert judgment retention function are as follows:

[0165]

[0166]

[0167] in, This represents the consistency-maximizing objective function. This represents the preset constant attenuation coefficient, which can be taken as 10 in this specification. This represents the consistency ratio of the modified judgment matrix. This represents the retention function determined by experts.

[0168] Finally, the server can determine the optimization model based on the consistency maximization objective function, the expert judgment retention function, the corrected judgment matrix, and the corrected weights of the evaluation results.

[0169] The expression for the optimization model is:

[0170]

[0171] S5: Use a genetic algorithm to solve the optimization model and determine the weights of the evaluation results.

[0172] S6: Determine the credibility of the intelligent sensing system based on the evaluation result weight and the evaluation result.

[0173] In one or more embodiments of this specification, the server can modify the factors into optimization variables, employ a genetic algorithm to optimize and solve the optimization model, and determine the weights of the evaluation results. Then, based on the evaluation result weights and the evaluation results, the reliability of the intelligent sensing system is determined.

[0174] Specifically, a credibility score can be calculated for each test scenario. Then, based on the number of simulations or actual test scenarios implemented, the credibility score of the intelligent sensing system is comprehensively determined, as follows: As shown, the reliability of the intelligent sensing system is represented. The reliability levels, indicating the degree of reliability, can be found in Table 3.

[0175]

[0176] in, For the evaluation results The corresponding evaluation result weights.

[0177]

[0178] in, The credibility score of the intelligent sensing system.

[0179] Table 3. Mapping Relationship between Credibility Score and Credibility Level

[0180]

[0181] Therefore, in this specification, simulation tests can be conducted based on various basic and extended test simulation scenarios to obtain the numerical values ​​of each evaluation result under each scenario. Furthermore, based on 10 invited domain experts from universities, vehicle manufacturers, and third-party testing institutions, a judgment matrix set is constructed, and the weight of each evaluation result is calculated, as shown in Table 4.

[0182] Table 4. Evaluation results weight calculation results based on simulation test

[0183]

[0184] Finally, the numerical values ​​of each evaluation result under each test scenario and their corresponding single-scenario credibility scores are shown in Table 5. By averaging the credibility scores of all scenarios, the overall credibility score of the perception system is finally obtained as 0.761. Based on the preset credibility level classification standard in Table 3, the credibility level of the perception system is determined to be "relatively reliable".

[0185] Table 5. Values ​​of each evaluation result and their corresponding single-scenario reliability scores for each test scenario.

[0186]

[0187] This manual also provides a parameter table (Table 6) for the simulated sensor system. The simulated sensor system mainly includes a stereo camera, a depth camera, and a LiDAR model. The parameter settings for each sensor are shown in Table 6. The depth camera parameters are the same as those for the left camera in the stereo camera setup. This simulated sensor system can acquire stereo images, depth images, and LiDAR point cloud data from the scene.

[0188] Table 6 Parameter Table of Simulated Sensor System

[0189]

[0190] In one or more embodiments of this specification, the use of a genetic algorithm in step S5 can be referred to the prior art and does not involve the inventive points of this invention; it is merely an illustrative example here.

[0191] The server uses a genetic algorithm to optimize the model, with the correction factor as the optimization variable, to determine the optimized correction factor. Based on this optimized correction factor, an optimized correction judgment matrix is ​​then determined. Next, using the optimized correction judgment matrix and referring to the expression for determining the correction weights of the evaluation results, the evaluation result weights are calculated. Finally, the reliability of the intelligent sensing system can be determined based on the evaluation result weights and the evaluation results themselves.

[0192] Specifically, first, generate a file containing... Q An initial search population of individuals. Each individual in the initial search population has a corresponding chromosome vector. The expression for the set of chromosome vectors of the constructed initial search population is:

[0193]

[0194] In the formula, This represents the set of chromosomes of the individuals in the initial search population. Indicates the first [number] in the initial search group q The chromosome vector of each individual is determined by the fact that the upper triangular part of the correction judgment matrix (excluding the diagonal part) contains 45 elements and there are 45 corresponding correction factors. ( i < j ),therefore, Given a 45-dimensional vector, Its expression is:

[0195]

[0196] In the formula, This indicates that it will satisfy i < j 45 correction factors for the conditions The reordered serial number is In the initial search population, the individual chromosome vector gene values ​​are integers. .in, and correction factor ( i < j The mapping relationship is as follows: The 45 correction factors corresponding to the 45 elements in the upper triangular part (excluding the diagonal part) of the correction judgment matrix are sorted, following the... and The smaller the value, Serial number The smaller the rule.

[0197] Then, the chromosome vectors of individuals in the initial search population are initialized. Specifically, the chromosome vectors of individuals in the initial search population are initialized. The initialization is performed by randomly sampling from a uniform distribution U(-0.1, 0.1), which preserves the adjustment space for iterative optimization and avoids excessive deviation from the expert's initial judgment.

[0198] After that, The two functions are linearly weighted to construct the comprehensive fitness function, which is expressed as follows:

[0199]

[0200] In the formula, Represents the overall fitness function. Indicates the first t The dynamic weighting coefficients of the objective function are adjusted as the number of iterations increases. The design of these dynamic weighting coefficients follows these principles: in the early stages of optimization iteration, emphasis should be placed on retaining expert judgments to avoid premature over-correction; in the later stages of optimization iteration, emphasis should be placed on improving consistency to ensure the rationality of the final solution. The expression is as follows:

[0201]

[0202] In the formula, This represents the initial weight of the objective function, which can be 0.4 in this invention. This represents the final weight of the objective function, which can be 0.8 in this invention. This indicates the current iteration number during the optimization process. T This represents the maximum number of iterations preset during the optimization process.

[0203] Then, based on the selection, crossover, and mutation operations of the genetic algorithm, the population is iteratively updated to explore the solution space of the weight optimization problem. First, in the... t During the next iteration of optimization, the current search population... Sampling with replacement is used. Each sample is randomly and uniformly selected from the current population. For each individual, the fitness value of the selected individual is calculated based on the expression of the comprehensive fitness function and the expression of the dynamic weight coefficient of the objective function. The individual with the highest comprehensive fitness is placed into the mating pool. This process of repeated sampling, comprehensive fitness calculation and comparison, and the entry of the individual with the highest comprehensive fitness into the mating pool continues until the mating pool contains... Q There are 1 individuals, and the set of newly obtained individuals in the mating pool is denoted as . In this invention, The possible value is 3.

[0204] Then gather the individuals in the mating pool. Individuals within each group are randomly paired in pairs, with a preset crossover probability. Simulated binary crossover is performed on the chromosome vectors of paired individuals to generate two offspring vectors.

[0205] For example: Suppose there is a set of individuals in a mating pool. The vectors of the two paired somatic chromosomes are respectively and Crossover generates chromosome vectors for two offspring individuals. and The calculation process for the chromosome vector gene values ​​of its offspring individuals is as follows:

[0206]

[0207]

[0208] In the formula, and The sequence number of the first individual in the assigned pair is The chromosome vector gene values ​​and the sequence number of the second individual are: Chromosome vector gene values. This indicates that after pairing, the sequence number of the first newly generated offspring is... Chromosome vector gene values. This indicates that after pairing, the sequence number of the newly generated second offspring is... Chromosome vector gene values. This represents the cross-distribution index, which is independent for each dimension.

[0209] Secondly, perform mutation operations on the newly generated offspring. t In the next iteration of optimization, the set of individuals in the mating pool The set of offspring generated after the crossover is denoted as . , and Same size The number of individuals is also Q Based on the probability of mutation The offspring are subjected to mutation. Specifically, for the individuals selected for mutation, the genes on their chromosome vectors are determined with probability. Perform mutation, assume These are genes that mutate through probabilistic selection, and the mutation process is as follows:

[0210]

[0211] In the formula, This represents the gene value after the mutation operation. The variable length is determined by the mutation strategy adopted (such as polynomial mutation) to ensure that the mutated gene value still falls within the range of [-0.5, 0.5].

[0212] No. t During the iteration optimization process, the newly generated child set after each iteration is denoted as . .

[0213] Finally, from the first t In the next iteration of optimization, from the current search population Select the top 10% based on fitness. The individual is replaced by the newly generated offspring after the iteration. The bottom 10% of medium fitness ranking The individual, obtains the first t Generation population This completes the iterative update of the population.

[0214] Repeat the above operations until any iteration termination condition is met.

[0215] Iteration termination condition 1: The number of iterations reaches the preset maximum number of iterations. T .

[0216] Iteration termination condition two: The improvement of the maximum value of the overall fitness of an individual in the population over 10 consecutive generations is less than the threshold of 0.001.

[0217] Iteration termination condition three: The individual corresponding to the maximum overall fitness of an individual in a certain generation of the population has a consistency ratio of less than 0.1 and an expert retention rate. Greater than 0.7.

[0218] After the iteration termination condition is met, the chromosome gene vector of the individual corresponding to the individual with the maximum overall fitness in the current population is mapped according to the above mapping relationship (the above... and correction factor ( i < jThe mapping relationship involved is restored to the correction factor of the elements in the upper triangular part of the correction judgment matrix.

[0219] Furthermore, following the principle that the diagonal elements of the modified judgment matrix are all 1s and the elements in the upper and lower triangular parts are reciprocals of each other, an optimized modified judgment matrix is ​​obtained. Then, based on the optimized modified judgment matrix, the weights of the evaluation results are calculated. .

[0220] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 This paper presents a multi-dimensional testing and evaluation method for the reliability of intelligent sensing systems.

[0221] This instruction manual also provides Figure 6 The diagram shows a schematic structural representation of the electronic device. Figure 6 As shown, at the hardware level, this electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above. Figure 1 This paper presents a multi-dimensional testing and evaluation method for the reliability of intelligent sensing systems.

[0222] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0223] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0224] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0225] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0226] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0227] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0228] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0229] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0230] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0231] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0232] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0233] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic or disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0234] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0235] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0236] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0237] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0238] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A multi-dimensional test and evaluation method for the reliability of an intelligent sensing system, characterized in that, include: S1. Obtain the preset basic test scenario data; S2. Based on the basic test scenario data, and based on the preset interference scenario factors, determine the extended test scenario data corresponding to each basic test scenario data; S3. Based on the data from each basic test scenario and each extended test scenario, conduct autonomous driving tests on the vehicle to determine the perception data of the intelligent perception system of the autonomous vehicle. Based on the perception data of the intelligent perception system, determine the evaluation result of the intelligent perception system according to the preset evaluation method. The evaluation result includes the mean average accuracy, the mean intersection-over-union ratio, the ranging accuracy, the speed measurement accuracy, the perception delay, the attenuation rate score, the coefficient of variation, the consistency of multiple sensors, the stability of a single sensor, and the perception uncertainty. S4. Determine the aggregated fuzzy number judgment matrix based on the set of judgment matrices pre-constructed by experts; The aggregated fuzzy number judgment matrix is ​​defuzzified to determine an initial judgment matrix; the consistency ratio of the initial judgment matrix is ​​determined based on the initial judgment matrix; if the consistency ratio of the initial judgment matrix meets a preset standard, a preset correction factor is introduced into the initial judgment matrix to determine a corrected judgment matrix. Based on the initial judgment matrix and the corrected judgment matrix, the optimization model is determined; S5. Use a genetic algorithm to solve the optimization model and determine the weights of the evaluation results; S6. Determine the credibility of the intelligent sensing system based on the evaluation result weight and the evaluation result.

2. The multi-dimensional test and evaluation method for the reliability of an intelligent sensing system as described in claim 1, characterized in that, S4 determines the optimization model based on the initial judgment matrix and the modified judgment matrix, specifically including: Based on the modified judgment matrix, determine the consistency ratio of the modified judgment matrix and the modified weight of the evaluation result; Based on the consistency ratio of the modified judgment matrix, a consistency maximization objective function is constructed; and based on the initial judgment matrix and the modified judgment matrix, an expert judgment retention function is constructed. An optimization model is determined based on the consensus maximization objective function, the expert judgment retention function, the corrected judgment matrix, and the corrected weights of the evaluation results.

3. The multi-dimensional test and evaluation method for the reliability of an intelligent sensing system as described in claim 2, characterized in that, The expression for the set of judgment matrices is: in, The set of judgment matrices includes A fuzzy number judgment matrix; For the first A fuzzy number judgment matrix pre-constructed by experts , It is a positive integer; For the first The evaluation results were predetermined by the experts. Compared to the evaluation results Importance assessment results; For the first The evaluation results were predetermined by the experts. Compared to the evaluation results The minimum value of the importance assessment result; For the first The evaluation results were predetermined by the experts. Compared to the evaluation results Reference values ​​for the importance assessment results; For the first The evaluation results were predetermined by the experts. Compared to the evaluation results The maximum value of the importance assessment result; , , It is a positive integer. For positive integers, when hour, ,when hour, ; For the first One evaluation result, For the first One evaluation result.

4. The multi-dimensional test and evaluation method for the reliability of an intelligent sensing system as described in claim 3, characterized in that, The expression for the aggregated fuzzy number judgment matrix is: in, The aggregated fuzzy number judgment matrix; To The result after aggregation; To The result after aggregation processing , Indicates to The result after aggregation processing , Indicates to The result after aggregation processing .

5. The multi-dimensional test and evaluation method for the reliability of an intelligent sensing system as described in claim 4, characterized in that, The expression for the initial judgment matrix is: in, This is the initial judgment matrix; for The result after deblurring.

6. The multi-dimensional test and evaluation method for the reliability of an intelligent sensing system as described in claim 5, characterized in that, The expression for determining the consistency ratio of the initial judgment matrix is: in, CR The consistency ratio of the initial judgment matrix; The largest eigenvalue of the initial judgment matrix; RI This is the preset average random consistency constant.

7. The multi-dimensional test and evaluation method for the reliability of an intelligent sensing system as described in claim 6, characterized in that, The expression for the corrected judgment matrix is: in, The correction judgment matrix is ​​as follows; For the first in the modified judgment matrix i Line number j Column elements; In the initial judgment matrix The preset correction factor, .

8. The multi-dimensional test and evaluation method for the reliability of an intelligent sensing system as described in claim 7, characterized in that, The expression for determining the corrected weights of the evaluation results is: in, This represents the geometric mean of the elements in each row of the modified judgment matrix; Indicates the evaluation result Corrected weights; This indicates the corrected weight of the evaluation result.

9. The multi-dimensional test and evaluation method for the reliability of an intelligent sensing system as described in claim 8, characterized in that, The expressions for the consensus maximization objective function and the expert judgment retention function are as follows: in, This represents the consistency maximization objective function; This represents the preset constant attenuation coefficient; This represents the consistency ratio of the modified judgment matrix; This represents the expert's decision retention function.

10. The multi-dimensional test and evaluation method for the reliability of an intelligent sensing system as described in claim 9, characterized in that, The expression for the optimization model is: 。

Images