Multi-sensor sensing confidence evaluation and arbitration method and system in aided driving and medium

Abstract

The invention provides a multi-sensor sensing confidence evaluation and arbitration method and system in auxiliary driving and a medium, and belongs to the technical field of intelligent driving, and the method comprises the steps: obtaining a sensing result of each sensor, sensor state data and environment information, and carrying out the preprocessing; calculating a local confidence score of the sensing result of each sensor based on the signal quality of the sensing result of each sensor, the sensor state data and the environment information; identifying conflicts among sensing results of different sensors, determining conflict types, performing conflict arbitration on conflicts of different conflict types by using corresponding arbitration strategies based on local confidence scores, and generating fusion target information; the conflict types comprise target existence conflict arbitration, target type conflict arbitration and target state conflict arbitration; and outputting the fused target information to a vehicle controller so as to carry out intelligent auxiliary driving. The sensing accuracy is improved, the environmental adaptability is enhanced, and the driving control is optimized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent driving, and particularly relates to a multi-sensor perception confidence assessment and arbitration method, system and medium in assisted driving. BACKGROUND

[0002] With the development of intelligent assisted driving, multi-sensor fusion has become a key means to improve the accuracy of environmental perception. As core sensors, cameras and millimeter wave radars have their own unique advantages and limitations: cameras can provide rich texture and semantic information, but are significantly affected by light and weather; millimeter wave radars have high precision in distance measurement and speed measurement and strong anti-interference ability, but have low angle resolution and weak target classification ability. In actual application, the perception results of the same scene by the two often conflict, for example: the camera misses detection due to strong light glare, while the radar continuously detects the target; or the camera identifies the target as a "pedestrian", while the radar determines it as a "bicycle" due to its motion characteristics. If these conflicts are not properly handled, it will directly lead to the output of incorrect or unstable information by the perception system, thereby threatening driving safety.

[0003] The current mainstream method for handling such conflicts has significant limitations: first, the confidence assessment dimension is single, usually only relying on the original confidence score output by the perception algorithm, without comprehensively quantifying the specific influence of sensor real-time signal quality, its own working state (such as dirt, failure) and environmental factors (such as rain, fog, backlight) on the perception performance, resulting in one-sided evaluation results. Second, the conflict arbitration strategy is rigid, and fixed rules (such as sensor priority or fixed weight fusion) are often used, which cannot make adaptive decisions and are prone to misjudgment or missed detection in complex scenarios. Third, the weight adjustment mechanism is static and sluggish, and it is difficult to respond in real time to the dynamically changing driving environment (such as sudden weather changes, tunnel entry and exit), resulting in insufficient stability of the system during the transition period. Finally, the decision-making process lacks explainability, and is highly dependent on preset thresholds and empirical rules, forming a "black box" that is not conducive to system debugging, verification and functional safety compliance.

[0004] Existing improvement schemes, such as fusion result verification based on quality supervision model, or scene discrimination through correlation coefficients between sensors, have their own focuses, but none of them fundamentally build a refined, quantifiable and adaptive confidence assessment and conflict arbitration system.

[0005] Therefore, there is an urgent need for a method that can deeply integrate multi-dimensional information, accurately identify conflict types and implement intelligent arbitration, in order to improve the accuracy of the perception system. SUMMARY

[0006] In a first aspect, the embodiments of the present application provide a multi-sensor perception confidence assessment and arbitration method in assisted driving, comprising the following steps: S1. Obtain each sensor perception result, sensor state data and environment information, and pre-process them; S2. Calculate the local confidence score of each sensor perception result based on the signal quality of each sensor perception result, sensor state data and environment information; S3. Identify the conflicts between different sensor perception results, determine the conflict type, and then based on the local confidence score, use the corresponding arbitration strategy for conflict arbitration of different conflict types to generate fusion target information; the conflict type includes target existence conflict arbitration, target type conflict arbitration and target state conflict arbitration; S4. Output the fusion target information to the vehicle controller for intelligent auxiliary driving.

[0007] Further, the specific steps of step S1 are as follows: S11. Obtain camera perception results, which include target position information, target classification label, motion state, existence probability, target bounding box and size; S12. Obtain millimeter wave radar perception results, which include target position information, azimuth angle, point cloud cluster feature, radar scattering cross section value, target motion state and crossable flag; S13. Obtain sensor state data, which includes camera dirty or occlusion state identification, ambient light intensity value, radar received signal strength indication, self-diagnosis state and calibration state; S14. Obtain environment information, which includes weather state, lighting condition, road type and slope information.

[0008] Further, the specific steps of step S2 are as follows: S21. Calculate target feature confidence based on the signal quality of the perception result

[0009] Among them, for camera perception results, is the image quality score calculated based on target bounding box sharpness and classification probability; for millimeter wave radar perception results, is the radar echo quality score calculated based on point cloud cluster density and radar scattering cross section value stability; and is a normalized weight coefficient, and when the sensor types are different, only the corresponding quality score item is enabled; S22. Calculate sensor state confidence based on sensor state data ​​

[0010] wherein, is a binary fault identification determined based on the self-diagnosis state and the calibration state, is an occlusion impact factor determined based on the dirt / occlusion state identification, is a signal strength factor obtained by normalizing the received signal strength indication or the ambient light intensity value; S23. Calculate the environmental signal confidence based on the environmental information :

[0011] wherein, is a quantitative impact value of the kth environmental impact factor, is a sensitivity coefficient of the corresponding environmental impact factor to the current sensor type; the quantitative impact value and the sensitivity coefficient are obtained through a pre-calibrated mapping table; S24. Calculate the local confidence score of each sensor perception result according to the following formula :

[0012] wherein, , , is a dynamically configurable weight coefficient.

[0013] Further, the specific steps of identifying conflicts between different sensor perception results in step S3 are as follows: correlate and match the perception results of different sensors in the same space-time range, and then identify whether there is a conflict and determine the conflict type by comparing the attribute information of the correlated targets; if only one sensor reports the existence of the target, and the remaining associated sensors do not report the target, it is determined as a target existence conflict; if multiple sensors all report the existence of the target, but the reported target types are inconsistent, it is determined as a target type conflict; if multiple sensors all report the existence of the target and the target types are consistent, but the difference between the reported target state information exceeds the preset error tolerance, it is determined as a target state conflict.

[0014] Further, the specific steps of step S3 are as follows: S31. When there is a conflict between different sensor perception results, obtain the local confidence score of each sensor related to the conflict target; S32. Invoke the corresponding arbitration strategy according to the determined conflict type, and combine the obtained local confidence score to arbitrate and generate the fused target information.

[0015] Further, the step S32 specifically comprises the following steps: S321. judging the determined conflict type; If the conflict type is the target existence conflict, go to step S322; If the conflict type is the target type conflict, go to step S323; If the conflict type is the target state conflict, go to step S323; S322. judging whether the sensor that does not recognize the target exists fault according to the sensor state data; If the fault exists, arbitrating that the target exists, and outputting the perception result of the sensor that recognizes the target; If the fault does not exist, comparing the local confidence scores of the sensors of the conflict parties, and adopting the perception result of the sensor with the higher score; Go to step S4; S323. calculating the local confidence score difference of the sensors of the conflict parties :

[0016] Wherein, is the higher local confidence score of the conflict parties, is the lower local confidence score; If the local confidence score difference is greater than a preset threshold, adopting the target type of the sensor with the higher local confidence score; If the local confidence score difference is less than or equal to the preset threshold, performing the physical attribute arbitration by comprehensively considering the motion state and action trajectory information of the target; Go to step S4; S324. using a dynamic weighted fusion algorithm based on the local confidence score and the measurement data variance to fuse the state information of the conflict, and going to step S4.

[0017] Further, the dynamic weighted fusion algorithm in step S324 is used to fuse the same target state parameter of the conflict, and the fusion formula is as follows:

[0018] Wherein, and are the perception values of the target state parameter by the first sensor and the second sensor respectively; and are the weight coefficients of the target state parameter corresponding to the first sensor and the second sensor respectively; The calculation formula of the weight coefficient of any sensor is:

[0019] In the formula, Therefore, the sensor corresponds to the feature confidence level of the current target. Therefore, the sensor corresponds to the state confidence level of the current target. Therefore, the sensor measures the variance of the target state parameters over a set time period. This is the measurement value of the sensor in this measurement.

[0020] Furthermore, the specific steps of step S4 are as follows: S41. Integrate the target information into a standard format data frame containing the target global ID, target presence status, target type, target location, target velocity, and target acceleration, and send it to the vehicle controller; S42. The vehicle controller performs trajectory prediction and risk assessment on the received standard format data frames and generates vehicle control commands; S43. The vehicle controller executes vehicle control commands to perform at least one of the following, lane changing, braking, or steering intelligent driver assistance controls.

[0021] Secondly, embodiments of this application also provide a multi-sensor perception confidence assessment and arbitration system for assisted driving, including: The data acquisition and preprocessing module is used to acquire the sensing results, sensor status data and environmental information from each sensor, and to perform preprocessing. The local confidence calculation module is used to calculate the local confidence score of each sensor's sensing results based on the signal quality, sensor status data, and environmental information of each sensor's sensing results. The conflict arbitration module is used to identify conflicts between the perception results of different sensors and determine the conflict type. Then, based on the local confidence score, it uses the corresponding arbitration strategy to arbitrate the conflicts of different conflict types and generate fused target information. The conflict types include target existence conflict arbitration, target type conflict arbitration, and target state conflict arbitration. The control output module is used to output the fused target information to the vehicle controller for intelligent assisted driving.

[0022] Thirdly, embodiments of this application also provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the multi-sensor perception confidence assessment and arbitration method for assisted driving as described in the first aspect.

[0023] As can be seen from the above technical solutions, this application has the following advantages: The method, system, and medium for multi-sensor perception confidence assessment and arbitration in assisted driving provided in this application comprehensively consider the signal quality, sensor status, and environmental information of sensor perception results to perform refined confidence assessment of sensor perception results. It also adopts corresponding arbitration strategies based on different conflict types, resolving the problem of multi-sensor perception result conflicts, improving the accuracy and reliability of the intelligent assisted driving system's environmental perception, and reducing the probability of false detections and missed detections. This application can dynamically adjust weighting coefficients according to different environmental conditions, enabling the system to possess strong environmental adaptability and maintain stable performance even in complex and changing driving scenarios, thus improving the applicability and stability of the intelligent assisted driving system in various practical applications. Furthermore, by accurately fusing multi-sensor perception information and outputting it to the vehicle controller, this application provides information support for intelligent assisted driving control such as following, lane changing, braking, or steering, improving driving safety and comfort, and promoting the development and application of intelligent assisted driving. Attached Figure Description

[0024] To more clearly illustrate the technical solution of this application, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of the overall process of the multi-sensor perception confidence assessment and arbitration method in assisted driving according to the present invention.

[0026] Figure 2 This is a schematic diagram illustrating the specific process of the multi-sensor perception confidence assessment and arbitration method in assisted driving according to the present invention.

[0027] Figure 3 This is a schematic diagram of the multi-sensor perception confidence assessment and arbitration system for assisted driving according to the present invention. Detailed Implementation

[0028] Various embodiments of this disclosure will be described more fully in the following detailed description of the specific steps of the multi-sensor perception confidence assessment and arbitration method in assisted driving. This disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of this disclosure to the specific embodiments disclosed herein, but rather this disclosure should be understood to cover all adjustments, equivalents, and / or alternatives falling within the spirit and scope of the various embodiments of this disclosure.

[0029] This embodiment provides a method for multi-sensor perception confidence assessment and arbitration in assisted driving, which refines confidence assessment, resolves sensor conflicts, improves perception accuracy, enhances environmental adaptability, and optimizes intelligent driving control.

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] Please see Figure 1 The diagram shows a flowchart of a multi-sensor perception confidence assessment and arbitration method in assisted driving according to a specific embodiment. The method includes the following steps: S1. Acquire the sensing results, sensor status data, and environmental information from each sensor, and perform preprocessing; It should be noted that this step obtains a comprehensive and accurate foundation of input data. Preprocessing ensures the usability and consistency of the data, providing a data source for confidence assessment and conflict arbitration. S2. Based on the signal quality, sensor status data, and environmental information of each sensor's sensing results, calculate the local confidence score of each sensor's sensing results; It should be noted that this step, by comprehensively considering information from multiple dimensions, conducts a refined confidence assessment of the sensor perception results, making the assessment results more comprehensive, objective, and reliable. This provides a reasonable and accurate basis for conflict arbitration and is a key step in resolving sensor perception conflicts. S3. Identify conflicts between the perception results of different sensors and determine the conflict type. Then, based on the local confidence score, use the corresponding arbitration strategy to arbitrate the conflicts of different conflict types and generate fused target information. The conflict types include target existence conflict arbitration, target type conflict arbitration, and target state conflict arbitration. It should be noted that this step can accurately identify conflicts between sensor perception results and adopt targeted arbitration strategies according to different conflict types, thus resolving the sensor perception conflict problem and generating accurate and reliable fused target information, which directly determines the improvement of the perception performance of the intelligent assisted driving system. S4. The fused target information is output to the vehicle controller for intelligent assisted driving; It should be noted that applying the fused target information after precise arbitration to the intelligent assisted driving control of the vehicle realizes the connection from perception to control, ensuring the accuracy and reliability of the intelligent assisted driving system, and improving driving safety and comfort.

[0032] This embodiment resolves sensor conflicts, improves perception accuracy and reliability, enhances environmental adaptability, and optimizes intelligent assisted driving control through multi-dimensional confidence assessment and intelligent arbitration.

[0033] Furthermore, as a refinement and extension of the specific implementation methods described above, and to fully illustrate the specific implementation process in this embodiment, another method for multi-sensor perception confidence assessment and arbitration in assisted driving is provided, such as... Figure 2 As shown, the method includes the following steps: S1. Acquire the sensing results, sensor status data, and environmental information from each sensor, and perform preprocessing; the specific steps of step S1 are as follows: S11. Obtain camera perception results, the camera perception results including target location information, target classification label, motion state, existence probability, target bounding box and size; S12. Obtain millimeter-wave radar sensing results, which include target location information, azimuth angle, point cloud cluster characteristics, radar cross section value, target motion state, and traversable marker positions; S13. Acquire sensor status data, which includes a camera dirt or obstruction status indicator, ambient light intensity value, radar received signal strength indication, self-diagnosis status, and calibration status. S14. Obtain environmental information, including weather conditions, lighting conditions, road type, and slope information; S2. Based on the signal quality, sensor status data, and environmental information of each sensor's sensing results, calculate the local confidence score of each sensor's sensing results; the specific steps of step S2 are as follows: S21. Calculate the target feature confidence level based on the signal quality of the perception results. :

[0034] Regarding the camera's perception results, The image quality score is calculated based on the sharpness of the target bounding box and the classification probability; for millimeter-wave radar perception results, The radar echo quality score is calculated based on the stability of point cloud cluster density and radar cross section value. and These are normalized weighting coefficients, and only the corresponding quality scoring items are enabled when the sensor type is different; S22. Calculate sensor state confidence level based on sensor state data. :

[0035] in, It is a binary fault identifier (0 indicates normal, 1 indicates fault) determined based on self-diagnostic status and calibration status. The occlusion impact factor (range 0 to 1) is determined based on the dirt / occlusion status indicator. The signal strength factor (ranging from 0 to 1) is obtained by normalizing the Received Signal Strength Indication (RSSI) or ambient light intensity value. S23. Calculate the confidence level of environmental signals based on environmental information. :

[0036] in, Let be the quantified impact value of the k-th environmental impact factor (including weather conditions, lighting conditions, road type, and slope). The sensitivity coefficient of the current sensor type to corresponding environmental influencing factors; quantified influence value. With sensitivity coefficient Obtained through a pre-calibrated mapping table; S24. Calculate the local confidence score of the sensing results of each sensor according to the following formula. :

[0037] in, , , These are dynamically configurable weighting coefficients; Specifically, based on the weather conditions and lighting conditions in the environmental information, the corresponding weight coefficient combinations are dynamically loaded from a pre-set configuration strategy table; among them, the weight coefficients for environmental signal confidence are determined under rain, snow, fog, or backlight scenarios. Set to a level higher than the target feature confidence level Weighting coefficients; S3. Identify conflicts between the perception results of different sensors and determine the conflict type. Then, based on the local confidence score, use the corresponding arbitration strategy to arbitrate the conflicts of different conflict types and generate fused target information. The conflict types include target existence conflict arbitration, target type conflict arbitration, and target state conflict arbitration. The specific steps for identifying conflicts between the perception results of different sensors and determining the type of conflict in step S3 are as follows: The perception results of different sensors within the same spatiotemporal range are correlated and matched. Then, by comparing the attribute information of the correlated targets, the existence of conflicts is identified and the type of conflict is determined. If only one sensor reports the presence of a target, while the remaining associated sensors do not report the target, it is determined to be a target presence conflict; If multiple sensors report the presence of a target, but the reported target types are inconsistent, it is determined to be a target type conflict; If multiple sensors report the presence of a target and the target type is the same, but the difference between the reported target status information exceeds the preset error tolerance, it is determined to be a target status conflict. The specific steps of step S3 are as follows: S31. When there is a conflict between the perception results of different sensors, obtain the local confidence scores of each sensor related to the conflicting target; S32. Invoke the corresponding arbitration strategy according to the determined conflict type, and perform arbitration in conjunction with the obtained local confidence score to generate fusion target information; the specific steps of step S32 are as follows: S321. Determine the type of conflict; If the conflict type is a target existence conflict, proceed to step S322; If the conflict type is a target type conflict, proceed to step S323; If the conflict type is a target state conflict, proceed to step S323; S322. Determine whether the sensor that did not detect the target is faulty based on the sensor status data; If a fault exists, the arbitration assumes the target exists and outputs the perception result of the sensor that identified the target; If there is no fault, compare the local confidence scores of the conflicting sensors and adopt the perception result of the one with the higher score; Proceed to step S4; S323. Calculate the difference in local confidence scores between the sensors of the two conflicting parties. :

[0038] in, The higher local confidence score among the conflicting parties. The local confidence score is relatively low. If the local confidence score difference If the score is greater than the preset threshold, the target type with the higher local confidence score will be adopted. If the local confidence score difference If the value is less than or equal to a preset threshold, then physical attribute arbitration is performed based on the target's motion state and trajectory information. Proceed to step S4; S324. A dynamic weighted fusion algorithm based on local confidence scores and measurement data variance is used to fuse the conflicting state information, and then proceed to step S4; In step S324, the dynamic weighted fusion algorithm uses the following fusion formula for conflicting target state parameters:

[0039] in, and These are the perceived values ​​of the target state parameters by the first and second sensors, respectively. and These are the weighting coefficients for the target state parameters of the first and second sensors, respectively. Weighting coefficient of any sensor The calculation formula is:

[0040] In the formula, Therefore, the sensor corresponds to the feature confidence level of the current target. Therefore, the sensor corresponds to the state confidence level of the current target. Therefore, the sensor measures the variance of the target state parameters over a set time period. This is the measurement value measured by the sensor in this instance; For example, taking the fusion of vertical distance as an example, the formula is:

[0041] in, The longitudinal distance sensed by the camera. The longitudinal distance sensed by millimeter-wave radar; and These are the weighting coefficients for the camera and the millimeter-wave radar, respectively. ,in This represents the historical variance of the longitudinal distance of the sensor. This is the current distance measurement value; S4. Output the fused target information to the vehicle controller for intelligent assisted driving; the specific steps of step S4 are as follows: S41. Integrate the target information into a standard format data frame containing the target global ID, target presence status, target type, target location, target velocity, and target acceleration, and send it to the vehicle controller; S42. The vehicle controller performs trajectory prediction and risk assessment on the received standard format data frames and generates vehicle control commands; S43. The vehicle controller executes vehicle control commands to perform at least one of the following, lane changing, braking, or steering intelligent driver assistance controls.

[0042] Based on typical driving scenarios, the following explanations will focus on three core scenarios: target existence conflict, target type conflict, and target state conflict. The basic parameters are set as follows: Using a passenger vehicle intelligent driver assistance system as an example, the system is equipped with a high-definition camera and a 77GHz millimeter-wave radar, and presets weight coefficients for calculating various confidence levels: target feature confidence weights. =0.4, Sensor state confidence weight =0.3, environmental signal confidence weight =0.3; the threshold for the difference in confidence scores for target type conflicts is set to 10%; the historical measurement variance statistics duration in the dynamic weighted fusion algorithm is 5s; After the vehicle starts, the system enters normal working state, and the data acquisition and preprocessing module continuously collects the following basic data: Basic camera perception data: target bounding box sharpness score 0.85, classification probability 0.82, dirt status marked as "no dirt", ambient light intensity normalized value 0.7; Millimeter-wave radar sensing basic data: point cloud cluster density score 0.9, radar cross section (RCS) stability score 0.88, normalized received signal strength indication (RSSI) value 0.85, self-diagnostic status is "normal", calibration status is "calibrated"; Environmental information: Real-time weather and road type information can be obtained through vehicle-mounted meteorological sensors and high-precision maps, providing a basis for subsequent dynamic adjustment of confidence levels; Scenarios for conflict of interest arbitration: The scenario involves a vehicle traveling at 60 km / h on an urban expressway during a rainstorm. A stationary, disabled vehicle is located 120 meters ahead. The camera lens is partially obstructed by raindrops, preventing it from detecting the target ahead. However, the millimeter-wave radar remains unaffected by the rain and fog and reliably detects the disabled vehicle. Step S1: Data Acquisition and Preprocessing Camera perception results: No target detection results, only output of its own status data (dirt / occlusion status is marked as "partial occlusion", ambient light intensity value is normalized to 0.4); Millimeter-wave radar detection results: Target position detected (longitudinal distance 120m, lateral offset 0.5m), azimuth angle 0.8°, RCS value 15dBsm, target motion status is "stationary", and the passable marker is "non-passable"; Sensor status data: Camera self-diagnostic status is "normal (partial occlusion)" and calibration status is "calibrated"; Radar self-diagnostic status is "normal" and calibration status is "calibrated". Environmental information: Weather condition is "heavy rain", lighting conditions are "cloudy", road type is "urban expressway", and slope is "flat". Calculation of local confidence score in step S2 (1) Confidence of target features

[0043] The camera detected no target. =0; The millimeter-wave radar calculations are based on point cloud cluster density (0.9) and RCS value stability (0.88). =0.4×0.9+0.6×0.88=0.888 (Activate the radar corresponding scoring item, =0.6, =0); (2) Sensor state confidence

[0044] Camera: Fault indicator F=0 (no hardware fault), occlusion impact factor O=0.6 (partial occlusion), signal strength factor S=0.4 (insufficient lighting), then =(1-0)×(1-0.6)×0.4=0.16; Millimeter-wave radar: Fault indicator F=0, obstruction impact factor O=0 (no obstruction), signal strength factor S=0.85 (RSSI normal), then =(1-0)×(1-0)×0.85=0.85; (3) Confidence level of environmental signals

[0045] Camera sensitivity to heavy rain =0.8, the quantitative impact value of the rainstorm q=0.7, then =1-0.8×0.7=0.44; Sensitivity coefficient of millimeter-wave radar to rainstorm weather =0.1, then =1 - 0.1 × 0.7 = 0.93; (4) Total score of local confidence Camera: =0.4×0+0.3×0.16+0.3×0.44=0.18; Millimeter-wave radar: =0.4×0.888+0.3×0.85+0.3×0.93=0.895; Conflict arbitration in step S3: The system determines that there is a target presence conflict. First, it checks the status of the camera that did not identify the target: the camera has no hardware failure, but there is partial occlusion. Since the camera confidence level (0.18) is much lower than that of the radar (0.895), and the radar is in normal condition and has strong environmental adaptability, the arbitration result is that the target exists. The perception result of the millimeter-wave radar is adopted, and the target position, motion status and other information are output. Control output of step S4: The fused target information is encapsulated into a standard data frame and sent to the vehicle controller. The controller determines that there is a stationary obstacle ahead, triggers a warning and executes a deceleration strategy to avoid a rear-end collision. This demonstrates the value of the invention points of multi-dimensional confidence assessment and environmental adaptive weight adjustment.

[0046] Scenario for target type conflict arbitration The scenario involves a vehicle traveling on a suburban road at a speed of 40 km / h. A cyclist (actually an electric bicycle at 20 km / h) is 50 meters ahead. Due to backlighting, the camera identifies the target as a "pedestrian" (classification probability 0.65). The millimeter-wave radar, based on the target's motion characteristics and RCS value, identifies the target as a "bicycle" (confidence correlation index 0.72), triggering a target type conflict. Steps S1-S2: Data Acquisition and Confidence Calculation. After preprocessing and confidence calculation, the local confidence score of the camera is obtained. =0.62, local confidence level of millimeter-wave radar =0.75 (Due to the backlight environment, the confidence level of the camera environment is reduced, while the radar environment has strong adaptability and higher confidence level).

[0047] Conflict arbitration in step S3: (1) The conflict type is determined to be a target type conflict. First, the confidence score difference is calculated: =(0.75-0.62) / 0.62×100%≈20.97%, and this difference is greater than the preset threshold of 10%; (2) Preliminary Arbitration Logic According to the rules, the radar's "bicycle" classification result could have been directly adopted. However, in order to reflect the invention point of the arbitration method that integrates physical rules, the system further combined the target's motion state for verification: the target's motion speed was 20km / h, which far exceeded the normal walking speed of pedestrians (≤5km / h), and it met the motion characteristics of non-motorized vehicles. Therefore, the final arbitration target type was "bicycle", avoiding misjudgment based solely on confidence level. Control output of step S4 The vehicle controller adjusts the following safety distance to 30m based on the target type of "bicycle" and activates a special warning for pedestrians / non-motorized vehicles to ensure driving safety.

[0048] Scenario for conflict arbitration of target state The scenario involves a vehicle following another vehicle on a highway. The vehicle in front is traveling at 80 km / h, and the current distance between the two vehicles is 80 m. The camera outputs a longitudinal distance of 82 m and a lateral speed of 0.2 m / s for the vehicle in front through visual ranging. The millimeter-wave radar outputs a longitudinal distance of 78 m and a lateral speed of 0.15 m / s. The discrepancy between the two sets of information exceeds the preset error tolerance, triggering a target state conflict. Data acquisition and confidence calculation in steps S1-S2: Calculations show that the local confidence level of the camera... =0.8 (high image clarity, normal lighting), local confidence level of millimeter-wave radar =0.9 (strong ranging stability, no interference); simultaneously, the variance of the longitudinal distance measurement between the two devices within 5 seconds, and the camera variance are statistically analyzed. =1.2m, radar variance =0.3m; Conflict arbitration in step S3: (1) Dynamic weight calculation based on the weight formula Calculate the weights of both separately: Camera weight =0.78×0.82×(1-1.2 / 82)≈0.63; Radar weight =0.88×0.85×(1-0.3 / 78)≈0.74; (2) Dynamic weighted fusion Perform fusion calculation on longitudinal distance: =(82×0.63+78×0.74) / (0.63+0.74)≈79.8m\); After fusing the lateral velocity, it is 0.17m / s, resulting in more accurate target state information.

[0049] Control output of step S4: The vehicle controller adjusts the parameters of the adaptive cruise control (ACC) system based on the fused distance and speed information to maintain a safe following distance of 80m and achieve smooth following.

[0050] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0051] like Figure 3 As shown, the following are embodiments of the multi-sensor perception confidence assessment and arbitration system for assisted driving provided in this disclosure. This system and the multi-sensor perception confidence assessment and arbitration methods for assisted driving in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the multi-sensor perception confidence assessment and arbitration system for assisted driving, please refer to the embodiments of the multi-sensor perception confidence assessment and arbitration methods for assisted driving described above.

[0052] The system includes: The data acquisition and preprocessing module is used to acquire the sensing results, sensor status data and environmental information from each sensor, and to perform preprocessing. The local confidence calculation module is used to calculate the local confidence score of each sensor's sensing results based on the signal quality, sensor status data, and environmental information of each sensor's sensing results. The conflict arbitration module is used to identify conflicts between the perception results of different sensors and determine the conflict type. Then, based on the local confidence score, it uses the corresponding arbitration strategy to arbitrate the conflicts of different conflict types and generate fused target information. The conflict types include target existence conflict arbitration, target type conflict arbitration, and target state conflict arbitration. The control output module is used to output the fused target information to the vehicle controller for intelligent assisted driving.

[0053] This embodiment improves the accuracy and reliability of the intelligent assisted driving system's perception of complex environments, enhances the system's environmental adaptability, optimizes driving control, and improves driving safety and comfort through the interactive collaboration of the data acquisition and preprocessing module, the local confidence calculation module, the conflict arbitration module, and the control output module.

[0054] The storage medium provided in this application stores a program product capable of implementing a multi-sensor perception confidence assessment and arbitration method in assisted driving.

[0055] The multi-sensor perception confidence assessment and arbitration method in assisted driving includes: acquiring the perception results, sensor status data, and environmental information of each sensor, and performing preprocessing; calculating the local confidence score of each sensor perception result based on the signal quality, sensor status data, and environmental information; identifying conflicts between different sensor perception results and determining the conflict type; then, based on the local confidence score, using corresponding arbitration strategies to arbitrate conflicts of different conflict types to generate fused target information; the conflict types include target existence conflict arbitration, target type conflict arbitration, and target state conflict arbitration; and outputting the fused target information to the vehicle controller for intelligent assisted driving.

[0056] In some possible implementations, the multi-sensor perception confidence assessment and arbitration method for assisted driving disclosed herein can be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0057] The storage medium disclosed herein may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0058] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for assessing and arbitrating the confidence level of multi-sensor perception in assisted driving, characterized in that, Includes the following steps: S1. Acquire the sensing results, sensor status data, and environmental information from each sensor, and perform preprocessing; S2. Based on the signal quality, sensor status data, and environmental information of each sensor's sensing results, calculate the local confidence score of each sensor's sensing results; S3. Identify conflicts between the perception results of different sensors and determine the conflict type. Then, based on the local confidence score, use the corresponding arbitration strategy to arbitrate the conflicts of different conflict types and generate fused target information. The conflict types include target existence conflict arbitration, target type conflict arbitration, and target state conflict arbitration. S4. The fused target information is output to the vehicle controller for intelligent assisted driving.

2. The method for multi-sensor perception confidence assessment and arbitration in assisted driving according to claim 1, characterized in that, The specific steps of step S1 are as follows: S11. Obtain camera perception results, the camera perception results including target location information, target classification label, motion state, existence probability, target bounding box and size; S12. Obtain millimeter-wave radar sensing results, which include target location information, azimuth angle, point cloud cluster characteristics, radar cross section value, target motion state, and traversable marker positions; S13. Acquire sensor status data, which includes a camera dirt or obstruction status indicator, ambient light intensity value, radar received signal strength indication, self-diagnosis status, and calibration status. S14. Obtain environmental information, including weather conditions, lighting conditions, road type, and slope information.

3. The method for multi-sensor perception confidence assessment and arbitration in assisted driving according to claim 1, characterized in that, The specific steps of step S2 are as follows: S21. Calculate the target feature confidence level based on the signal quality of the perception results. : Regarding the camera's perception results, The image quality score is calculated based on the sharpness of the target bounding box and the classification probability; for millimeter-wave radar perception results, The radar echo quality score is calculated based on the stability of point cloud cluster density and radar cross section value. and These are normalized weighting coefficients, and only the corresponding quality scoring items are enabled when the sensor type is different; S22. Calculate sensor state confidence level based on sensor state data. : in, A binary fault identifier determined based on self-diagnostic status and calibration status. The occlusion impact factor is determined based on the dirt / occlusion status indicator. The signal strength factor is obtained by normalizing the received signal strength indication or ambient light intensity value. S23. Calculate the confidence level of environmental signals based on environmental information. : in, This represents the quantified impact value of the kth environmental impact factor. The sensitivity coefficient of the current sensor type to corresponding environmental influencing factors; quantified influence value. With sensitivity coefficient Obtained through a pre-calibrated mapping table; S24. Calculate the local confidence score of the sensing results of each sensor according to the following formula. : in, , , These are dynamically configurable weighting coefficients.

4. The method for multi-sensor perception confidence assessment and arbitration in assisted driving according to claim 1, characterized in that, The specific steps for identifying conflicts between the perception results of different sensors and determining the type of conflict in step S3 are as follows: The perception results of different sensors within the same spatiotemporal range are correlated and matched. Then, by comparing the attribute information of the correlated targets, the existence of conflicts and the type of conflicts are identified. If only one sensor reports the presence of a target, while the remaining associated sensors do not report the target, it is determined to be a target presence conflict; If multiple sensors report the presence of a target, but the reported target types are inconsistent, it is determined to be a target type conflict; If multiple sensors report the presence of a target and the target type is the same, but the difference between the reported target status information exceeds the preset error tolerance, it is determined to be a target status conflict.

5. The method for multi-sensor perception confidence assessment and arbitration in assisted driving according to claim 4, characterized in that, The specific steps of step S3 are as follows: S31. When there is a conflict between the perception results of different sensors, obtain the local confidence scores of each sensor related to the conflicting target; S32. Invoke the corresponding arbitration strategy according to the determined conflict type, and perform arbitration in combination with the obtained local confidence score to generate fusion target information.

6. The method for multi-sensor perception confidence assessment and arbitration in assisted driving according to claim 5, characterized in that, The specific steps of step S32 are as follows: S321. Determine the type of conflict; If the conflict type is a target existence conflict, proceed to step S322; If the conflict type is a target type conflict, proceed to step S323; If the conflict type is a target state conflict, proceed to step S323; S322. Determine whether the sensor that did not detect the target is faulty based on the sensor status data; If a fault exists, the arbitration assumes the target exists and outputs the perception result of the sensor that identified the target; If there is no fault, compare the local confidence scores of the conflicting sensors and adopt the perception result of the one with the higher score; Proceed to step S4; S323. Calculate the difference in local confidence scores between the sensors of the two conflicting parties. : in, The higher local confidence score among the conflicting parties. The local confidence score is relatively low. If the local confidence score difference If the score is greater than the preset threshold, the target type with the higher local confidence score will be adopted. If the local confidence score difference If the value is less than or equal to a preset threshold, then physical attribute arbitration is performed based on the target's motion state and trajectory information. Proceed to step S4; S324. A dynamic weighted fusion algorithm based on local confidence scores and measurement data variance is used to fuse conflicting state information, and then proceed to step S4.

7. The method for multi-sensor perception confidence assessment and arbitration in assisted driving according to claim 6, characterized in that, In step S324, the dynamic weighted fusion algorithm uses the following fusion formula for conflicting target state parameters: in, and These are the perceived values ​​of the target state parameters by the first and second sensors, respectively. and These are the weighting coefficients for the target state parameters of the first and second sensors, respectively. Weighting coefficient of any sensor The calculation formula is: In the formula, Therefore, the sensor corresponds to the feature confidence level of the current target. Therefore, the sensor corresponds to the state confidence level of the current target. Therefore, the sensor measures the variance of the target state parameters over a set time period. This is the measurement value of the sensor in this measurement.

8. The method for multi-sensor perception confidence assessment and arbitration in assisted driving according to claim 1, characterized in that, The specific steps of step S4 are as follows: S41. Integrate the target information into a standard format data frame containing the target global ID, target presence status, target type, target location, target velocity, and target acceleration, and send it to the vehicle controller; S42. The vehicle controller performs trajectory prediction and risk assessment on the received standard format data frames and generates vehicle control commands; S43. The vehicle controller executes vehicle control commands to perform at least one of the following, lane changing, braking, or steering intelligent driver assistance controls.

9. A multi-sensor perception confidence assessment and arbitration system for assisted driving, characterized in that, include: The data acquisition and preprocessing module is used to acquire the sensing results, sensor status data and environmental information from each sensor, and to perform preprocessing. The local confidence calculation module is used to calculate the local confidence score of each sensor's sensing results based on the signal quality, sensor status data, and environmental information of each sensor's sensing results. The conflict arbitration module is used to identify conflicts between the perception results of different sensors and determine the conflict type. Then, based on the local confidence score, it uses the corresponding arbitration strategy to arbitrate the conflicts of different conflict types and generate fused target information. The conflict types include target existence conflict arbitration, target type conflict arbitration, and target state conflict arbitration. The control output module is used to output the fused target information to the vehicle controller for intelligent assisted driving.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the multi-sensor perception confidence assessment and arbitration method for assisted driving as described in any one of claims 1 to 8.

Images