Method for road surface condition assessment based on multiple sensors and vehicle

Abstract

The application provides a multi-sensor-based road surface state evaluation method and a vehicle. The method comprises the following steps: determining signal confidence of a plurality of sensors, and determining scene classification probability based on measurement signals output by the sensors respectively; the signal confidence represents the measurement reliability of the sensors; the scene classification probability represents the classification probability of the sensors in each low adhesion scene; based on the signal confidence and the scene classification probability, a fusion distribution probability corresponding to each low adhesion scene is determined; based on the fusion distribution probability, a trust degree corresponding to each low adhesion scene is determined; and based on the trust degree, a road surface state result of a road on which the vehicle travels is determined. The method can avoid the problem of weak scene adaptability caused by single-sensor perception dimension and isolated processing of different sensor data, and can improve the evaluation accuracy of a wet and slippery road to a certain extent.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a road surface condition assessment method and vehicle based on multiple sensors. Background Technology

[0002] With the development of intelligent driving technology, a vehicle's ability to perceive road conditions in its driving environment has become a key factor affecting driving safety. Slippery road surfaces (such as water, ice, snow, etc.) significantly reduce tire adhesion, leading to longer braking distances and decreased handling stability, which is one of the main causes of traffic accidents.

[0003] In related technologies, the identification or assessment of slippery road surfaces mainly adopts single sensor passive response mode, vision-led identification technology, and cloud-based early warning based on historical data. However, in these solutions, because data from different sensors are often processed in isolation, there are problems such as single perception dimension and weak scene adaptability. The accuracy of identifying or assessing slippery road surfaces is low, which leads to vehicles frequently triggering false interventions or missing alarms in complex environments. Summary of the Invention

[0004] To address the aforementioned issues, this application provides at least one method, apparatus, vehicle, medium, and computer program product for road surface condition assessment based on multiple sensors. This solution has the technical effect of improving the accuracy of assessment in low-adhesion scenarios.

[0005] The technical solution of this application is implemented as follows: Firstly, this application provides a road surface condition assessment method based on multiple sensors. This method includes at least: determining the signal confidence levels of multiple sensors; and determining scene classification probabilities based on the measurement signals output by each sensor. The signal confidence level characterizes the measurement reliability of the sensors; the scene classification probability characterizes the classification probability of each sensor in each low-adhesion scene; based on the signal confidence levels and scene classification probabilities, determining the fusion allocation probability corresponding to each low-adhesion scene; based on the fusion allocation probability, determining the trust level corresponding to each low-adhesion scene; and based on the trust levels, determining the road surface condition result of the road where the vehicle is traveling.

[0006] Secondly, this application provides a road surface condition assessment device based on multiple sensors, which includes at least a first determining module, a second determining module, a third determining module and a fourth determining module; The first determining module is used to determine the signal confidence levels of multiple sensors, and to determine the scene classification probability based on the measurement signals output by each sensor; the signal confidence level characterizes the measurement reliability of the sensor; the scene classification probability characterizes the classification probability of each sensor in each low-adhesion scene; the second determining module is used to determine the fusion allocation probability corresponding to each low-adhesion scene based on the signal confidence levels and the scene classification probabilities; the third determining module is used to determine the trust level corresponding to each low-adhesion scene based on the fusion allocation probability; the fourth determining module is used to determine the road surface state result of the vehicle driving road based on the trust levels.

[0007] Thirdly, this application provides a vehicle, which includes a vehicle body, a processor and a memory. The memory stores a computer program or instructions. When the computer program or instructions are run by the processor, they implement the multi-sensor-based road condition assessment method provided in the first aspect.

[0008] Fourthly, this application also provides a storage medium storing a computer program or instructions, which, when executed by a processor, implements any of the multi-sensor-based road condition assessment methods provided in the first aspect.

[0009] Fifthly, this application also provides a computer program product, which includes a computer program or instructions, which, when executed by a processor, implement any of the multi-sensor-based road condition assessment methods provided in the first aspect.

[0010] In this scheme, by calculating the fusion allocation function of each sensor for each low-adhesion scenario, the degree of trust of each sensor for each low-adhesion scenario can be expressed through the fusion allocation function; by calculating the trust degree corresponding to the fusion allocation function, the degree of support of multiple sensors for each low-adhesion scenario can be expressed through the trust degree; finally, based on the trust degree, the road surface state of the vehicle's driving road is determined. It can obtain the most likely road surface state of the vehicle's driving road based on the degree of support of multiple sensors for each low-adhesion scenario, which can avoid the problem of weak scene adaptability caused by the single sensor's limited perception dimension and the isolated processing of data from different sensors, and can improve the assessment accuracy of slippery road surfaces to a certain extent. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of a first optional process for a multi-sensor-based road surface condition assessment method provided in an embodiment of this application. Figure 2 A flowchart illustrating a first optional specific implementation of the multi-sensor-based road surface condition assessment method provided in this application embodiment; Figure 3 A flowchart illustrating a first optional specific implementation of the multi-sensor-based road surface condition assessment method provided in this application embodiment; Figure 4 This is a schematic diagram of an optional structure of a multi-sensor-based road condition assessment device provided in an embodiment of this application.

[0012] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the specific technical solutions of the application will be further described in detail below with reference to the accompanying drawings of the embodiments of this application. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application.

[0014] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0015] In the following description, the terms "first," "second," and "third" are used only to distinguish different objects and do not represent a specific order of objects, nor are they constituting a chronological order. It is understood that "first," "second," and "third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0016] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0017] This application provides a method, apparatus, vehicle, storage medium, and program product for road surface condition assessment based on multiple sensors. The following describes various embodiments of the method, apparatus, vehicle, storage medium, and program product for road surface condition assessment based on multiple sensors provided in this application.

[0018] To facilitate understanding, some technical terms will be explained first.

[0019] DS (Dempster) Shafer's Evidence Theory: Shafer's evidence theory is a mathematical theory for handling uncertain information. Its core advantage lies in its ability to directly express and process "uncertain" and "unknown" information, and it is a generalization of classical probability theory. Shafer's evidence theory includes the following core concepts: 1) Frame of Discernment: A frame of discernment is a set of mutually exclusive and complete basic propositions (assumptions) representing all possible answers to a problem, with only one answer being true. The mathematical representation of a frame of discernment is: ; in, This is a basic assumption. (Set) The set of all subsets of is called the power set, expressed as .

[0020] 2) Basic Probability Assignment (BPA): Basic probability assignment represents the distribution of probability values ​​from the power set. arrive mapping Also called the mass function, it satisfies: ,and ; in, It indicates the exact degree of support of the evidence for proposition A, rather than the probability.

[0021] 3) Belief Function (Bel): The belief function represents the minimum level of belief in proposition A, and is the minimum level of belief in all subsets of A. The sum of mass functions means "at least one The reasoning that "A is true" reflects conservative reasoning.

[0022] 4) Dempster Combination Rule: The Dempster Combination Rule is used to merge the BPAs of two or more independent sources of evidence (such as multiple sensors or multiple experts) to obtain a composite BPA, which reflects the combined effect of the evidence.

[0023] Kalman filtering is an algorithm that uses the state equations of a linear system to optimally estimate the system state using observed input and output data. Since the observed data includes noise and interference from the system, the optimal estimation can also be viewed as a filtering process. Kalman filtering can estimate the state of a dynamic system from a series of data containing measurement noise, provided the measurement variance is known. Because it is easy to implement in computer programming and can update and process field-acquired data in real time, Kalman filtering is currently the most widely used filtering method and has found good applications in communication, navigation, guidance, and control, among other fields.

[0024] This application provides a multi-sensor-based road surface condition assessment method. This control method can be executed by a multi-sensor-based road surface condition assessment device, which can be deployed in electronic devices or vehicles and implemented through a program stored in the memory of a processor.

[0025] This application uses a vehicle application scenario as an example to explain the technical solution of this application. However, it should be noted that the solution of this application can also be applied to other sensing scenarios, such as road icing warning, road slipperiness sensing, etc.

[0026] The following section uses the vehicle controller as an example to illustrate this multi-sensor-based road condition assessment method.

[0027] refer to Figure 1 The process may include, but is not limited to, S101 to S104 described below. S101. Determine the signal confidence of multiple sensors, and determine the scene classification probability based on the measurement signals output by each sensor; the signal confidence characterizes the measurement reliability of the sensor; the scene classification probability characterizes the classification probability of each sensor in each low-adhesion scene.

[0028] Multiple sensors may include, but are not limited to, at least two of the following: millimeter-wave radar, vision sensor, wheel-end sensor, temperature sensor, humidity sensor, rain sensor, inertial measurement unit, and V2I roadside sensor.

[0029] Millimeter-wave radar, in particular, operates in the millimeter-wave band. Millimeter waves typically refer to the 30–300 GHz frequency range (wavelength 1–10 mm). The shorter wavelength of millimeter waves corresponds to a smaller optical area, making them more suitable for detecting small targets compared to microwave radar. Except for long-range millimeter-wave radars used for special space target observation, millimeter-wave radars are generally suitable for short-range detection below 30 km.

[0030] The main function of vision sensors is to acquire enough raw images for machine vision systems to process, such as various cameras or other cameras. Wheel-end sensors are sensor clusters installed on wheel hubs / bearings / steering knuckles. They are responsible for collecting physical quantities such as wheel speed, steering, tire pressure, temperature, load, and slip ratio in real time and outputting them to controllers such as ABS / ESP / EPB / ADAS / VCU. They are the basic perception source for vehicle active safety, chassis control, and autonomous driving. Examples include magnetoelectric (electromagnetic induction) wheel speed sensors, Hall effect wheel speed sensors, indirect / direct tire pressure monitoring, and wheel-end temperature / load sensors.

[0031] Temperature sensors are used to measure ambient temperature for accurate perception of the surrounding environment; humidity sensors measure ambient humidity; rain sensors, including vehicle-mounted rain sensors or conventional rain sensors, detect the amount of rainfall in the current environment. An Inertial Measurement Unit (IMU) measures an object's three-axis attitude angles (or angular rates) and acceleration, acquiring dynamic characteristics of a vehicle's attitude and speed. Vehicle-to-Infrastructure (V2I) roadside communication systems are a core component of vehicle-road cooperation, referring to clusters of intelligent infrastructure deployed along roadsides responsible for perception, computation, communication, and decision-making, providing vehicles with real-time information on road conditions, signals, and hazards. Multiple sensors can be integrated into a vehicle or communicate with it to provide various types of environmental perception information.

[0032] The vehicles mentioned here can include, but are not limited to, electric vehicles, hybrid vehicles, and gasoline-powered vehicles. Signals measured by sensors can be transmitted to the vehicle controller within the vehicle control system, allowing the controller to jointly process signals from multiple sensors. Alternatively, the sensors can communicate remotely with processors in other data processing centers, enabling the processors to jointly process signals from multiple sensors.

[0033] For the same sensor, its reliability or measurement signal confidence level varies in different scenarios due to signal quality and environmental influences. Therefore, to ensure the vehicle obtains accurate environmental parameters, the signal confidence levels of multiple onboard sensors can be determined based on the type of environment the vehicle is in. For example, the signal confidence level of a rain sensor in a parking garage or sunny environment is negligible; its signal confidence level is higher in rainy conditions, but lower in snowy, foggy, or nighttime conditions. Similarly, a temperature sensor has higher signal confidence level outdoors, but its signal confidence level is lower indoors due to poor air circulation and potential temperature variations in different areas. Signal confidence level characterizes the measurement reliability of a sensor. The higher the signal confidence level, the higher the measurement reliability of the sensor.

[0034] Low-adhesion scenarios refer to scenarios where the coefficient of adhesion between the vehicle and the ground is low, such as dry scenarios, water accumulation scenarios, icy scenarios, snow accumulation scenarios, and muddy scenarios.

[0035] Furthermore, the vehicle processor can run classification software / programs / models, capable of independently determining scene classification probabilities based on the measurement signals output by each sensor. That is, each sensor can correspond to a probability of various scenes. The scene classification probability characterizes the classification probability of each sensor in various low-adhesion scenarios. For example, for a humidity sensor, if its output humidity ranges widely, its scene classification probability is higher for rain / snow / fog scenes and lower for sunny / underground parking scenes. For a vision sensor, if its output image has high brightness / luminosity, its scene classification probability is higher for sunny / snow scenes and lower for rain / fog / night / underground parking scenes. Specific values ​​can be determined based on experience or deep learning model methods.

[0036] S102. Based on the confidence level of each signal and the classification probability of each scene, determine the fusion allocation probability corresponding to each low-attachment scene.

[0037] The fusion allocation probability can represent the degree of common support of different sensors for shared scenarios in different low-adhesion environments. The specific process for determining the fusion allocation probability can be as follows: First, based on the confidence level of each signal and the classification probability of each scenario, the allocation probability of each sensor in each low-adhesion scenario is calculated using basic probability allocation theory. Second, based on this allocation probability, the fusion allocation probability corresponding to each low-adhesion scenario is calculated.

[0038] For example, for two sensors and Their mass functions are respectively and The fusion method is shown in Formula 1: (Formula 1); in, K is the conflict coefficient, representing the possibility that scenario B and scenario C do not share scenario A. The value of K ranges from [0,1). When K=1, it defaults to complete conflict, and the rule does not apply. Indicates sensor For scenario B, the quality function or probability assignment Indicates sensor For the quality function or probability assignment of scenario C, sum the products of all quality functions of B and C that support shared scenario A, then divide by the normalization factor 1-K to obtain the quality function of the fused shared scenario A; the symbol " "Indicates integration."

[0039] For multiple sensors, an iterative fusion approach can be used. First, the quality functions or allocation probabilities of the first two sensors are fused to obtain a new quality function or allocation probability. Then, this new quality function or allocation probability is fused with the quality function or allocation probability of the third sensor, and so on, until the quality functions or allocation probabilities of multiple sensors are combined into a total fused allocation probability, as shown in Formula 2 below: (Formula 2); in, This represents the probability of fusion allocation, and N represents the number of sensors.

[0040] S103. Based on the fusion allocation probability, determine the trust level corresponding to each low-attachment scenario.

[0041] Trust level is the trust function. The function value is calculated as shown in Formula 3: (Formula 3); Where B represents all subsets contained in set A; This indicates the level of trust in scenario A. This represents the probability of fusion assignment for scenario B. For example, if A represents a combined scenario of water accumulation and ice formation, then B can be water accumulation, ice formation, or an empty set.

[0042] S104. Based on each level of trust, determine the road surface condition of the road where the vehicle is traveling.

[0043] Based on the respective trust levels corresponding to each low-adhesion scenario, the road surface condition of the road on which the vehicle travels can be determined. This road surface condition can represent the actual condition of the road surface, such as dryness, water accumulation, icing, or snow accumulation; it can also include specific state parameter values, such as the adhesion coefficient, water film thickness, and snow thickness. For example, if the trust level for a snow accumulation scenario is higher than that for other scenarios, the road surface condition can be considered snow accumulation.

[0044] In addition, considering that the noise level of sensor signals may vary in different scenarios, the road condition results determined directly based on the confidence level may not meet the requirements of precise control in terms of accuracy. To improve the accuracy of the road condition results, the Kalman filtering method can be used to calculate the specific state parameter values ​​based on the confidence level and the measured values ​​of various road condition parameters of the road where the vehicle is traveling.

[0045] Based on the embodiments disclosed in this application, by calculating the fusion allocation function of each sensor for each low-adhesion scenario, the degree of trust of each sensor for each low-adhesion scenario can be expressed through the fusion allocation function; by calculating the trust degree corresponding to the fusion allocation function, the degree of support of multiple sensors for each low-adhesion scenario can be expressed through the trust degree; finally, based on the trust degree, the road surface state of the vehicle's driving road is determined. The most likely road surface state of the vehicle's driving road can be obtained based on the degree of support of multiple sensors for each low-adhesion scenario, which can avoid the problem of weak scene adaptability caused by the single sensor's perception dimension and the isolated processing of data from different sensors, and can improve the assessment accuracy of slippery road surfaces to a certain extent.

[0046] In some embodiments, the road surface condition results include road surface condition assessment values; the following is a detailed description of S104, determining the road surface condition results of the vehicle's driving road based on various confidence levels: S141. Obtain the measured values ​​of various road surface condition parameters.

[0047] Road surface condition parameters characterize the condition of the road surface and can include factors such as the coefficient of adhesion, water film thickness, smoothness, curvature, and the probability of ice presence. Among these, a higher coefficient of adhesion indicates greater vehicle grip, greater smoothness indicates less road surface undulation, and greater curvature indicates greater road curvature (i.e., larger curves).

[0048] Among the measured values ​​of road surface condition parameters, some values ​​can be predetermined, such as the adhesion coefficient and curvature of various road surfaces (dry road surface, waterlogged road surface, icy road surface); others can be measured in real time, such as water film thickness which can be measured by millimeter-wave radar, and smoothness which can be measured by visual sensors. Here, "road" refers to the road on which vehicles travel, and may include location information (i.e., the road where the vehicle is located).

[0049] S142. Based on each confidence level and each measurement value, determine the observed values ​​of each pavement state parameter at the current time step.

[0050] Observations refer to the most likely measurements in a real-world scenario. For a combined scenario, the observed values ​​cannot be simply assumed to be the measurements from any one of the scenarios included in that combined scenario. Therefore, measurements can be combined with confidence levels to jointly determine the observed values ​​of each road surface state parameter at the current time step.

[0051] The confidence level can act as a weighting value. Based on the confidence level, the measured values ​​of each road surface state parameter can be weighted and summed to obtain the observed value of each road surface state parameter at the current time step.

[0052] S143. Based on the observed values, determine the pavement condition assessment values ​​of each pavement condition parameter at the current time step.

[0053] Since the observed values ​​represent the most likely measured values ​​in the actual scenario, they can be directly used as the road surface condition evaluation values ​​for each road surface condition parameter at the current time step. In addition, considering that the sensor will carry process noise and observation noise during the measurement process, the Kalman filter method can be used to process the observed values, so that the road surface condition evaluation values ​​for each road surface condition parameter at the current time step can be calculated.

[0054] Based on the above embodiments disclosed in this application, the confidence level can be combined with the measured values ​​of various road surface condition parameters to calculate the corresponding observation values. Based on the observation values, the road surface condition evaluation value can be obtained as the road surface condition result through direct determination or Kalman filtering, which can improve the accuracy and readability of the road surface condition result to a certain extent.

[0055] In some embodiments, the following provides a detailed description of S143, determining the pavement state evaluation value of each pavement state parameter at the current time step based on the observed values: S1431. Based on the fusion allocation probability, determine the overall confidence level of fusing multiple sensors.

[0056] The overall confidence score is used to represent the overall reliability of the fusion result. Its calculation method is shown in Formula 4 below: (Formula 4); in," "" indicates "other", that is, other scenarios that are not within the defined range of low-attachment scenarios. The probability of fusion allocation for other scenarios can be calculated using Formula 2.

[0057] S1432. When the overall confidence level is greater than or equal to the confidence level threshold, determine the observation noise covariance matrix based on the overall confidence level.

[0058] The confidence threshold is used to determine whether the fusion result should be applied to low-attachment scene identification or estimation, and is usually set manually. If the overall confidence score is greater than or equal to the confidence threshold, it indicates that the fusion result is reliable, and the observation noise covariance matrix of the fusion result can be determined based on the overall confidence score. Specifically, the overall confidence score can be used to determine the ratio between the observation noise covariance matrix of the fusion result and the calibration observation noise covariance matrix. Based on this ratio and the calibration observation noise covariance matrix, the observation noise covariance matrix of the fusion result can be calculated, as shown in Formula 5 below: Formula 5; in, This indicates the minimum confidence level limit (e.g., it can be set to a small value such as 0.1 or 0.2). Represents the observation noise covariance matrix. This represents the calibration observation noise covariance matrix, which can be determined based on the sensor type and application scenario.

[0059] S1433. Based on the observation noise covariance matrix, Kalman filtering is applied to the observation values ​​to obtain the road surface condition assessment value at the current time step.

[0060] The specific mathematical form of Kalman filtering on the observations is shown in Equations 6 to 10 below: Formula 6; Formula 7; Formula 8; Formula 9; Formula 10; in, This represents the predicted state value at the current time step k. Let A be the optimal state value at the previous time step k-1; let B be the state transition matrix; and let B be the control matrix. This is the state control variable for the current time step k; This represents the predicted covariance value; Represents the observation matrix; Indicates Kalman gain; Represents the observed value; This represents the state estimate at the current time step k; This represents the covariance at the current time step k; This represents the observation noise covariance matrix; the subscript "-1" indicates the inverse matrix. Among them, and As variables, therefore, according to and This allows us to obtain the road surface condition assessment value at the current time step after Kalman filtering.

[0061] Based on the embodiments disclosed in this application, the optimal road surface condition assessment value can be calculated by using the Kalman filtering method in combination with the observation noise covariance matrix. To a certain extent, the road surface condition assessment value can be made unaffected by sensor observation noise, thereby improving the accuracy of the road surface condition assessment value.

[0062] In some embodiments, the following provides a detailed explanation of S1433, which involves performing Kalman filtering on the observed values ​​based on the observation noise covariance matrix to obtain the road surface condition assessment value at the current time step: S14331. Determine the environmental stability index of the driving environment.

[0063] In Kalman filtering, in addition to correcting observation noise, it can also correct for sensor process noise. Process noise, also known as "state noise," refers to the random fluctuations of the measured physical quantity (i.e., process variable) over time. It is independent of the sensor's measurement capability and reflects the inherent dynamic characteristics of the measured object. Observation noise is caused by factors such as sensor accuracy, electromagnetic interference, and quantization errors, reflecting the random error between the sensor's measured value and the true state of the system.

[0064] In this embodiment, the vehicle is in motion, which causes the environment in which the sensor is located to change frequently, thereby amplifying the incompleteness of the sensor measurement model. Therefore, the more stable the environment, the less process noise.

[0065] For the purpose of accurately evaluating environmental stability, this application introduces an environmental stability index, the value of which can be calculated from the change in environmental parameters between two time steps. These environmental parameters are provided by measurements taken by various sensors.

[0066] S14332. Based on the environmental stability index, determine the process noise covariance matrix that is common to all sensors.

[0067] The process noise covariance matrix measures the difference in process noise among multiple sensors. Based on the environmental stability index, the process noise covariance matrix common to all sensors can be calculated, as shown in Formula 11 below: Formula 11; in, Represents the process noise covariance matrix. To calibrate the process noise covariance matrix, calibration is performed according to the sensor type and application scenario. This represents the environmental stability index. This is a scaling factor, which is environment-dependent. The greater the environmental differences, the larger the scaling factor. and They exhibit consistent increase and decrease characteristics.

[0068] S14333: Based on the process noise covariance matrix and the observation noise covariance matrix, Kalman filtering is applied to the observation values ​​to obtain the road surface condition assessment value at the current time step.

[0069] The process noise covariance matrix can also be used in Kalman filtering to correct for process noise in the estimates. After introducing the process noise covariance matrix, Equation 7 above can be modified to Equation 12, as follows: Formula 12; in, This represents the process noise covariance matrix.

[0070] After introducing the process noise covariance matrix, the observed values ​​can be Kalman filtered using formulas 6, 8 to 10 and 12 based on the process noise covariance matrix and the observation noise covariance matrix, thereby calculating the road surface condition assessment value at the current time step.

[0071] Based on the above embodiments disclosed in this application, a process noise covariance matrix related to environmental stability can be introduced into the Kalman filtering process, which can, to a certain extent, make the road surface condition assessment value unaffected by environmental changes and improve the accuracy of the road surface condition assessment value.

[0072] In some embodiments, the following provides a detailed description of S14331, determining the environmental stability index of the driving environment: Step A1: Determine the temperature and humidity changes of the driving environment at the current time step from the measurement signal.

[0073] The temperature change at the current time step refers to the absolute difference between the temperature value at the current time step and the temperature value at the previous time step. The humidity change at the current time step refers to the absolute difference between the humidity value at the current time step and the humidity value at the previous time step. These are expressed as follows: and Where t represents the current time step.

[0074] The measurement signal includes the temperature signal output by the temperature sensor and the humidity signal output by the humidity sensor. These signals can have the same measurement period, so their output time steps can also correspond one-to-one. Therefore, the temperature and humidity values ​​of the current time step and the previous time step can be obtained from the temperature and humidity signals, and then the temperature change and humidity change can be calculated.

[0075] Step A2: Determine the environmental stability index of the driving environment based on the changes in temperature and humidity.

[0076] The specific calculation method for the driving environment stability index based on temperature and humidity changes is shown in Formula 13 below: Formula 13; Where T represents the time step span, and k represents the current time step. This application mainly considers the impact of temperature and humidity on environmental changes, and does not consider other environmental parameters.

[0077] Based on the embodiments disclosed in this application, the environmental stability index at the current time step can be calculated from the temperature and humidity changes, which have a significant impact on environmental changes. This can reduce the computational complexity of the environmental stability index and improve its computational efficiency to some extent.

[0078] Based on the above implementation method, the low-adhesion scene evaluation method based on multiple sensors may further include steps S1434 and S1435: S1434. If the overall confidence level is less than the confidence level threshold, determine the historical road surface condition assessment value of the previous time step.

[0079] If the overall confidence level is less than the confidence threshold, the observation result can be considered unreliable. To obtain the road surface condition estimate at the current time step, the historical road surface condition assessment value at the previous time step can be obtained first and then predicted.

[0080] S1435. Based on historical pavement condition assessment values, determine the pavement condition assessment value of the road at the current time step.

[0081] The method for predicting the road surface condition assessment value at the current time step based on historical road surface condition assessment values ​​is shown in Formula 14 below: Formula 14; in, Here is the state transition matrix. This is process noise; This represents the predicted road surface condition assessment value at the current time step. This represents the historical pavement condition assessment value at the previous time step. This historical pavement condition assessment value is the optimal pavement condition assessment value at the previous time step.

[0082] Based on the above embodiments disclosed in this application, when the overall confidence level is less than the confidence level threshold, the historical road surface condition assessment value of the previous time step can be used to predict the road surface condition assessment value of the current time step. This can, to some extent, avoid the problem of low accuracy of the road surface condition assessment value due to low overall confidence level and improve the accuracy of the final output road surface condition assessment value.

[0083] In some embodiments, pavement condition parameters include water film thickness. The following provides a detailed explanation of S142, which involves determining the observed values ​​of each pavement condition parameter at the current time step based on various confidence levels and measurements: S1421. Determine the radar signal output by the millimeter-wave radar in the sensor from the measurement signal, and determine the first confidence level of the water film scene from the confidence level.

[0084] Road surface condition parameters may include water film thickness, which can include both the depth of liquid water and the depth of snow accumulation.

[0085] The measurement signal contains signals from multiple sensors; therefore, the radar signal output by the millimeter-wave radar can be obtained by filtering from the measurement signal. Simultaneously, the confidence level includes confidence levels for various low-adhesion scenarios; therefore, the first confidence level for the water film scenario can be determined from multiple confidence levels.

[0086] S1422. Based on radar signals, determine the thickness measurement value of the water film.

[0087] The basic principle of radar signal measurement is to determine distance based on the phase difference between the transmitted and received signals. Water films also cause changes in phase difference; therefore, the thickness of the water film can be calculated from the radar signal. The calculation method can be found in the mathematical modeling methods commonly used in radar systems, and will not be elaborated here.

[0088] S1423. Based on the first confidence level and the thickness measurement value, determine the thickness observation value of the water film at the current time step.

[0089] Based on the first confidence level and the thickness measurement, the observed thickness of the water film at the current time step is calculated using the following formula 15: Formula 15; in, The thickness observation value at the current time step. The first level of trust for road flooding. The first level of trust is based on the snow accumulation on the road surface. This is the measured value of the water film thickness. This is a measurement of snow thickness.

[0090] In some embodiments, pavement state parameters include the adhesion coefficient. The following provides a detailed explanation of S142, which involves determining the observed values ​​of each pavement state parameter at the current time step based on various confidence levels and measurements: S1424. Determine the second trust level corresponding to each road attachment scenario from the trust level.

[0091] Road surface condition parameters may also include the coefficient of adhesion. The coefficient of adhesion, also known as the coefficient of friction, measures the ability of a vehicle's wheel hub to prevent relative slippage between the wheel hub and the road surface.

[0092] Similar to the water film scenario mentioned above, the trust level includes trust levels for various low-adhesion scenarios. Therefore, the second trust level for an adhesion scenario can be determined from multiple trust levels.

[0093] S1425. Based on the measured values ​​of the adhesion coefficient corresponding to each second level of trust and each road adhesion scenario, determine the observed value of the adhesion coefficient at the current time step.

[0094] Road adhesion scenarios can include at least four types: dry asphalt pavement, wet asphalt pavement, flooded pavement, and icy surface. Each scenario has its own corresponding adhesion coefficient measurement value, which can be calibrated by the vehicle wheel manufacturer or determined experimentally. Correspondingly, each scenario also has its own corresponding second confidence level (which can be 0).

[0095] The specific method for calculating the observed value of the adhesion coefficient at the current time step based on the second confidence level and the adhesion coefficient measurement is shown in Formula 16 below. Formula 16; in, This represents the observed value of the adhesion coefficient at the current time step; , , , These represent the second level of trust for dry asphalt pavement, wet asphalt pavement, icy surface, and waterlogged surface, respectively. , , , These represent the measured values ​​of the adhesion coefficient for dry asphalt pavement, wet asphalt pavement, icy surface, and waterlogged surface, respectively.

[0096] In some embodiments, road surface state parameters include the probability of ice presence. The following provides a detailed explanation of S142, which involves determining the observed values ​​of each road surface state parameter at the current time step based on various confidence levels and measurements: S1426. Determine the third level of trust for the ice surface scene from the level of trust.

[0097] Road surface condition parameters may also include the probability of ice presence. The probability of ice presence is used to characterize the likelihood of road icing. Similar to the above embodiments, the trust level includes trust levels for various low-adhesion scenarios; therefore, a third trust level for the ice surface scenario can be determined from multiple trust levels.

[0098] S1427. Determine the third level of trust as the probability of ice surface existence at the current time step.

[0099] Since the third confidence level can characterize the conservative estimate of the ice surface's existence probability, it can be defined as the probability of the ice surface's existence at the current time step.

[0100] Based on the above embodiments disclosed in this application, when the road surface condition parameters include water film scenario and / or adhesion scenario and / or ice surface scenario, the observed values ​​can be calculated for the measurement values ​​corresponding to each scenario, which can improve the accuracy of the observed values ​​of the road surface condition parameters to a certain extent.

[0101] In some embodiments, low-adhesion scenarios include at least one of dry scenarios, water film scenarios, ice surface scenarios, and snow-covered scenarios; the following provides a detailed explanation of S102, which involves determining the fusion allocation probability corresponding to each low-adhesion scenario based on the confidence level of each signal and the classification probability of each scenario: S121. Based on the confidence level of each signal and the classification probability of each scene, determine the allocation probability of each sensor in each target scene; the target scenes include dry scenes, water film scenes, ice surface scenes, snow scenes and combinations thereof; The allocation probability is used to represent the degree of support of the measurement signal for each scenario, and its calculation method is shown in Formula 17 below: Formula 17; in, Indicates sensor Signal confidence; Let A represent the scene classification probability of scene A; scene B is included in the entire set. All subsets of scenes; Indicates sensor The probability of scenario A (also called the quality function); Indicates sensor The probability of assigning other scenarios. Other scenarios refer to scenarios other than the target scenario, that is, scenarios outside the sensor's perception range.

[0102] S122. Based on the allocation probability, determine the fusion allocation probability corresponding to each target scene.

[0103] The fusion allocation probability corresponding to each target scene can be obtained by fusing the allocation probabilities of the target scene from multiple sensors. For the specific calculation method, please refer to Formulas 1 and 2 and their parameter explanations, which will not be repeated here.

[0104] Based on the above embodiments disclosed in this application, the fusion allocation probability of multi-sensor fusion can be calculated for multiple target scenes respectively, which can improve the accuracy of the fusion allocation probability to a certain extent.

[0105] In some embodiments, the following provides a detailed description of S101, determining the signal confidence levels of multiple sensors: S111. Obtain the signal-to-noise ratio and environmental impact parameters of each sensor in the current environment; the current environment characterizes the driving environment of the vehicle equipped with each sensor.

[0106] The current environment characterizes the driving environment of the vehicle equipped with each sensor, such as rainy, tunnel, sunny, or icy conditions. The signal reliability of the sensors is affected by their own signal-to-noise ratio (SNR) and the environment. Therefore, the SNR of each sensor and the environmental impact parameters of the current environment can be obtained by looking up a table. The SNR data table is shown in Table 1 below: Table 1

[0107] The environmental impact parameters are shown in Table 2 below:

[0108] in, Indicates the signal-to-noise ratio. Indicates sensor Environmental impact parameters.

[0109] S112. Based on the signal-to-noise ratio and environmental impact parameters, determine the signal confidence level of each sensor.

[0110] Based on the signal-to-noise ratio and environmental impact parameters, the specific method for calculating the signal confidence of each sensor is shown in Equation 18 below: Formula 18; in, Indicates sensor The signal confidence level and These represent the weights of the signal-to-noise ratio and the environmental impact, respectively. Indicates sensor The signal quality index value can be calculated from the signal-to-noise ratio. Indicates sensor Environmental impact parameters under the current environment. Among them, and The following information can be obtained by looking up Table 3: Table 3

[0111] Based on the embodiments disclosed in this application, the signal confidence level of each sensor can be calculated according to the signal-to-noise ratio and environmental influence parameters of each sensor, which fully considers the influence of the environment on the sensor signal strength and noise intensity, and can improve the accuracy of signal confidence level to a certain extent.

[0112] In some embodiments, the following provides a detailed description of S112, which involves determining the signal confidence level of each sensor based on the signal-to-noise ratio and environmental influence parameters: S1121. Determine the response characteristic parameters corresponding to each sensor.

[0113] The response characteristic parameters are the sensor's characteristic parameters, reflecting the sensor's response to the signal-to-noise ratio, and can determine the saturation rate of the signal quality function. Specific values ​​can be found in Table 3. A column. By looking up the values ​​in Table 3, the response characteristic parameters corresponding to each sensor can be obtained.

[0114] S1122. Based on the response characteristic parameters and signal-to-noise ratio, determine the signal quality index values ​​corresponding to each sensor.

[0115] The method for calculating the signal quality index values ​​corresponding to each sensor based on the response characteristic parameters and signal-to-noise ratio is shown in Formula 19 below: Formula 19; in, This represents the signal quality index value. Indicates sensor The response characteristic parameters, Indicates sensor The signal-to-noise ratio.

[0116] S1123. Determine the signal confidence level of each sensor based on the signal quality index value, the first preset weight value of the signal quality index value, the environmental impact parameter, and the second preset weight value of the environmental impact parameter.

[0117] The calculation method for the signal confidence of each sensor can be found in Formula 18. This is the first preset weight value for the signal quality index. This is the second preset weight value for the environmental impact parameters.

[0118] Based on the above embodiments disclosed in this application, the signal quality index value of the sensor can be calculated by the response characteristic parameters and the signal-to-noise ratio. The signal confidence level of each sensor can be calculated by combining the first preset weight value of the signal quality index value and the second preset weight value of the environmental influence parameter. This can fully consider the measurement bias caused by different scenarios and different sensors, thereby improving the accuracy and reliability of the signal confidence level to a certain extent.

[0119] This application provides a vehicle, which includes a vehicle body, a processor, and a memory. The memory stores a computer program or instructions. When the processor runs the computer program or instructions, it implements the various steps of the above-described method embodiment of the multi-sensor-based road condition assessment method.

[0120] The following example illustrates the process of road condition assessment based on multiple sensors.

[0121] With the development of intelligent driving technology, a vehicle's ability to perceive road conditions has become a key factor affecting driving safety. Slippery road surfaces (such as water, ice, and snow) significantly reduce tire grip, leading to longer braking distances and decreased handling stability, and are a major cause of traffic accidents. Currently, the industry primarily relies on the following technical solutions for identifying slippery road surfaces: (1) Single sensor passive response mode; Current mass-produced vehicles generally use rain sensors or wheel speed sensors as the primary basis for judgment. Among them, the rain sensor solution indirectly infers the possibility of slippery road surface by the intensity of rainfall on the windshield, but this solution has obvious defects: it cannot distinguish the type of precipitation (rain / snow / hail) and the actual road surface condition (such as the road surface being dry after light rain); it is completely ineffective in scenarios with no precipitation and low adhesion (such as morning dew, water accumulation in underground parking garages, water stains left by road sprinkler trucks); and it cannot identify key risk types (black ice, thin oil film, etc.).

[0122] Wheel speed sensor solutions trigger intervention after the ABS / ESC system detects wheel slippage. Essentially, this is a reactive mechanism, intervening only after loss of control has occurred, and cannot achieve risk prediction or preventative control.

[0123] (2) Vision-driven recognition technology; Some vehicle models have attempted to incorporate cameras for road image analysis to identify slippery road surfaces. However, the technology is limited by the fact that it is highly dependent on lighting conditions, and the recognition rate drops sharply at night, in backlight, or in rainy or foggy weather. In addition, it is difficult to distinguish visually similar scenes, such as slippery asphalt and thin ice, or water reflection and oil film. Furthermore, it lacks certain quantitative capabilities and cannot measure key physical parameters such as water film thickness and ice layer hardness.

[0124] (3) Cloud-based early warning based on historical data; Some navigation systems provide historical weather or accident-prone road section alerts, but these have problems such as poor timeliness, low coverage, and coarse granularity. They cannot reflect real-time road conditions, rely on manual reporting, and cannot locate specific slippery areas.

[0125] Summary of core defects in existing technologies: Existing solutions generally suffer from problems such as limited perception dimensions, weak scene adaptability, and high false alarm rates, and cannot provide early warnings before slippage occurs. Furthermore, because data from different sensors is often processed in isolation and lacks a dynamic weighted multi-source information fusion mechanism, the system frequently triggers false alarms or misses alarms in complex environments.

[0126] To address the aforementioned issues, this application proposes a multi-source sensor fusion architecture based on dynamic confidence weights. Through dynamic fusion of multiple sensors and adaptive Kalman filtering correction, the real-time confidence of each sensor is dynamically calculated based on the signal quality of each sensor and the current environmental conditions. Then, basic probability allocation fusion is performed to obtain a preliminary road condition confidence distribution. Based on the overall confidence and environmental stability index of the road condition confidence distribution, the process noise covariance matrix and observation noise covariance matrix of the Kalman filter are adaptively adjusted to perform temporal filtering and correction on the preliminary road condition confidence distribution, outputting the final road condition estimation result. This establishes a triple-coupled decision model of environment-road-vehicle, achieving a paradigm shift from "passive response" to "active early warning."

[0127] The core of the dynamic weighted multi-source sensor fusion architecture includes: determining low-adhesion scenarios involves the fusion and judgment of multiple sensor sources, such as rain sensors, temperature sensors, humidity sensors, inertial measurement units, cameras, millimeter-wave radar, wheel-end sensors, cloud-based sensors, and road-end sensors. To avoid using a single sensor as the primary weight, a sensor fusion strategy is employed. All sensors can provide judgment criteria, and the algorithm comprehensively assesses the data to arrive at a high-confidence road condition. Since the reliability and confidence of different sensors change dynamically under different conditions, a dynamic confidence weight is used to dynamically evaluate the confidence of each sensor input and adjust its weight in the fusion process accordingly.

[0128] The process of the dynamic weight fusion algorithm is as follows: Figure 2 and Figure 3 As shown.

[0129] Figure 2 The process includes six steps, S201 to S206: S201, Raw data from multiple sensors; This step is used to acquire raw data collected by multiple sensors.

[0130] S202, Multi-sensor confidence assessment; This step is used to calculate the signal confidence level of each sensor.

[0131] S203, Basic Probability Assignment; This step is used to calculate the allocation probability of each sensor in each low-adhesion scenario based on the signal confidence of each sensor using the basic probability allocation theory.

[0132] S204, Dempster combination rules; This step is used to determine the fusion allocation probability for multi-sensor fusion in each low-attachment scene according to the Dempster combination rule.

[0133] S205, Kalman filter dynamic correction; This step is used to dynamically correct the observed values ​​of the road surface state parameters of the road where the vehicle is traveling by using Kalman filtering based on the process noise covariance matrix and observation noise covariance matrix that are common to each sensor, thereby obtaining the estimated value of the road surface state.

[0134] S206, Confidence-based decision output.

[0135] This step is used to determine whether the final output value is the road surface state estimate obtained after dynamic correction by Kalman filtering or the state prediction value calculated based on the relationship between the comprehensive confidence level of multi-sensor fusion and the confidence threshold.

[0136] For details of steps S201 to S206, please refer to the relevant content of the foregoing embodiments, which will not be repeated here.

[0137] Figure 3 Including S301 to S305: S301, Raw Data; This step is the same as S201 and is used to acquire the raw data obtained from multiple sensors. S302, Sensor Confidence Assessment; This step is the same as S202 and is used to calculate the signal confidence of the sensor. S303. Determine the confidence type corresponding to each sensor; This step is used to determine the confidence level type for each sensor. For example, the reliability of a rain sensor depends on the intensity of precipitation; cameras are mainly used to assess illumination / occlusion / image sharpness; millimeter-wave radar is used to detect signal-to-noise ratio and path loss; temperature sensors focus on the confidence level of ambient-road temperature difference; and wheel-end data focuses on the confidence level of slip ratio.

[0138] S304 and DS fusion algorithms; This step is used to fuse scene classification probabilities from multiple sensors using the DS evidence theory algorithm.

[0139] S305, output road surface condition and overall confidence level.

[0140] This step is used to calculate the overall confidence level and road condition estimate of the multi-sensor fusion based on the fusion assignment probability obtained after fusion.

[0141] For details of steps S301 to S305, please refer to the relevant content of the foregoing embodiments, which will not be repeated here.

[0142] The specific plan is as follows: (1) Define the set of sensors included in the system: Where N is the number of sensors, and each sensor This represents one sensor type. See Table 1 for specific correspondences. The number of sensor types can be increased or decreased in a practical system.

[0143] (2) For each sensor Its confidence level Determined by both signal quality and environmental factors:

[0144] in and These are weighting coefficients, satisfying... The value can be set according to the type of sensor, and the value is referenced in Table 3.

[0145] It is a sensor The signal-to-noise ratio, or signal quality metric, varies depending on the sensor type and the application environment. The values ​​also differ. The reference values ​​are shown in Table 1.

[0146] It is a signal quality function, positively correlated with signal quality, and its expression is: ,in, For sensors The characteristic parameters reflect the sensor's response to the signal-to-noise ratio and determine the saturation rate of the signal quality function. Reference values ​​are shown in Table 2. It is an environmental impact function. For example, for cameras, its value is lower during heavy rain or at night. See Table 2 for reference values.

[0147] The response characteristic parameter ranges of each sensor are shown in Table 4 below, and the values ​​can be referenced from Table 3: Table 4

[0148] (3) Construction of Basic Probability Assignment (BPA). BPA is a core tool of Dempster-Shafer evidence theory, used to quantify the degree of trust a sensor has in a particular proposition. Unlike traditional probability, it assigns "probability quality" to a subset of the identification frame, thereby expressing uncertainty and unknowns. Here, the identification frame refers to the set of road surface states (the entire set). ,in This refers to the number of road surface conditions, including dry, waterlogged, icy, snowy, and underground parking, etc. Specific types can be added or removed based on actual categories. For each sensor... Its basic probability allocation function At the same time satisfy Its specific construction method is as follows:

[0149] in It is a sensor On the proposition The probability of support, i.e., the probability distribution output by the sensor algorithm itself, is as follows: For example, on a flooded road, the probability of the water category output by the vision system through the classification model is... In the summation formula In the diagram, B is a loop variable representing the traversal recognition framework. Given all possible subsets (e.g., {dry}, {waterlogged}, {ice surface}, {dry, waterlogged}, ..., Θ), sum the probabilities P(B) of these subsets to obtain a normalized denominator. For sensors that assign probabilities only to a single subset, DS theory allows assignment to any subset.

[0150] (4) Dempster-Shafer evidence fusion (DS fusion). For two sensors and Their BPA values ​​are respectively and The fusion formula is: ,in The conflict coefficient, Where B and C are subsets of the quality allocated to m1 and m2, respectively. It indicates the degree to which two pieces of evidence completely conflict (i.e., the propositions they support have no overlap). The range of values ​​for is [0, 1). If there is a complete conflict, the rule does not apply. Only when the intersection of two subsets B and C supported by two pieces of evidence equals A do they jointly support A. We sum the products of the masses of all B and C that support A, and then divide by the normalization factor (1-). Then, the quality of the fused A is obtained. For example, if sensor 1 outputs {water accumulation} and sensor 2 outputs {ice surface}, then it is necessary to calculate the collision coefficient of the two sensors. Then, the quality of each subset after merging is calculated separately. For {Sekishui}: only B={Sekishui} and C= When the intersection is {accumulated water}, For {ice surface}: only B = When C = {ice surface}, the intersection is {ice surface}. ,for{ }: When B=oth and C= When, the intersection is { }, ,and Where 'oth' represents other scenarios.

[0151] For multiple sensors, an iterative fusion approach is required. First, the BPAs of the first two sensors are fused to obtain a new BPA. Then, this new BPA is fused with the BPA of the third sensor, and so on, until the BPAs of multiple sensors are merged into a unified fusion allocation probability.

[0152] The fusion allocation probability is an assignment to the recognition framework. The quality function of all subsets. This leads to the confidence function Bel(A), used to assess the overall confidence level in proposition A. The confidence function represents the sum of evidence for all explicitly supported subsets of A, representing the most conservative estimate; the overall confidence level is an assessment of the overall reliability of this fusion result. (5) Adaptive Kalman Filter Correction (AKF Correction). After the initial fusion of multiple sensors is completed by the DS evidence theory, the DS fusion result is used as the observation value. The state is estimated by adaptive Kalman filtering. The fusion result at the current moment can be combined with the historical state information. The system dynamics model is used for prediction and correction, making the state estimation smoother and more stable, and avoiding estimation jitter caused by noise in single frame data.

[0153] In Kalman filtering, the observation noise covariance matrix R quantifies the uncertainty of the observations. The larger the value of R, the more unreliable the observations are, and the smaller the Kalman gain K will be. This causes the system to trust its own predictions more than the observed values ​​when updating the state.

[0154] The output of the DS evidence theory precisely provides a quantitative indicator of this uncertainty: Bel(A): Provides the observed value z k The content (what is the road surface condition).

[0155] Overall confidence level: Provides the overall reliability of the observation (how credible this judgment is).

[0156] Therefore, the core logic of the transformation is: the lower the overall confidence level, the higher the observation uncertainty, and the larger the value of the R matrix.

[0157] When the overall confidence level is 1.0 (completely confident), R k =R0. The system considers the observations to be highly reliable and assigns them high weight in the Kalman filter update.

[0158] When the overall confidence level is 0.5 (half-convinced, half-doubtful), R k =2×R0. The observation noise variance doubles, and the Kalman filter, assuming increased uncertainty in the observations, reduces their weight during updates.

[0159] When the overall confidence level → 0+ (extremely unreliable), R k →+∞. This will cause the Kalman gain K→0, and the system will completely ignore the current observation and only trust the state prediction.

[0160] Additionally, Bel(A) can be used for observations. In, for example: 1) The friction coefficient μ is calculated by weighting the typical friction coefficients under different road surface conditions (dry, wet, waterlogged, icy, etc.). The weights are the confidence levels corresponding to each condition.

[0161] Example of friction coefficient under typical road surface conditions: c dry =0.8 (dry asphalt) c wet =0.6 (wet asphalt) c water =0.4 (flooded road surface) c ice =0.1 (ice surface) 2) Observed values ​​of water film thickness:

[0162] When the system determines that there is water or snow accumulation on the road surface, it provides a thickness estimate based on its confidence level in this condition. The formula uses h... water and h snow It can be a preset typical value, or it can be directly estimated by a sensor (such as millimeter-wave radar).

[0163] 3) Probability observations of ice surface : The most direct mapping method is adopted, using the degree of confidence in the proposition "ice surface" in evidence theory (DS theory) directly as the observed value of the ice surface probability, that is... =Bel({ice}).

[0164] (6) The lower the confidence level, the greater the observation noise, and the smaller the weight that Kalman filter assigns to the observation value in the update step. In practical applications, the state vector can be set according to the characteristics of each sensor. The reference state vector of this patent is defined as follows: This is an estimated value for the road surface adhesion coefficient (0-1). Estimate the thickness of the water film / snow layer on the road surface. The probability estimate (0-1) of the road surface being ice: Using state equations Indicates the predicted state from the previous time step. Evolved to the current moment The changes, among which It is the state transition matrix, which can be set as the identity matrix. This indicates that the expected state will not change drastically, and that the state at the next moment is assumed to be the same as the state at the previous moment. For process noise, It follows a mean of 0 and a covariance of 0. The normal distribution indicates the inaccuracy of the prediction model.

[0165] Using observation equations Describes the observation vector of the sensor. It mainly comes from the fusion allocation probability transformation of DS fusion. It is the observation matrix, which can be set as the identity matrix. This indicates the state of direct observation. To observe the noise, It follows a mean of 0 and a covariance of 0. It follows a normal distribution.

[0166] The main innovation of adaptive Kalman filtering lies in the process noise covariance. and observation noise covariance The adaptive process noise covariance is dynamically adjusted based on environmental stability, while the observation noise covariance is dynamically adjusted based on the confidence level of the fusion result.

[0167] Process noise represents the degree of confidence in the state equation model. When the environment changes drastically, the accuracy of the model's predictions will decrease, and in this case, process noise should be increased. , It is the baseline process noise covariance matrix, which is set according to the sensor type and application scenario; which sensors are considered comprehensively to provide a baseline. It is a scaling factor that controls the intensity of the adaptive effect. The scaling factor is set differently depending on the environment. The greater the difference in environment, the larger the scaling factor. It is the environmental stability index, and its calculation formula is: , and These are the changes in temperature and humidity between consecutive time steps. From this, it can be seen that... The larger the value, the more unstable the environment, thus... The larger the change, the more the filter tends to believe the new observations rather than the old state predictions. .

[0168] Observation noise represents the system's level of confidence in the observed values. The lower the sensor confidence, the greater the observation noise should be. Filters will reduce the current observation value. The weights of the observation noise covariance. , It is the benchmark observation noise covariance matrix, which is set according to the sensor type and application scenario. It is a diagonal matrix with diagonal elements.

[0169] The adaptive Kalman filter correction follows the standard Kalman filter framework, using adaptive methods at each step. and Ultimately, the smoothed road surface condition estimate can be used for the final vehicle control decision.

[0170] The meanings and calculation methods of the parameters involved in the above implementation can be found in Formulas 1 to 19, and will not be repeated here.

[0171] This embodiment has the following technical effects: (1) By fusing multi-source sensor information with cloud / V2I information, risks can be identified before the vehicle enters a slippery area, providing a key decision-making time window for predictive vehicle control (such as pre-loading braking force and linearly limiting torque).

[0172] (2) It greatly enhances the robustness and reliability of the system and improves the perception effect in complex environments. Through the dynamic confidence weighting mechanism, the system can automatically determine the reliability of each sensor and dynamically adjust its decision weight. For example, in heavy rain, the weight of the visual sensor is automatically reduced and the weight of the rain gauge and millimeter-wave radar is increased; in a dry reservoir, the system relies on wheel-end baseline comparison and radar dielectric constant measurement.

[0173] (3) Achieve fine-grained identification of slippery conditions. This invention classifies different types such as water accumulation, ice surface, and snow, and makes complex judgments for different scenarios.

[0174] This application provides a road surface condition assessment device based on multiple sensors, with reference to... Figure 4 The content shown indicates that the road surface condition assessment device 40 based on multiple sensors may include at least: a first determining module 401, a second determining module 402, a third determining module 403, and a fourth determining module 404; The first determining module 401 is used to determine the signal confidence levels of multiple sensors, and to determine the scene classification probability based on the measurement signals output by each sensor; the signal confidence level characterizes the measurement reliability of the sensor; the scene classification probability characterizes the classification probability of each sensor in each low-adhesion scene; the second determining module 402 is used to determine the fusion allocation probability corresponding to each low-adhesion scene based on the signal confidence levels and the scene classification probabilities; the third determining module 403 is used to determine the trust level corresponding to each low-adhesion scene based on the fusion allocation probability; the fourth determining module 404 is used to determine the road surface state result of the vehicle driving road based on the trust levels.

[0175] In some embodiments, the road surface condition result includes a road surface condition evaluation value; the fourth determining module 404 is further configured to: obtain the measured values ​​of each road surface condition parameter of the road; determine the observed value of each road surface condition parameter at the current time step based on each confidence level and each measured value; and determine the road surface condition evaluation value of each road surface condition parameter at the current time step based on the observed value.

[0176] In some embodiments, the fourth determining module 404 is further configured to: determine the overall confidence level of the fusion of the multiple sensors based on the fusion allocation probability; determine the observation noise covariance matrix based on the overall confidence level if the overall confidence level is greater than or equal to a confidence level threshold; and perform Kalman filtering on the observation values ​​based on the observation noise covariance matrix to obtain the road surface condition evaluation value of the road at the current time step.

[0177] In some embodiments, the fourth determining module 404 is further configured to: determine the environmental stability index of the driving environment; determine the process noise covariance matrix common to each of the sensors based on the environmental stability index; and perform Kalman filtering on the observed values ​​based on the process noise covariance matrix and the observation noise covariance matrix to obtain the road surface state evaluation value of the road at the current time step.

[0178] In some embodiments, the fourth determining module 404 is further configured to: determine the temperature change and humidity change of the driving environment at the current time step from the measurement signal; and determine the environmental stability index of the driving environment based on the temperature change and humidity change.

[0179] In some embodiments, the device 40 further includes: The fifth determining module is used to: determine the historical road surface condition assessment value at the previous time step when the overall confidence level is less than the confidence level threshold; and determine the road surface condition assessment value at the current time step based on the historical road surface condition assessment value.

[0180] In some embodiments, the road surface state parameters include water film thickness. The fourth determining module 404 is further configured to: determine the radar signal output by the millimeter-wave radar in the sensor from the measurement signal, and determine a first confidence level of the water film scene from the confidence level; determine a thickness measurement value of the water film thickness based on the radar signal; and determine an observed thickness value of the water film thickness at the current time step based on the first confidence level and the thickness measurement value. And / or, The road surface state parameters include the adhesion coefficient. The fourth determining module 404 is further configured to: determine the second confidence level corresponding to each road adhesion scenario from the confidence level; and determine the observed value of the adhesion coefficient at the current time step based on each second confidence level and the measured value of the adhesion coefficient corresponding to each road adhesion scenario. And / or, The road surface state parameters include the probability of ice surface existence. The fourth determining module 404 is further configured to: determine a third trust level of the ice surface scenario from the trust level; and determine the third trust level as the probability of ice surface existence at the current time step.

[0181] In some embodiments, the low-adhesion scenario includes at least one of a dry scenario, a water film scenario, an ice surface scenario, and a snow accumulation scenario; the second determining module 402 is further configured to: determine the allocation probability of each sensor in each target scenario based on the confidence level of each of the signals and the classification probability of each of the scenarios; the target scenarios include the dry scenario, the water film scenario, the ice surface scenario, the snow accumulation scenario, and combinations thereof; and determine the fusion allocation probability corresponding to each target scenario based on the allocation probability.

[0182] In some embodiments, the first determining module 401 is further configured to: acquire the signal-to-noise ratio of each sensor in the current environment and the environmental impact parameters of the current environment; the current environment characterizes the driving environment of the vehicle equipped with each sensor; and determine the signal confidence level of each sensor based on the signal-to-noise ratio and the environmental impact parameters.

[0183] In some embodiments, the first determining module 401 is further configured to: determine the response characteristic parameters corresponding to each of the sensors; determine the signal quality index values ​​corresponding to each of the sensors based on the response characteristic parameters and the signal-to-noise ratio; and determine the signal confidence level of each sensor based on the signal quality index values, a first preset weight value of the signal quality index values, the environmental impact parameters, and a second preset weight value of the environmental impact parameters.

[0184] It should be noted that the communication device of the application provided in this application embodiment includes all the units included, which can be implemented by a processor in an electronic device; of course, it can also be implemented by specific logic circuits; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), or a field-programmable gate array (FPGA), etc.

[0185] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0186] It should be noted that, in the embodiments of this application, if the above-described vehicle driving control method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0187] This application provides a storage medium, namely a computer-readable storage medium, on which a computer program or instructions are stored. When the computer program or instructions are executed by a processor, they implement the steps in any of the multi-sensor-based low-adhesion scene evaluation methods provided in the first aspect of the above embodiments.

[0188] This application provides a computer program product, which includes a computer program or instructions. When the computer program or instructions are executed by a processor, they implement the steps in any of the multi-sensor-based low-adhesion scene evaluation methods provided in the first aspect of the above embodiments.

[0189] It should be noted that the descriptions of the above embodiments of storage media, devices, apparatuses, and program products are similar to the descriptions of the above method embodiments and have similar beneficial effects. For technical details not disclosed in the embodiments of storage media, devices, apparatuses, and program products of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0190] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0191] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0192] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another electronic device, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0193] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0194] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0195] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0196] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0197] The above are merely embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A road surface condition assessment method based on multiple sensors, characterized in that, The method includes: The signal confidence levels of multiple sensors are determined, and the scene classification probability is determined based on the measurement signals output by each sensor; the signal confidence levels characterize the measurement reliability of the sensors; the scene classification probability characterizes the classification probability of each sensor in each low-adhesion scene. Based on the confidence level of each signal and the classification probability of each scene, the fusion allocation probability corresponding to each low-attachment scene is determined. Based on the fusion allocation probability, the trust level corresponding to each low-attachment scenario is determined. Based on each of the aforementioned trust levels, the road surface condition of the road on which the vehicle travels is determined.

2. The method according to claim 1, characterized in that, The road surface condition results include road surface condition assessment values; determining the road surface condition results of the road where the vehicle travels based on each of the trust levels includes: Obtain the measured values ​​of various road surface state parameters of the road; Based on each of the trust levels and each of the measured values, the observed values ​​of each road surface state parameter at the current time step are determined; Based on the observed values, the pavement condition evaluation values ​​of each pavement condition parameter at the current time step are determined.

3. The method according to claim 2, characterized in that, The step of determining the pavement state evaluation value of each pavement state parameter at the current time step based on the observed values ​​includes: Based on the fusion allocation probability, the overall confidence level of the fusion of the multiple sensors is determined; If the overall confidence level is greater than or equal to the confidence level threshold, the observation noise covariance matrix is ​​determined based on the overall confidence level. Based on the observation noise covariance matrix, Kalman filtering is performed on the observation values ​​to obtain the road surface condition assessment value at the current time step.

4. The method according to claim 3, characterized in that, The step of performing Kalman filtering on the observed values ​​based on the observed noise covariance matrix to obtain the road surface condition assessment value at the current time step includes: Determine the environmental stability index of the driving environment; Based on the environmental stability index, determine the process noise covariance matrix that is common to all the sensors; Based on the process noise covariance matrix and the observation noise covariance matrix, Kalman filtering is performed on the observation values ​​to obtain the road surface condition assessment value at the current time step.

5. The method according to claim 4, characterized in that, Determining the environmental stability index of the driving environment includes: The temperature and humidity changes of the driving environment at the current time step are determined from the measurement signals. Based on the temperature change and humidity change, the environmental stability index of the driving environment is determined.

6. The method according to claim 3, characterized in that, The method further includes: If the overall confidence level is less than the confidence threshold, determine to obtain the historical road surface state assessment value of the previous time step; Based on the historical road surface condition assessment values, the road surface condition assessment value at the current time step is determined.

7. The method according to any one of claims 2 to 6, characterized in that, The road surface condition parameters include water film thickness. The determination of the observed values ​​of each road surface condition parameter at the current time step, based on each confidence level and each measured value, includes: The radar signal output by the millimeter-wave radar in the sensor is determined from the measurement signal, and a first confidence level of the water film scene is determined from the confidence level. Based on the radar signal, the thickness measurement value of the water film is determined; Based on the first level of confidence and the thickness measurement value, the observed thickness value of the water film at the current time step is determined; And / or, The road surface condition parameters include the adhesion coefficient. Determining the observed values ​​of each road surface condition parameter at the current time step based on each confidence level and each measured value includes: From the trust level, determine the second trust level corresponding to each road attachment scenario; Based on the second level of trust and the measured value of the adhesion coefficient corresponding to each road adhesion scenario, the observed value of the adhesion coefficient at the current time step is determined. And / or, The road surface state parameters include the probability of ice presence. The determination of the observed values ​​of each road surface state parameter at the current time step, based on each confidence level and each measured value, includes: A third level of trust for the ice surface scenario is determined from the aforementioned level of trust. The third level of trust is determined as the probability of the ice surface existing at the current time step.

8. The method according to any one of claims 1 to 6, characterized in that, The low-adhesion scenarios include at least one of dry scenarios, water film scenarios, ice surface scenarios, and snow-covered scenarios; determining the fusion allocation probability corresponding to each low-adhesion scenario based on the confidence level of each signal and the classification probability of each scenario includes: Based on the confidence levels of each signal and the classification probabilities of each scene, the allocation probability of each sensor in each target scene is determined; the target scenes include the dry scene, the water film scene, the ice surface scene, the snow scene, and combinations thereof; Based on the allocation probability, the fusion allocation probability corresponding to each target scene is determined.

9. The method according to any one of claims 1 to 6, characterized in that, Determining the signal confidence levels of multiple sensors includes: The signal-to-noise ratio of each sensor in the current environment and the environmental impact parameters of the current environment are obtained respectively; the current environment represents the driving environment of the vehicle equipped with each sensor. Based on the signal-to-noise ratio and the environmental impact parameters, the signal confidence level of each of the sensors is determined.

10. The method according to claim 9, characterized in that, The step of determining the signal confidence level of each sensor based on the signal-to-noise ratio and the environmental impact parameters includes: Determine the response characteristic parameters corresponding to each of the aforementioned sensors; Based on the response characteristic parameters and the signal-to-noise ratio, the signal quality index values ​​corresponding to each of the sensors are determined respectively; The signal confidence level of each sensor is determined based on the signal quality index value, the first preset weight value of the signal quality index value, the environmental impact parameter, and the second preset weight value of the environmental impact parameter.

11. A vehicle, characterized in that, The vehicle includes a vehicle body, a processor, and a memory. The memory stores computer programs or instructions. When the processor executes the computer programs or instructions, it implements the multi-sensor-based road condition assessment method according to any one of claims 1 to 10.

Images