Physical mechanism prior guided real-time credibility assessment method for multi-modal sensor

Abstract

The physical mechanism priori guided multimodal sensor real-time credibility evaluation method relates to the technical field of robot multimodal environment perception and credible computing, and solves the problems of poor real-time performance and lack of physical explainability of existing uncertainty estimation methods. The method uses a credibility evaluation network to evaluate the original image data of the RGB camera, depth camera and laser radar of the autonomous mobile robot, and outputs the credibility measurement results and confidence interval of each modality sensor. The network includes an explicit physical quantity extraction unit, a multimodal feature encoding unit, a feature alignment unit, a physical degradation operator parameterization estimation unit, an uncertainty estimation unit and a credibility mapping unit, which realizes the joint modeling of physical degradation state perception and data-driven uncertainty by fusing physical priori quantity and deep learning features. The application is mainly applied to the perception system of the autonomous mobile robot, and is used for real-time credibility evaluation of the multimodal sensor.

Description

Technical Field

[0001] This invention relates to the field of robot multimodal environmental perception and trusted computing technology. Background Technology

[0002] With the large-scale application of autonomous mobile robots in indoor scenarios such as warehousing and logistics, inspection and security, and medical care, robots typically deploy multimodal sensors, including RGB cameras, depth cameras, LiDAR, and inertial measurement units (IMUs), to form a heterogeneous perception network that coordinates visual, geometric, and motion information, supporting environmental mapping, target recognition, and autonomous navigation. However, in long-term actual deployment of robots, sensor observations are inevitably affected by sudden changes in lighting, occlusion, dust, hardware aging, collisions, and other factors, which can easily lead to a surge in output noise, measurement drift, and even data loss. Furthermore, different types of sensors have significantly different physical failure mechanisms: RGB cameras are susceptible to weak light, strong light, motion blur, and occlusion; depth cameras are prone to long-distance attenuation and measurement holes caused by transparent and reflective objects; LiDAR is susceptible to false points caused by highly reflective materials and multipath reflection interference; and IMUs suffer from zero-bias drift and cumulative errors. Once these uncertainties occur, they will directly lead to distortion of local feature representations and generate cascading error amplification along the "perception-modeling-planning" chain, ultimately jeopardizing the safety of the robot's motion decision-making.

[0003] To address the aforementioned problems, existing technologies typically employ uncertainty estimation methods to characterize the reliability of observational data. However, existing uncertainty estimation methods suffer from the following main shortcomings:

[0004] First, existing methods mostly employ a dual-branch modeling approach, combining accidental and cognitive uncertainties, and rely on Bayesian inference or multiple Monte Carlo sampling to estimate the output distribution. Such methods require multiple forward samplings of the model, resulting in high computational costs and significant inference delays, making it difficult to meet the stringent real-time requirements of autonomous mobile robots.

[0005] Secondly, most existing methods are purely data-driven and lack modeling of the physical failure mechanism of sensors. They cannot characterize the intrinsic mapping relationship between "physical state - data degradation - feature deviation". The uncertainty of their output lacks physical interpretability and it is difficult to point out the physical cause of the decline in credibility.

[0006] In summary, how to construct a multimodal sensor credibility assessment scheme that is physically interpretable, can continuously quantify credibility, and can predict its evolution trend while ensuring real-time performance has become an important technical problem that urgently needs to be solved in this field. Summary of the Invention

[0007] The purpose of this invention is to address the problems of poor real-time performance and lack of physical interpretability in existing uncertainty estimation methods. This invention provides a real-time reliability assessment method for multimodal sensors guided by prior knowledge of physical mechanisms.

[0008] A real-time reliability assessment method for multimodal sensors guided by prior knowledge of physical mechanisms is proposed. This method utilizes a reliability assessment network to evaluate the raw observation data of multimodal sensors of an autonomous mobile robot, obtaining reliability metrics and confidence intervals for each sensor mode. The raw observation data consists of images, and the multimodal sensors include at least two of RGB cameras, depth cameras, and LiDAR. The reliability assessment network includes:

[0009] The explicit physical quantity extraction unit is used to perform physical quantity calculation and stitching on the images received from each modal sensor to obtain the physical prior quantity q(m) characterizing the physical degradation state of the modal sensor; m=1,…,3, where m is the modal type number, and when m takes values ​​from 1 to 3, it represents the RGB camera mode, depth camera mode and lidar mode, respectively.

[0010] The multimodal feature encoding unit is used to extract high-dimensional features from the received images from each modal sensor to obtain the high-dimensional feature map F(m) of each modality.

[0011] The feature alignment unit is used to perform spatial and semantic alignment processing on the high-dimensional feature map F(m) of each modality to obtain the aligned feature map F_Align(m) of each modality;

[0012] The physical degradation operator parameter estimation unit estimates the physical degradation operator parameters P_deg(m) for each mode based on the physical prior quantity q(m) and the aligned multimodal feature maps F_Align(m).

[0013] The uncertainty estimation unit is used to perform uncertainty quantization estimation on each aligned modal feature map F_Align(m) to obtain the uncertainty estimate U(m);

[0014] The confidence mapping unit, based on the physical degradation operator parameter P_deg(m) and uncertainty estimate U(m), obtains the confidence measurement result C(m) and confidence interval I(m) for each modal sensor through forward operation mapping.

[0015] Preferably, the explicit physical quantity extraction unit performs physical quantity calculations and stitches together the images received from each modal sensor to obtain the physical prior quantity q(m) characterizing the physical degradation state of the modal sensor. The implementation methods include:

[0016] For an image in RGB camera mode, it is first converted into a brightness map. The brightness mean, contrast and gradient magnitude are calculated based on the brightness map and then stitched together to obtain the physical prior quantity q(1) of RGB camera mode.

[0017] For images in the depth camera mode, the depth distance is extracted pixel by pixel and the average of the effective pixel depth is taken. After the proportion of holes in the transparent and reflective areas is counted, the images are stitched together to obtain the physical prior quantity q(2) of the depth camera mode.

[0018] For images in lidar mode, the mean reflection intensity of point clouds in the image and the proportion of outliers are statistically analyzed and stitched together to obtain the physical prior quantity q(3) of lidar mode.

[0019] Preferably, the physical degradation operator parameter estimation unit estimates the physical degradation operator parameter P_deg(m) for each mode based on the physical prior quantity q(m) and the aligned multimodal feature maps F_Align(m) as follows:

[0020] First, the physical prior quantity q(m) of each modality is mapped to a C-dimensional embedding vector through a fully connected layer, and then broadcast along the spatial dimension as a physical embedding feature map of size H×W×C; H and W represent the spatial height and width, respectively, and C is the dimension of the embedding vector;

[0021] Next, the physical embedding feature map is concatenated with the aligned corresponding modal feature map F_Align(m) along the channel dimension to obtain a fused feature map of size H×W×(d+C); the aligned corresponding modal feature map F_Align(m) has a size of H×W×d, where d is the channel dimension of the aligned modal feature map F_Align(m);

[0022] Then, the fused feature map is downsampled and feature extracted sequentially through two convolutional units, and then global average pooling is used to obtain the channel description vector;

[0023] Finally, the channel description vector is regressed through another fully connected layer to obtain the physical degradation operator parameter P_deg(m) for the corresponding m-th mode.

[0024] Preferably, each convolutional unit includes a 3×3 convolutional layer, batch normalization, and ReLU activation function cascaded in sequence.

[0025] Preferably, the uncertainty estimation unit performs uncertainty quantization on the aligned modal feature maps F_Align(m) to obtain the uncertainty estimate U(m). The implementation methods include:

[0026] The aligned modal feature maps F_Align(m) are used for feature mapping through a multilayer perceptron to obtain the uncertainty estimate U(m) of the corresponding modality.

[0027] Preferably, the confidence mapping unit obtains the confidence measurement result C(m) and confidence interval I(m) for each modal sensor through forward operation mapping based on the physical degradation operator parameter P_deg(m) and uncertainty estimate U(m), including the following implementation methods:

[0028] The scalar physical degradation degree g(m) obtained after preprocessing the physical degradation operator parameter P_deg(m) is concatenated with the uncertainty estimate U(m) by tensor to obtain the joint degradation characterization;

[0029] The distribution parameters are obtained by performing a two-layer forward fully connected operation on the joint degradation characterization.

[0030] The mean value μ_net(m) is obtained by using the mean head to perform mean operation on the distribution parameters, and the reliability log odds μ(m) is obtained by combining g(m). The reliability metric C(m) is obtained by mapping μ(m).

[0031] Variance calculation is performed on the distribution parameters using the variance head to obtain the non-negative variance v(m), and then μ(m) is combined with the variance v(m) to map the variance to obtain the confidence interval I(m).

[0032] Preferably, C(m) = Sigmoid(μ(m)), I(m) = [Sigmoid(μ(m)-z·s(m)), Sigmoid(μ(m)+z·s(m))];

[0033] μ(m)=μ_net(m)-(λ_1·U(m)+λ_2·g(m));

[0034] Where Sigmoid() is a monotonically increasing function, s(m) is the square root of the non-negative variance v(m), z is the coefficient corresponding to the preset confidence level, and λ_1 and λ_2 are the non-negative balance coefficients, respectively.

[0035] Preferably, the real-time reliability assessment method for multimodal sensors guided by prior knowledge of physical mechanisms further includes a time-series evolution early warning step. In this step, the continuous reliability measurement result C(m) of the current continuous output is extrapolated and predicted to be the continuous reliability measurement result C(m) of the future time. This prediction is also used to determine whether to issue a failure early warning based on the prediction result.

[0036] Preferably, the extrapolation prediction is implemented using a pre-trained temporal attention network;

[0037] The pre-trained temporal attention network extracts the temporal change trend of the credibility measurement result C(m) of the currently received continuous output, and extrapolates and predicts the continuous credibility measurement result C(m) of future time based on the temporal change trend.

[0038] Preferably, the method for determining whether to issue a failure warning based on the prediction results is as follows:

[0039] When the prediction result is lower than the preset warning threshold, a failure warning will be issued.

[0040] The beneficial effects of this invention are:

[0041] This invention provides a real-time reliability assessment method for multimodal sensors guided by prior physical mechanisms, primarily targeting multimodal perception tasks of autonomous mobile robots in complex indoor environments. This method organically combines physical failure mechanism modeling of sensors with deep representation learning, constructing a reliability assessment framework driven by a "physical state—data degradation—feature bias" mapping, thereby achieving continuous quantification and interpretable characterization of the reliability of multimodal observation data.

[0042] Specifically, this invention uses an explicit physical quantity extraction unit to directly calculate the physical prior quantities characterizing the physical degradation state of the sensor from the original observation data of each modality, and uses a physical degradation operator parameterization estimation unit to estimate the physical degradation operator parameters of each modality, so that the credibility assessment has clear physical interpretability and can indicate the physical causes of the credibility decline, which is different from the existing pure data-driven methods.

[0043] This invention, through an uncertainty estimation unit and a confidence mapping unit, based on physical degradation operator parameters and uncertainty estimation, deterministically outputs continuous confidence measurement results and confidence intervals through forward operation mapping, without the need for multiple sampling. This significantly reduces inference latency while ensuring evaluation accuracy, thus meeting the real-time requirements of autonomous mobile robots.

[0044] As a preferred embodiment, the present invention also models and extrapolates the temporal change trend of continuous reliability measurement results through a time-series evolution early warning step, which can provide a failure warning before the sensor completely fails, providing a key basis for the system's early response and safety decision-making.

[0045] The continuous reliability measurement results and confidence intervals output by this invention can provide continuous, quantifiable, and fine-grained reliability basis for downstream multimodal fusion and topology fault-tolerant decision-making, thereby improving the robustness of the entire sensing system under sensor degradation or failure conditions. Attached Figure Description

[0046] Figure 1This is a schematic diagram illustrating the principle of how the credibility assessment network described in this invention evaluates the raw observation data of each modal sensor.

[0047] Figure 2 This is a schematic diagram of the structure of the parameterized estimation unit of the physical degradation operator;

[0048] Figure 3 This is a schematic diagram of the credibility mapping unit. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0051] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but this is not intended to limit the scope of the invention.

[0052] Specific Implementation Method 1: Combination Figure 1 This embodiment describes a real-time reliability assessment method for multimodal sensors guided by prior knowledge of physical mechanisms. This method utilizes a reliability assessment network to evaluate the raw observation data of multimodal sensors on an autonomous mobile robot, obtaining reliability metrics and confidence intervals for each sensor mode. The raw observation data consists of images, and the multimodal sensors include at least two of RGB cameras, depth cameras, and LiDAR. The reliability assessment network includes:

[0053] The explicit physical quantity extraction unit is used to perform physical quantity calculation and stitching on the images received from each modal sensor to obtain the physical prior quantity q(m) characterizing the physical degradation state of the modal sensor; m=1,…,3, where m is the modal type number, and when m takes values ​​from 1 to 3, it represents the RGB camera mode, depth camera mode and lidar mode, respectively.

[0054] The multimodal feature coding unit is used to extract high-dimensional features from the images received from each modal sensor to obtain the high-dimensional feature map F(m) of each modality.

[0055] The feature alignment unit is used to perform spatial and semantic alignment processing on the high-dimensional feature map F(m) of each modality to obtain the aligned feature map F_Align(m) of each modality;

[0056] The physical degradation operator parameter estimation unit estimates the physical degradation operator parameters P_deg(m) for each mode based on the physical prior quantity q(m) and the aligned multimodal feature maps F_Align(m).

[0057] The uncertainty estimation unit is used to perform uncertainty quantization estimation on each aligned modal feature map F_Align(m) to obtain the uncertainty estimate U(m);

[0058] The confidence mapping unit, based on the physical degradation operator parameter P_deg(m) and uncertainty estimate U(m), obtains the confidence measurement result C(m) and confidence interval I(m) for each modal sensor through forward operation mapping.

[0059] In terms of real-time performance, the method of this invention does not employ traditional Bayesian networks or Monte Carlo dropout methods, which require a large number of iterative samplings. Instead, it directly calculates physical priors from sensor images. These low-dimensional numerical calculations are extremely low-cost, and these physical priors play a guiding role in the subsequent network, allowing the network to skip feature calculations in many irrelevant regions. The entire evaluation process only requires one forward propagation to obtain the result, thus ensuring computational efficiency in actual deployment.

[0060] Regarding physical interpretability, the method of this invention explicitly quantifies the physical degradation state of the sensor (e.g., image darkening due to insufficient lighting, holes caused by depth loss, abnormal laser reflection, etc.) and incorporates these physical quantities as part of the network input. The subsequent network estimates the degradation operator parameters for each sensor based on these physical priors, and the final confidence score and confidence interval are directly linked to these observable physical degradation levels. Therefore, when the evaluation method indicates a low confidence level for a sensor, operators can clearly understand the source of image degradation based on the mapping relationship between the confidence score and confidence interval and the degradation type, rather than simply obtaining a black-box value with no traceable cause.

[0061] Furthermore, the explicit physical quantity extraction unit performs physical quantity calculations and stitches together the images received from each modal sensor to obtain the physical prior quantity q(m) characterizing the physical degradation state of the modal sensor. The implementation methods include:

[0062] For an image in RGB camera mode, it is first converted into a brightness map. The brightness mean, contrast and gradient magnitude are calculated based on the brightness map and then stitched together to obtain the physical prior quantity q(1) of RGB camera mode.

[0063] For images in the depth camera mode, the depth distance is extracted pixel by pixel and the average of the effective pixel depth is taken. After the proportion of holes in the transparent and reflective areas is counted, the images are stitched together to obtain the physical prior quantity q(2) of the depth camera mode.

[0064] For images in lidar mode, the mean reflection intensity of point clouds in the image and the proportion of outliers are statistically analyzed and stitched together to obtain the physical prior quantity q(3) of lidar mode.

[0065] The preferred approach specifically outlines three methods for calculating the physical prior quantities of the sensors. For RGB cameras, extracting the mean brightness, contrast, and gradient magnitude simultaneously reflects changes in ambient lighting and image texture clarity—both factors directly determining the discernibility of visual features. For depth cameras, extracting the mean effective depth and calculating the proportion of voids quantifies data loss due to transparent objects, reflective surfaces, or excessively long detection distances. For LiDAR, calculating the mean reflection intensity and outlier ratio characterizes changes in the reflectivity of the target surface material to laser light and environmental noise interference. These physical quantities are all based on the sensor's own operating mechanism, are easy to calculate, and have clear physical meaning, providing a reliable degradation basis for subsequent evaluation.

[0066] See Figure 2 The physical degradation operator parameter estimation unit estimates the physical degradation operator parameters P_deg(m) for each mode based on the physical prior quantity q(m) and the aligned multimodal feature maps F_Align(m). The implementation method is as follows:

[0067] First, the physical prior quantity q(m) of each modality is mapped to a C-dimensional embedding vector through a fully connected layer, and then broadcast along the spatial dimension as a physical embedding feature map of size H×W×C; H and W represent the spatial height and width, respectively, and C is the dimension of the embedding vector;

[0068] Next, the physical embedding feature map is concatenated with the aligned corresponding modal feature map F_Align(m) along the channel dimension to obtain a fused feature map of size H×W×(d+C); the aligned corresponding modal feature map F_Align(m) has a size of H×W×d, where d is the channel dimension of the aligned modal feature map F_Align(m);

[0069] Then, the fused feature map is downsampled and feature extracted sequentially through two convolutional units, and then global average pooling is used to obtain the channel description vector; each convolutional unit includes a cascaded 3×3 convolutional layer, batch normalization and ReLU activation function;

[0070] Finally, the channel description vector is regressed through another fully connected layer to obtain the physical degradation operator parameter P_deg(m) for the corresponding m-th mode.

[0071] The above describes the specific estimation process for the physical degradation operator parameters. In this process, the physical prior is first mapped to an embedding vector through a fully connected layer, and then concatenated with the aligned feature map. This is equivalent to using the physical prior to activate the degradation-related channel regions in the feature map. Subsequently, features are extracted through convolutional downsampling, and then the degradation operator parameters are obtained through global average pooling and fully connected layer regression. The ingenuity of this design lies in the fact that the physical prior does not simply influence the output at the end of the network, but rather permeates the feature extraction process, actively guiding the network to focus on which features are sensitive to the current degradation state. The two convolutional units use 3×3 convolutions combined with batch normalization and ReLU activation. This configuration maintains a low computational cost while ensuring feature extraction capabilities.

[0072] Furthermore, the uncertainty estimation unit performs uncertainty quantization on the aligned modal feature maps F_Align(m) to obtain the uncertainty estimate U(m). The implementation methods include:

[0073] The aligned modal feature maps F_Align(m) are used for feature mapping through a multilayer perceptron to obtain the uncertainty estimate U(m) of the corresponding modality.

[0074] This preferred method involves a specific implementation of uncertainty estimation. The method directly maps the aligned feature map to the uncertainty estimate via a multilayer perceptron, without employing the Bayesian approximation method requiring multiple random samplings, nor introducing additional random variables. This deterministic mapping method ensures both computational efficiency and maintains the dependency of the uncertainty estimate on the previously extracted physical features.

[0075] See Figure 3 The confidence mapping unit, based on the physical degradation operator parameter P_deg(m) and uncertainty estimate U(m), obtains the confidence measurement result C(m) and confidence interval I(m) for each modal sensor through forward operation mapping. The implementation methods include:

[0076] The scalar physical degradation degree g(m) obtained after preprocessing the physical degradation operator parameter P_deg(m) is concatenated with the uncertainty estimate U(m) by tensor to obtain the joint degradation characterization; the physical degradation operator parameter P_deg(m) is normalized after preprocessing.

[0077] The distribution parameters are obtained by performing a two-layer forward fully connected operation on the joint degenerate representation using a mapping network. Figure 3 In the mapping network, each forward fully connected operation is implemented using a fully connected layer combined with a ReLU activation;

[0078] The mean value μ_net(m) is obtained by using the mean head to perform mean operation on the distribution parameters, and the reliability log odds μ(m) is obtained by combining g(m). The reliability metric C(m) is obtained by mapping μ(m).

[0079] Variance calculation is performed on the distribution parameters using the variance head to obtain the non-negative variance v(m), and then μ(m) is combined with the variance v(m) to map the variance to obtain the confidence interval I(m).

[0080] C(m)=Sigmoid(μ(m)), I(m)=[Sigmoid(μ(m)-z·s(m)), Sigmoid(μ(m)+z·s(m))];

[0081] μ(m) = μ_net(m) - (λ_1·U(m) + λ_2·g(m)); where Sigmoid() is a monotonically increasing function, s(m) is the square root of the non-negative variance v(m), z is the coefficient corresponding to the preset confidence level, and λ_1 and λ_2 are the non-negative balance coefficients.

[0082] The above describes the core mechanism of confidence mapping. The process concatenates the physical degradation degree with the uncertainty estimate, obtains the distribution parameters through a two-layer fully connected network, and then calculates the reliability log-odds and variance using the mean header and variance header, respectively, ultimately mapping to a confidence score and confidence interval. The key here is that the physical degradation degree has a direct negative moderating effect on the final result. When physical degradation is severe, even if the mean of the network output is high, the final confidence will be lowered. The confidence interval considers the fluctuation range of both physical degradation and uncertainty. The formula introduces a pre-set confidence level coefficient z and a balance coefficient λ, allowing the width of the confidence interval to be adaptively adjusted. Users can set different confidence levels according to their actual needs.

[0083] Furthermore, the physical mechanism prior-guided real-time reliability assessment method for multimodal sensors also includes a time-series evolution early warning step. In this step, the continuous reliability measurement result C(m) of the current continuous output is extrapolated and predicted to be the continuous reliability measurement result C(m) of the future time. It is also used to determine whether to issue a failure early warning based on the prediction result.

[0084] Extrapolation prediction is achieved using a pre-trained temporal attention network;

[0085] The pre-trained temporal attention network extracts the temporal variation trend of the credibility metric C(m) of the currently received continuous outputs, and extrapolates and predicts the continuous credibility metric C(m) for future time steps based on this temporal variation trend. The training of the temporal attention network is implemented using existing technologies.

[0086] The method for determining whether to issue a failure warning based on the prediction results is as follows: when the prediction results are lower than the preset warning threshold, a failure warning is issued.

[0087] This preferred approach introduces a temporal evolution warning function. This method extrapolates and predicts based on the continuous output sequence of a confidence metric, going beyond the current moment's evaluation to predict future trends in sensor confidence. A pre-trained temporal attention network is used for prediction, capturing dynamic patterns of confidence changes over time. This warning mechanism is valuable for practical robot operation because sensor performance typically doesn't collapse instantaneously but rather undergoes a degradation process; early detection of impending failure allows for system switching or safe shutdown.

[0088] While the invention has been described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the invention. Therefore, it should be understood that many modifications can be made to the exemplary embodiments, and other arrangements can be designed without departing from the spirit and scope of the invention as defined by the appended claims. It should be understood that different dependent claims and features described herein can be combined in ways different from those described in the original claims. It is also understood that features described in conjunction with individual embodiments can be used in other described embodiments.

Claims

1. A real-time reliability assessment method for multimodal sensors guided by prior knowledge of physical mechanisms. This method utilizes a reliability assessment network to evaluate the raw observation data of multimodal sensors of an autonomous mobile robot, obtaining reliability metrics and confidence intervals for each modality of sensor. The raw observation data is images, and the multimodal sensors include at least two of RGB cameras, depth cameras, and LiDAR. Its characteristic is that... The credibility assessment network includes: The explicit physical quantity extraction unit is used to perform physical quantity calculation and stitching on the images received from each modal sensor to obtain the physical prior quantity q(m) characterizing the physical degradation state of the modal sensor; m=1,…,3, where m is the modal type number, and when m takes values ​​from 1 to 3, it represents the RGB camera mode, depth camera mode and lidar mode, respectively. The multimodal feature encoding unit is used to extract high-dimensional features from the received images from each modal sensor to obtain the high-dimensional feature map F(m) of each modality. The feature alignment unit is used to perform spatial and semantic alignment processing on the high-dimensional feature map F(m) of each modality to obtain the aligned feature map F_Align(m) of each modality; The physical degradation operator parameter estimation unit estimates the physical degradation operator parameters P_deg(m) for each mode based on the physical prior quantity q(m) and the aligned multimodal feature maps F_Align(m). The uncertainty estimation unit is used to perform uncertainty quantization estimation on each aligned modal feature map F_Align(m) to obtain the uncertainty estimate U(m); The confidence mapping unit, based on the physical degradation operator parameter P_deg(m) and uncertainty estimate U(m), obtains the confidence measurement result C(m) and confidence interval I(m) for each modal sensor through forward operation mapping.

2. The real-time reliability assessment method for multimodal sensors guided by prior knowledge of physical mechanisms according to claim 1, characterized in that, The explicit physical quantity extraction unit performs physical quantity calculations and stitches together the images received from each modal sensor to obtain the physical prior quantity q(m) characterizing the physical degradation state of the modal sensor. The implementation methods include: For an image in RGB camera mode, it is first converted into a brightness map. The brightness mean, contrast and gradient magnitude are calculated based on the brightness map and then stitched together to obtain the physical prior quantity q(1) of RGB camera mode. For images in the depth camera mode, the depth distance is extracted pixel by pixel and the average of the effective pixel depth is taken. After the proportion of holes in the transparent and reflective areas is counted, the images are stitched together to obtain the physical prior quantity q(2) of the depth camera mode. For images in lidar mode, the mean reflection intensity of point clouds in the image and the proportion of outliers are statistically analyzed and stitched together to obtain the physical prior quantity q(3) of lidar mode.

3. The real-time reliability assessment method for multimodal sensors guided by prior knowledge of physical mechanisms according to claim 1, characterized in that, The parameterized estimation unit for the physical degradation operator estimates the physical degradation operator parameters P_deg(m) for each mode based on the physical prior q(m) and the aligned multimodal feature maps F_Align(m). The implementation method is as follows: First, the physical prior quantity q(m) of each modality is mapped to a C-dimensional embedding vector through a fully connected layer, and then broadcast along the spatial dimension as a physical embedding feature map of size H×W×C; H and W represent the spatial height and width, respectively, and C is the dimension of the embedding vector; Next, the physical embedding feature map is concatenated with the aligned corresponding modal feature map F_Align(m) along the channel dimension to obtain a fused feature map of size H×W×(d+C); the aligned corresponding modal feature map F_Align(m) has a size of H×W×d, where d is the channel dimension of the aligned modal feature map F_Align(m); Then, the fused feature map is downsampled and feature extracted sequentially through two convolutional units, and then global average pooling is used to obtain the channel description vector; Finally, the channel description vector is regressed through another fully connected layer to obtain the physical degradation operator parameter P_deg(m) for the corresponding m-th mode.

4. The real-time reliability assessment method for multimodal sensors guided by prior knowledge of physical mechanisms according to claim 3, characterized in that, Each convolutional unit consists of a cascaded 3×3 convolutional layer, batch normalization, and the ReLU activation function.

5. The real-time reliability assessment method for multimodal sensors guided by prior knowledge of physical mechanisms according to claim 1, characterized in that, The uncertainty estimation unit performs uncertainty quantization on the aligned modal feature maps F_Align(m) to obtain the uncertainty estimate U(m). Implementation methods include: The aligned modal feature maps F_Align(m) are used for feature mapping through a multilayer perceptron to obtain the uncertainty estimate U(m) of the corresponding modality.

6. The real-time reliability assessment method for multimodal sensors guided by prior knowledge of physical mechanisms according to claim 1, characterized in that, The confidence mapping unit, based on the physical degradation operator parameter P_deg(m) and the uncertainty estimate U(m), obtains the confidence measurement result C(m) and confidence interval I(m) for each modal sensor through forward operation mapping. The implementation methods include: The scalar physical degradation degree g(m) obtained after preprocessing the physical degradation operator parameter P_deg(m) is concatenated with the uncertainty estimate U(m) by tensor to obtain the joint degradation characterization; The distribution parameters are obtained by performing a two-layer forward fully connected operation on the joint degradation characterization. The mean value μ_net(m) is obtained by using the mean head to perform mean operation on the distribution parameters, and the reliability log odds μ(m) is obtained by combining g(m). The reliability metric C(m) is obtained by mapping μ(m). Variance calculation is performed on the distribution parameters using the variance head to obtain the non-negative variance v(m), and then μ(m) is combined with the variance v(m) to map the variance to obtain the confidence interval I(m).

7. The real-time reliability assessment method for multimodal sensors guided by prior knowledge of physical mechanisms according to claim 6, characterized in that, C(m)=Sigmoid(μ(m)), I(m)=[Sigmoid(μ(m)-z·s(m)), Sigmoid(μ(m)+z·s(m))]; μ(m)=μ_net(m)-(λ_1·U(m)+λ_2·g(m)); Where Sigmoid() is a monotonically increasing function, s(m) is the square root of the non-negative variance v(m), z is the coefficient corresponding to the preset confidence level, and λ_1 and λ_2 are the non-negative balance coefficients, respectively.

8. The real-time reliability assessment method for multimodal sensors guided by prior knowledge of physical mechanisms according to claim 1, characterized in that, It also includes a time-series evolution early warning step, in which the continuous credibility measurement result C(m) of the current continuous output is extrapolated and predicted to be the continuous credibility measurement result C(m) of the future time, and it is also used to determine whether to issue a failure early warning based on the prediction result.

9. The real-time reliability assessment method for multimodal sensors guided by prior knowledge of physical mechanisms according to claim 8, characterized in that, Extrapolation prediction is achieved using a pre-trained temporal attention network; The pre-trained temporal attention network extracts the temporal change trend of the credibility metric C(m) of the currently received continuous output, and extrapolates and predicts the continuous credibility metric C(m) of future times based on this temporal change trend.

10. The real-time reliability assessment method for multimodal sensors guided by prior knowledge of physical mechanisms according to claim 8, characterized in that, The method for determining whether to issue a failure warning based on the prediction results is as follows: When the prediction result is lower than the preset warning threshold, a failure warning will be issued.

Images