Multi-sensor contribution degree evaluation method and system, electronic device and storage medium

Abstract

The application relates to the technical field of sensor information processing, and discloses a multi-sensor contribution degree evaluation method and system, an electronic device and a storage medium. The multi-sensor contribution degree evaluation method comprises the following steps: obtaining a current environment state feature of a target sensor, inputting the current environment state feature into a pre-trained contribution degree evaluation model, and obtaining a current contribution degree of the target sensor; wherein the contribution degree evaluation model is trained with historical environment state features as input and contribution degree labels as expected output; the contribution degree labels are determined according to state prediction error changes corresponding to different sensor combinations; and the state prediction error is obtained by inputting mask observation data corresponding to different sensor combinations into a pre-trained information fusion model. The application can quickly determine the real-time contribution degree of a sensor based on a current environment state feature.

Description

Technical Field

[0001] This application relates to the field of sensor information processing technology, specifically to multi-sensor contribution assessment methods, systems, electronic devices, and storage media. Background Technology

[0002] In multi-sensor data processing, accurately assessing the contribution of each sensor is crucial for resource scheduling and redundant node removal. However, current technologies typically rely on real-time calculations based on raw sensor measurements each time contribution is required, using methods such as information theory metrics, statistical filtering analysis, or model ablation experiments. These methods are complex and time-consuming, failing to meet the need for efficient acquisition of sensor contributions under diverse operating environments.

[0003] Therefore, there is an urgent need for a multi-sensor contribution assessment method to solve the problem of low efficiency in sensor contribution assessment in related technologies. Summary of the Invention

[0004] This application provides a method, system, electronic device, and storage medium for evaluating the contribution of multiple sensors, in order to solve the problem of low efficiency in sensor contribution evaluation in related technologies.

[0005] Firstly, this application provides a method for evaluating the contribution of multiple sensors, the method comprising: Acquire the current environmental state characteristics of multiple target sensors; The current environmental state features are input into a pre-trained contribution evaluation model to obtain the current contribution of the target sensor. The contribution evaluation model is trained by taking historical environmental state characteristics as input and the corresponding contribution label as the expected output. The contribution label is determined based on the change in state prediction error corresponding to different sensor combinations. The state prediction error is obtained by inputting the masked observation data corresponding to different sensor combinations into a pre-trained information fusion model.

[0006] In one alternative implementation, the contribution evaluation model is trained through the following steps: Multiple sets of historical environmental state feature data and the contribution labels corresponding to each set of historical environmental state feature data are obtained to construct a training sample set; The historical environmental state feature data in the training sample set is used as the input of the machine learning model, and the corresponding contribution label is used as the expected output of the machine learning model. The machine learning model is trained to obtain the trained machine learning model. The trained machine learning model is used as the contribution evaluation model.

[0007] In one alternative implementation, the contribution label is determined through the following steps: Acquire observational data that includes features from the target sensor and other sensors; For different combinations of sensors to be evaluated, masking is performed on sensor features not included in the sensor combinations to be evaluated, and masked observation data for the corresponding combinations are obtained. The masked observation data is input into a pre-trained information fusion model to obtain the state prediction results corresponding to each sensor combination to be evaluated. Based on the difference between the state prediction results and the actual physical state of the target object, determine the state prediction error corresponding to each sensor combination to be evaluated; The marginal utility of the target sensor is determined based on the improvement in state prediction error before and after its addition to the combination. The contribution label of the target sensor is determined based on marginal utility.

[0008] In one optional implementation, masking is performed on sensor features not included in the sensor combination to be evaluated to obtain masked observation data for the corresponding combination, including: In the observation data, identify the feature data corresponding to sensors that are not included in the sensor combination to be evaluated; The identified feature data is replaced with preset placeholder values ​​to obtain masked observation data; the preset placeholder values ​​are used to characterize the state where the sensor does not participate in the current fusion process.

[0009] In one alternative implementation, the environmental state characteristics include at least one of the following: the relative distance of the target object to the target sensor, the signal-to-noise ratio, and the visibility index of the observation environment.

[0010] In one alternative implementation, determining the contribution label of the target sensor based on marginal utility includes: Randomly generate the order in which multiple sensor groups join the alliance; For each arrangement order, each sensor is added to the current alliance in turn, and the marginal utility of each sensor under each arrangement order is calculated based on the change in the state prediction error of each sensor combination to be evaluated before and after the addition. The Shapley value of the target sensor is obtained by weighting and averaging the marginal utility of the same sensor in multiple arrangement orders. The Shapley value is used as a label for the contribution of the target sensor.

[0011] In one optional implementation, the information fusion model includes multiple independent segmented sub-models; the training process of the information fusion model includes: Based on the different distance ranges between the target object and the sensor, the historical multi-sensor observation data is divided into multiple corresponding data subsets; For each distance interval, a corresponding segmented sub-model is independently trained using the corresponding data subset; When inputting masked observation data into the information fusion model, the current relative distance of the target object in the observation data is obtained, and the segmented sub-model that matches the distance interval of the current relative distance is called for prediction.

[0012] Secondly, this application provides a multi-sensor contribution assessment system, which includes: The acquisition module is used to acquire the current environmental state characteristics of multiple target sensors; The evaluation module is used to input the current environmental state features into a pre-trained contribution evaluation model to obtain the current contribution of the target sensor. The contribution evaluation model is trained by taking historical environmental state characteristics as input and the corresponding contribution label as the expected output. The contribution label is determined based on the change in state prediction error corresponding to different sensor combinations. The state prediction error is obtained by inputting the masked observation data corresponding to different sensor combinations into a pre-trained information fusion model.

[0013] Thirdly, this application provides an electronic device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the multi-sensor contribution evaluation method described in the first aspect or any corresponding embodiment.

[0014] Fourthly, this application provides a computer-readable storage medium storing computer instructions for causing a computer to perform the multi-sensor contribution assessment method described in the first aspect or any of its corresponding embodiments.

[0015] The multi-sensor contribution evaluation method provided in this application trains a contribution evaluation model using historical environmental state features as input and contribution labels as the expected output. This transfers the complex contribution calculation for different sensor combinations to the offline training stage. In the online application stage, only the current environmental state features need to be input into the trained contribution evaluation model to directly output the sensor contribution, eliminating the need for repeated inference of the information fusion model in the online stage, thus achieving rapid evaluation of multi-sensor contributions. Furthermore, the contribution label is determined based on the change in state prediction error obtained after inputting masked observation data corresponding to different sensor combinations into the information fusion model, which can objectively reflect the actual impact of each sensor on the system state prediction performance. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram illustrating an application scenario according to an embodiment of this application; Figure 2 This is a schematic diagram of the first process of the multi-sensor contribution evaluation method according to an embodiment of this application; Figure 3 This is a schematic diagram of the second process of the multi-sensor contribution evaluation method according to the embodiments of this application; Figure 4 This is a schematic diagram of a multi-sensor contribution assessment system based on marginal utility and neural networks according to an embodiment of this application; Figure 5 This is a structural block diagram of a multi-sensor contribution assessment system according to an embodiment of this application; Figure 6 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] It is understood that before using the technical solutions disclosed in the various embodiments of this application, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this application in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0020] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0021] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained.

[0022] Multi-source heterogeneous sensors refer to multiple sensors that operate in parallel within the same system, employing different physical principles or sampling mechanisms. Examples include radar (detecting targets by emitting radio waves and receiving echoes), infrared cameras (imaging targets by sensing their thermal radiation), and satellites (performing wide-area detection of ground targets via an orbital platform). Due to the differences in sampling frequency, data format, and measurement range among the sensors, their data typically require unified preprocessing before fusion.

[0023] Information fusion: The process of integrating observation data from multiple sensors according to certain rules to obtain a more accurate and complete target state estimation result than any single sensor.

[0024] Metamodel: A surrogate model used to approximate the behavior of complex systems, typically replacing the original complex model with a lower computational cost to complete prediction or evaluation tasks.

[0025] Deep Multi-Layer Perceptron (MLP): A deep neural network structure consisting of multiple fully connected layers stacked together. All neurons in each layer are interconnected with all neurons in the next layer, enabling it to learn complex features in the input data through layer-by-layer nonlinear transformations.

[0026] Batch Normalization (BN) is a technique used during neural network training to normalize the outputs of each layer of each batch of training data to zero mean and unit variance. It is used to alleviate the problem of internal covariate bias, which helps to accelerate network convergence and improve training stability.

[0027] Dropout: A neural network regularization technique that randomly sets the output of some neurons to zero with a certain probability during training to prevent the model from becoming overly dependent on specific neurons, thereby reducing the risk of overfitting.

[0028] ReLU activation function (Rectified Linear Unit): A commonly used activation function for neural networks, expressed as f(x) = max(0,x), meaning it outputs zero for negative values ​​and retains the original value for positive values. This function is simple to compute and can alleviate the gradient vanishing problem in deep networks to some extent.

[0029] Mean Squared Error (MSE) loss function: A loss function that measures the difference between predicted and true values. It calculates the mean of the squares of the prediction errors of all samples. The smaller the value, the more accurate the model prediction.

[0030] Adam optimizer (Adaptive Moment Estimation): A gradient descent optimization algorithm with an adaptive learning rate. It automatically adjusts the learning rate of each parameter by comprehensively utilizing the first-order moment estimate and the second-order moment estimate of the gradient. It has the characteristics of fast convergence speed and insensitivity to hyperparameters.

[0031] Cooperative Game Theory: This study investigates how to rationally distribute the total revenue generated by an alliance when multiple participants cooperate to form one.

[0032] Shapley Value: A payoff distribution scheme in cooperative game theory. Its core idea is to define a participant's contribution as the average marginal utility gained before and after joining any possible alliance. Shapley values ​​satisfy four axioms: efficiency, symmetry, dummy elements, and additivity.

[0033] Feature masking mechanism: a technique that masks the influence of specific input information by replacing or setting the value of a specified feature position in the input data. It is often used to simulate the state where certain input features are unavailable or do not participate.

[0034] Monte Carlo approximation sampling: a numerical calculation method that approximates the expectation or integral of complex mathematical problems by using a large number of random samples. It can transform the original exhaustive calculation into a statistical approximation based on random samples while ensuring a certain level of accuracy.

[0035] Marginal utility refers to the incremental change in the overall benefit of a particular alliance after a participant joins it, compared to before the participant joined. It reflects the magnitude of the participant's incremental contribution in the current combination scenario.

[0036] Standardization processor: A commonly used data preprocessing tool that performs a linear transformation on input features to zero mean (subtract the feature mean) and unit variance (divide by the feature standard deviation) to eliminate numerical scale inconsistencies caused by differences in units between different features.

[0037] Early stopping mechanism: A training control strategy to prevent overfitting of neural networks. During training, it continuously monitors the loss changes on the validation set. When the validation set loss no longer improves in several consecutive iterations, it automatically terminates training and saves the model weights with the best validation performance.

[0038] As one optional application scenario in the embodiments of this application, such as Figure 1 As shown, this multi-sensor contribution assessment method can be run in a system that includes at least one processing device. Figure 1 The system is illustrated in the example, including an edge computing node 101, an onboard computer 102, and a server 103, and the processing devices such as the edge computing node 101 and the onboard computer 102 can communicate with the server 103 via a network 110.

[0039] Specifically, the processing equipment can be an embedded computing platform, an edge computing node, an industrial control computer, or a vehicle-mounted terminal, an airborne computer, a UAV flight control unit, or other devices with data processing capabilities. Server 103 can be a standalone physical server, a server cluster or a distributed system, or a cloud server providing cloud computing services. Network 110 can be a wired or wireless network, examples of which include, but are not limited to, local area networks (LANs), wide area networks (WANs), industrial Ethernet, data links, and combinations thereof.

[0040] It should be noted that, Figure 1 This is merely an example of an application scenario and does not limit the scope of protection of this application.

[0041] In related technologies, existing multi-sensor contribution assessment methods in multi-sensor information fusion typically require complex real-time calculations based on raw measurement data for each assessment. They lack a pre-established correspondence between environmental state characteristics and contribution levels, leading to the technical problem of difficulty in efficiently determining the contribution of each sensor based on current environmental state characteristics. To address the above technical problem, this application provides a multi-sensor contribution assessment method to quickly determine the contribution of sensors based on current environmental state characteristics.

[0042] According to an embodiment of this application, a method for evaluating the contribution of multiple sensors is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0043] This embodiment provides a multi-sensor contribution evaluation method, which can be used in the aforementioned processing devices, such as edge computing nodes, airborne computers, servers, and other computing platforms with data processing capabilities. Figure 2 This is a flowchart of a multi-sensor contribution evaluation method according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps: Step S201: Obtain the current environmental state characteristics of multiple target sensors.

[0044] A target sensor refers to the sensor whose contribution needs to be evaluated. In a multi-sensor fusion system, multiple target sensors can exist simultaneously, such as radar sensors, infrared sensors, and satellite navigation sensors, each of which can be evaluated independently as a target sensor.

[0045] Environmental state characteristics refer to a set of quantifiable indicators that reflect the current working environment of the target sensor, such as the relative distance between the target object and the sensor, the signal-to-noise ratio of the current communication link, and the level of external electromagnetic interference. These indicators describe the physical environmental conditions faced by the sensor at the current moment from a macroscopic perspective.

[0046] Acquisition refers to the process by which the processing device reads or extracts the aforementioned environmental state characteristics from operational data. Specifically, the processing device can periodically collect current operational parameters and organize them into a structured feature vector, which serves as input for subsequent steps. For example, at a certain moment, if the processing device reads that the current distance between the target object and the sensor is 300 kilometers and the current signal-to-noise ratio is 15 dB, then these two values ​​can be combined into the current environmental state feature vector.

[0047] The purpose of this step is to extract key information that characterizes the working environment, laying the input foundation for subsequent contribution queries based on this information. It should be noted that the specific dimensions and value ranges of the environmental state characteristics can be flexibly determined according to the sensor type and working conditions in the actual application scenario; this application does not impose any restrictions on this.

[0048] Step S202: Input the current environmental state features into the pre-trained contribution evaluation model to obtain the current contribution of the target sensor.

[0049] The contribution evaluation model is a pre-trained offline model that establishes a mapping between environmental state characteristics and sensor contributions. For example, the contribution evaluation model can employ a lightweight neural network structure, such as a shallow multilayer perceptron (MLP), to ensure low-latency online inference on resource-constrained edge computing platforms. Current contribution refers to the quantified value of the target sensor's contribution to the system state prediction performance under current environmental conditions, reflecting the sensor's actual value to the overall system at the current moment.

[0050] This step involves directly inputting the acquired environmental state feature vectors into the contribution evaluation model for forward inference. For example, feature vectors containing parameters such as target relative distance and signal-to-noise ratio are fed into the deployed contribution evaluation model. After the model's forward calculation, the contribution values ​​corresponding to each target sensor are output.

[0051] The core of this step lies in utilizing a pre-trained contribution evaluation model to simplify the contribution calculation process, which originally required a large amount of repetitive inference, into a single lightweight forward inference process. This allows for the rapid output of the sensor's current contribution during the online application phase. It's important to note that the contribution evaluation model is trained offline; the online phase only performs inference without updating or recalculating the model. Therefore, this step has an extremely short execution time, meeting the requirements for real-time evaluation.

[0052] The contribution evaluation model is trained with historical environmental state characteristics as input and corresponding contribution labels as expected output. The contribution labels are determined based on the change in state prediction error corresponding to different sensor combinations. The state prediction error is obtained by inputting masked observation data corresponding to different sensor combinations into a pre-trained information fusion model.

[0053] Masked observation data refers to data obtained by processing the original sensor observation data according to a specific sensor combination. The sensor data belonging to the current combination retains the original values, while the sensor data not belonging to the current combination is replaced with preset label values. This simulates the state where some sensors do not participate in the current fusion process without retraining the information fusion model.

[0054] The information fusion model is a pre-trained model that takes multi-sensor observation data as input and system state prediction as output, and its parameters remain fixed during the contribution evaluation process.

[0055] State prediction error refers to the deviation between the predicted output of the information fusion model and the actual state of the system, reflecting the overall prediction performance of the system under a specific sensor combination.

[0056] The process of determining contribution labels involves quantifying the marginal impact of each sensor on the system's prediction performance by comparing the changes in state prediction error of the information fusion model under different sensor combinations. This marginal impact serves as the true value label for the contribution. For example, if removing a sensor from the current combination significantly increases the state prediction error of the information fusion model, then that sensor has a high contribution; conversely, if it doesn't, its contribution is low. By statistically analyzing and summarizing the error changes under various sensor combinations in historical data, the contribution labels for each sensor under different historical environmental conditions can be obtained.

[0057] A training sample library is constructed by pairing historical environmental state features with corresponding contribution labels. Based on this library, a contribution evaluation model is trained using supervised learning, enabling the model to learn the mapping pattern from macroscopic environmental states to sensor contributions. After offline training, the model can be directly used for rapid inference in the online phase, without requiring repeated calls to the information fusion model for extensive computational combinations during the online stage.

[0058] In summary, the multi-sensor contribution evaluation method provided in this application involves inputting masked observation data corresponding to different sensor combinations into a pre-trained information fusion model, constructing contribution labels based on the obtained state prediction error changes, and offline training the contribution evaluation model using historical environmental state features and contribution labels as training data. In the online application stage, it is only necessary to obtain the current environmental state features and input them into the trained contribution evaluation model to directly output the current sensor contribution, without requiring any repeated reasoning of the information fusion model in the online stage, thereby achieving rapid evaluation of the contribution of multiple sensors.

[0059] This embodiment provides a multi-sensor contribution evaluation method, which can be used in the aforementioned processing devices, such as edge computing nodes, airborne computers, servers, and other computing platforms with data processing capabilities. Figure 3 This is a flowchart of a multi-sensor contribution evaluation method according to an embodiment of this application, such as... Figure 3 As shown, the process includes the following steps: Step S301: Obtain the current environmental state characteristics of multiple target sensors.

[0060] The specific composition of environmental state characteristics has a significant impact on the accuracy of contribution assessment. Optionally, environmental state characteristics include at least one of the following: the relative distance of the target object to the target sensor, the signal-to-noise ratio, and the visibility index of the observation environment.

[0061] The relative distance refers to the spatial distance between the target object and the sensor platform. This indicator directly affects the sensor's detection accuracy and signal strength. For example, the farther the target is, the greater the radar's detection noise usually is, and the lower its data quality becomes.

[0062] Signal-to-noise ratio (SNR) is the ratio of the effective signal strength received by a sensor to the background noise strength. The higher the SNR, the more reliable the sensor data and the greater its contribution to state estimation.

[0063] Visibility index refers to the optical transparency of the current observation environment, and this index has a particularly significant impact on the performance of infrared and photoelectric sensors. For example, if at a certain moment the processing device reads that the current relative distance to the target object is 300 kilometers, the current signal-to-noise ratio is 15 dB, and the current visibility level is medium, then these three values ​​can be combined into a current environmental state feature vector [300, 15, 2]. The specific dimensions and value range of the environmental state features can be flexibly expanded according to the sensor type and operating conditions in the actual application scenario, and this application does not impose any restrictions on this.

[0064] Step S302: Input the current environmental state features into the pre-trained contribution evaluation model to obtain the current contribution of the target sensor.

[0065] The contribution evaluation model is a model pre-trained offline. Its essence is to establish a mapping relationship from macroscopic environmental state characteristics to the contribution of each sensor. The current contribution refers to the quantitative value of the target sensor's contribution to the system state prediction performance under the current environmental conditions, reflecting the sensor's actual value to the overall fusion system at the current moment.

[0066] In the online application phase, the processing device directly feeds the environmental state feature vector obtained in step S301 into the contribution evaluation model for forward inference, and outputs the current contribution of the target sensor. Since the training process of the contribution evaluation model has been completed in the offline phase, only one lightweight forward inference is performed in the online phase. There is no need to repeatedly call or combine calculations on the information fusion model. Therefore, the execution time of this step is extremely short, which can meet the needs of real-time evaluation.

[0067] To further illustrate the construction process of the contribution evaluation model, in one optional implementation, the contribution evaluation model can be trained through the following steps: First, multiple sets of historical environmental state feature data and the contribution labels corresponding to each set of historical environmental state feature data are obtained to construct a training sample set.

[0068] Among them, historical environmental state characteristic data refers to macroscopic physical quantity records describing the sensor's working environment collected during historical operation tasks, such as parameters like the target relative distance and signal-to-noise ratio at various moments in the historical task; contribution labels are supervision signals paired one-to-one with each set of historical environmental state characteristic data, reflecting the actual contribution quantification value of each sensor under the corresponding historical environmental conditions, and their specific determination method will be explained in detail in step S303. By pairing historical environmental state characteristic data with contribution labels, a large-scale training sample set is constructed, providing sufficient training data for subsequent supervised learning.

[0069] Secondly, the historical environmental state feature data in the training sample set is used as the input of the machine learning model, and the corresponding contribution label is used as the expected output of the machine learning model. The machine learning model is then trained to obtain the trained machine learning model.

[0070] The specific form of the machine learning model can be flexibly selected. For example, a lightweight shallow multilayer perceptron (MLP) can be used, which has fewer hidden layers and fewer parameters, and can achieve extremely low latency inference while ensuring nonlinear mapping capabilities. Alternatively, other machine learning models with nonlinear fitting capabilities, such as support vector regression and random forest, can also be used.

[0071] During training, mean squared error can be used as the loss function to measure the deviation between the predicted contribution value and the contribution label of the model output, and the model parameters can be iteratively updated using a gradient descent-type optimizer. An early stopping mechanism can also be configured to automatically terminate training if the validation set loss fails to improve after several consecutive iterations, preventing overfitting and thus obtaining optimal weights with good generalization ability. Furthermore, before inputting data into the model, standardization can be performed to normalize the feature vectors to zero mean and unit variance, eliminating the influence of differences in the dimensions of different physical quantities on the training process, further improving the model's training stability and convergence speed.

[0072] Finally, the trained machine learning model is deployed as a contribution evaluation model in the online application phase for real-time inference. Through the above training process, the trained contribution evaluation model can quickly output the contribution prediction values ​​of each sensor based solely on the current environmental state characteristics in the online phase.

[0073] Step S303, the process of determining contribution labels.

[0074] Contribution labels are the supervisory signals required for training the contribution evaluation model, and their quality directly determines the evaluation accuracy of the model. To ensure that the contribution labels objectively reflect the actual contribution of each sensor to the system performance, in one optional implementation, the contribution labels can be determined through the following steps: Acquire observational data that includes features from the target sensor and other sensors.

[0075] Observational data refers to the raw measurement data collected by a multi-sensor system during its historical operational missions, containing characteristic values ​​of all sensors in the system at each moment. For example, observational data may include the target's three-dimensional coordinates measured by radar sensors, the target's azimuth angle output by infrared cameras, and position correction information provided by satellite communication receivers. It should be noted that, since the sampling frequencies and timestamps of each sensor are usually not perfectly synchronized, time alignment processing can be performed on the acquired observational data to ensure consistency across different sensors in the time dimension.

[0076] For different combinations of sensors to be evaluated, masking is performed on sensor features not included in the sensor combination to be evaluated, resulting in masked observation data for the corresponding combination.

[0077] The sensor combination to be evaluated refers to a subset of sensors selected from all sensors in the system, used to simulate a scenario where the system uses only sensors from this subset for state prediction. Masking involves replacing the feature data of sensors not selected in the current sensor combination to mask their impact on the information fusion model output. This physically simulates the state where specific sensors "do not participate" in the current fusion process without retraining the information fusion model. Masked observation data is the result of performing the above replacement operation on the original observation data.

[0078] In one optional implementation, the masking process may include: identifying feature data corresponding to sensors not included in the sensor combination to be evaluated in the observation data; replacing the identified feature data with preset placeholder values ​​to obtain masked observation data; wherein the preset placeholder values ​​are used to characterize the state of the sensor not participating in the current fusion process.

[0079] The selection of the preset placeholder value is crucial. If "0" is chosen as the placeholder value, it may be confused with the case where the actual physical measurement value of the sensor is zero (for example, a component of the target's three-dimensional coordinates is exactly zero). This would cause the information fusion model to be unable to correctly distinguish between the two fundamentally different states of "the normal sensor measurement value is zero" and "the sensor is not involved." Therefore, the preset placeholder value should be a special value that deviates from the actual range of the sensor's physical measurement, so that the information fusion model can identify it as a clear "absence" marker, rather than a valid sensor measurement value.

[0080] For example, the preset placeholder value can be a very large value that far exceeds the normal measurement range of each sensor to ensure that it will not be confused with any real physical measurement value. By introducing a preset placeholder value to replace "0" for masking, the ambiguity between the mask state and the real zero value is effectively resolved, ensuring the accuracy of subsequent contribution label calculation.

[0081] In addition, to enable the information fusion model to correctly process masked observation data containing a large number of preset placeholders, a random feature masking training mechanism can be introduced simultaneously during the training phase of the information fusion model. This allows the model to fully learn the data distribution when some sensor features are masked, thereby enabling robust forward propagation of masked observation data containing a large number of placeholders during the contribution calculation phase and outputting reasonable state prediction results.

[0082] The masked observation data is input into a pre-trained information fusion model to obtain the state prediction results corresponding to each sensor combination to be evaluated.

[0083] The information fusion model is a pre-trained model that takes multi-sensor observation data as input and outputs the system's state prediction. Its parameters remain fixed during the calculation of the contribution label, and it is used as a performance simulator. By sequentially feeding masked observation data corresponding to different sensor combinations into the information fusion model, the system state prediction results for each sensor combination can be quickly obtained without retraining the model.

[0084] To further improve the prediction accuracy of the information fusion model under different observation conditions, in one optional implementation, the information fusion model may include multiple independent segmented sub-models. The training process may include: dividing historical multi-sensor observation data into multiple corresponding data subsets according to the different distance intervals in which the relative distance between the target object and the sensor is located; independently training the corresponding segmented sub-model for each distance interval using the corresponding data subset; when inputting masked observation data into the information fusion model, obtaining the current relative distance of the target object in the observation data, and calling the segmented sub-model that matches the distance interval in which the current relative distance is located for prediction.

[0085] The reason for introducing segmented sub-models is that the observation noise characteristics and signal quality of sensors often vary significantly with the distance between the target object and the sensor, making it difficult for a single model to maintain good prediction accuracy across the entire measurement range. For example, the observation data can be divided into three main intervals—far-end, mid-end, and near-end—based on the relative distance to the target, and a dedicated segmented sub-model can be trained independently for each interval. If the mid-end has a large amount of data and complex system state changes, it can be further subdivided into several continuous sub-intervals, with corresponding sub-models trained separately for each, achieving more refined adaptive modeling. During online inference, the system automatically selects the segmented sub-model for the corresponding interval based on the current relative distance to the target, effectively overcoming the performance degradation problem that easily occurs with single fusion models under cross-scale distance observations, and ensuring the accuracy of contribution label calculation in highly dynamic scenarios.

[0086] Based on the difference between the state prediction results and the actual physical state of the target object, the state prediction error corresponding to each sensor combination to be evaluated is determined.

[0087] State prediction error refers to the deviation between the predicted value output by the information fusion model under a specific sensor combination and the actual physical state of the target object. It reflects the overall prediction performance of the system under that sensor combination. The actual physical state refers to the true state value of the target object calibrated or recorded by high-precision means in historical tasks, serving as a benchmark for evaluating prediction accuracy. For example, state prediction error can be measured by the mean square error between the predicted output and the actual state. The smaller the state prediction error, the better the fusion performance under the corresponding sensor combination.

[0088] The marginal utility of the target sensor is determined based on the improvement in state prediction error before and after its addition to the combination.

[0089] Marginal utility refers to the improvement in system state prediction error resulting from adding a target sensor to a sensor combination; that is, the change in state prediction error before and after the addition. A larger marginal utility indicates a more significant contribution of the sensor's addition to system performance improvement. A negative marginal utility indicates that adding the sensor actually leads to a decrease in system performance, possibly due to the sensor introducing noise or erroneous information under current conditions. By calculating the marginal utility of a target sensor under different sensor combination scenarios, we can comprehensively characterize the synergistic or antagonistic relationships between the sensor and other sensors, rather than just considering its performance when working alone. This provides a more objective reflection of the sensor's actual value in multi-sensor collaborative scenarios.

[0090] The contribution label of the target sensor is determined based on marginal utility.

[0091] To further improve the accuracy of contribution labels, in one optional implementation, contribution labels can be determined based on marginal utility in the following manner: Multiple sensor combinations are randomly generated to determine their order of joining the consortium. A consortium refers to a subset of sensors currently participating in the fusion process; the order refers to the sequence in which each sensor joins the consortium. By randomly generating multiple different orders, each sensor has the opportunity to be evaluated under various combinations, thus avoiding evaluation bias caused by a fixed evaluation order.

[0092] For each arrangement order, each sensor is added to the current alliance in sequence, and the marginal utility of each sensor under each arrangement order is calculated based on the change in the state prediction error of each sensor combination to be evaluated before and after addition.

[0093] The Shapley value of a target sensor is obtained by weighted averaging of the marginal utilities of the same sensor across multiple permutations. Derived from cooperative game theory, the Shapley value quantifies a participant's fair contribution share by statistically analyzing the average marginal contribution of a participant to the alliance's value across all possible joining orders. The Shapley value possesses axiomatic properties such as efficiency, symmetry, virtuality, and additivity, theoretically ensuring the fairness of sensor contribution allocation and effectively eliminating the interference of complex nonlinear cooperative and interactive effects between sensors on the contribution evaluation results. It should be noted that accurately calculating the Shapley value requires traversing all possible sensor permutations, and the computational complexity increases exponentially with the number of sensors. Therefore, an approximation method using Monte Carlo random sampling can be employed: by randomly generating a finite number of permutations for sampling, the weighted average of the marginal utilities under each permutation is used to approximate the precise Shapley value, significantly reducing computational complexity while maintaining evaluation accuracy.

[0094] Finally, the Shapley value is used as the contribution label for the target sensor. Thus, by using the contribution quantification result based on the Shapley value from cooperative game theory as the training label for the contribution evaluation model, the mapping target learned by the contribution evaluation model has the theoretical guarantee of axiomatic fairness, thereby ensuring that the contribution prediction value output by the model in the online phase also inherits this fairness characteristic.

[0095] In summary, the multi-sensor contribution evaluation method provided in this application embodiment only needs to obtain the current environmental state features of the target sensor and input them into the pre-trained contribution evaluation model during the online application stage. It can directly output the current contribution of each sensor without performing any repeated reasoning or combined traversal calculations on the information fusion model during the online stage. This transforms the originally computationally complex contribution evaluation process into a lightweight model forward inference, realizing a rapid and dynamic evaluation of the contribution of multiple sensors.

[0096] The contribution label obtains the change in state prediction error by inputting masked observation data corresponding to different sensor combinations into an information fusion model with fixed parameters. This simulates and quantifies the system performance under different sensor combinations without retraining the information fusion model. Furthermore, by using preset placeholder values ​​instead of zero values ​​for masking, the ambiguity between the masked state and the actual zero-value measurement is effectively resolved, ensuring the accuracy of the state prediction error calculation and thus guaranteeing the quality of the contribution label. Building on this, by introducing a Shapley value calculation mechanism based on Monte Carlo random sampling, the contribution label theoretically possesses axiomatic fairness, objectively reflecting the true marginal contribution of each sensor under different combination backgrounds. This avoids the evaluation bias caused by traditional single ablation experiments neglecting the synergistic effect between sensors. The Monte Carlo approximate sampling strategy, while ensuring evaluation accuracy, reduces the computational complexity of the Shapley value from exponential to an acceptable range, improving the feasibility of offline label generation.

[0097] The information fusion model adopts a multi-segment sub-model architecture based on distance intervals. To address the issue of significant differences in sensor characteristics between the target object and the sensor at different relative distances, the model trains dedicated sub-models independently for each distance interval. This ensures that the information fusion model can output accurate state prediction results across the entire range, thereby further guaranteeing the reliability of the contribution labels calculated based on the model under different operating conditions and providing high-quality training supervision signals for the contribution evaluation model.

[0098] To better illustrate the multi-sensor contribution evaluation method of this application, a preferred embodiment will be provided below. This embodiment is intended to detail the implementation process of this application, but is not intended to limit the scope of protection of this application.

[0099] This embodiment uses a multi-source heterogeneous fusion system composed of radar, infrared cameras, and satellite sensors in a multi-target tracking scenario as a specific example to detail the complete implementation process of the quantitative evaluation method for sensor contributions. This scenario was chosen as an example because it encompasses typical engineering challenges such as significant differences in sensor types, inconsistent sampling frequencies, high probability of missing channels, and large target distance spans, making it highly representative and illustrative. It should be noted that this embodiment is not limited to the specific scenario or sensor type mentioned above. Any application scenario involving the collaborative work of multi-source heterogeneous sensors and requiring dynamic evaluation of the contribution of each sensor (including autonomous driving, industrial quality inspection, structural health monitoring, etc.) can be adapted and implemented by referring to the technical principles of this embodiment. Furthermore, this embodiment can either run independently on a single computing platform as a method or be deployed as a system in a distributed processing architecture containing multiple functional modules; the specific implementation form can be flexibly selected according to actual application needs.

[0100] In the aforementioned multi-target tracking scenario, radar provides the target's distance and velocity information, infrared cameras provide the target's angle and thermal imaging features, and satellites provide target position information with wide-area coverage. After information fusion, the system outputs a real-time estimate of the target's three-dimensional coordinates. However, as the distance between the target and the sensors dynamically changes, the actual contribution of the three types of sensors changes significantly: when the target is at a distance, radar signal attenuation is significant but it remains the primary source of information; as the target approaches the mid-to-near distance, the accuracy advantage of the infrared camera gradually becomes apparent. If the current contribution of each sensor cannot be quantified accurately and in real-time, the system will be unable to allocate computational resources rationally or promptly remove redundant sensor channels with low contribution or even negative effects.

[0101] In related technologies, traditional contribution assessment methods either rely on linear assumptions, making them difficult to adapt to nonlinear dynamic environments; or require repeated retraining of the neural network for each assessment, leading to cumbersome computation; or suffer from exponential computational complexity due to the Shapley value, failing to meet real-time requirements. Therefore, this embodiment proposes a multi-sensor contribution assessment system based on marginal utility and neural networks.

[0102] Figure 4 This is a schematic diagram of the multi-sensor contribution evaluation system based on marginal utility and neural networks provided in this embodiment. Figure 4 As shown, the system is divided into two parts: the offline training and evaluation phase at the top and the online deployment and application phase at the bottom. It contains four functional modules. The data processing flow proceeds sequentially from left to right. The accurate evaluation results of the offline phase are finally transmitted downwards in the form of labels, driving the training and deployment of the online lightweight network.

[0103] Module 1: Preprocessing of multi-source heterogeneous data.

[0104] like Figure 4 As shown in Module 1, the goal of this stage is to organize the raw observation data from radar, infrared cameras, and satellites into a unified, time-aligned fusion input feature matrix for use by subsequent neural network models.

[0105] The three types of sensors have different sampling frequencies. For example, radar collects 100 frames of data per second, infrared cameras collect 50 frames per second, and satellites collect 10 frames per second. Their timestamps are naturally not aligned. In this embodiment, the time series of the sensor with the highest sampling frequency and the most stable timing (radar in this embodiment) is selected as the main time axis. The raw observation data sequences of the other sensors are mapped to the main time axis using linear interpolation. The basic principle of linear interpolation is: if the time... The observed value is ,time The observed value is Then at any time in between The interpolation result is: ; Through the above processing, the data from all sensors are uniformly aligned to various moments on the main time axis, forming a multi-channel synchronized feature sequence.

[0106] In practical engineering, when a sensor fails to detect a target at a specific moment (e.g., the target enters a radar blind zone or its infrared beam is obscured by clouds), the corresponding data channel for that moment is lost. This embodiment uses a preset maximum value (the value is...). The maximum value is used as a placeholder for the missing channel, rather than zero. The reason for using a maximum value instead of zero is that the actual physical measurement of the sensor may be exactly zero (e.g., the coordinate of a certain dimension of the target is 0). If a zero value is used as a placeholder, the neural network cannot distinguish between the actual zero measurement and the missing state, thus causing ambiguity; while the maximum value... Far exceeding the normal physical measurement range of all sensors, it can be clearly identified by the network as a missing marker. Meanwhile, It is a bounded value, which will not cause numerical overflow during the forward propagation of the neural network, thus ensuring the numerical stability of the gradient during training. Before the data officially enters the first layer of the neural network, the system uniformly performs a conditional replacement operation: wherever the element value in the feature matrix is ​​detected to be equal to a maximum placeholder, it is replaced with a value that the network can process normally, in order to maintain the numerical stability of the forward propagation.

[0107] After the above processing, a structured fusion input feature matrix is ​​output, with rows corresponding to each moment on the main time axis and columns corresponding to the feature values ​​of each channel of all sensors. There are no missing positions in the matrix, and all missing positions have been filled by placeholders and processed by conditional replacement.

[0108] Module 2: Construction and training of segmented information fusion meta-model.

[0109] like Figure 4 As shown in Module 2, this stage constructs and trains an information fusion meta-model based on the fusion input feature matrix output from Module 1.

[0110] Considering that the signal-to-noise ratio and data quality of each sensor change drastically with distance as the target flies from a distant point to a near point, using a single model to process the full-range data often leads to performance degradation across distance intervals. This embodiment divides the complete time observation sequence into three main stages based on the changing pattern of the relative distance between the target and the sensors: the distant stage (time interval...). ), middle stage (time interval) ) and proximal phase (time interval) Since the system state changes in the middle stage are the most complex and last the longest, this embodiment further subdivides it into 6 consecutive sub-intervals, thus forming a total of 8 time periods. A segmented information fusion meta-model is trained independently for each time period, resulting in a total of 8 independent models.

[0111] Each segmented meta-model adopts a fully connected deep multilayer perceptron (Deep MLP) architecture, with the following specific structure: There are 4 hidden layers, with the number of neurons compressed layer by layer, configured as 512, 256, 128 and 64 respectively; Batch Normalization layers are configured between the linear output and activation function of each hidden layer to alleviate internal covariate shift during training and accelerate convergence; The ReLU function is uniformly used for the activation function of the hidden layers; To suppress overfitting, a random dropout mechanism is introduced in each layer, with the dropout probability set to 0.3; The number of nodes in the output layer is 3, directly corresponding to the three-dimensional true state coordinates (x, y, z) of the target.

[0112] Mean squared error (MSE) is used as the loss function to measure the deviation between the predicted output and the actual system state. The Adam adaptive optimizer is used for end-to-end training with a batch size of 64, an initial learning rate of 0.003, and a segmented decay strategy of 0.5 every 30 training epochs. The maximum number of training epochs is set to 60.

[0113] During training, this embodiment simultaneously introduces a channel random deactivation and random feature mask training mechanism. Specifically, during training, the feature values ​​of some sensor channels are randomly assigned to preset placeholder values ​​to simulate various sensor missing combinations. This mechanism enables the meta-model to robustly propagate forward even when faced with masked inputs where a large number of channel features are masked, laying the foundation for distributed generalization in Module 3 by simulating federated states using feature masks.

[0114] In this embodiment, taking the far-end stage as an example: when the target is at a distance, the observation signal attenuates significantly. The system selects sensors 1 to 6 as stable input sources and configures four effective data channels for each type of sensor to participate in the network training for this stage. After the eight segmented sub-models complete their training independently, their parameters are frozen and used as a performance simulator for subsequent contribution evaluation.

[0115] Module 3: Offline contribution quantification assessment based on Shapley value.

[0116] like Figure 4 As shown in Module 3, after the training of each segmented meta-model converges and the parameters are frozen, this stage uses the Shapley value method based on cooperative game theory to accurately quantify and evaluate the contribution of each sensor offline.

[0117] Cooperative game modeling: Each independent data channel within a sensor is considered a participant in a cooperative game, and the negative correlation form of the prediction error in the information fusion meta-model is defined as a feature function. : ; in, Indicates only use of alliances The predicted output of the neural network when given input features. For real labels, To prevent extremely small positive numbers with unstable values, a larger characteristic function value indicates better fusion performance of the system under that alliance.

[0118] To evaluate the performance of various sensor combinations without retraining the neural network, this embodiment innovatively introduces a feature masking mechanism to physically simulate the participation state of the consortium: for sensor channels included in the current consortium S, their corresponding input features retain the original observed values; for sensor channels not included in consortium S, their corresponding input features are uniformly replaced with preset maximum placeholders. The input tensor after masking is directly fed into the frozen parameter meta-model for forward inference, thus obtaining the system prediction error under this consortium combination, without requiring any retraining of the meta-model.

[0119] Since the computational complexity of traversing all possible alliance combinations increases exponentially with the number of participants, this embodiment employs a Monte Carlo approximation sampling method for efficient estimation. The specific steps are as follows: Randomly generate the addition order of M groups of sensor channels; for each group, calculate the change in the characteristic function value of each channel before and after joining the alliance, using this as the marginal utility of that channel in that group; take a weighted average of the marginal utilities of the same channel in the M groups to obtain the approximate Shapley value of that channel.

[0120] By summing and normalizing the Shapley values ​​of all data channels under the same sensor, the offline comprehensive contribution of the sensor can be obtained, which can be used as a ground truth label for training in the subsequent module four.

[0121] Module 4: Construction and Online Inference of a General Contribution Evaluation Network.

[0122] like Figure 4 As shown in Module 4, this stage pairs the offline contribution labels generated in Module 3 with the historical environmental state feature vectors to build a training sample library, and then trains a lightweight general contribution evaluation network to achieve fast contribution inference in the online stage.

[0123] Macroscopic environmental state feature vectors under different physical cross-sections are extracted from historical operational data. These include quantifiable macroscopic indicators such as target relative distance, current communication link signal-to-noise ratio (SNR), and environmental interference level. These indicators are key attributes that significantly influence sensor contribution, summarized from historical examples. The aforementioned feature vectors are then paired one-to-one with the corresponding offline contribution labels (Shapley values) calculated in Module 3 to construct a large-scale supervised offline training sample library.

[0124] The general contribution evaluation network employs a simplified shallow multilayer perceptron (MLP) architecture to achieve extremely low inference latency while ensuring nonlinear mapping capabilities. Specifically, the number of nodes in the input layer is consistent with the dimension of the macroscopic environmental state feature vector; before entering the network, the input data undergoes normalization preprocessing with zero mean and unit variance using a standard scaler; the hidden layer adopts a typical dual-hidden-layer structure, with the number of neurons configured as 64 and 32 respectively; the number of nodes in the output layer is consistent with the total number of sensors or independent data channels participating in the evaluation in the system, and the network output is further truncated and vector normalized to ensure that the real-time contribution prediction values ​​of all channels are non-negative and their sum is always equal to 1.

[0125] During the offline training phase, the mean squared error (MSE) is used to measure the deviation between the predicted value and the Shapley true value. The initial learning rate is set to 0.001, the batch size is set to 16, and an early stopping mechanism is configured: when the validation set loss does not improve after 40 consecutive iterations, the training is automatically terminated and the optimal weights are saved to prevent overfitting and to obtain the optimal model parameters for cross-domain mapping with high generalization ability.

[0126] The essence of this universal contribution evaluation network is a cross-domain rapid calculation mapping function from macroscopic physical states to microscopic sensor contributions. In the online application phase, the system collects the current macroscopic environmental state feature vector in real time and directly inputs it into the trained and deployed lightweight network. The network performs a single forward inference and outputs real-time dynamic contribution predictions for multi-source heterogeneous sensors within microseconds. Based on this output, the resource scheduling module can identify and prune redundant sensor data streams with low contributions in real time, thereby significantly reducing the data storage and transmission volume of the underlying computing platform and achieving dynamic optimization scheduling of system hardware resources.

[0127] In summary, the multi-sensor contribution evaluation system based on marginal utility and neural networks provided in this embodiment organically combines the axiomatic fairness evaluation framework of cooperative game theory with the efficient reasoning capability of lightweight neural networks, forming a closed-loop evaluation system with accurate offline labeling and fast online mapping. In the offline stage, the alliance participation state is physically simulated through a feature masking mechanism, avoiding repeated retraining of the meta-model. Combined with Monte Carlo approximation sampling, the computational complexity of Shapley value is reduced from exponential to a controllable level. In the online stage, a pre-trained lightweight evaluation network achieves real-time contribution output at the microsecond level, significantly reducing the computing power burden on edge computing platforms such as airborne and missile-borne systems. This allows the system to dynamically prune redundant sensor channels with low contributions, effectively reducing the data storage and transmission overhead of the underlying hardware and achieving efficient utilization of computing resources.

[0128] This embodiment also provides a multi-sensor contribution assessment system, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the systems described in the following embodiments are preferably implemented in software, hardware implementations, or a combination of software and hardware, are also possible and contemplated.

[0129] This embodiment provides a multi-sensor contribution assessment system, such as... Figure 5 As shown, it includes: The acquisition module 501 is used to acquire the current environmental state characteristics of multiple target sensors; Evaluation module 502 is used to input the current environmental state features into a pre-trained contribution evaluation model to obtain the current contribution of the target sensor; The contribution evaluation model is trained by taking historical environmental state characteristics as input and the corresponding contribution label as the expected output. The contribution label is determined based on the change in state prediction error corresponding to different sensor combinations. The state prediction error is obtained by inputting the masked observation data corresponding to different sensor combinations into a pre-trained information fusion model.

[0130] In one alternative implementation, the system further includes: Module 503 is used to acquire multiple sets of historical environmental state feature data and the contribution labels corresponding to each set of historical environmental state feature data, in order to construct a training sample set. The historical environmental state feature data in the training sample set is used as the input of the machine learning model, and the corresponding contribution label is used as the expected output of the machine learning model. The machine learning model is trained to obtain the trained machine learning model. The trained machine learning model is used as the contribution evaluation model.

[0131] In one alternative implementation, the construction module 503 is configured to: Acquire observational data that includes features from the target sensor and other sensors; For different combinations of sensors to be evaluated, masking is performed on sensor features not included in the sensor combinations to be evaluated, and masked observation data for the corresponding combinations are obtained. The masked observation data is input into a pre-trained information fusion model to obtain the state prediction results corresponding to each sensor combination to be evaluated. Based on the difference between the state prediction results and the actual physical state of the target object, determine the state prediction error corresponding to each sensor combination to be evaluated; The marginal utility of the target sensor is determined based on the improvement in state prediction error before and after its addition to the combination. The contribution label of the target sensor is determined based on marginal utility.

[0132] In one alternative implementation, the construction module 503 is configured to: In the observation data, identify the feature data corresponding to sensors that are not included in the sensor combination to be evaluated; The identified feature data is replaced with preset placeholder values ​​to obtain masked observation data; the preset placeholder values ​​are used to characterize the state where the sensor does not participate in the current fusion process.

[0133] In one alternative implementation, the environmental state characteristics include at least one of the following: the relative distance of the target object to the target sensor, the signal-to-noise ratio, and the visibility index of the observation environment.

[0134] In one alternative implementation, the construction module 503 is configured to: Randomly generate the order in which multiple sensor groups join the alliance; For each arrangement order, each sensor is added to the current alliance in turn, and the marginal utility of each sensor under each arrangement order is calculated based on the change in the state prediction error of each sensor combination to be evaluated before and after the addition. The Shapley value of the target sensor is obtained by weighting and averaging the marginal utility of the same sensor in multiple arrangement orders. The Shapley value is used as a label for the contribution of the target sensor.

[0135] In one alternative implementation, the information fusion model includes multiple independent segmented sub-models; the construction module 503 is used for: Based on the different distance ranges between the target object and the sensor, the historical multi-sensor observation data is divided into multiple corresponding data subsets; For each distance interval, a corresponding segmented sub-model is independently trained using the corresponding data subset; When inputting masked observation data into the information fusion model, the current relative distance of the target object in the observation data is obtained, and the segmented sub-model that matches the distance interval of the current relative distance is called for prediction.

[0136] The multi-sensor contribution assessment system provided in this application can execute the multi-sensor contribution assessment method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0137] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0138] The following is a detailed reference. Figure 6 This diagram illustrates a suitable structural schematic for implementing the electronic device described in the embodiments of this application. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 601, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 602 or a program loaded from memory 608 into random access memory (RAM) 603. The RAM 603 also stores various programs and data required for the operation of the electronic device. The processor 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0139] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0140] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a memory 608, or installed from a ROM 602. When the computer program is executed by the processor 601, it performs the functions defined in the multi-sensor contribution assessment method of embodiments of this application.

[0141] Figure 6 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0142] This application also provides a computer-readable storage medium. The methods described above according to this application can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the multi-sensor contribution assessment method shown in the above embodiments is implemented.

[0143] A portion of this application can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to this application through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0144] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A multi-sensor contribution assessment method, characterized by, The method includes: Acquire the current environmental state characteristics of multiple target sensors; The current environmental state features are input into a pre-trained contribution evaluation model to obtain the current contribution of the target sensor. The contribution evaluation model is trained using historical environmental state features as input and corresponding contribution labels as expected output. The contribution label is determined based on the change in state prediction error corresponding to different sensor combinations. The state prediction error is obtained by inputting the masked observation data corresponding to different sensor combinations into a pre-trained information fusion model.

2. The method according to claim 1, characterized in that, The contribution evaluation model is trained through the following steps: Multiple sets of historical environmental state feature data and the contribution labels corresponding to each set of historical environmental state feature data are obtained to construct a training sample set; The historical environmental state feature data in the training sample set is used as the input of the machine learning model, and the corresponding contribution label is used as the expected output of the machine learning model. The machine learning model is then trained to obtain the trained machine learning model. The trained machine learning model is used as the contribution evaluation model.

3. The method according to claim 1 or 2, characterized in that, The contribution label is determined through the following steps: Acquire observation data containing features of the target sensor and other sensors; For different combinations of sensors to be evaluated, masking is performed on sensor features not included in the sensor combination to be evaluated to obtain masked observation data for the corresponding combination. The masked observation data is input into a pre-trained information fusion model to obtain the state prediction results corresponding to each sensor combination to be evaluated. Based on the difference between the state prediction result and the actual physical state of the target object, determine the state prediction error corresponding to each sensor combination to be evaluated; The marginal utility of the target sensor is determined based on the improvement in state prediction error before and after the target sensor is added to the combination. The contribution label of the target sensor is determined based on the marginal utility.

4. The method according to claim 3, characterized in that, The step of performing masking processing on sensor features not included in the sensor combination to be evaluated to obtain masked observation data for the corresponding combination includes: In the observation data, identify the feature data corresponding to sensors that are not included in the sensor combination to be evaluated; The identified feature data is replaced with a preset placeholder value to obtain the masked observation data; wherein the preset placeholder value is used to characterize the state in which the sensor does not participate in the current fusion process.

5. The method according to claim 1, characterized in that, The environmental state characteristics include at least one of the following: the relative distance of the target object to the target sensor, the signal-to-noise ratio, and the visibility index of the observation environment.

6. The method according to claim 3, characterized in that, The determination of the contribution label of the target sensor based on the marginal utility includes: Randomly generate the order in which multiple sensor groups join the alliance; For each arrangement order, each sensor is added to the current alliance in turn, and the marginal utility of each sensor under each arrangement order is calculated based on the change in the state prediction error of each sensor combination to be evaluated before and after the addition. The Shapley value of the target sensor is obtained by weighted averaging of the marginal utility of the same sensor in multiple sets of the aforementioned arrangement orders. The Shapley value is used as a contribution label for the target sensor.

7. The method according to claim 3, characterized in that, The information fusion model includes multiple independent segmented sub-models; the training process of the information fusion model includes: Based on the different distance ranges between the target object and the sensor, the historical multi-sensor observation data is divided into multiple corresponding data subsets; For each distance interval, the corresponding segmented sub-model is independently trained using the corresponding data subset; When the masked observation data is input into the information fusion model, the current relative distance of the target object in the observation data is obtained, and the segmented sub-model that matches the distance interval of the current relative distance is called to make a prediction.

8. A multi-sensor contribution assessment system, characterized in that, The system includes: The acquisition module is used to acquire the current environmental state characteristics of multiple target sensors; The evaluation module is used to input the current environmental state features into a pre-trained contribution evaluation model to obtain the current contribution of the target sensor. The contribution evaluation model is trained using historical environmental state features as input and corresponding contribution labels as expected output. The contribution label is determined based on the change in state prediction error corresponding to different sensor combinations. The state prediction error is obtained by inputting the masked observation data corresponding to different sensor combinations into a pre-trained information fusion model.

9. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the multi-sensor contribution assessment method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the multi-sensor contribution assessment method according to any one of claims 1 to 7.

Images