A method and device for multi-sensor information fusion

Abstract

This application provides a multi-sensor information fusion method and device. Preprocessed data is obtained by performing a spatiotemporal preprocessing algorithm based on production rules on sensor layer data. The preprocessed data is then subjected to feature-layer fusion using a neural network multi-classifier to obtain feature-fusion data. The feature-fusion data is then subjected to decision-layer fusion using a support-corrected iterative fusion method to obtain decision-layer data. The decision-layer data is then uploaded for decision-making. This application utilizes a multi-level fusion framework to eliminate redundancy and inconsistencies among massive amounts of heterogeneous, multi-source data, resulting in more reliable decision results.

Description

Technical Field

[0001] The present application relates to the field of information processing technology, and in particular to a multi-sensor information fusion method and device. Background Art

[0002] With the rapid development of IoT technology, more and more devices are able to connect and communicate with each other. This enables the deployment of multiple and diverse sensors within smart city environments to collect environmental information. By processing this data, more reliable and comprehensive support is provided for smart city environmental monitoring, operational management, and decision-making.

[0003] In this context, the fusion and processing of multi-source, heterogeneous data has become a crucial aspect of the development of the Smart City Internet of Things. However, with the continuous increase in monitoring scenarios, the data volume has become massive, the data sources and formats are diverse and heterogeneous, and data timeliness is also crucial. Furthermore, data uncertainty also presents challenges. These factors can lead to data redundancy and inconsistencies, reducing the efficiency of fusion of massive amounts of data from different data formats. Summary of the Invention

[0004] This application provides a multi-sensor information fusion method and device. The technical solution of this application is as follows:

[0005] According to a first aspect of an embodiment of the present application, a multi-sensor information fusion method is provided, comprising:

[0006] Processing the sensor layer data using a spatiotemporal preprocessing algorithm based on production rules to obtain preprocessed data; wherein the sensor layer data is obtained through multiple sensors;

[0007] Using a neural network multi-classifier to perform feature layer fusion on the preprocessed data to obtain feature fusion data;

[0008] Performing decision-layer fusion on the feature fusion data using a support correction iterative fusion method to obtain decision-layer data;

[0009] The decision-making layer data is uploaded for decision-making.

[0010] According to a second aspect of an embodiment of the present application, a resource allocation device for a cognitive industrial Internet of Things in a dynamic uncertain scenario is provided, comprising:

[0011] A data preprocessing module is used to process the sensor layer data using a spatiotemporal preprocessing algorithm based on production rules to obtain preprocessed data; wherein the sensor layer data is obtained through multiple sensors;

[0012] A feature fusion module uses a neural network multi-classifier to perform feature layer fusion on the preprocessed data to obtain feature fusion data;

[0013] A decision layer fusion module, which performs decision layer fusion on the feature fusion data using a support correction iterative fusion method to obtain decision layer data;

[0014] The decision-making layer data uploading module uploads the decision-making layer data for decision-making.

[0015] According to a third aspect of an embodiment of the present application, there is provided a non-volatile storage device comprising: a processor, and a memory communicatively connected to the processor;

[0016] The memory stores computer-executable instructions;

[0017] The processor executes the computer-executable instructions stored in the memory to implement the method provided by the first aspect.

[0018] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the method provided in the first aspect.

[0019] Beneficial effects:

[0020] This application provides a multi-sensor information fusion method and device. Preprocessed data is obtained by performing a spatiotemporal preprocessing algorithm based on production rules on sensor layer data. The preprocessed data is then subjected to feature-layer fusion using a neural network multi-classifier to obtain feature-fusion data. The feature-fusion data is then subjected to decision-layer fusion using a support-corrected iterative fusion method to obtain decision-layer data. The decision-layer data is then uploaded for decision-making. This application utilizes a multi-level fusion framework to eliminate redundancy and inconsistencies among massive amounts of heterogeneous, multi-source data, resulting in more reliable decision results.

[0021] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification are used to explain the principles of the present application, and do not constitute an improper limitation on the present application.

[0023] Figure 1 The figure is a schematic diagram showing the steps of a multi-sensor information fusion method according to an exemplary embodiment.

[0024] Figure 2 The figure is a schematic diagram of a data preprocessing method based on production rules according to an exemplary embodiment.

[0025] Figure 3 Box line diagram for removing outliers according to an exemplary embodiment Figure 4 Schematic diagram of the percentile range.

[0026] Figure 4 The figure is a schematic structural diagram of a multi-sensor information fusion device according to an exemplary embodiment. DETAILED DESCRIPTION

[0027] In order to enable ordinary people in the art to better understand the technical solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0028] It should be noted that the terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0029] Figure 1 This is a flow chart of a multi-sensor information fusion method provided by an exemplary embodiment of the present application. Figure 1 As shown in Figure 2, the multi-sensor information fusion method specifically includes:

[0030] S1, performing a spatiotemporal preprocessing algorithm based on production rules on the sensor layer data to obtain preprocessed data, wherein the sensor layer data is heterogeneous data acquired by multiple sensors.

[0031] Specifically, before the device performs data fusion, the raw data needs to be preprocessed to ensure the reliability of the results because the data types collected by each sensor are different.

[0032] The sensing layer data is data collected by multiple sensors. Exemplarily, the data collected by multiple sensors include at least one of PM2.5, PM10, SO2, CO, NO2, O3, etc.

[0033] In some embodiments, the pre-processing algorithm based on production rules mainly selects different data pre-processing algorithms according to different input types. This application uses a production system to describe the process. The production system consists of three parts: a production rule base 21, an inference engine 22 and a dynamic database 23. The production rules of this application have the following structure: Figure 2 shown.

[0034] The production rule base 21 is a production rule set consisting of a series of rules that determine the data type of input data. The rules within a production rule set, based on their logical relationships, form a structure called an inference network. Exemplarily, the production rule base includes at least one of the following: NO2 sensor rules, O3 sensor rules, pressure sensor rules, video image sensor rules, and light sensor rules.

[0035] The inference engine 22 is the control execution mechanism, responsible for testing or matching the preconditions of production rules, scheduling and selecting rules, and interpreting and executing the rule body. In other words, the inference engine implements reasoning and controls reasoning, and is also the rule interpreter.

[0036] The dynamic database 23 includes a database and an algorithm library, wherein the database includes a global database, a comprehensive database, a context, etc. It is a dynamic data structure for storing initial fact data, intermediate results, final results, etc.; the algorithm library is used to store spatiotemporal preprocessing algorithms.

[0037] Exemplarily, the spatiotemporal preprocessing algorithm includes at least one of an interquartile range algorithm for outlier detection, a least squares rule for time alignment, and a Kalman filter algorithm for denoising.

[0038] In some embodiments, if the spatiotemporal preprocessing data is data collected by a single sensor in the time dimension, the interquartile range algorithm and the Kalman filter algorithm are used to remove outliers and reduce noise between data; if the data is heterogeneous data collected between different sensors in the spatiotemporal dimension, the least squares rule algorithm of time matching is used to process the data, with the aim of adapting the data collected by sensors with different acquisition frequencies.

[0039] In some embodiments, if the data is collected by a general sensor, an interquartile range algorithm is used to detect outliers.

[0040] like Figure 3 As shown, the interquartile range algorithm involved in this application measures variability by dividing the data set into quartiles. The data is sorted in ascending order and divided into 4 equal parts. Q1, Q2, and Q3 are called the first, second, and third quartiles, which are the values ​​that separate the 4 equal parts.

[0041] Q1 represents the 25th percentile of the data; Q2 represents the 50th percentile of the data; Q3 represents the 75th percentile of the data.

[0042] The IQR is the range between the first and third quartiles, i.e., IQR = Q3 – Q1. Data points below Q1 – 1.5*IQR or above Q3 + 1.5*IQR are outliers.

[0043] The interquartile range reflects the degree of dispersion of the middle 50% of the data. The smaller the value, the more concentrated the data in the middle; the larger the value, the more dispersed the data in the middle. This method can effectively detect outliers in the data. After finding the outliers, use the mean or median to replace the outliers to complete the outlier filtering operation.

[0044] In some embodiments, during multi-sensor data fusion, due to the varying precision of the sensors involved, their target measurement cycles are not synchronized, and they have varying transmission delays. This leads to data asynchrony in the measurement data received by the fusion center from different sensors. These measurement data cannot reflect the target's motion state at the same moment. Therefore, before fusing multi-sensor data, the multi-sensor data should be temporally aligned. Otherwise, fusing unaligned multi-sensor data may yield worse results than a single sensor, and may even generate some untrue, false information. In other words, the significance of temporal alignment lies in applying a specific algorithm to the target observation data collected by each sensor, extrapolating each measurement value to a unified observation time point.

[0045] Based on the above reasons, this application uses the least squares method to compensate for the time alignment between asynchronous information in order to compress and "align" the data to any time. Specifically,

[0046] Assume that the sampling periods of the first and second sensors are τ and T, respectively, and that their ratio is an integer, that is, n = τ / T. If the first sensor's most recent target state update is (k-1) / τ, the next update time is k = [(k-1) / τ + nT]. This means that between two consecutive target state updates, the second sensor has n measurements. Therefore, using the least squares rule, these n measurements are fused into a single virtual value, which serves as the second sensor's measurement at time k and is then fused with the first sensor's measurement.

[0047] In some embodiments, Z n =[z1z2…z n ] T represents the set of n measurement values ​​of the second sensor from time (k-1) to time k, Z n And the first sensor measurement value at time k is synchronized. represents z1,z2,…z n , after fusion of the measured value and its derivative, the measured value Z of the second sensor is i Expressed as:

[0048]

[0049] Among them, v i represents the measurement noise.

[0050] In some embodiments, the vector is formalized as: n =W n U+V n .

[0051] in Its mean is zero and its variance is In the formula is the noise variance of the measurement before fusion.

[0052] Using the least squares principle, we introduce the function To make J the smallest, the two sides of J Taking the partial derivative and setting it to zero, we get:

[0053]

[0054] Therefore, the least squares solution is:

[0055]

[0056] And the corresponding variance matrix estimate is:

[0057]

[0058] For the n measurement values ​​of the second sensor at time k, the fused measurement value is:

[0059]

[0060] The measurement noise variance is:

[0061]

[0062] Where: c1 = -2 / n, c2 = 6 / [n(n+1)].

[0063] This application's least-squares temporal registration algorithm uses minimizing the sum of squared measurement residuals as a performance criterion. It fits the measurements of the high-rate sensor to the time points of the low-rate sensor, then fuses these virtual measurements with the low-rate sensor's. Because the algorithm compresses the data from the second sensor, reducing the amount of fused data, it offers relatively simple computations and high registration accuracy.

[0064] In some embodiments, the Kalman filter method is a method for linear filtering and prediction problems, which can estimate and correct the target in real time based on the measurement information. Specifically,

[0065] Discrete Kalman filter system equation, the state equation is:

[0066] X(k+1)=Φ(k+1|k)X(k)+G(k)w(k)

[0067] The measurement equation is:

[0068] Z(k)=H(k)X(k)+v(k)

[0069] Where X(k+1) is the state estimation vector of the system at time k+1, Φ is the state transfer matrix of the system, G(k) is the state noise weighting matrix, w(k) is the system noise vector, Z(k) is the measurement vector at time k, H(k) is the measurement transfer matrix, and v(k) is the measurement noise vector at time k.

[0070] Kalman filtering requires that w(k) and v(k) are independent zero-mean Gaussian white noise sequences and satisfy:

[0071]

[0072] Among them, Q(k) and R(k) are the state noise matrix and measurement noise matrix of k respectively, δ ki is a Kroneckor function with δ ki =0(k≠i),δ ki =1(k=i).

[0073] S2, using a neural network multi-classifier to perform feature layer fusion on the preprocessed data to obtain feature fusion data.

[0074] In the smart city environmental monitoring system, data comes from a variety of different sensors. There are redundancies and contradictions in the massive data. Therefore, a neural network multi-classifier is used to fuse multi-sensor data to obtain the overall environmental assessment results.

[0075] For example, the neural network multi-classifier includes at least one of the back propagation neural network (BP), the radial basis function neural network (RBF) and the extreme learning machine neural network (ELM). Among them, the BP neural network adjusts the weights and thresholds through the back propagation algorithm and has a strong nonlinear modeling ability. The RBF neural network models the input data based on the radial basis function and shows good generalization performance. The ELM neural network uses a method of randomly initializing the hidden layer weights to quickly solve the output layer weights. The above three neural network algorithms have excellent performance in multi-classification problems. By integrating the prediction results of these three classifiers, we can make full use of their respective characteristics and advantages, mine data and extract features from different angles, and improve the accuracy and robustness of multi-classification tasks.

[0076] S3, using the support correction iterative fusion method to perform decision layer fusion on the feature fusion data to obtain decision layer data.

[0077] In some embodiments, the feature fusion data decision layer is fused into the DS evidence theory combination rule using a support modified iterative fusion method to obtain decision layer data.

[0078] In evidence theory, the entire set of objects of study is called an identification frame. Elements in the frame are discrete and mutually exclusive. Each element is called a primitive, and the set consisting of all subsets of the frame is called the power set of the frame. The core problem of evidence theory is: given a known identification frame, determine the degree to which an undetermined element in the frame belongs to a subset of zero based on given information. This introduces the foundation of evidence theory—the basic probability assignment function (BPA). It is defined as follows:

[0079] Define 1Θ as a frame to be identified, Θ={μ1,μ2,…,μ n}, if there is a mapping m:2 Θ →[0, 1], and satisfies where m is the basic probability assignment function of A on the framework Θ, also known as the mass function. m(A) represents the precise degree of confidence in proposition A, and m(B) represents the precise degree of confidence in proposition B.

[0080] Definition 2 In the recognition framework Θ={μ1,μ2,…,μ n}, there are two pieces of evidence E1 and E2, m1 and m2 are their corresponding basic probability assignment functions, and the DS evidence combination rule is:

[0081]

[0082]

[0083]

[0084] in:

[0085] The value of K reflects the degree of conflict between the two pieces of evidence, often called the conflict coefficient, and ranges from [0 to 1]. The closer it is to 1, the greater the conflict between the two pieces of evidence; conversely, the closer it is to 0, the less conflict there is. The coefficient 1 / 1-K is a normalization factor designed to prevent non-zero values ​​from being assigned to empty sets.

[0086] When evidence conflicts are minor, the DS evidence theory combination rules can continuously concentrate the evidence confidence towards the proposition with higher certainty. However, when evidence conflicts are large or completely contradictory, DS evidence theory discards all conflicts, losing its ability to integrate them, and the resulting conclusions often contradict the actual situation. This application addresses the conflicts and computational complexity issues of DS evidence theory by using a support-corrected iterative fusion method to solve these problems.

[0087] Starting from the data source level, we use the volatility of node error data and introduce node trust correction parameters to correct the evidence conflicts caused by error data, thus reducing the probability of evidence conflicts from the source. For example,

[0088] S31, the feature fusion data of each node is used to perform evidence correction using variance, and the DS theory is used to fuse the various evidence data to obtain n groups of initial evidence;

[0089] S32, fusing the initial evidence to obtain revised evidence, and using the revised group of evidence as the initial reference evidence for the iterative algorithm.

[0090] S33, obtaining a discount factor for each piece of initial evidence by calculating the distance, angle cosine, and conflict amount between the n groups of initial evidence fusion results;

[0091] S34, using the discount factor to re-correct the evidence body, and then fuse it, and iterate continuously until the preset accuracy is reached to obtain the final fusion result.

[0092] In some embodiments, there are currently n sets of initial evidence A fusion result set up represents the evidence of the i-th evidence after the j-th iteration correction; Expressing support for evidence The weights are modified in the jth iteration, where j = 1, 2, ..., n and i = 1, 2, ..., n.

[0093] The method for obtaining the final fusion result includes:

[0094] S41, calculation and Josselme distance:

[0095]

[0096] in: is a 2 N ×2 N phalanx.

[0097] S42, calculation and The cosine of the angle:

[0098]

[0099] S43, calculation and The conflict amount k i , and calculate the support parameter,

[0100]

[0101] Among them: a, b, c represent the importance adjustment parameters of a single attribute respectively.

[0102] S44, parameter of evidence support Normalize and update evidence The weight of:

[0103]

[0104] S45, using the modified weight Correct the j-th evidence to get

[0105]

[0106]

[0107] S46, iterate until Stop the iteration and get the final fusion result.

[0108] A hierarchical fusion model is established to address the diverse characteristics of aggregating massive amounts of heterogeneous data from multiple sources in smart city environmental monitoring scenarios. This model processes the large amounts of raw data collected by wireless sensor networks within the network, eliminating redundant information between multiple sensors, reducing data redundancy and conflicts, and enhancing the credibility of the data and the reliability of the fusion results.

[0109] S4. Uploading the decision-making layer data for decision-making.

[0110] Specifically, decisions are made based on the results of hierarchical fusion, and the results are uploaded and displayed to detect changes in the collected data in real time.

[0111] The decision results are uploaded to the cloud via the narrowband Internet of Things (NB-IoT); the cloud has a visual interface that can monitor the status of the current environment in real time and report abnormal information to the relevant departments to facilitate the execution of corresponding operations.

[0112] In some embodiments, data from air quality application scenarios are fused. First, the neural network multi-classifier mentioned in S2 is trained using historical air quality index AQI data to obtain three models to construct a multi-level fusion framework for air quality application scenarios. Secondly, the fusion framework is applied to actual scenarios. In actual applications, the data collected by the air quality sensor is first passed through the spatiotemporal preprocessing algorithm of the production rule of S1 to obtain high-quality preprocessed data. Secondly, the preprocessed data is passed through the neural network multi-classifier of S2 to obtain multiple sets of feature fusion data. These multiple sets of feature fusion data are then iteratively fused through the support correction iterative fusion method of S3 to perform multiple sets of feature evidence. Finally, the fused results are uploaded to the cloud.

[0113] This application provides a multi-sensor information fusion method. This method processes sensor-layer data using a spatiotemporal preprocessing algorithm based on production rules to obtain preprocessed data. The sensor-layer data is acquired through multiple sensors. A neural network multi-classifier is used to perform feature-layer fusion on the preprocessed data to obtain feature-fusion data. The feature-fusion data is then fused at the decision-layer using a support-corrected iterative fusion method to obtain decision-layer data. The decision-layer data is then uploaded for decision-making. This application utilizes a multi-level fusion framework to eliminate redundancy and inconsistencies among massive amounts of heterogeneous, multi-source data, resulting in more reliable decision results.

[0114] In some embodiments, a historical AQI dataset from a city since 2013 is selected. A portion of the dataset is shown in Table 1. It is assumed that six sensors are deployed in the environment, monitoring PM2.5, PM10, SO2, CO, NO2, and O3, respectively. The dataset is divided into a training set, a test set, and a validation set. The six sensor data are used as input, and the quality level or AQI is used as the label. The training and validation set data are then used to train and model three neural network classifiers, respectively. The test set data simulates a real-world environment to test the multi-level fusion framework.

[0115] Table 1 Historical AQI data

[0116] date AQI Quality Grade PM2.5 PM10 SO2 CO NO2 O3_8h 2013 / 12 / 2 142 Light pollution 109 138 61 2.6 88 11 2013 / 12 / 3 86 good 64 86 38 1.6 54 45 2013 / 12 / 4 109 Light pollution 82 101 42 2 62 23 2013 / 12 / 5 56 good 39 56 30 1.2 38 52 2013 / 12 / 6 169 Moderate pollution 128 162 48 2.5 78 15 2013 / 12 / 7 291 Severe pollution 241 285 64 4.2 98 6 2013 / 12 / 8 223 Severe pollution 173 189 47 2.9 60 41 2013 / 12 / 9 26 excellent 11 16 10 0.6 22 51 2013 / 12 / 10 45 excellent 21 45 14 1 29 52 2013 / 12 / 11 30 excellent 19 30 15 0.7 30 45 2013 / 12 / 12 29 excellent 16 29 11 0.8 25 56 2013 / 12 / 13 66 good 48 63 29 1.3 45 29 2013 / 12 / 14 56 good 40 48 29 1.2 41 46

[0117] The test set data was hierarchically fused using the multi-level fusion framework algorithm, and the accuracy of the decision-level fusion results was compared with the output of the single classifier. The following table shows the comparison of the decision accuracy.

[0118] Table 2 Effect comparison table

[0119] Accuracy Error rate BP 93.33％ 6.67％ RBF 93.33％ 6.67％ ELM 86.67％ 13.33％ Decision-making level integration 96.00％ 4.00％

[0120] The following two tables show how the decision layer considers the outputs of the three classifiers and ultimately obtains the correct classification.

[0121] Table 3 Comparison of evidence revisions

[0122] Probability value Decision Value BP 0,0.884,0.115,0,0,0 Category II RBF 0,0.415,0.582,0.002,0，0 Category 3 ELM 0.112,0.542,0.051,0.081,0.1,0.101 Category II Decision-making level integration 0,0.982,0.017,0,0,0 Category II True value 0,1,0,0,0,0 Category II

[0123] Table 4 Comparison of evidence revisions

[0124] Probability value Decision Value BP 0,0.009,0.598,0.367,0.024,0 Category 3 RBF 0,0.001,0.682,0.303,0.012,0 Category 3 ELM 0.201,0.093,0.173,0.268,0.127,0.135 Category 4 Decision-making level integration 0,0,0.702,0.296,0,0 Category 3 True value 0,0,1,0,0,0 Category 3

[0125] Through the multi-level fusion of data through the above-mentioned example simulation, the redundancy and contradiction between data were eliminated, and ultimately a more reliable decision-making result was obtained.

[0126] The test set data is fused using a multi-level fusion framework. First, the raw data is preprocessed. Then, the six-dimensional heterogeneous data is input into three neural network multi-classifiers for feature layer fusion, and three sets of raw evidence are obtained. These three sets of evidence are then fused at the decision layer to obtain the final set of evidence. The accuracy of these four sets of evidence is compared as shown in Table 2, and with reference to Tables 3 and 4, the results show that the fusion results of the multi-sensor information fusion method involved in this application have a high accuracy rate, eliminate redundancy and contradictions between data, and ultimately obtain more reliable decision results.

[0127] Figure 4This is a schematic diagram of a multi-sensor information fusion device provided by an exemplary embodiment of the present application. Figure 4 As shown, the present application provides a multi-sensor information fusion device comprising:

[0128] A data preprocessing module is used to process the sensor layer data using a spatiotemporal preprocessing algorithm based on production rules to obtain preprocessed data; wherein the sensor layer data is obtained through multiple sensors;

[0129] A feature fusion module uses a neural network multi-classifier to perform feature layer fusion on the preprocessed data to obtain feature fusion data;

[0130] A decision layer fusion module, which performs decision layer fusion on the feature fusion data using a support correction iterative fusion method to obtain decision layer data;

[0131] The decision-making layer data uploading module uploads the decision-making layer data for decision-making.

[0132] The device provided in the embodiment of the present application can be specifically used to perform the above Figure 1 The solutions, specific functions and technical effects that can be achieved by the corresponding method embodiments will not be described in detail here.

[0133] An embodiment of the present invention further provides a non-volatile storage device comprising: a processor, and a memory communicatively connected to the processor;

[0134] Memory stores computer-executable instructions;

[0135] The processor executes the computer-executable instructions stored in the memory to implement the solution provided by any of the above method embodiments, and the specific functions and technical effects that can be achieved are not described in detail here. The electronic device can be the server mentioned above.

[0136] An embodiment of the present application also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by a processor, they are used to implement the solution provided by any of the above-mentioned method embodiments. The specific functions and technical effects that can be achieved are not repeated here.

[0137] An embodiment of the present application also provides a computer program product, which includes: a computer program, which is stored in a readable storage medium. At least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the solution provided by any of the above method embodiments. The specific functions and technical effects that can be achieved are not repeated here.

[0138] The application scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field can know that with the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0139] Those skilled in the art will appreciate that various aspects of the present application may be implemented as systems, methods, or program products. Therefore, various aspects of the present application may be implemented in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits," "modules," or "systems."

[0140] In some possible implementations, an electronic device according to the present application may include at least one processor and at least one memory. The memory stores program code that, when executed by the processor, causes the processor to perform the operational data management methods according to various exemplary embodiments of the present application described above. For example, the processor may perform the steps described in the operational data management method.

[0141] It should be noted that although several units or subunits of the device are mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, depending on the embodiment of the application, the features and functions of two or more units described above can be embodied in a single unit. Conversely, the features and functions of a single unit described above can be further divided and embodied by multiple units.

[0142] Furthermore, although the operations of the method of the present application are described in a particular order in the accompanying drawings, this does not require or imply that the operations must be performed in this particular order, or that all illustrated operations must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0143] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0144] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable image scaling device to produce a machine, so that the instructions executed by the processor of the computer or other programmable image scaling device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0145] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable image scaling device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, the instruction device being implemented in the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0146] These computer program instructions may also be loaded onto a computer or other programmable image scaling device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide for implementing the process described in the flow. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0147] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0148] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A multi-sensor information fusion method, characterized in that: include: Processing the sensor layer data using a spatiotemporal preprocessing algorithm based on production rules to obtain preprocessed data; wherein the sensor layer data is obtained through multiple sensors; Using a neural network multi-classifier to perform feature layer fusion on the preprocessed data to obtain feature fusion data; Using the support correction iterative fusion method to perform decision layer fusion on the feature fusion data to obtain decision layer data; Uploading the decision-making layer data for decision-making; The processing of the sensor layer data by a spatiotemporal preprocessing algorithm based on production rules to obtain preprocessed data includes: If the sensor layer data is data collected by a single sensor in the time dimension, the interquartile range algorithm and the Kalman filter algorithm are used for processing to remove outliers and reduce noise between the data; If the sensor layer data is heterogeneous data collected by different sensors in the spatiotemporal dimension, the least squares rule algorithm of time alignment is used to process the data collected by sensors with different acquisition frequencies to adapt them; If the sensor layer data is data collected by a general sensor, the interquartile range algorithm is used to detect abnormal values.

2. The multi-sensor information fusion method according to claim 1, wherein: The method of performing feature layer fusion on the pre-processed data using a neural network multi-classifier to obtain feature fusion data includes: The pre-processed data is input into a BP back propagation neural network, an RBF radial basis function neural network and an ELM extreme learning machine neural network for training.

3. The multi-sensor information fusion method according to claim 1, wherein: The feature fusion data is subjected to a decision layer analysis using a support correction iterative fusion method to obtain decision layer data, including: The support modified iterative fusion method is used to fuse the feature fusion data decision layer into the DS evidence theory combination rule to obtain the decision layer data.

4. The multi-sensor information fusion method according to claim 3, wherein: The support modified iterative fusion method is used to fuse the feature fusion data decision layer into the DS evidence theory combination rule to obtain the decision layer data, including: The feature fusion data of each node is used to perform evidence correction using variance, and the DS theory is used to fuse each evidence data to obtain n groups of initial evidence; fusing the initial evidence to obtain revised evidence, and using the revised evidence as initial reference evidence for the iterative algorithm; Obtaining a discount factor for each piece of initial evidence by calculating the distance, angle cosine, and conflict amount between the n sets of initial evidence fusion results; The correction evidence is re-corrected using the discount factor, and then fused, and iterated continuously until the preset accuracy is reached to obtain the final fusion result.

5. A device for a multi-sensor information fusion method, characterized in that: include: A data preprocessing module is used to process the sensor layer data using a spatiotemporal preprocessing algorithm based on production rules to obtain preprocessed data; wherein the sensor layer data is obtained through multiple sensors; A feature fusion module uses a neural network multi-classifier to perform feature layer fusion on the preprocessed data to obtain feature fusion data; The decision layer fusion module uses the support correction iterative fusion method to perform decision layer fusion on the feature fusion data to obtain decision layer data; A decision-making layer data uploading module uploads the decision-making layer data for decision-making; The processing of the sensor layer data by a spatiotemporal preprocessing algorithm based on production rules to obtain preprocessed data includes: If the sensor layer data is data collected by a single sensor in the time dimension, the interquartile range algorithm and the Kalman filter algorithm are used for processing to remove outliers and reduce noise between the data; If the sensor layer data is heterogeneous data collected by different sensors in the spatiotemporal dimension, the least squares rule algorithm of time alignment is used to process the data collected by sensors with different acquisition frequencies to adapt them; If the sensor layer data is data collected by a general sensor, the interquartile range algorithm is used to detect abnormal values.

6. A non-volatile storage device, characterized in that: include: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 4 when executed by a processor.

Images