A method for generating weights of multiple sensor fusion for target recognition

Abstract

An embodiment of the present invention discloses a method for generating multiple sensor fusion weights for target recognition, including: analyzing the factors affecting the detection and recognition performance of various sensors in a multi-sensor target fusion recognition system, establishing a detection and recognition capability table and a detection and recognition capability model for each sensor under each influencing factor, and obtaining the detection and recognition capability confidence of each sensor; mapping the detection and recognition weight level of each sensor based on the detection and recognition capability confidence, and obtaining training sample data for a weight level prediction network; establishing a weight level prediction network corresponding to each sensor, using the training sample data to perform network training to obtain network parameters, and achieving effective prediction of the fusion weights of each sensor under the combined effects of various influencing factors. The technical solution provided by the embodiment of the present invention solves the problem that existing target recognition systems deviate from actual results due to relying solely on a single sensor to collect target information.

Description

Technical Field

[0001] The present invention relates to, but is not limited to, the field of target recognition technology, and in particular to a method for generating weights of multiple sensor fusions for target recognition. Background Art

[0002] With the advancement of modern science and technology and production levels, automation and intelligent technology are increasingly being applied in a growing number of fields, such as industry, scientific research, and medicine. In practical applications, the primary issue for automated system operation is the identification of the object being manipulated, specifically the use of sensors to achieve target recognition. Therefore, research on target recognition technology is of great practical significance.

[0003] Target recognition uses technical processing to analyze target characteristics to determine their qualitative or quantitative properties. Due to the complex and diverse operating environments of existing target recognition systems, the varying recognition accuracy of different sensors, and the overlap in measurement data between target types, relying solely on a single sensor to gather target information often results in deviations from actual results. Summary of the Invention

[0004] Purpose of the present invention: An embodiment of the present invention provides a method for generating multiple sensor fusion weights for target recognition, so as to solve the problem that the existing target recognition system deviates from the actual result because it relies on only a single sensor to collect target information.

[0005] Technical solution of the present invention: An embodiment of the present invention provides a method for generating multiple sensor fusion weights for target recognition, comprising:

[0006] Analyze the factors affecting the detection and recognition performance of various sensors in the multi-sensor target fusion recognition system, establish a table and model of the detection and recognition capabilities of each sensor under various influencing factors, and obtain the confidence level of each sensor's detection and recognition capabilities; map the detection and recognition weight level of each sensor based on the detection and recognition confidence level, and obtain training sample data for the weight level prediction network;

[0007] A weight level prediction network supporting multiple sensor fusion weight levels is established, and the network parameters are obtained by network training using training sample data to achieve effective prediction of the fusion weights of each sensor under the combined effects of various influencing factors.

[0008] Optionally, the method for generating multiple sensor fusion weights for target recognition as described above includes the following steps:

[0009] Step 1: Identify the types of sensors involved in target recognition processing in the multi-sensor target fusion recognition system and identify the factors that affect the detection and recognition performance of each type of sensor;

[0010] Step 2: Based on the analysis results of the influencing factors of each sensor, a table of the detection and recognition capabilities of each sensor under the conditions of each influencing factor and a detection and recognition capability model of each sensor are established;

[0011] Step 3: Calculate the confidence level of each sensor's detection and recognition capability based on the detection and recognition capability table and detection and recognition capability model of each sensor;

[0012] Step 4: Establish a mapping relationship between the detection and recognition weight levels of each sensor based on the confidence level of the detection and recognition capabilities of each sensor, and obtain training sample data for the weight level prediction network;

[0013] Step 5: construct a weight level prediction network for each sensor, which is used to predict the detection and recognition weight of each sensor in the multi-sensor target fusion recognition system based on the weight level prediction network;

[0014] Step 6: Using the training sample data obtained in step 4, the weight level prediction network constructed in step 5 is trained to obtain network parameters that meet the actual expected output results.

[0015] Step 7: Map the weight level information of each sensor predicted in step 6 to normalized weight information.

[0016] Optionally, in the above-mentioned method for generating weights of multiple sensor fusion for target recognition,

[0017] The multi-sensor target fusion recognition system is a geostationary target recognition system, and the sensor types determined in step 1 include: millimeter wave radar sensor, lidar sensor and photoelectric sensor;

[0018] The factors that affect the detection and recognition performance of each sensor in the geostationary target recognition system include weather factors, terrain factors, target size and target distance factors.

[0019] Optionally, in the above-mentioned method for generating multiple sensor fusion weights for target recognition, step 2 includes:

[0020] Step 21: Based on the influence of weather factors and terrain factors on each sensor in the multi-sensor target fusion recognition system, a table of detection and recognition capabilities of each sensor under different weather factors and terrain factors is established;

[0021] Step 22: Establish sensor detection and recognition capability models for different target sizes and different target distances:

[0022]

[0023]

[0024] Among them, f S (x) represents the sensor detection capability under different target sizes, a and b represent the minimum and maximum obstacle sizes, respectively, and M and N are coefficients;

[0025] f L (X) represents the sensor detection capability at different target distances, A and B represent the closest and farthest detection distances of different sensors, respectively. P and Q are coefficients, and when the value is A, f L (X)=10, when the value is B, f L (X)=1.

[0026] Optionally, in the above-mentioned method for generating multiple sensor fusion weights for target recognition, step 3 includes:

[0027] Step 31: Express the detection and recognition capabilities of each sensor under different weather conditions as:

[0028] F={F 11 ,…,F ij}, where i represents the i-th weather type and j represents the j-th type of sensor;

[0029] Step 32: Express the detection and recognition capabilities of each sensor under different terrain factors as follows:

[0030] E={E 11 ,…,E kj}, where k represents the kth terrain type and j represents the jth type of sensor;

[0031] Step 33: Define the confidence level of the detection and recognition capability of the jth type of sensor under different environments as:

[0032] P ikj =K(F ij ×E kj ×f S (x)×f L (X));

[0033] Where K is the normalization coefficient.

[0034] Optionally, in the above-mentioned method for generating multiple sensor fusion weights for target recognition, step 4 includes:

[0035] Step 41: for the target type set Z = {Z1, Z2, ..., Z n ,…,Z N}, the number of target types in Z is C=N, n=1,2,…,N, and the jth sensor is sensitive to any sample Z nThe prediction results are expressed as:

[0036]

[0037] Among them, c jn Indicates that the actual target type of the jth sensor is Z n The prediction results of the sample, and the prediction confidence is P ikj ;c jm (m≠n) represents the remaining target types except the predicted target type of the j-th sensor;

[0038] Step 42, for target type Z n For samples of , the entropy distance of the jth sensor is:

[0039]

[0040] Step 43: Calculate the sensor entropy distance corresponding to each sample in the target type set and determine the maximum entropy distance E max and minimum entropy distance E min , the entropy distance of the j-th type sensor corresponding to N samples is mapped to the sensor weight level, and the weight level mapping relationship is expressed as:

[0041]

[0042] Optionally, in the above-mentioned method for generating multiple sensor fusion weights for target recognition, step 5 includes:

[0043] A fully connected neural network is used to construct corresponding weight level prediction networks for various types of sensors;

[0044] Among them, each influencing factor in the training sample data is used as the training sample input of each weight level prediction network, and the weight level of each sensor corresponding to each group of training sample inputs is used as the reference label of the corresponding weight level prediction network output; in the weight level prediction network of each type of sensor, the first layer is the input layer, which is used to receive the feature vector of the influencing factor information, and the last layer is the output layer, and the dimension of the output is the same as the number of weight levels.

[0045] Optionally, in the above-mentioned method for generating multiple sensor fusion weights for target recognition, step 6 includes:

[0046] Step 61, performing network training on the weight level prediction network using training sample data, including forward information propagation and backward error propagation;

[0047] (1) In the process of forward information propagation, the intermediate input I l and intermediate output O lTo transmit information;

[0048] Among them, by setting the net input of the lth layer to I l , then I l Expressed as:

[0049] I l =w l-1 O l-1 +b l-1 ;

[0050] The network parameters include connection weights w and biases b, w l-1 is the connection weight matrix from the previous layer network l-1 to the l layer, O l-1 is the output matrix of the previous layer of network, b l-1 Corresponding to the bias from the previous layer l-1 to the lth layer; and output O l Expressed as:

[0051]

[0052] (2) During the backward error propagation process, the network parameters are adjusted by calculating the network loss between the output layer and the expected value so that the output result meets the actual expectation; the network loss is expressed as:

[0053] L=αL CE +βL reg ;

[0054] Among them, L CE represents the cross entropy loss between the sensor's predicted output label for one of the samples and the reference label corresponding to the sample;

[0055] L reg Represents the result of regularizing the output of the weighted level prediction network;

[0056] α and β are weight coefficients for balancing cross entropy loss and regularization.

[0057] Step 62, using the gradient descent method to find the adjustment path of the network parameters is:

[0058]

[0059]

[0060] Where s is the current number of iterations, η is the step size, that is, the learning rate;

[0061] Step 63, set the connection weight connecting the pth neural unit in the l-1th layer and the qth neural unit in the lth layer to be expressed as Then the gradient matrix L is used to weight each connection The partial derivative of The adjustment path of network parameters is expressed as:

[0062]

[0063]

[0064] In step 64, for each sample submitted to the weight level prediction network, all connection weights and all bias values are updated once using the network parameter adjustment path in step 63 until the error values of all samples are less than the preset threshold, completing the network training and obtaining network parameters that meet the network training requirements.

[0065] Optionally, in the above-mentioned method for generating weights of multiple sensor fusion for target recognition, in step 7,

[0066] Step 71: Divide the sensor's weight level into five different levels: very high, high, medium, low, and very low. The weight level is expressed as:

[0067] D = {D1, D2, D3, D4, D5} = {very high, high, medium, low, very low};

[0068] Step 72: Map the weight level D to different weight information. The mapping relationship is:

[0069]

[0070] Among them, d1 <d2<d3<d4<d5,d5=1;

[0071] Step 73 : According to the weight level output by each sensor in real time, the weight information of each different sensor is obtained from the mapping relationship.

[0072] Optionally, in the above-mentioned method for generating weights of multiple sensor fusion for target recognition, before step 7, the method further includes:

[0073] Step 8: Expand and update the weight level prediction network. The expansion and update includes:

[0074] The actual detection and identification data of each sensor is processed to obtain more training sample data, so as to further train the weight level prediction network of each sensor to obtain more accurate network parameters.

[0075] Beneficial Effects of the Invention: The embodiments of the present invention provide a method for generating multiple sensor fusion weights for target recognition. By analyzing the factors influencing the detection and recognition performance of various sensors in a multi-sensor target fusion recognition system, a table of detection and recognition capabilities of each sensor under various influencing factors and a detection and recognition capability model are established, and the detection and recognition capability confidence of each sensor is obtained. Based on the detection and recognition capability confidence, the detection and recognition weight levels of each sensor are mapped and processed, and training sample data for a weight level prediction network is obtained. A weight level prediction network supporting multiple sensor fusion weight levels is established, and network parameters are obtained through network training using the training sample data, thereby effectively predicting the fusion weights of each sensor under the combined effects of various influencing factors. Compared with the prior art, the technical solution provided by the embodiments of the present invention has the following beneficial effects: it comprehensively considers the impact of various influencing factors on the detection and recognition capabilities of various types of sensors. When performing multi-sensor target recognition information fusion processing, the recognition results of each sensor are assigned corresponding weights based on their actual detection and recognition capabilities, thereby reducing the negative impact of sensors with poor recognition capabilities on the fusion recognition results, improving the accuracy of fusion recognition, and having strong engineering feasibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present invention and do not constitute a limitation on the technical solution of the present invention.

[0077] Figure 1 A flowchart of a method for generating multiple sensor fusion weights for target recognition provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0078] To make the purpose, technical solutions and advantages of the present invention more clearly understood, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that, unless there is a conflict, the embodiments and features in the embodiments of the present application can be combined with each other in any manner.

[0079] As explained in the above background technology, existing target recognition systems have complex and diverse working environments, different sensors have different recognition accuracies, and the measurement data of different target types are overlapping. Relying on a single sensor to collect target information usually deviates from the actual results.

[0080] In response to the problems existing in the above-mentioned existing target recognition systems, an embodiment of the present invention provides a method for generating multiple sensor fusion weights for target recognition. On the one hand, it considers using multi-sensor measurements to identify and process targets from multiple target feature dimensions, and uses information fusion technology to fuse multi-sensor recognition information to achieve more accurate and comprehensive understanding of the target; on the other hand, considering that the detection and recognition performance of different categories of sensors in different working environments are different, the degree of trust in the detection and recognition results of different sensors should be different when performing fusion processing. Different categories of sensors need to be configured with different weights when performing target recognition information fusion processing.

[0081] The embodiment of the present invention analyzes the recognition performance of different types of sensors under different working environments, establishes a detection and recognition capability table of each sensor under various influencing factors and a detection and recognition capability model of each sensor, calculates the confidence level of the detection and recognition capability of each sensor, and thus establishes a weight level mapping relationship of sensor detection data; and realizes sensor fusion weight generation based on a fully connected neural network, thereby reducing the influence of sensors with poor recognition performance in different working environments on the target fusion recognition processing results, and improving the accuracy of target recognition.

[0082] The present invention provides the following specific embodiments that can be combined with each other. The same or similar concepts or processes may not be described in detail in some embodiments.

[0083] An embodiment of the present invention provides a method for generating weights for fusion of multiple sensors for target recognition. The following first describes the design concept of the method for generating weights for fusion of multiple sensors for target recognition provided by an embodiment of the present invention:

[0084] In the field of target recognition, due to the complex operating environments of target recognition systems, relying solely on target recognition information provided by a single sensor can, in some cases, lead to discrepancies between the actual results. Therefore, to overcome the inherent limitations of using only a single sensor for target recognition, embodiments of the present invention consider configuring multiple sensor types within the target recognition system, thereby utilizing these multiple sensor types to fuse target recognition results and improve target recognition accuracy. Currently, target recognition systems configured with multiple sensor types typically employ fixed weightings.

[0085] However, in a multi-sensor target fusion recognition system, the working principles of each sensor are different, and under different working conditions, the target recognition performance of each sensor is also different. If the target recognition data of each sensor is not treated differently and is directly fused, the poor sensor target recognition results will affect the multi-sensor fusion recognition results and even lead to incorrect conclusions. Therefore, in a system where multiple sensors participate in the target fusion recognition process (i.e., a multi-sensor target fusion recognition system), it is necessary to find an effective method to enable the multi-sensor target fusion recognition system to assign different weights to different sensors according to different working conditions, reduce the weight of sensors with poor recognition performance under certain working conditions in the fusion process, and reduce their impact on the final target recognition results of the multi-sensor target fusion recognition system, thereby effectively improving the target recognition accuracy.

[0086] Based on the above analysis of the target recognition system configured with multiple types of sensors and the differences in the working mechanism and target recognition performance of multiple sensors in the system, as well as the explanation that poor sensor target recognition results are an important reason affecting the multi-sensor fusion recognition results, it can be seen that: the technical problem to be solved by the embodiment of the present invention is: how to configure different weights for different sensors according to different working conditions, reduce the weight of sensors with poor recognition performance participating in the fusion processing of the multi-sensor target fusion recognition system under certain working conditions, reduce their influence on the final target recognition results of the multi-sensor target fusion recognition system, and improve the target recognition accuracy.

[0087] To solve the above technical problems, an embodiment of the present invention provides a method for generating weights of multiple sensor fusions for target recognition. The method for generating weights of multiple sensor fusions provided by an embodiment of the present invention includes:

[0088] Analyze the factors affecting the detection and recognition performance of various sensors in the multi-sensor target fusion recognition system, establish a table and model of the detection and recognition capabilities of each sensor under various influencing factors, and obtain the confidence level of each sensor's detection and recognition capabilities; map the detection and recognition weight level of each sensor based on the detection and recognition confidence level, and obtain training sample data for the weight level prediction network;

[0089] A weight level prediction network supporting multiple sensor fusion weight levels is established, and the network parameters are obtained by network training using training sample data to achieve effective prediction of the fusion weights of each sensor under the combined effects of various influencing factors.

[0090] like Figure 1 FIG. 1 is a flow chart of a method for generating weights of multiple sensor fusions for target recognition provided by an embodiment of the present invention. The method for generating weights of multiple sensor fusions mainly includes the following steps:

[0091] Step 1: Identify the types of sensors involved in target recognition processing in the multi-sensor target fusion recognition system and identify the factors that affect the detection and recognition performance of each type of sensor;

[0092] Step 2: Based on the analysis results of the influencing factors of each sensor, a table of the detection and recognition capabilities of each sensor under the conditions of each influencing factor and a detection and recognition capability model of each sensor are established;

[0093] Step 3: Calculate the confidence level of each sensor's detection and recognition capability based on the detection and recognition capability table and detection and recognition capability model of each sensor;

[0094] Step 4: Establish a mapping relationship between the detection and recognition weight levels of each sensor based on the confidence level of the detection and recognition capabilities of each sensor, and obtain training sample data for the weight level prediction network;

[0095] Step 5: construct a weight level prediction network for each sensor, which is used to predict the detection and recognition weight of each sensor in the multi-sensor target fusion recognition system based on the weight level prediction network;

[0096] Step 6: Using the training sample data obtained in step 4, the weight level prediction network constructed in step 5 is trained to obtain network parameters that meet the actual expected output results.

[0097] Step 7: Map the weight level information of each sensor predicted in step 6 to normalized weight information.

[0098] The method for generating weights for multiple sensor fusion for target recognition provided by an embodiment of the present invention mainly includes two parts:

[0099] (1) First, the types of sensors involved in the fusion and recognition processing in the multi-sensor target fusion and recognition system are identified, and the factors affecting the detection and recognition performance of various sensors in the multi-sensor target fusion and recognition system are analyzed. Based on the analysis results, a table of the detection and recognition capabilities of each sensor under each influencing factor and a detection and recognition capability model are established. Based on the table of the detection and recognition capabilities of each sensor and the detection and recognition capability model, the confidence level of the detection and recognition capability of each sensor is obtained. Finally, based on the detection and recognition capability confidence level, a mapping relationship is established for the detection and recognition weight level of each sensor, and the training sample data for the weight level prediction network is obtained.

[0100] (2) A weight level prediction network that supports multiple sensor fusion weight levels is established, and the generated weight level prediction network is trained based on the training sample data of the weight level prediction network; in addition, the actual detection and recognition data of each sensor is processed to obtain more training sample data, and then the weight level prediction network of multiple sensors is further trained to obtain more accurate network parameters and improve the network's weight level prediction ability; finally, the fusion weight level of each sensor is normalized and mapped to obtain the final multiple sensor fusion weight.

[0101] The embodiment of the present invention provides a method for generating multiple sensor fusion weights for target recognition. By analyzing the factors influencing the detection and recognition performance of various sensors in a multi-sensor target fusion recognition system, a table of detection and recognition capabilities of each sensor under various influencing factors and a detection and recognition capability model are established, and the detection and recognition capability confidence of each sensor is obtained. Based on the detection and recognition capability confidence, the detection and recognition weight levels of each sensor are mapped and processed, and training sample data for a weight level prediction network is obtained. A weight level prediction network supporting multiple sensor fusion weight levels is established, and network parameters are obtained through network training using the training sample data. This effectively predicts the fusion weights of each sensor under the combined effects of various influencing factors. Compared with the prior art, the technical solution provided by the embodiment of the present invention has the following beneficial effects: it comprehensively considers the impact of various influencing factors on the detection and recognition capabilities of various types of sensors. When performing multi-sensor target recognition information fusion processing, the recognition results of each sensor are assigned corresponding weights based on their actual detection and recognition capabilities, thereby reducing the negative impact of sensors with poor recognition capabilities on the fusion recognition results, improving the accuracy of fusion recognition, and having strong engineering feasibility.

[0102] The following uses a specific implementation example to illustrate the specific implementation of the method for generating weights for multiple sensor fusion for target recognition provided by an embodiment of the present invention. This implementation example is implemented based on the technical solution of the present invention and provides a detailed implementation method and specific operation process. However, the scope of protection of the present invention is not limited to the following implementation example.

[0103] Implementation Example

[0104] This example embodiment proposes a method for calculating the fusion weights of multiple sensor detection and recognition information during target recognition. The method primarily includes two steps: establishing a sensor detection and recognition weight level mapping based on a sensor detection and recognition capability model and constructing training sample data based on this mapping; and establishing a neural network to predict the sensor fusion weight level. Specifically, this example embodiment includes the following steps:

[0105] Step 1: First, confirm the types of sensors involved in target recognition processing in the multi-sensor target fusion recognition system, and confirm the various influencing factors that affect the detection and recognition performance of each type of sensor.

[0106] In this implementation example, a geostationary target recognition system is used as an example. In this step, millimeter-wave radar sensors, lidar sensors, and photoelectric sensors are generally used in geostationary target recognition systems to detect and identify ground stationary targets. The first two are active sensors, and the third is a passive sensor. Their working mechanisms are different. Weather factors, terrain factors, target size, and target distance all affect the detection and recognition performance of each sensor to varying degrees. The following details the working principles of the above three types of sensors and the effects of weather factors, terrain factors, target size, and target distance on each sensor:

[0107] 1) Millimeter-wave radar sensor

[0108] Millimeter-wave radar sensors operate in the millimeter-wave frequency band. Their operating principle is to transmit radio waves (radar waves), receive echoes, and measure the target's position based on the time difference between transmission and reception. The transmitted millimeter waves are a section of radio waves, ranging from 30 to 300 GHz (wavelength 1 to 10 mm). This range of wavelengths overlaps microwaves and far-infrared waves, thus combining characteristics of both spectrums. Factors such as weather, terrain, target size, and distance from the millimeter-wave radar sensor can all affect the detection and recognition performance of millimeter-wave radar sensors.

[0109] Weather factors: Millimeter waves propagate through the atmosphere and are affected by the atmosphere, special substances, and precipitation. Therefore, the impact of atmospheric conditions on millimeter wave propagation must be considered.

[0110] The main weather factors affecting millimeter-wave radar sensors include rain and hail. Rain, with its high dielectric constant, has the greatest impact on the radar. Millimeter-wave radar sensors cannot function effectively in moderate rain or above, and performance drops by approximately 30% in light to moderate rain. Hail and rain have roughly the same impact on sensor recognition, but other weather conditions are unaffected, ensuring all-weather operation.

[0111] Terrain factors: The detection and recognition performance of millimeter-wave radar sensors will also be affected by the background interference near the target. The degree of interference varies with different terrain features. The recognition performance is best on plains, followed by plateaus, mountains, and hills. The recognition performance of millimeter-wave radar sensors is poor under desert terrain conditions.

[0112] Target size and target distance: The larger the target size and the closer the distance, the better the detection and recognition performance of the millimeter-wave radar sensor.

[0113] 2) LiDAR sensor

[0114] A lidar sensor, short for laser detection and ranging system, refers to a radar system operating in the infrared to ultraviolet spectrum. The laser beam in a common mechanical lidar sensor is near-infrared light with a wavelength of approximately 900 nm. Its principle and construction are very similar to those of a laser rangefinder. Target detection technology based on lidar sensors is a non-contact, active measurement technology that applies laser ranging technology to rapidly measure large spatial areas. The function of a lidar sensor is to accurately measure a target's position (distance and angle), motion (speed, vibration, and attitude), and shape. Factors such as weather and terrain, target size, and distance from the lidar sensor can all affect the detection and recognition performance of a lidar sensor.

[0115] Weather factors: Laser propagation through the atmosphere can be unstable due to the complexity of atmospheric composition and other factors, such as weather. The impact of dust storms is similar to that of smog. Since dust particles are larger than smog particles, dust storms have a more severe impact on lidar sensors than smog. Rainy weather, due to its higher dielectric constant, has the greatest impact on lidar sensors. LiDAR sensor recognition performance in different weather conditions is as follows: snow > hail > smog > dust storm > rain.

[0116] Terrain: LiDAR sensor detection and recognition performance is affected by the color and surface type (matt, glossy, smooth, rough, etc.) of an object. Light-colored objects reflect more light than darker ones. Deciduous trees, rubble, and dry sand have a higher reflectivity to lasers. Therefore, LiDAR sensor detection is better in flat areas such as plains and deserts than in hilly and mountainous areas.

[0117] Target size and target distance: The larger the target size and the closer the distance, the better the lidar sensor's detection and recognition performance.

[0118] 3) Photoelectric sensor

[0119] Photoelectric sensors utilize an infrared detector, an optical imaging lens, and an optomechanical scanning system to receive the infrared radiation energy distribution pattern of the target being measured, which is then reflected onto the infrared detector's photosensitive element. Between the optical system and the infrared detector, an optomechanical scanning mechanism scans the infrared thermal image of the object being measured and focuses it onto a unit or spectroscopic detector. The detector converts the infrared radiation energy into an electrical signal, which is then amplified, processed, converted, or displayed as a standard video signal on a television screen or monitor. Factors such as weather, terrain, target size, and distance from the millimeter-wave radar sensor can all affect the detection and recognition performance of photoelectric sensors.

[0120] Weather factors: The impact of weather factors on infrared imaging is, on the one hand, the scattering and absorption of infrared radiation by floating particles and some gases in the air; on the other hand, strong winds, rain, snow and other weather conditions cause the loss of surface temperature of the measured target and thus affect infrared radiation.

[0121] Terrain factors: Photoelectric sensors rely on visible light and infrared detection to detect targets, so they are more adaptable to terrain and generally have little impact. However, when there is terrain obstruction, it will affect the detection of the target.

[0122] Target size and target distance: The larger the target size and the closer the distance, the better the detection and recognition performance of the photoelectric sensor.

[0123] Step 2: Based on the analysis results of the influencing factors of each sensor, a table of the detection and recognition capabilities of each sensor under the conditions of each influencing factor and a detection and recognition capability model of each sensor are established.

[0124] In this step, based on the analysis of factors influencing the detection and recognition performance of millimeter-wave radar, lidar, and infrared sensors, a table and model of their detection and recognition capabilities were developed under different weather, terrain, target size, and distance conditions. The following two tables show the detection and recognition capabilities of each sensor under different weather and terrain conditions, respectively. In the table, "0" indicates that the sensor cannot recognize the target, while "1-10" indicates the sensor's recognition ability, with higher values indicating better recognition capability.

[0125] Table 1 Detection and recognition capabilities of each sensor under different weather factors

[0126]

[0127]

[0128] Table 2 Detection and recognition capabilities of each sensor under different terrain factors

[0129]

[0130] Since the target size (e.g., the width and height of the target) and the distance between the target and each sensor also affect the sensor's detection and recognition capabilities, in this step, the sensor detection and recognition capability model for different target sizes and different target distances is established as follows:

[0131]

[0132]

[0133] In the above formulas (1) and (2), f S (x) and f LThe value range of (X) is 0 to 10, where 0 means no detection and a larger value means better detection capability. S (x) represents different target sizes (width*height, unit: m) 2 ) sensor detection capability under the condition of the obstacle. The larger the size, the better the detection effect. The parameters M, N, a, and b can be set according to different types of sensors. Among them, a and b represent the minimum and maximum values of the obstacle size, respectively. For example, a and b are set to 6 and 532. M and N are coefficients and satisfy f S (6) = 1, f S (532) = 10. f L (X) represents the sensor detection capability at different target distances (in meters). The closer the distance within a certain range, the better the detection effect. Among them, the parameters P, Q, A, and B can be set according to different types of sensors. A and B represent the closest and farthest detection distances of different sensors, respectively. P and Q are coefficients, and when the value is A, f L (X)=10, when the value is B, f L (X)=1.

[0134] Specifically, the P, Q, A, and B of the millimeter-wave radar sensor are 9 / 13369, 10.7, 400, and 4000 respectively; the P, Q, A, and B of the lidar sensor are 9 / 6142, 10.7, 200, and 2000 respectively; the P, Q, A, and B of the low-light-level night vision sensor are 9 / 1657, 10.1, 10, 600 respectively; and the P, Q, A, and B of the infrared sensor are 9 / 10231, 10.2, 100, and 3000 respectively.

[0135] Step 3: Calculate the confidence level of each sensor's detection and recognition capability based on the detection and recognition capability table and detection and recognition capability model of each sensor.

[0136] In this step, the detection and recognition capabilities of various types of sensors under different weather conditions are expressed as: F = {F 11 ,…,F ij}, where i = 1, 2, ..., 12 represents 12 weather types, and j = 1, 2, 3 represents 3 different types of sensors. In addition, the detection and recognition capabilities of each sensor under different terrain factors are expressed as: E = {E 11 ,…,E kj}, where k = 1, 2, ..., 6 represents 6 terrain types and j = 1, 2, 3 represents 3 different sensors. Based on the above definition, the confidence level of the detection and recognition capability of the jth type of sensor under different environments can be defined as:

[0137] P ikj =K(F ij ×E kj ×f S(x)×f L (X)); (3)

[0138] Among them, K is the normalization coefficient, the purpose of which is to obtain the normalized sensor confidence.

[0139] Step 4: Establish a mapping relationship between the detection and recognition weight levels of each sensor based on the confidence level of the detection and recognition capabilities of each sensor, and obtain training sample data for the weight level prediction network. The training sample data includes input and output data of the weight level prediction network, such as weather, terrain, target size and target distance as input, and weight levels corresponding to various combinations of influencing factors as output. This implementation example takes 5 weight levels as an example for illustration.

[0140] In this step, first, let Z = {Z1, Z2, ..., Z n ,…,Z N} represents different types of targets to be identified, the number of target types in Z is C=N, n=1,2,…,N. According to step 3, under certain conditions, the prediction results of the three types of sensors are [c jn :P ikj ], where c jn Indicates that the actual target type of the jth sensor is Z n The prediction results of the sample, and the prediction confidence given is P ikj In addition, the remaining confidence (ie 1-P ikj ) are evenly distributed to other target types, then the jth sensor will n The prediction results can be expressed as:

[0141] where c jm (m≠n) represents the remaining target types except the predicted target type of the j-th sensor. Assume that sample Z n The true weight level label is c true , then for the sample Z n , the entropy distance of the j-th sensor can be expressed as follows:

[0142]

[0143] It can be seen that when the target type predicted by the sensor is the same as the true target type, the entropy distance is negatively correlated with the prediction confidence. The higher the confidence, the smaller the entropy distance, and the greater the weight assigned to the sensor. When the target type predicted by the sensor is different from the true target type, the entropy distance is positively correlated with the prediction confidence. The higher the confidence of the wrong target type, the larger the entropy distance, indicating that the weight assigned to the sensor needs to be reduced. According to the above rules, the entropy distance to sensor weight mapping can adopt a linear mapping method, that is, for all available training samples, the sensor entropy distance corresponding to each sample is calculated, that is, N entropy distances are obtained, and the maximum entropy distance E is determined. max and minimum entropy distance E min , then, the entropy distance of the j-th type sensor corresponding to N samples is mapped to the sensor weight level, and the weight level mapping relationship can be expressed as:

[0144]

[0145] Using this method, the constructed data can be labeled with sensor weight levels, thereby constructing training sample data that can be used for the weight level prediction network. After obtaining the training sample data, in the subsequent step of network training, weather, terrain, target size, distance information under different conditions and corresponding sensor recognition weight level data can be input into the weight level prediction network to achieve network training.

[0146] Step 5: Construct a weight level prediction network for each sensor;

[0147] In this step, the purpose of constructing the weight level prediction network is to realize the prediction of the detection and recognition weights of each sensor in the geostationary target recognition system based on the weight level prediction network.

[0148] In this step, the weight level of each sensor is predicted and generated by a method based on a fully connected neural network. A fully connected neural network is a multi-layer feedforward neural network, which is mainly composed of an input layer, an output layer and several hidden layers. Among them, the input layer is composed of several neuron nodes, which correspond to the vector dimension of the input sample. The number of nodes in the output layer also corresponds to the number of outputs expected by the user. The hidden layer is mainly used for feature extraction and analysis of the input vector, and the output layer mainly outputs the network's analysis results of the input sample. The main features of a fully connected neural network are forward signal transmission and error back propagation. In the process of forward signal propagation, the input signal is processed from the input layer through the hidden layer to the output layer, and the neuron state of each layer only affects the neuron state of the next layer. If the output layer does not get the expected output, it turns to error back propagation, and adjusts the network weights and thresholds according to the prediction error, so that the predicted output of the fully connected neural network continues to approach the expected output.

[0149] It should be noted that different sensors have different emphases on scene factors, such as some are more affected by weather factors, while others need to focus on factors such as terrain. Therefore, the specific implementation method of this step is to use three fully connected neural networks to construct weight level prediction networks for the three types of sensors according to the number of sensors. After constructing the weight level prediction network of each sensor, the influencing factor information of the external environment in the training sample data (for example, including: weather, terrain, target size and target distance) can be used as the training sample input of each weight level prediction network, and the weight levels corresponding to the three sensors are used as reference labels for the output of the corresponding weight level prediction network.

[0150] The weight level prediction network in this implementation example is composed of a cascade of five fully connected layers (including an input layer, 3 hidden layers and an output layer), where the first layer is the input layer, which is used to receive the feature vector of influencing factor information, and the last layer is the output layer. The dimension of the output is the same as the number of weight levels, and the weight levels are divided into five different levels: very high, high, medium, low, and very low.

[0151] Step 6: In order to achieve effective prediction of the weight level of each sensor, it is necessary to train the weight level prediction network to obtain the network parameters.

[0152] Consider the input and output of layer l in the weighted level prediction network, assuming that the net input to layer l is I l , then I l It can be expressed as:

[0153] I l =w l-1 O l-1 +b l-1 ;

[0154] Among them, the network parameters include connection weight w and bias b, w l-1 is the connection weight matrix from the previous layer network l-1 to the l layer, O l-1 is the output matrix of the previous layer of network, b l-1 Corresponding to the bias from the previous layer of network l-1 to the lth layer, this bias will be used as a threshold to determine the activation characteristics of the neural unit.

[0155] Given the net input I of layer l l , the output O of the lth layer l It can be calculated by activation function. Here we use Sigmoid activation function, and the output is O l It can be expressed as:

[0156]

[0157] I l and O lIt is the intermediate input and intermediate output of network training.

[0158] The above network training is a progressive forward calculation, where the input feature vector passes through the neural units of each layer and finally obtains the calculation result at the output layer. In order to make the output result meet the actual expectations, it is necessary to continuously adjust the connection weights w between the neural units in each layer. l- 1 and bias b l-1 , the connection weights and biases are adjusted using backward error propagation.

[0159] The basic idea of backward error propagation is to adjust network parameters by calculating the network loss between the output layer and the expected value, thereby reducing the network loss. In this implementation example, the network loss constructed in the project mainly considers the following two aspects:

[0160] On the one hand, consider the cross entropy loss between the predicted output label of each sensor for one of the samples and the reference label corresponding to the sample:

[0161]

[0162] Among them, M is the number of samples in a training batch, y(x i ) represents the sample x i The corresponding true label (one-hot encoding), P net (x i ; θ) represents the network response to sample x when the parameter is θ i The cross entropy loss is used to measure the closeness between the network prediction output and the true label. The closer the predicted label is to the true label, the smaller the loss value is, and the smaller the change in the network parameter θ is.

[0163] On the other hand, in order to prevent the network from i The prediction output is too much assigned to a certain weight level, using a unified prior distribution π (such as π c =1 / C) to regularize the output of the weighted level prediction network:

[0164]

[0165] in, Indicates that the input sample is x i When , the weight level prediction network outputs the confidence probability at the cth weight level, and C represents the total number of weight levels.

[0166] Furthermore, the overall loss of the weighted level prediction network is:

[0167] L=αL CE +βL reg ;

[0168] Among them, α and β are the weight coefficients for balancing the two parts of loss.

[0169] In the specific implementation, considering the network's predicted output P net (x i ; θ) is calculated step by step through the weight parameters between each layer of the network. The weight parameters will be used as variables to be optimized, so the gradient descent method can be used to find the best path for adjusting the network parameters:

[0170]

[0171]

[0172] Where s is the current iteration number, η is the step size, that is, the learning rate. If we assume that the connection weight connecting the pth neural unit in the l-1th layer and the qth neural unit in the lth layer is expressed as Then the gradient matrix L is used to weight each connection The partial derivative of Composition, the above formula can be rewritten as:

[0173]

[0174]

[0175] For each sample submitted to the weighted level prediction network, all connection weights and all bias values are updated once using the above formula until the error values of all samples are less than a preset threshold, at which point the training is completed.

[0176] Step 7: Expand and update the weight level prediction network.

[0177] This step seven is optional. In the above steps, after training with sensor characteristic simulation data, the weight level prediction network has the basic sensor weight prediction capability. However, in actual scenarios, due to the complex and changeable combination of factors such as weather, terrain, target size and target distance, the data simulated based on the sensor prior characteristics cannot accurately reflect the actual complex scenarios.

[0178] Based on the above analysis, it is necessary to expand and update the weight level prediction network to continuously use the real-time collected data to generate training sample sets, further enhance the network's fitting ability, and thus obtain a more confident weight level prediction. In the process of generating the training sample set of actual data, the most critical thing is to generate the weight level label corresponding to the sample. The entropy measurement method in the dataset construction is used to convert the sensor prediction results (prediction results are [c i :p i ]: where ci Indicates that the i-th sensor predicts the category of the sample, and the prediction confidence given is p i ) and the true result are mapped to the sensor weight level. This method can be used to label the actual collected data with sensor weight levels, thereby constructing training data that can be used for the weight level prediction network. Further parameter updates and training of the network can be performed to continuously improve the network's prediction ability.

[0179] Step 8: Sensor fusion weight level normalization mapping processing.

[0180] After the weight level prediction network predicts the sensor weight level information, it is necessary to map the weight level information to normalized weight information for use in the subsequent multi-sensor detection and identification information fusion process. The sensor weight level is divided into five different levels: very high, high, medium, low, and very low. For convenience, they are represented as:

[0181] D={D1,D2,D3,D4,D5}={very high, high, medium, low, very low}

[0182] The weight level D is mapped to different weight information, and the mapping relationship is: According to the real-time weight level output of the three sensors, the weight information of the three different sensors is obtained from the mapping relationship. For example, assuming that the current real-time weight level output of the three sensors is: D = {D1, D3, D5}, then the weight mapping of the three sensors and the final weight information are:

[0183] {0.2, 0.6, 1} / (0.2+0.6+1)={0.1, 0.3, 0.6}.

[0184] Although the embodiments disclosed herein are as described above, the contents are merely provided to facilitate understanding of the present invention and are not intended to limit the present invention. Any person skilled in the art may make any modifications and variations in the form and details of the embodiments without departing from the spirit and scope of the present invention. However, the scope of patent protection of the present invention shall remain subject to the scope defined by the appended claims.

Claims

1. A method for generating weights of multiple sensor fusion for target recognition, characterized in that: include: Analyze the factors affecting the detection and recognition performance of various sensors in the multi-sensor target fusion recognition system, establish the detection and recognition capability table and detection and recognition capability model of each sensor under various influencing factors, and obtain the confidence level of each sensor's detection and recognition capability; Mapping the detection and recognition weight levels of each sensor based on the confidence level of detection and recognition capabilities, and obtaining training sample data for the weight level prediction network; Establish a weight level prediction network that supports multiple sensor fusion weight levels, use training sample data to train the network to obtain network parameters, and achieve effective prediction of each sensor fusion weight under the combined effect of various influencing factors; The analysis of factors influencing the detection and recognition performance of various sensors in the multi-sensor target fusion recognition system and the establishment of a detection and recognition capability table and a detection and recognition capability model for each sensor under various influencing factors include: S21, based on the influence of weather factors and terrain factors on each sensor in the multi-sensor target fusion recognition system, establish a table of the detection and recognition capabilities of each sensor under different weather factors and different terrain factors; S22, establish the sensor detection and recognition capability models for different target sizes and different target distances: ； ； in, represents the sensor detection capability under different target sizes, a and b represent the minimum and maximum obstacle sizes respectively, and M and N are coefficients; Indicates the sensor detection capability at different target distances. A and B represent the closest and farthest detection distances of different sensors, respectively. P and Q are coefficients, and when the value is A, , when the value is B, .

2. The method for generating weights of multiple sensor fusion for target recognition according to claim 1, characterized in that: The following steps are involved: Step 1: Identify the types of sensors involved in target recognition processing in the multi-sensor target fusion recognition system and identify the factors that affect the detection and recognition performance of each type of sensor; Step 2: Based on the analysis results of the influencing factors of each sensor, a table of the detection and recognition capabilities of each sensor under the conditions of each influencing factor and a detection and recognition capability model of each sensor are established; Step 3: Calculate the confidence level of each sensor's detection and recognition capability based on the detection and recognition capability table and detection and recognition capability model of each sensor; Step 4: Establish a mapping relationship between the detection and recognition weight levels of each sensor based on the confidence level of the detection and recognition capabilities of each sensor, and obtain training sample data for the weight level prediction network; Step 5: construct a weight level prediction network for each sensor, which is used to predict the detection and recognition weight of each sensor in the multi-sensor target fusion recognition system based on the weight level prediction network; Step 6: Using the training sample data obtained in step 4, the weight level prediction network constructed in step 5 is trained to obtain network parameters that meet the actual expected output results. Step 7: Map the weight level information of each sensor predicted in step 6 to normalized weight information.

3. The method for generating weights of multiple sensor fusion for target recognition according to claim 2, characterized in that: The multi-sensor target fusion recognition system is a geostationary target recognition system, and the sensor types determined in step 1 include: millimeter wave radar sensor, lidar sensor and photoelectric sensor; The factors that affect the detection and recognition performance of each sensor in the geostationary target recognition system include weather factors, terrain factors, target size and target distance factors.

4. The method for generating weights of multiple sensor fusion for target recognition according to claim 3, characterized in that: The step 3 comprises: Step 31: Express the detection and recognition capabilities of each sensor under different weather conditions as: ,in, represents the i-th weather type, Indicates the j Types of sensors; Step 32: Express the detection and recognition capabilities of each sensor under different terrain factors as follows: ,in, represents the kth terrain type, Indicates the j Types of sensors; Step 33: Define the confidence level of the detection and recognition capability of the jth type of sensor under different environments as: ； Where K is the normalization coefficient.

5. The method for generating weights of multiple sensor fusion for target recognition according to claim 4, characterized in that: The step 4 comprises: Step 41: for the target type set There are different types of targets in Z, the number of target types C=N, n=1,2,…,N, j For any sample The prediction result is expressed as: ； in, Indicates the j The actual target type of the sensor is The prediction results of the sample, and the prediction confidence is ; Indicates the j The remaining target types of each sensor except the predicted target type; Step 42, for target type For samples of , the entropy distance of the jth sensor is: ； Step 43: Calculate the sensor entropy distance corresponding to each sample in the target type set and determine the maximum entropy distance and minimum entropy distance , the entropy distance of the j-th type sensor corresponding to N samples is mapped to the sensor weight level, and the weight level mapping relationship is expressed as: 。 6. The method for generating weights of multiple sensor fusion for target recognition according to claim 5, characterized in that: The step 5 comprises: A fully connected neural network is used to construct corresponding weight level prediction networks for various types of sensors; Among them, each influencing factor in the training sample data is used as the training sample input of each weight level prediction network, and the weight level of each sensor corresponding to each group of training sample inputs is used as the reference label of the corresponding weight level prediction network output; in the weight level prediction network of each type of sensor, the first layer is the input layer, which is used to receive the feature vector of the influencing factor information, and the last layer is the output layer, and the dimension of the output is the same as the number of weight levels.

7. The method for generating weights of multiple sensor fusion for target recognition according to claim 6, characterized in that: The step 6 comprises: Step 61, performing network training on the weight level prediction network using training sample data, including forward information propagation and backward error propagation; (1) During the forward information propagation process, the intermediate input in the network training and intermediate output To transmit information; Among them, by setting the The net input to the layer is ,but Expressed as: ； Network parameters include connection weights and bias , The upper layer network To the The connection weight matrix of the layer, is the output matrix of the previous layer network, Corresponding to the previous layer of network To The bias of the layer; and the output Expressed as: ； (2) During the backward error propagation process, the network parameters are adjusted by calculating the network loss between the output layer and the expected value so that the output result meets the actual expectation; the network loss is expressed as: ； in, represents the cross entropy loss between the sensor's predicted output label for one of the samples and the reference label corresponding to the sample; Represents the result of regularizing the output of the weighted level prediction network; and The weight coefficient for balancing cross entropy loss and regularization; Step 62, using the gradient descent method to find the adjustment path of the network parameters is: ； ； in, is the current iteration number, is the step size, also known as the learning rate; Step 63, set the connection l-1 Tier p The neural unit and l Layer q The connection weights of the neural units are expressed as , then the gradient matrix Depend on For each connection weight The partial derivative of The adjustment path of network parameters is expressed as: ； ； In step 64, for each sample submitted to the weight level prediction network, all connection weights and all bias values are updated once using the network parameter adjustment path in step 63 until the error values of all samples are less than the preset threshold, completing the network training and obtaining network parameters that meet the network training requirements.

8. The method for generating weights of multiple sensor fusion for target recognition according to claim 7, characterized in that: In the step 7, Step 71: Divide the sensor's weight level into five different levels: very high, high, medium, low, and very low. The weight level is expressed as: ； Step 72: Map the weight level D to different weight information. The mapping relationship is: ； in, ; Step 73 : According to the weight level output by each sensor in real time, the weight information of each different sensor is obtained from the mapping relationship.

9. The method for generating weights of multiple sensor fusion for target recognition according to any one of claims 2 to 8, characterized in that: After step 7, the method further includes: Step 8: Expand and update the weight level prediction network. The expansion and update includes: The actual detection and identification data of each sensor is processed to obtain more training sample data, so as to further train the weight level prediction network of each sensor to obtain more accurate network parameters.

Images