Multi-modal remote sensing data fusion method based on probability voting

Abstract

The invention provides a multi-modal remote sensing data fusion method based on probability voting, aiming at the target association fusion problem of dense platform formation in a multi-modal remote sensing means collaborative observation scene. The invention provides a multi-modal remote sensing data fusion method based on probability voting, aiming at solving the problems of fuzzy single-mode fusion and unstable association effect during target association fusion in dense platform formation and relatively stable formation configuration scenes. The objective of the invention is to realize multi-modal data automatic association based on position topology and identification topology by constructing position probability and identification probability histograms in a plurality of observation modes, and improve target fusion and identification accuracy. According to the method provided by the invention, on one hand, a position feature vector based on a position histogram is constructed based on a CRLB theory to realize target positioning deviation correction, and on the other hand, a recognition feature vector is constructed based on a single-mode means recognition model, and cross-modal association fusion of formation targets is realized through comprehensive matching of the position feature vector and the recognition feature vector.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of multi-modal remote sensing data processing, and relates to a multi-modal remote sensing data fusion method based on probability voting. BACKGROUND

[0002] With the increasingly mature construction of space-ground observation resources, the demand for data fusion is increasingly urgent. The traditional mainstream method mainly relies on single-mode means (such as optical images or electromagnetic signals) to independently process, complete individual identification, and then perform decision-level fusion, which has certain limitations. On the one hand, optical identification stops at the platform type for the same type of platform due to high-angle observation and resolution constraints; on the other hand, there is a multi-layer one-to-many relationship of "radiation source parameter-radiation source type-platform type-individual" in the electromagnetic signal identification process, which leads to a high degree of dependence on the adjacent situation for individual identification. Therefore, after independent processing by single-mode means, it is difficult to complete stable associated fusion relying on distance constraints. How to break through the bottleneck problem of associated fusion after independent processing by traditional single-mode means, fully exert the advantages of multi-modal remote sensing collaborative observation, realize target associated fusion in the scene of dense platform formation and relatively stable formation configuration, and improve the accuracy of target associated fusion and lay a foundation for subsequent high-precision identification is one of the key research directions. SUMMARY

[0003] The application proposes a multi-modal remote sensing data fusion method based on probability voting to solve the problems of single-mode means fusion ambiguity and unstable associated effect in target associated fusion in the scene of dense platform formation and relatively stable formation configuration. The method aims to realize multi-modal data automatic association based on position topology and identification topology by constructing position probability and identification probability histograms in multiple observation modes, and improve the accuracy of target fusion and identification.

[0004] The technical solution of the application is as follows: A multi-modal remote sensing data fusion method based on probability voting, comprising the following steps:

[0005] Step 1: According to the collected electromagnetic remote sensing and image remote sensing data, the azimuth observation γ yaw , the elevation observation γ pitch , the time difference of arrival TDOA, and the Doppler frequency FDOA are obtained, and the number of each target data group is required to be greater than 100; the observation station position information is collected to form an observation station position matrix (x s , y s , z s ), and the sampling period is not greater than 1 second, and the process proceeds to step 2.

[0006] Step 2: Based on the GDOP model and the CRLB error model, the azimuth observation γ yaw , the elevation observation γ pitch, observation station position matrix (x s ,y s ,z s ), construct a positioning observation equation based on the observation station position and the target observation position, combine the earth ellipse equation, calculate the positioning coverage range of optical remote sensing and electromagnetic remote sensing, and realize the grid-based statistics of the observation probability in the positioning coverage range based on the probability constraint, and form the corresponding position histogram in each grid, and go to step 3.

[0007] Step 3, based on the identification model, the probability of optical remote sensing and electromagnetic remote sensing platform type identification is determined, and each target forms an identification histogram for different platform types and their probabilities:

[0008] The identification model includes an optical remote sensing image texture identification model and an electromagnetic remote sensing identification model, the platform type identification and confidence of the target are estimated based on the optical remote sensing image texture identification model; the multi-layer probability tree of 'radiation source parameter-radiation source type-platform type' is constructed based on the electromagnetic remote sensing identification model, the probability of electromagnetic remote sensing platform type identification is determined, each target forms an identification histogram for different platform types and their probabilities, and goes to step 4.

[0009] Step 4, the identification histogram formed by each target for different platform types and their probabilities is used to construct an identification feature vector, in order to overcome the geometric deformation caused by the rotation of the target formation configuration process and the slight displacement of the target, the position feature vector of the observation target is constructed by three targets, the position feature vector angle is obtained, and goes to step 5.

[0010] Step 5, by comparing the optical remote sensing means and the electromagnetic remote sensing means, the position correlation of the target is determined by combining the containing relationship of the position feature vector angle of the target; by comparing the intersection relationship of the platform types of the target, the correlation of the target on the platform type identification result is determined, and the identification confidence of the associated target is given.

[0011] Compared with the prior art, the advantages of the present application are:

[0012] (1) Based on the position histogram, the cross-modal means position feature vector is constructed, and the formation target cross-modal correlation fusion accuracy is improved: the CRLB theory is applied to optical and electromagnetic remote sensing signal processing, the target space distribution probability is quantified based on the ellipse error principle, a target space distribution probability calculation method under multi-dimensional observation conditions is proposed, the target positioning deviation is corrected, and the target position histogram and position feature vector are constructed, and the formation target cross-modal correlation fusion based on the position feature vector is realized.

[0013] (2) Constructing a cross-modal means recognition feature vector based on an identification histogram to improve the cross-modal correlation fusion accuracy of the formation target: quantifying the confidence of different means platform type recognition to form an identification histogram, constructing an identification feature vector through the identification histogram, and realizing cross-modal correlation fusion of the formation target based on the identification feature vector. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 A logic block diagram of the embodiment of the present application.

[0015] Figure 2 A positioning probability distribution diagram of the embodiment of the present application.

[0016] Figure 3 A grid positioning probability distribution diagram of the embodiment of the present application.

[0017] Figure 4 A target multi-group observation probability distribution diagram of the embodiment of the present application.

[0018] Figure 5 An identification histogram construction diagram of the embodiment of the present application. DETAILED DESCRIPTION

[0019] The technical solutions of the present application will be described in detail below in combination with the drawings and embodiments.

[0020] The present application proposes a multi-modal remote sensing data fusion method based on probability voting, which fully utilizes the advantages of star cluster and aircraft cluster cooperative observation, and significantly improves the target correlation fusion accuracy in the scene of dense platform formation and relatively stable formation configuration. The present application constructs a position probability histogram and an identification probability histogram under multiple observation modes, and completes the automatic correlation fusion of multi-modal data based on the position topology and identification topology, effectively reduces the correlation ambiguity, and improves the fusion precision.

[0021] The airborne multi-modal remote sensing data fusion method based on probability voting provided by the embodiment of the present application includes the following steps:

[0022] Step 1: According to the collected electromagnetic remote sensing and image remote sensing data, the azimuth observation γ yaw , the elevation observation γ pitch , the time difference of arrival TDOA and the Doppler frequency FDOA are obtained, and the number of each target data group is required to be greater than 100; the observation station position information is collected to form an observation station position matrix (x s , y s , z s ), and the sampling period is not greater than 1 second.

[0023] Go to step 2.

[0024] Step 2, based on the GDOP model and the CRLB error model, combined with the azimuth observation γ yaw , the elevation observation γ pitch , the observation station position matrix (x s , y s , z s ), the positioning observation equation based on the observation station position and the target observation position is constructed, combined with the Earth ellipsoid equation, the positioning coverage range of optical remote sensing and electromagnetic remote sensing is calculated, and based on the probability constraint, the grid statistical observation probability in the positioning coverage range is realized, and the corresponding position histogram is formed in each grid, as follows:

[0025] The principles of optical remote sensing positioning and electromagnetic remote sensing positioning are to complete target positioning through angle measurement, time difference and frequency difference, and the positioning observation equation is as follows:

[0026]

[0027] In the formula, (x, y, z) represents the target observation position information to be solved, (x s , y s , z s ) represents the observation station position matrix, f1 and f2 represent the observation equation, f3 is the Earth ellipsoid equation, and the target observation position information is solved by combining f1, f2 and f3, and α and β are observations.

[0028] When the positioning system is a direction finding system, α and β represent the azimuth observation γ yaw and the elevation observation γ pitch , respectively.

[0029] When the positioning system is a time difference system, α represents the time difference TDOA 1,2 between the first observation station and the second observation station, and β represents the time difference TDOA 1,3 between the first observation station and the third observation station.

[0030] When the positioning system is a time-frequency difference, α represents the time difference TDOA 1,2 between the first observation station and the second observation station, and β represents the frequency difference FDOA 1,2 between the first observation station and the second observation station.

[0031] And since the observations of angle, time difference and frequency difference have errors, the target positioning result has a deviation, and the positioning deviation is represented by CRLB:

[0032] Assuming that the observation error of α is ε α , the observation error of β is ε β , , the partial differential operation is represented by:

[0033]

[0034] The CRLB error covariance matrix M is:

[0035]

[0036] Where pinv() means to find the generalized inverse matrix, T means the transpose of the matrix, σ x , σ y and σ z They represent the standard deviation of x, y, and z respectively.

[0037] For ground targets or sea targets, assuming the height is 0, the CRLB error covariance matrix M is converted to the WGS84 coordinates according to the coordinate conversion matrix. BLH It can be expressed as:

[0038]

[0039] R is the coordinate transformation matrix from the WGS84 coordinate system to the Earth-fixed coordinate system, σ lat ,σ lon Represent the standard deviation of the corresponding latitude and longitude respectively.

[0040] Assume ε α , ε β All obey Gaussian distribution and are independent of each other. According to Bayesian unbiased estimation, the probability that the positioning point falls within the error ellipse can be expressed as η. The error ellipse radii a and b are:

[0041]

[0042] Wherein, constant C = -2ln(1-η).

[0043] like Figure 2 As shown in , the larger the ellipse radius, the greater the probability that the positioning point falls within the ellipse. The probability of the positioning point falling within the innermost ellipse is 0.5, and the probability of falling in the middle area is the probability of the middle ellipse minus the probability of the inner ellipse, which is 0.3. Similarly, the probability of the positioning point falling within the outermost elliptical ring is 0.1; Figure 3 As shown, for each set of observations of the target, a set of positioning ellipses can be determined, the center of the ellipse is the target positioning point, and the ellipse angle α1 is:

[0044]

[0045] For the above elliptical area, the probability η of the positioning point falling on each grid is approximately expressed as:

[0046]

[0047] η l is the probability of each grid for the same target multi-group observation

[0048]

[0049] where N is the total number of observations, η m is the probability of each grid for the same target multi-group observation, 0

[0050] As Figure 4 shown, multiple observations (more than or equal to 2 times) are made on the same target and probability statistics are made, and with the increase of the number of observations, the probability of each grid is the average of multiple observation results, and a corresponding position histogram can be formed in each grid.

[0051] Go to step 3.

[0052] Step 3, based on the identification model, determine the probability of optical remote sensing and electromagnetic remote sensing platform type identification, and form an identification histogram for each target for different platform types and their probabilities:

[0053] The identification model includes an optical remote sensing image texture identification model and an electromagnetic remote sensing identification model. The optical remote sensing image texture identification model is used to estimate target platform type identification and confidence. The electromagnetic remote sensing identification model is used to construct a multi-layer probability tree of "radiation source parameters-radiation source type-platform type" to determine the probability of electromagnetic remote sensing platform type identification, and an identification histogram is formed for each target for different platform types and their probabilities.

[0054] In the embodiment, the specific process of step 3 is as follows:

[0055] S3.1, based on the optical remote sensing image texture identification model, estimate the target platform type identification and confidence:

[0056] For the target detected in the image, an optical remote sensing image texture identification model of all platform types ShipClass i is constructed based on a deep learning neural network, where 1≤i≤N0, N0 is the number of platform type types, and the confidence P j of each platform type of the target Target j,i to be identified in the optical remote sensing image is obtained through the above optical remote sensing image texture identification model, 1≤j≤M1, M1 is the number of platforms in the formation in the image.

[0057] S3.2, based on the electromagnetic remote sensing identification model, construct a multi-layer probability tree of "radiation source parameters-radiation source type-platform type" to determine the probability of electromagnetic remote sensing platform type identification:

[0058] Electromagnetic parameter of a target to be identified in electromagnetic remote sensing data k k k k k ,…), where 1≤k≤M2, M2 is the number of formation platforms in the electromagnetic remote sensing data, and rf, pri, pw, and pa represent the frequency, pulse repetition period, pulse width, and pulse amplitude of the radiation source, respectively. The model of the radiation source is identified through the above electromagnetic parameters, and a possible radiation source model set Radartype k is obtained, where Radartype k,s represents the s-th element in the set, 1≤s≤M k , and M k is the size of the set Radartype k . The confidence Q_radartype k,s of the identification of the radiation source model Radartype k,s is obtained during the identification process.

[0059] Further, through the association between the radiation source model and the platform model in the platform knowledge, the possible platform model information set ShipClass k,s of Radartype k,s is obtained, where ShipClass k,s,r represents the r-th element in the set, 1≤r≤M k,s , and M k,s is the size of the set ShipClass k,s . The confidence Q_shipclass k,s,r of the identification of the platform model ShipClass k,s,r is obtained during the identification process, and the confidences of these platform models are equal in probability.

[0060] ShipClass(k) is the set of all possible platform models of Target k , and the elements thereof are unique, i.e., for Target k , ShipClass(k) t represents the t-th element of the platform model set ShipClass(k), and for ShipClass(k) t , the set I(k,t)={(k,s,r)|ShipClass k,s,r =ShipClass(k) t} is defined, and then Target k ​​​​The probability of each platform type corresponding to the possible platform types is

[0061] As shown in Figure 5 , the Target j probability of different platform types is P j,i , the Target k probability of different platform types is Q k,t , and thus for each target and the identification probability of the platform type, an identification histogram as shown in Figure 5 is formed.

[0062] Go to step 4.

[0063] Step 4, use the identification histogram formed by each target for different platform types and their probabilities to construct an identification feature vector, in order to overcome the geometric deformation caused by rotation and slight displacement of the target formation configuration process, construct a position feature vector for the observed target through three targets, obtain the position feature vector angle, as follows:

[0064] In the embodiment, the specific process of step 4 is as follows:

[0065] S4.1, use the identification histogram formed by each target for different platform types and their probabilities to construct an identification feature vector:

[0066] In the optical remote sensing means, the identification feature vector of Target j is represented as P j ={P j,i};

[0067] In the electromagnetic remote sensing means, the identification feature vector of Target k is represented as Q k ={Q k,t}.

[0068] S4.2, construct a position feature vector based on the position histogram in step 2:

[0069] Based on the multiple observations of the target by the remote sensing means in step 2, the probabilities of the target Target j , Target k distributed grid area are respectively represented as η j,l , η k,l′ , the first subscript represents the target number, the second subscript represents the grid number of the target, η j,l represents the probability of the jth grid of the jth target in the optical remote sensing means; η k,l′ represents the probability of the l'th grid of the kth target in the optical remote sensing means.

[0070] For the same remote sensing means collected data, the first target as the center, the first target and the second target may form a feature vector probability. That is, by traversing all the grids of the first target and all the grids of the second target, the two-dimensional feature vector probability that may be formed is calculated. Assuming that the number of grids of the first target is N1, and the number of grids of the second target is N2, the number of possible feature vectors is N1N2, and the feature vector probability formed by the first target grid l1 and the second target grid l2 can be represented as:

[0071]

[0072] Corresponding feature vector is represented as: lat, lon are the latitude and longitude of the corresponding grid center, respectively.

[0073] Since the target formation process involves rotation and slight displacement of the target, resulting in geometric deformation, a triangular feature vector is constructed by combining three targets to represent the spatial relationship of the formation. For the gth target, g>2, the number of triangular feature vectors formed by the first target and the second target can be represented as N1N2N g , N g is the number of grids of the gth target;

[0074] The probability of the triangular feature vector can be represented as:

[0075]

[0076] The feature vectors between each other in the above triangular feature vector satisfy the following relationship:

[0077]

[0078] θ represents the angle between the position feature vectors of the target.

[0079] Go to step 5.

[0080] Step 5, in step 5, the position correlation of the target is determined by comparing the optical remote sensing means and the electromagnetic remote sensing means, and combining the inclusion relationship of the position feature vector angle of the target. The association of the target on the platform type identification result is determined by comparing the intersection relationship of the platform type of the target, and the identification confidence of the associated target is given, as follows:

[0081] S5.1, match the position feature vector:

[0082] In the optical remote sensing means, the position feature vector angle formed by the target Target j and the target Target u is θ ju, 1≤u≤M1,u≠j; in the electromagnetic remote sensing means, the target Target k and Target v forms a position feature vector included angle β kv , 1≤v≤M2,v≠k;

[0083] When θ ju and β kv satisfy the following decision condition:

[0084] So that or

[0085] Target j and Target k are considered to have relevance in position; in the actual scene, the difference of multiple observations must be considered when making the decision, and the decision times of u can be reduced by 1-2 times.

[0086] S5.2, matching the recognition feature vector:

[0087] If Target j and Target k have relevance in the recognition dimension, then ShipClass(j,k)≠φ is satisfied, wherein ShipClass(j,k)=ShipClass(j)∩ShipClass(k), ShipClass(j) is the set of all possible platform types of Target j , ShipClass(k) is the set of all possible platform types of Target k ; φ represents an empty set.

[0088] ShipClass(j,k) w represents the wth element of the set, which is the w1th element of the set corresponding to ShipClass(j), and which is the w2th element of the set corresponding to ShipClass(k). For the relevant target Target j and Target k , the platform type recognition confidence is expressed as

[0089] The specific embodiments described in the present application are only examples of the spirit of the present application. Those skilled in the art of the present application can make various modifications or supplements to the described specific embodiments or replace them with similar ways, but will not deviate from the spirit of the present application or exceed the scope defined by the appended claims.

Claims

1. A multimodal remote sensing data fusion method based on probability voting, characterized in that: Here are the steps: Step 1: Obtain the azimuth observation value γ based on the collected electromagnetic remote sensing and image remote sensing data yaw , pitch angle observation γ pitch , time difference of arrival TDOA, Doppler frequency FDOA, requiring the number of each target data group to be greater than 100; collect observation station location information to form the observation station location matrix (x s ,y s ,z s ), the sampling period is no more than 1 second, go to step 2; Step 2: Based on the GDOP model and CRLB error model, combined with the azimuth observation γ yaw , pitch angle observation γ pitch , observation station location matrix (x s ,y s ,z s ), construct a positioning observation equation based on the observation station position and the target observation position, combine it with the earth ellipse equation, calculate the optical remote sensing and electromagnetic remote sensing positioning coverage, and based on the probability constraint, realize the grid statistics of the observation probability within the positioning coverage, form the corresponding position histogram in each grid, and go to step 3; Step 3: Based on the recognition model, determine the probability of optical remote sensing and electromagnetic remote sensing platform model recognition. For each target, form an identification histogram for different platform models and their probabilities: The recognition model includes an optical remote sensing image texture recognition model and an electromagnetic remote sensing recognition model. The optical remote sensing image texture recognition model is used to estimate the target platform model identification and confidence level. A multi-layer probability tree of "radiation source parameters - radiation source model - platform model" is constructed based on the electromagnetic remote sensing recognition model to determine the probability of electromagnetic remote sensing platform model identification. For each target, an identification histogram is generated for different platform models and their probabilities, and the process proceeds to step 4. Step 4: Use the recognition histograms formed for each target for different platform models and their probabilities to construct an identification feature vector. To overcome the geometric deformation caused by the rotation and slight displacement of the target during the target formation configuration process, the position feature vector of the observed target is constructed by combining the three targets. The angle of its position feature vector is obtained, and then proceed to step 5. Step 5: Determine the position correlation of the target by cyclically comparing optical remote sensing means and electromagnetic remote sensing means, combined with the inclusion relationship of the target's position feature vector angle; determine the correlation of the target in the platform model recognition result by comparing the intersection relationship of the target's platform model, and give the recognition confidence of the target after correlation.

2. The multimodal remote sensing data fusion method based on probabilistic voting according to claim 1, characterized in that: In step 2, based on the GDOP model and the CRLB error model, the azimuth observation γ is combined yaw , pitch angle observation γ pitch , observation station location matrix (x s ,y s ,z s ), construct a positioning observation equation based on the observation station position and the target observation position, combine it with the earth ellipse equation, calculate the optical remote sensing and electromagnetic remote sensing positioning coverage, and based on the probability constraint, realize the grid statistics of the observation probability within the positioning coverage, and form the corresponding position histogram in each grid, as follows: The principles of optical remote sensing positioning and electromagnetic remote sensing positioning are to achieve target positioning through angle measurement, time difference and frequency difference. The positioning observation equation is as follows: In the formula, (x, y, z) represents the target observation position information to be solved, (x s ,y s ,z s ) represents the observation station position matrix, f1 and f2 represent the observation equations, f3 is the earth ellipsoid equation, and the target observation position information is obtained by combining f1, f2, and f3. α and β are the observation quantities. When the positioning system is a direction finding system, α and β represent the azimuth observations γ yaw , pitch angle observation γ pitch ; When the positioning system is the time difference system, α represents the time difference TDOA between the first observation station and the second observation station 1,2 , β represents the time difference TDOA between the first and third observation stations 1,3 ; When the positioning system is time-frequency difference, α represents the time difference TDOA between the first observation station and the second observation station 1,2 , β represents the frequency difference FDOA between the first observation station and the second observation station 1,2 ; And because there are errors in the observation quantities of angle measurement, time difference, and frequency difference, there is a deviation in the target positioning result. The positioning deviation is expressed by CRLB: Assume that the error of the observed quantity α is ε α , the observed quantity β error is ε β , Represents the partial differential operation, then the Jacobian matrix J is expressed as: The CRLB error covariance matrix M is: Where pinv() means to find the generalized inverse matrix, T means the transpose of the matrix, σ x , σ y and σ z They correspond to the standard deviations of x, y, and z respectively; For ground targets or sea targets, assuming the height is 0, the CRLB error covariance matrix M is converted to the WGS84 coordinates according to the coordinate transformation matrix. BLH Expressed as: R is the coordinate transformation matrix from the WGS84 coordinate system to the Earth-fixed coordinate system, σ lat , σ lon They correspond to the standard deviation of latitude and longitude respectively; Assume ε α , ε β All obey Gaussian distribution and are independent of each other. According to Bayesian unbiased estimation, the probability that the positioning point falls within the error ellipse is expressed as η, and the error ellipse radii a and b are respectively: Wherein, constant C = -2ln(1-η); The larger the ellipse radius, the greater the probability that the positioning point falls within the ellipse. The probability of the positioning point falling within the innermost ellipse is 0.5, and the probability of falling in the middle area is the probability of the middle ellipse minus the probability of the inner ellipse, which is 0.

3. Similarly, the probability of the positioning point falling within the outermost ellipse ring is 0.

1. For each set of observations of the target, a set of positioning ellipses can be determined, with the center of the ellipse being the target positioning point and the ellipse angle α1 being: For the above elliptical area, the probability η of the positioning point falling on each grid is approximately expressed as: For multiple sets of observations of the same target, the probability η of the positioning point falling on each grid l for: Where N is the total number of observations, η m is the probability that each observation falls on the grid, 0<l<l0, l0 is the total number of grids in the gridded area; Multiple observations are made on the same target, and probability statistics are carried out. As the number of observations increases, a corresponding position histogram is formed in each grid.

3. The multimodal remote sensing data fusion method based on probabilistic voting according to claim 1, characterized in that: In step 3, the recognition model includes an optical remote sensing image texture recognition model and an electromagnetic remote sensing recognition model. The optical remote sensing image texture recognition model is used to estimate the target platform model identification and confidence. The electromagnetic remote sensing recognition model is used to construct a multi-layer probability tree of "radiation source parameters-radiation source model-platform model" to determine the probability of electromagnetic remote sensing platform model identification, as follows: S3.

1. Estimating target platform model identification and confidence based on optical remote sensing image texture recognition model: Build all platform models ShipClass based on deep learning neural network for the targets detected in the image i The optical remote sensing image texture recognition model is used, where 1≤i≤N0, N0 is the number of platform types, and the target to be identified in the dense platform formation in the optical remote sensing image is obtained by the above optical remote sensing image texture recognition model. j The confidence level P of each platform model j,i , 1≤j≤M1, M1 is the number of formation platforms in the image; S3.

2. Construct a multi-layer probability tree of "radiation source parameters - radiation source model - platform model" based on the electromagnetic remote sensing identification model to determine the probability of electromagnetic remote sensing platform model identification: Target of the radiation source to be identified in electromagnetic remote sensing data k The electromagnetic parameters are (rf k ,pri k ,pw k ,pa k ,…), where 1≤k≤M2, M2 is the number of platforms in a formation in the electromagnetic remote sensing data; rf, pri, pw, and pa correspond to the frequency, pulse repetition period, pulse width, and pulse amplitude of the radiation source respectively; the model of the radiation source is identified by the above electromagnetic parameters, and the possible radiation source model set Radartype is obtained. k , Radartype k,s Represents the sth element in the set, 1≤s≤M k , M k Is the collection Radartype k The size of the radiation source can be obtained during the identification process. k,s Identification confidence Q_radartype k,s ; Further understand Radartype through the relationship between "radiation source model-platform model" in platform knowledge k,s Possible platform model information collection ShipClass k,s , ShipClass k,s,r Represents the rth element in the set, 1≤r≤M k,s , M k,s Is a collection of ShipClass k,s Size, while the identification process can obtain the platform model ShipClass k,s,r Identification confidence Q_shipclass k,s,r , the confidence levels of these platform models are equally probable, ShipClass(k) is Target k The set of all possible platform models, and its elements are unique, that is, for Target k , ShipClass(k) t Represents the tth element of the platform model set ShipClass(k). For ShipClass(k) t , define the set I(k,t)={(k,s,r)|ShipClass k,s,r =ShipClass(k) t }, then Target k The probability corresponding to each possible platform model is Each target forms a recognition histogram for different platform models and their probabilities.

4. The multimodal remote sensing data fusion method based on probabilistic voting according to claim 1, characterized in that: In step 4, the recognition feature vector is constructed using the recognition histogram formed by each target for different platform models and their probabilities. To overcome the geometric deformation caused by the rotation of the target formation configuration process and the slight displacement of the target, the position feature vector of the observed target is constructed by combining the three targets, and the angle of its position feature vector is obtained as follows: S4.

1. Use the recognition histogram formed by each target for different platform models and their probabilities to construct the recognition feature vector: In optical remote sensing, Target j The identification feature vector is represented as P j ={P j,i }; In electromagnetic remote sensing, Target k The identification feature vector is represented as Q k ={Q k,t }; S4.

2. Construct a position feature vector based on the position histogram in step 2: After multiple observations of the target using remote sensing in step 2, the target j Target k The probability of the distribution grid area is expressed as η j,l ,η k,l′ , the first subscript indicates the target number, the second subscript indicates the grid number of the target, η j,l represents the probability of the lth grid of the jth target in optical remote sensing; η k,l′ represents the probability of the l′th grid of the kth target in optical remote sensing means; For the data collected by the same remote sensing method, with the first target as the center, the probability of the possible eigenvector composed of the first target and the second target is calculated; that is, by traversing all the grids of the first target and all the grids of the second target, the probability of the possible two-dimensional eigenvector is calculated; Assuming that the number of grids for the first target is N1 and the number of grids for the second target is N2, the possible number of feature vectors is N1N2. The probability of the feature vector composed of the l1th grid of the first target and the l2th grid of the second target is Expressed as: Corresponding eigenvector Expressed as: lat and lon are the latitude and longitude of the corresponding grid center respectively; Since the target formation configuration process has geometric deformation caused by rotation and slight displacement of the target, a triangular eigenvector is constructed with three targets to represent the spatial relationship of the formation; for the g-th target, g>2, the number of triangular eigenvectors formed with the first target and the second target is expressed as N1N2N g , N g is the grid number of the g-th target; Then the probability of the triangular eigenvector is Expressed as: The relationship between the two eigenvectors in the above triangular eigenvectors is as follows: θ represents the angle of the position feature vector corresponding to the target.

5. The multimodal remote sensing data fusion method based on probabilistic voting according to claim 1, characterized in that: In step 5, the position correlation of the target is determined by cyclically comparing the optical remote sensing means and the electromagnetic remote sensing means, combined with the inclusion relationship of the target's position feature vector angle. The correlation of the target in the platform model recognition result is determined by comparing the intersection relationship of the target's platform model, and the recognition confidence of the target after correlation is given as follows: S5.

1. Matching position feature vector: In optical remote sensing, the target j Target u The angle of the position feature vector formed is θ ju , 1≤u≤M1,u≠j; in electromagnetic remote sensing, the target k with Target v The angle between the position eigenvectors is β kv , 1≤v≤M2,v≠k: When θ ju and β kv The following judgment conditions are met: Make or Target j with Target k Relevance in terms of location; S5.2, Matching identification feature vector: If Target j with Target k If there is relevance in the identification dimension, then ShipClass(j,k)≠φ, where ShipClass(j,k)=ShipClass(j)∩ShipClass(k), and ShipClass(j) is Target j The set of all possible platform models, ShipClass(k) is Target k The set of all possible platform models; φ represents the empty set; ShipClass(j,k) w Indicates the wth element of the set, the w1th element in the corresponding ShipClass(j) of the set, the w2th element in the corresponding ShipClass(k) of the set, for the associated target Target j with Target k The confidence level of platform model recognition is expressed as 6. The multimodal remote sensing data fusion method based on probabilistic voting according to claim 5, characterized in that: In actual scenarios, when S5.1 makes a decision, the differences among multiple observations must be considered, and the number of decisions on u can be reduced by 1 to 2.

Images