Target comprehensive identification method based on mixed information fusion model

Abstract

The invention discloses a target comprehensive identification method based on a mixed information fusion model. The method comprises the following steps: acquiring original information collected by multiple types of sensors; the multiple types of sensors support radar sensors, photoelectric sensors and electric detection sensors; performing mixed information fusion on the original information, and processing the original information into fusion information Df; converting the fusion information Df into target time sequence data through time accumulation window processing; loading the target comprehensive recognition model, inputting target time sequence data to the target comprehensive recognition model, and obtaining a target recognition result; wherein the structure of the target comprehensive identification model adopts an FTTransformer without a LayerNorm layer, and the real category labels of the samples of the target comprehensive identification model comprise a miniature / light and small rotor unmanned aerial vehicle, a miniature / light and small fixed-wing unmanned aerial vehicle, a bird and a patrolling bomb. According to the technical scheme, the model can learn the high-dimensional representation of the sensor information, the data value is fully mined, and the recognition capability of the model on the unmanned aerial vehicle target is improved.

Description

Technical Field

[0001] The present invention relates to the field of UAV countermeasures, and in particular to a target comprehensive recognition method based on a hybrid information fusion model. Background Art

[0002] Due to the characteristics of low cost, easy control and strong concealment, drone countermeasures are needed based on public safety and other factors.

[0003] Currently, the commonly used drone countermeasure technologies each have their own limitations: for example, traditional target detection and countermeasure solutions are limited by limitations such as communication and the time sensitivity of recognition results; rule-based multi-sensor decision-level target comprehensive recognition algorithms are widely used in the industry, and can be improved and optimized through field experiments and manual parameter adjustment based on expert experience. However, in the context of drone countermeasures, sensors are deployed in a distributed manner, and the sensor system and number increase. It is difficult to adjust the algorithm parameters through expert experience to achieve local or global optimization of sensor information fusion recognition; the industrial drone countermeasure recognition technology combined with artificial intelligence is based on traditional manual feature engineering, and the model parameters are manually adjusted based on the recognition results for drone recognition. However, in actual environments, since the vast majority of targets are birds, the sample data structure is extremely unbalanced, which makes the model mostly tend to identify all targets as birds to improve the overall accuracy of the model.

[0004] Therefore, in the context of drone countermeasures, this application proposes a drone target comprehensive recognition algorithm, which aims to improve the recognition effect of drones by the countermeasure system under multi-system and multi-number detection sensors through the target comprehensive recognition algorithm. Summary of the Invention

[0005] To achieve the above object, the present invention provides a method for comprehensive target recognition based on a hybrid information fusion model, comprising the following steps:

[0006] Acquire raw information collected from multiple types of sensors; multiple types of sensors support radar sensors, photoelectric sensors, and electronic detection sensors;

[0007] Perform mixed information fusion on the original information and process it into fused information D f ;

[0008] Through time accumulation window processing, the fusion information D f Convert to target time series data;

[0009] The target comprehensive recognition model is loaded, and the target time series data is input into the target comprehensive recognition model to obtain the target recognition result. The structure of the target comprehensive recognition model adopts the FTTransformer with the LayerNorm layer removed. The real category labels of the samples of the target comprehensive recognition model include: micro / light small rotor UAV, micro / light small fixed-wing UAV, bird, and cruise missile. The target recognition result is consistent with the range defined by the category label.

[0010] Before hybrid information fusion, fusion information sub-models of sensors of different systems are constructed, and the set of fusion information sub-models constitutes the fusion information model, which is expressed as:

[0011] Among them, D ld ,D gd ,D dz are the characteristic information sets of radar, photoelectric and electric detection sensors respectively, d is the original information collected in step S110, and F is the mixed information fusion processing; f ld It is a radar sensor information fusion model used to generate radar fusion information f gd It is an information fusion model for photoelectric sensors, used to generate photoelectric fusion information f dz It is an information fusion model for electronic reconnaissance sensors, used to fuse electronic reconnaissance fusion information.

[0012] The time accumulation window processing includes: setting the time accumulation window to w;

[0013] Accumulate the fusion information D within the w time period f , generate target time series data starting from time t, expressed as:

[0014]

[0015] Furthermore, before loading the target comprehensive recognition model, the target comprehensive recognition model is constructed, including:

[0016] Define the structure of the target comprehensive recognition model; determine the model loss of the target comprehensive recognition model using negative log-likelihood loss and add weight w label Improve the sensitivity of the loss function to the recognition of the specified target;

[0017] The input parameters of the target comprehensive recognition model are defined as the high-dimensional representation vector D w , the output result is the target recognition result s.

[0018] Among them, the high-dimensional representation vector D w Generated by spatial transformation of target time series data.

[0019] The spatial conversion process includes:

[0020] Extract the fusion information at time t from the target time series data, expressed as

[0021] Separate the fused information at time t into discrete data and continuous data Expressed as

[0022] Different spatial transformation operators are used for discrete data and continuous data to obtain high-dimensional representation of the target, which can be expressed as: Among them, Emb c 、Emb t is the spatial transformation operator for discrete and continuous data, expressed as: Emb c (cat)=cat 1×n ·E c n×m , Emb t (ctn)=ctn 1×n ·E t n×m , where cat and ctn are discrete and continuous information vectors, E n×m is the spatial transformation matrix.

[0023] Furthermore, the loss function is expressed as:

[0024] Among them, p n,l is the prediction result of the model for the nth sample in the lth category, N is the batch size, and y is the true category label of the sample.

[0025] Further, radar fusion information is generated When the target is in discrete information, take the mode of the target in each radar discrete information. If there are multiple modes, take The largest mode is used as fusion information;

[0026] For continuous information, the radar fusion information Expressed as: Among them, b i is the weight bias of radar i, is the result of weight normalization of radar i.

[0027] Generate optoelectronic fusion information When the discrete information is used, the mode of the target in each photoelectric discrete information is taken. When multiple modes appear, take ψ i The largest mode is used as fusion information;

[0028] For continuous information, optoelectronic fusion information Expressed as Among them, b i is the weight bias of photoelectric i, ψ i is the result of normalized weight of photoelectric i.

[0029] Generate electronic reconnaissance fusion information When , for discrete information, the mode of the target in each discrete information of electronic reconnaissance is taken as the fusion information;

[0030] For continuous information, electronic detection fusion information Expressed as: Among them, the weight α of all electronic reconnaissance equipment is the same, and b is the weight bias of electronic reconnaissance.

[0031] This paper employs a hybrid information fusion model and FTTransformer architecture to address the problem of comprehensive target recognition under distributed sensor deployment conditions. This hybrid information fusion model fuses information from sensors of the same system before feeding it into the recognition model. Dedicated fusion models are designed for sensors of different systems, improving the quality of information fusion and reducing the computational complexity of the recognition model. The FTTransformer architecture allows the model to learn high-dimensional representations of sensor information, fully tapping into the data's value and enhancing its target recognition capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a flow chart of a method for comprehensive target recognition based on a hybrid information fusion model provided according to an embodiment of the present invention;

[0033] Figure 2 3. This is a schematic diagram showing a comparison of false alarm rates between the target comprehensive recognition method provided by an embodiment of the present invention and the target recognition method using the prior art;

[0034] Figure 3 3 is a schematic diagram comparing the accuracy of target recognition performed by the target comprehensive recognition method provided by an embodiment of the present invention and the prior art. DETAILED DESCRIPTION

[0035] The present invention uses the idea of ​​hybrid expert model to construct multiple information fusion sub-models, and selects the corresponding sub-model according to the corresponding system of the sensor to complete information fusion; on the basis of information fusion of different systems, time accumulation is performed, and discrete and continuous features are transformed and spliced ​​through different spatial transformation operators to obtain high-dimensional representation of fused features, fully mining the deep information of feature data; finally, the target recognition model designed through the FTTransformer architecture realizes the recognition of UAV targets based on high-dimensional representation.

[0036] The specific implementation of the present invention is described in detail below with reference to the accompanying drawings.

[0037] Figure 1 A target comprehensive recognition method process based on a hybrid information fusion model is provided, including the following steps:

[0038] Step S110: obtaining original information collected from multiple types of sensors;

[0039] The present invention determines the supportable detection sensor systems, including radar sensors, photoelectric sensors and electric detection sensors. In this step, the raw data from these types of detection sensors are collected and expressed as d.

[0040] Generally speaking, the targets identified by the raw information from multiple types of sensors include various types of drones, birds, and cruise missiles.

[0041] The original information d is combined with the artificial label and can be used as a training sample for the target comprehensive recognition model of the present invention; the target can also be recognized by the original information through the trained target comprehensive recognition model.

[0042] Step S120: Perform mixed information fusion on the original information to process it into fused information D f ;

[0043] Since the present invention supports multiple types of sensors including radar sensors, photoelectric sensors and electric detection sensors, before executing this step, execute step S121 to construct fusion information sub-models of sensors of different systems, and fuse them to generate radar fusion information respectively. Optoelectronic Fusion Information and electronic reconnaissance fusion information The fusion information sub-models of sensors of different systems constitute the fusion information model, which can be expressed as:

[0044] Among them, D ld ,D gd ,D dz are the characteristic information sets of radar, photoelectric, and electronic reconnaissance sensors respectively, d is the original information collected through step S110, and F is the mixed information fusion processing.

[0045] During the hybrid information fusion process, different fusion processing methods are adopted for the original information of different sensors:

[0046] 1)f ld It is a radar sensor information fusion model used to generate radar fusion information

[0047] The specific generation method includes: calculating the slant distance d of the target from the radar i according to the latitude and longitude height of the target detected by the radar i , integrated radar detection accuracy acci Perform weighted fusion on radar i, and the weight of radar i is expressed as: The weight normalization is recorded as

[0048] For discrete information, radar fusion information Take the mode of the target's discrete information in each radar. If there are multiple modes, take The largest mode is used as the fusion information.

[0049] For continuous information, radar fusion information Expressed as: Among them, b i is the weight bias of radar i, is the result of weight normalization of radar i.

[0050] 2)f gd It is an information fusion model for photoelectric sensors, used to generate photoelectric fusion information The specific generation method includes: according to the photoelectric slant distance d i 、Pixel height and low angle interval d v , horizontal angle interval of pixels d h and the number of target imaging pixels k to select the photoelectric image information I i , calculate the weight of each photoelectric sensor i Where Δ is a constant, normalizing the weights, denoted as ψ i .

[0051] For discrete information, optoelectronic fusion information Take the mode of the target in each photoelectric discrete information. If there are multiple modes, take ψ i The largest mode is used as the fusion information.

[0052] For continuous information, optoelectronic fusion information Expressed as Among them, b i is the weight bias of photoelectric i, ψ i is the result of normalized weight of photoelectric i.

[0053] 3)f dz It is an information fusion model for electronic reconnaissance sensors, used to fuse information from sensors of this type.

[0054] For discrete information, electronic detection fusion information The mode of the target in each discrete type of electronic reconnaissance information is taken as the fusion information.

[0055] For continuous information, electronic detection fusion information Expressed as: Among them, the weight α of all electronic reconnaissance equipment is the same, and b is the weight bias of electronic reconnaissance.

[0056] Step S130: Time accumulation window processing, the fusion information D f Convert to target time series data;

[0057] Time accumulation window processing includes the following processing steps:

[0058] Set the time accumulation window to w;

[0059] Accumulate the fusion information D within the w time period f , generate target time series data starting from time t, expressed as:

[0060]

[0061] Step S140: loading a target comprehensive recognition model, inputting target time series data into the target comprehensive recognition model, and obtaining a target recognition result;

[0062] Before loading the target comprehensive recognition model, step S141 is executed: constructing the target comprehensive recognition model, including:

[0063] 1) Define the structure of the target comprehensive recognition model:

[0064] The target comprehensive recognition model of the present invention adopts FTTransformer with LayerNorm layer removed and adopts Rotational Position Encoding (RoPE);

[0065] Specifically, the model loss of the target comprehensive recognition model adopts negative log-likelihood loss (Negative Log-Likelihood Loss), by adding weight w label , improve the recognition sensitivity of the loss function to the specified target, and the loss function is expressed as:

[0066]

[0067] Among them, p n,l is the prediction result of the model for the nth sample in the lth category (i.e., the output of Logsoftmax), N is the batch size, and y is the true category label of the sample.

[0068] Among them, the real category labels of the samples include: micro / light small rotorcraft UAV, micro / light small fixed-wing UAV, bird, and cruise missile. The target recognition results are consistent with the range defined by the category labels.

[0069] 2) Define the input parameters of the target comprehensive recognition model as a high-dimensional representation vector D w , the output result is the target recognition result s;

[0070] Among them, the high-dimensional representation vector D w Generated by the target time series data through the spatial conversion processing in step S142.

[0071] Specifically, when performing spatial conversion processing, the spatial conversion operator is selected according to the sensor fusion information type, and the target time series data generated by the time accumulation window processing is converted into a high-dimensional representation vector D w , including the following steps:

[0072] Extract the fusion information at time t from the target time series data, expressed as

[0073] Extract the fusion information at time t from the fusion information and separate it into discrete data and continuous data Expressed as

[0074] Different spatial transformation operators are used for the discrete data and continuous data to obtain a high-dimensional representation of the target, which can be expressed as:

[0075] Among them, Emb c 、Emb t is a spatial transformation operator for discrete and continuous data:

[0076] Emb c (cat)=cat 1×n ·E c n×m ,

[0077] Emb t (ctn)=ctn 1×n ·E t n×m ,

[0078] Among them, cat and ctn are discrete and continuous information vectors, E n×m is the spatial transformation matrix, m<n.

[0079] After transforming the fusion information of all times in the target time series data, a high-dimensional representation vector is obtained, which is expressed as: D w ={I t ,I t+1 ,...,I t+w}, which is used by the subsequent time series recognition model to distinguish the target type.

[0080] During the training process of the target comprehensive recognition model, step S142 is executed to calculate the parameter gradient, update the overall model parameters, and determine whether the target recognition meets expectations and whether the model performance meets expectations in combination with the target label; if the model performance does not meet expectations, the process returns to step S140, and the target time series data is input into the target comprehensive recognition model again to start a new round of training.

[0081] Specifically, in step S142, for the training of the target comprehensive recognition model, the full model parameters at time t are defined as: θ = {θ m ,θ f},

[0082] The gradient of loss with respect to the parameters is

[0083] The update of the parameter learning rate is defined as

[0084] g t Corrected to g through historical status t =g t +λθ t-1 ,

[0085] The squared gradient and the state update parameter are defined as

[0086] Then the full model parameter update formula is

[0087] Among them, γ is the learning rate, λ is the weight decay rate, η is the weight decay rate, G is the cumulative sum of squared gradients, t is the current time, It is a gradient calculation operator, which is used to update model parameters after calculating the gradient.

[0088] If the target comprehensive recognition model has been completed, the high-dimensional representation vector D is input to the target comprehensive recognition model. w Afterwards, as shown in step S143, the target recognition result can be calculated and output, and in the subsequent steps, it is determined whether to end the recognition task.

[0089] The present invention also provides a simulation experiment of a target comprehensive recognition method based on a hybrid information fusion model and an FTTransformer architecture, which further illustrates the technical effects of the present invention:

[0090] First, the simulation conditions used in the simulation experiment include: establishing a comprehensive target recognition model according to the above method, and conducting model simulation result verification. The original data uses UAV test data, which contains detection data from three radars, two optoelectronics, and two electronic reconnaissance units; setting the time accumulation window w = 3, the learning rate γ = 0.03, the weight decay rate λ = 0.01, and the weight decay rate η = 0.01.

[0091] The simulation content is: testing the performance of the target comprehensive recognition model in actual drone countermeasure data.

[0092] The simulation results show that the recognition accuracy is as follows: Figure 2 As shown in , 201 is the recognition accuracy of the present invention on six data subsets, and 202 is the recognition accuracy of the rule model based on expert experience;

[0093] The false alarm rate comparison between simulation results and commonly used solutions is as follows Figure 3 As shown, 301 is the average false alarm rate of radars on six data sets, 302 is the false alarm rate of the rule model based on expert experience, and 303 is the false alarm rate of the present invention.

[0094] From the above comparison, it is obvious that the recognition accuracy of the target comprehensive recognition method provided by the present invention has been improved, with an average increase of 22.944%; the target false alarm rate has been significantly reduced, and the average false alarm rate of the intelligent comprehensive recognition model is 1.155%, which is lower than the 2.288% of the rule model, and the data fluctuation is small, and the false alarm rate is more stable. Figure 2 and Figure 3 It can be seen from the data fluctuations that the quality of radar data will affect the final recognition effect of the model, but the target intelligent comprehensive recognition model is more stable and more robust than the traditional rule model.

[0095] The above disclosures are only a few specific embodiments of the present invention. However, the present invention is not limited thereto. Any changes that can be conceived by those skilled in the art should fall within the scope of protection of the present invention.

Claims

1. A target comprehensive recognition method based on a hybrid information fusion model, characterized in that: The following steps are involved: Acquire raw information collected from multiple types of sensors; the multiple types of sensors support radar sensors, photoelectric sensors, and electric detection sensors; The original information is mixed and fused to form fused information D f ; Through time accumulation window processing, the fusion information D f Convert to target time series data; A target comprehensive recognition model is loaded, target time series data is input into the target comprehensive recognition model, and a target recognition result is obtained; wherein, the structure of the target comprehensive recognition model adopts the FTTransformer with the LayerNorm layer removed, and the true category labels of the samples of the target comprehensive recognition model include: micro / light small rotor UAV, micro / light small fixed-wing UAV, bird, and cruise missile; the target recognition result is consistent with the range defined by the category label.

2. The target comprehensive identification method according to claim 1, characterized in that: Before the hybrid information fusion, fusion information sub-models of sensors of different systems are constructed. The fusion information sub-models are assembled to form a fusion information model, which is expressed as: Among them, D ld ,D gd ,D dz are the characteristic information sets of radar, optoelectronic and electronic reconnaissance sensors, d is the original information, and F is the mixed information fusion processing; f ld It is a radar sensor information fusion model used to generate radar fusion information f gd It is an information fusion model for photoelectric sensors, used to generate photoelectric fusion information f dz It is an information fusion model for electronic reconnaissance sensors, used to fuse electronic reconnaissance fusion information.

3. The target comprehensive identification method according to claim 1, characterized in that: The time accumulation window processing includes: Set the time accumulation window to w; Accumulate the fusion information D within the w time period f , generate target time series data starting from time t, expressed as:

4. The target comprehensive identification method according to claim 1, characterized in that: Before loading the target comprehensive recognition model, constructing the target comprehensive recognition model includes: Define the structure of the target comprehensive recognition model; determine the model loss of the target comprehensive recognition model using negative log-likelihood loss and add weight w label , to improve the recognition sensitivity of the loss function to the specified target; The input parameter of the target comprehensive recognition model is defined as a high-dimensional representation vector D w , the output result is the target recognition result s.

5. The target comprehensive identification method according to claim 4, characterized in that: The high-dimensional representation vector D w It is generated by spatial conversion processing of the target time series data.

6. The target comprehensive identification method according to claim 5, characterized in that: The space conversion process includes: Extract the fusion information at time t from the target time series data, expressed as Separate the fused information at time t into discrete data and continuous data Expressed as Different spatial transformation operators are used for the discrete data and continuous data to obtain a high-dimensional representation of the target, which can be expressed as: Among them, Emb c 、Emb t is the spatial transformation operator for discrete and continuous data, expressed as: Emb c (cat)=cat 1×n ·E c n×m , Emb t (ctn)=ctn 1×n ·E t n×m , where cat and ctn are discrete and continuous information vectors, E n×m is the spatial transformation matrix.

7. The target comprehensive identification method according to claim 4, characterized in that: The loss function is expressed as: Among them, p n,l is the prediction result of the model for the nth sample in the lth category, N is the batch size, and y is the true category label of the sample.

8. The target comprehensive identification method according to claim 2, characterized in that: Generate radar fusion information When the target is in discrete information, take the mode of the target in each radar discrete information. If there are multiple modes, take The largest mode is used as fusion information; For continuous information, the radar fusion information Expressed as: Among them, b i is the weight bias of radar i, is the result of weight normalization of radar i.

9. The target comprehensive identification method according to claim 2, characterized in that: Generate optoelectronic fusion information When the discrete information is used, the mode of the target in each photoelectric discrete information is taken. When multiple modes appear, take ψ i The largest mode is used as fusion information; For continuous information, optoelectronic fusion information Expressed as Among them, b i is the weight bias of photoelectric i, ψ i is the result of normalized weight of photoelectric i.

10. The target comprehensive identification method according to claim 2, characterized in that: Generate electronic reconnaissance fusion information When , for discrete information, the mode of the target in each discrete information of electronic reconnaissance is taken as the fusion information; For continuous information, electronic detection fusion information Expressed as: Among them, the weight α of all electronic reconnaissance equipment is the same, and b is the weight bias of electronic reconnaissance.

Images