A classification and recognition method based on convolutional neural network and multi-modal fusion

Abstract

The application discloses a kind of classification identification method based on convolutional neural network and multimodal fusion.The implementation scheme is: (1) the distance-doppler-amplitude data containing target is obtained by processing target original echo;(2) the distance-doppler image of 4×16 size around target is obtained, and different modal Doppler data is concatenated and fused, respectively form one-dimensional, two-dimensional dataset;(3) construct multimodal fusion model based on convolutional neural network, use one-dimensional, two-dimensional convolutional neural network to classify and identify;(4) train, test network by forward propagation and back propagation, and use gradient descent algorithm to optimize network model;(5) the classification results obtained by two kinds of modules are fused at decision level by natural Bayes algorithm, and the comprehensive result of target classification identification is obtained.The application improves the accuracy of target classification identification, reduces manual participation, and can be used for classification and identification of low, slow and small radar targets in urban environment.

Description

Technical Field

[0001] This invention belongs to the field of radar target recognition technology, and specifically relates to a classification and recognition method based on convolutional neural networks and multimodal fusion. Background Technology

[0002] Radar target classification and identification refers to a technique that extracts stable, meaningful, and robust target features from the reflected echo signals received by radar to determine the target type and provide a result. Due to the complexity of modern military warfare environments and the diversification of hostile targets and missions, the timely detection, identification, and effective recognition of targets in harsh environments are crucial for victory. Radar equipment, as an important long-range detection sensor, plays a vital role and is in high demand both militarily and civilianly. Target classification and identification technology has become an indispensable function in radar systems, playing a significant role.

[0003] Traditional radar target classification and identification processes are complex, requiring manual extraction of target features before classification using existing machine learning algorithms. This process is time-consuming and inaccurate. In recent years, deep learning has been increasingly researched and applied in the field of intelligent signal processing. Among them, convolutional neural networks are a highly efficient feature extraction network structure in the field of target recognition. They have good transferability and can extract deep-level features of targets, better representing the essential information of the targets.

[0004] Multimodal fusion technology in deep learning is the process by which models process data of different forms when performing analysis and recognition tasks. The fusion of multimodal information can provide more accurate and richer information for model decision-making, thereby improving the accuracy of the overall decision results. Multimodal information fusion recognition is also an important trend in the development of radar target classification and recognition. When dealing with redundant and incomplete sensor data, multimodal information fusion provides us with new ideas. Building, processing, and associating models from multiple modalities to construct a multimodal fusion recognition processing framework and achieving efficient fusion of diverse information is an important goal for future intelligent radar target classification and recognition. Therefore, researching a classification and recognition method based on convolutional neural networks and multimodal fusion, exploring how to combine multimodal information from target data, and using convolutional neural networks and natural Bayesian algorithms for decision fusion to improve the accuracy of target classification and recognition and enhance the overall performance of target classification and recognition has become a key research topic in the current technological field. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by proposing a classification and recognition method based on convolutional neural networks and multimodal fusion. This method integrates multiple modal information from the data and uses convolutional networks to automatically extract target features, thereby reducing the inaccuracies and blind spots caused by manual methods and improving the overall performance of target classification and recognition.

[0006] To achieve the above objectives, the technical solution adopted by this invention is: a classification and recognition method based on convolutional neural networks and multimodal fusion, comprising the following steps:

[0007] (1) Acquisition and processing of raw echo signals from radar targets;

[0008] (2) Target Doppler data fusion and acquisition of target distance-Doppler image;

[0009] (3) Construct a multimodal fusion model based on convolutional neural networks, wherein a one-dimensional convolutional neural network is used to extract features from the target Doppler data; and a two-dimensional convolutional neural network is used to extract features from the distance-Doppler image.

[0010] (4) Train and test the network model to obtain stable model and target category results;

[0011] (5) The classification results obtained from the two classification and recognition modules are fused at the decision level using the Natural Bayes algorithm to obtain the comprehensive result of the final classification and recognition of the target;

[0012] The construction of a multimodal fusion model based on a convolutional neural network is described in step (3). The specific process is as follows: The structure of the one-dimensional convolutional neural network and the two-dimensional convolutional neural network includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer. The convolutional layer extracts local features by sliding the convolutional kernel. In order to improve the nonlinear expression capability of the model, the local features output by the convolutional layer are processed by the activation function. The pooling layer is used to reduce the feature dimension and increase the receptive field range. Finally, the fully connected layer is used to achieve feature fusion and output the target category probability.

[0013] (3a) A convolutional layer is represented as follows: the input network data size is m×m, the convolutional layer includes K convolutional kernels, the kernel size is F×F, and the padding size is... If the stride is denoted by S, then the size of the output after convolution is...

[0014] For discrete sequences, the convolution process is represented as:

[0015]

[0016] For a two-dimensional image, the convolution process is represented as:

[0017]

[0018] Where x represents the input sequence, h represents the convolution kernel, k and j represent the length of the data, and * represents the convolution operation;

[0019] (3b) The activation function is a non-linear σ = ReLU() function, and the output is expressed as:

[0020] σ(G(m,n))=σ(x[m,n]*h[m,n])=max(0,x[m,n]*h[m,n]),

[0021] (3c) The pooling layer is represented as follows: Each submatrix of the input tensor is compressed, the pooling region size is set to k×k, and the pooling criterion is max pooling; if the input is m×m dimensional, the output is represented as...

[0022] (3d) A fully connected layer is represented as follows: each input neuron is fully connected to the next layer of neurons, and the output neurons represent the probability of each class. Setting the activation function and the number of neurons L in each fully connected layer, the activation function is typically σ = Sigmoid(). The output of the connected layer is then:

[0023] H = σ(G(m,n)),

[0024] The Sigmoid function is:

[0025]

[0026] Furthermore, the original target echo signal received by the radar in step (1) is described as follows:

[0027]

[0028] Where s r (t) represents the received signal, a is the amplitude of the received signal, f0 is the carrier frequency of the transmitted signal, and f D It's a Doppler shift. This is due to the phase shift of the received signal caused by the target's motion, which is achieved by mixing it with the transmitted signal according to the following formula:

[0029] s t (t) = acos(2πf0t),

[0030] After passing through a synchronous detector and a low-pass filter, a complex Doppler signal is generated as follows:

[0031]

[0032] The data is recalculated according to the magnitude, I-path, and Q-path arrangements to obtain the range-Doppler-amplitude data of each frame of the target echo; the data of the four range cells before and after the target and the 16 Doppler channels to the left and right of each frame are extracted to form a dataset.

[0033] The target Doppler data fusion described in step (2) involves directly concatenating different modal Doppler data within each range cell into a single data line, fusing it into one-dimensional target data. The result is as follows:

[0034] f(x)=concat[dis1, dis2, dis3,…, disn],

[0035] Where, disn refers to all one-dimensional Doppler channel data of the nth distance cell, which are fused from different modes by the concat concatenation function;

[0036] The concatenated one-dimensional Doppler data of the target constitutes the dataset for inputting a one-dimensional convolutional neural network, while the truncated 4×16-dimensional distance-Doppler image of the target constitutes the dataset for inputting a two-dimensional convolutional neural network.

[0037] The target classification and recognition training and testing based on the convolutional neural network in step (4) specifically includes the following steps:

[0038] (4a) Train the constructed convolutional neural network, including the forward propagation process and the back propagation process;

[0039] Forward propagation process: During the forward propagation process, the parameters of each layer of the convolutional neural network are convolved with the data of the same layer, and a bias is added. This process is then passed forward step by step to obtain the processing result of the entire network.

[0040] Backpropagation process: After the input data is forward propagated, the difference between it and the true label is calculated through the loss function, which is called the residual. The residual is backpropagated through gradient descent. According to the chain rule, the trainable weights and biases of each layer of the convolutional neural network are updated layer by layer. The learning rate is used to control the strength of the residual backpropagation.

[0041] (4b) The network parameters are continuously updated iteratively according to the loss function, and finally the trained network model is obtained;

[0042] The goal of training a convolutional neural network is to minimize the loss function, and a commonly used loss function is the cross-entropy loss function:

[0043]

[0044] Where y i Indicates sample x i The real label, h θ (xi () represents the probability that the sample is in the positive class;

[0045] Convolutional neural network training uses the gradient descent algorithm for iterative processing. The gradient vector formula is as follows:

[0046]

[0047] Training ends when the loss function converges to a sufficiently small value or the number of training iterations reaches the maximum, resulting in a convolutional neural network model that can be used for object classification and recognition.

[0048] (4c) Save the trained network model parameters and input test data to obtain the target classification and recognition results;

[0049] When saving the parameters of a convolutional neural network (CNN) model as a PowerBuilder (PB) file, the first step is to specify the session to be saved and a specific tag. Then, the save is performed using the `build` method in the `Saver` class of the Tensorflow deep learning framework, and the restore is performed using the `load` method. The `variables` folder contains all the variables and names of the CNN model, and the `saved model.pb` file contains information such as the structure of the CNN model. The saved CNN model is then called to process the test data and obtain the target classification and recognition results.

[0050] The Natural Bayesian fusion algorithm in step (5) assumes that there are n independent classifiers. Each classifier is applied to the training set to obtain a c×c confusion matrix CM. j :

[0051]

[0052] Where the subscript c represents the total number of categories in the sample, D represents the j-th classifier. j w in the training samples k Class samples are divided into w s Number of classes, CM j The sum of the sth column For classifier D j Classify the samples into categories w s The total number; there are CMs j The probability matrix LM can then be obtained. j , of which elements D represents j Classifier w k Class samples are divided into w s The probability of a class; that is Because of the independence between classifiers, the true class w is obtained. k The probability of:

[0053]

[0054] Ultimately, the sample's class is determined by taking the largest possible true class probability.

[0055] The classification and recognition method based on convolutional neural networks and multimodal fusion proposed in this invention has the following advantages compared with existing technologies:

[0056] (1) The classification and recognition method based on convolutional neural network has the ability to express multi-level features and the ability to fit nonlinear data. Compared with other target recognition methods, it has the advantages of extracting features more deeply and having stronger recognition ability. It also reduces the blindness and uncertainty brought about by human intervention and improves the accuracy of classification and recognition.

[0057] (2) By using a multimodal fusion recognition strategy, the target one-dimensional Doppler data and the target range-Doppler two-dimensional image are combined to integrate radar feature data of different dimensions, allowing the network to extract interrelated feature information in different representation subspaces and improve the precision of classification and recognition.

[0058] (3) The classification and recognition results under different modalities are fused at the decision level using the natural Bayesian fusion algorithm. By reasonably evaluating and assigning weights to each classifier, a better recognition effect than a single classifier is obtained.

[0059] This invention fully utilizes the multimodal information of target data, combines it with the automatic feature extraction of convolutional neural networks, and further performs decision fusion on the recognition results. It can be used to classify and identify low, slow, and small radar targets in urban environments. Attached Figure Description

[0060] Figure 1 This is a flowchart illustrating the implementation of this invention;

[0061] Figure 2 This is the multimodal data input format for drones and automobiles in this invention;

[0062] Figure 3 This is a schematic diagram of the one-dimensional convolutional neural network structure of the present invention;

[0063] Figure 4 This is a schematic diagram of the two-dimensional convolutional neural network structure of the present invention;

[0064] Figure 5 The curves showing the changes in recognition accuracy and loss value of the one-dimensional and two-dimensional convolutional neural networks of this invention with the number of iterations are shown.

[0065] Figure 6 This is the comprehensive result of multimodal fusion classification and recognition in this invention. Detailed Implementation

[0066] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings and specific embodiments.

[0067] Reference Figure 1-6 The present invention provides a classification and recognition method based on convolutional neural networks and multimodal fusion, comprising the following steps:

[0068] (1) Acquisition and processing of raw echo signals from radar targets;

[0069] After pulse compression and moving target detection, radar target echo data is used to obtain range-Doppler two-dimensional data.

[0070] (1a) Pulse compression processing refers to compressing a wide transmitted pulse signal into a narrow pulse signal, which is essentially matching filtering of the signal. A matched filter is the optimal linear filter under the criterion of maximizing the output signal-to-noise ratio. Assume the input waveform of the linear filter is a mixture of signal and noise:

[0071]

[0072] x(t) = s(t) + n(t)

[0073] Assuming the noise is white noise with a mean of 0, and the power spectral density is: P n Given that (w) = n0 / 2, the signal's spectral function is S(w), and the matched filter's transfer characteristic is H(w), then the optimal linear filter's transfer characteristic can be derived from the criterion of maximizing output signal-to-noise ratio:

[0074]

[0075] Where: k is the amplitude normalization constant, S * (W) is the complex conjugate of the signal S(w).

[0076] The output of the matched filter is:

[0077] (Frequency Domain)

[0078] (Time Domain)

[0079] Its pulse compression result output has an envelope of the sinc function, which is expressed as follows: The maximum value is obtained at the point, and a pulse appears, thus achieving the purpose of time-domain pulse compression.

[0080] (1b) Moving target detection of the signal, wherein the moving target detection system consists of a set of adjacent and partially overlapping filter banks, and a narrow-band Doppler filter bank covering the entire Doppler frequency range. The N adjacent Doppler filter banks are formed by N output transverse filters (N pulses and N-1 delay lines), which are then weighted and summed according to different values ​​of each pulse.

[0081] The frequency response function for each impulse response is:

[0082]

[0083] In fact, when the number of filter banks is an integer power of 2, the Fast Fourier Transform algorithm can be used to complete the moving target detection filter.

[0084] (1c) Obtain radar target range-Doppler two-dimensional data

[0085] Doppler radar uses the Doppler effect to measure the radial velocity of a target. The Doppler frequency shift is extracted by a quadrature phase detector. The received signal is described as follows:

[0086]

[0087] Where s r (t) represents the received signal, a is the amplitude of the received signal, f0 is the carrier frequency of the transmitted signal, and f D It's a Doppler shift. This is due to the phase shift of the received signal caused by the target's motion, which is achieved by mixing it with the transmitted signal according to the following formula:

[0088] s t (t)=acos(2πf0t)

[0089] After passing through a synchronous detector and a low-pass filter, a complex Doppler signal is generated as follows:

[0090]

[0091] The data was recalculated according to the magnitude, I-channel, and Q-channel arrangements to obtain the range-Doppler-amplitude data for each frame of the target echo. Data from the four range cells before and after the target and the 16 Doppler channels to the left and right of each frame were extracted to form a dataset.

[0092] (2) Target Doppler data fusion and acquisition of target distance-Doppler image;

[0093] Target Doppler data fusion involves directly concatenating different modal Doppler data within each range cell into a single data line, fusing it into one-dimensional target data, as follows:

[0094] f(x)=concat[dis1, dis2, dis3,…, disn]

[0095] Here, disn refers to the data of all one-dimensional Doppler channels in the nth distance cell, which are fused into different modes through the concat concatenation function.

[0096] The concatenated one-dimensional Doppler data of the target constitutes the dataset for inputting a one-dimensional convolutional neural network, while the truncated 4×16-dimensional distance-Doppler image of the target constitutes the dataset for inputting a two-dimensional convolutional neural network.

[0097] Reference Figure 2 This involves fusing one-dimensional Doppler data and two-dimensional distance-Doppler image data for vehicles and drones.

[0098] (3) Construct a multimodal fusion model based on convolutional neural networks, wherein a one-dimensional convolutional neural network is used to extract features from the target Doppler data; and a two-dimensional convolutional neural network is used to extract features from the distance-Doppler image.

[0099] Reference Figure 3 and Figure 4 The schematic diagrams of the one-dimensional and two-dimensional convolutional neural networks of the present invention include an input layer, a convolutional layer and a pooling layer, a fully connected layer and an output layer. The convolutional layer extracts local features by sliding the convolutional kernel. To improve the non-linear expressive power of the model, the local features output by the convolutional layer are processed by an activation function. The pooling layer is used to reduce the feature dimension and increase the receptive field. Finally, the fully connected layer achieves feature fusion and outputs the target class probability.

[0100] (3a) A convolutional layer is represented as follows: the input network data size is m×m, the convolutional layer includes K convolutional kernels, the kernel size is F×F, and the padding size is... If the stride is denoted by S, then the size of the output after convolution is...

[0101] For discrete sequences, the convolution process is represented as:

[0102]

[0103] For a two-dimensional image, the convolution process is represented as:

[0104]

[0105] Where x represents the input sequence, h represents the convolution kernel, k,j represents the length of the data, and * represents the convolution operation.

[0106] (3b) The activation function is a non-linear σ = ReLU() function, and the output is expressed as:

[0107] σ(G(m,n))=σ(x[m,n]*h[m,n])=max(0,x[m,n]*h[m,n])

[0108] (3c) The pooling layer is represented as follows: Each submatrix of the input tensor is compressed, the pooling region size is set to k×k, and the pooling criterion is max pooling. If the input is m×m dimensional, the output is represented as follows:

[0109] (3d) A fully connected layer is represented as follows: each input neuron is fully connected to the next layer of neurons, and the output neurons represent the probability of each class. Setting the activation function and the number of neurons L in each fully connected layer, the activation function is typically σ = Sigmoid(). The output of the connected layer is then:

[0110] H=σ(G(m,n))

[0111] The Sigmoid function is:

[0112]

[0113] (4) Train and test the network model to obtain stable model and target category results;

[0114] Specifically, the steps include the following:

[0115] (4a) Train the constructed convolutional neural network, including the forward propagation process and the back propagation process;

[0116] Forward propagation process: During the forward propagation process, the parameters of each layer of the convolutional neural network are convolved with the data of that layer, and a bias is added. This process is then passed forward step by step to obtain the processing result of the entire network.

[0117] Backpropagation process: After the input data is forward propagated, the difference between it and the true label is calculated through the loss function, which is called the residual. The residual is backpropagated through gradient descent. According to the chain rule, the trainable weights and biases of each layer of the convolutional neural network are updated layer by layer. The learning rate is used to control the strength of the residual backpropagation.

[0118] (4b) The network parameters are continuously updated iteratively according to the loss function, and finally the trained network model is obtained;

[0119] The goal of training a convolutional neural network is to minimize the loss function, and a commonly used loss function is the cross-entropy loss function:

[0120]

[0121] Where y i Indicates sample x i The real label, hθ (x i ) represents the probability that a sample is classified as positive.

[0122] Convolutional neural network training uses the gradient descent algorithm for iterative processing. The gradient vector formula is as follows:

[0123]

[0124] Training continues until the loss function converges to a sufficiently small value or the training reaches its maximum number of iterations, at which point the training ends, resulting in a convolutional neural network model that can be used for object recognition.

[0125] (4c) Save the trained network model parameters and input test data to obtain the target classification and recognition results.

[0126] This invention saves the parameters of a convolutional neural network (CNN) model as a PowerBuilder (PB) file. First, it specifies the session to be saved and certain tags. Then, it saves the model using the `build` method of the `Saver` class in the TensorFlow deep learning framework and restores it using the `load` method. The `variables` folder stores all variables and names of the CNN model; the `savedmodel.pb` file stores the structure and other information of the CNN model. The saved CNN model is then called to process the test data to obtain the target classification and recognition results.

[0127] (5) The classification results obtained from the two classification and recognition modules are fused at the decision level using the Natural Bayes algorithm to obtain the comprehensive result of the final classification and recognition of the target.

[0128] Decision-level fusion first involves identification, then parallelizing the various decision results to obtain a fused judgment. The fusion method employs the Natural Bayes algorithm because it requires less prior information and considers the differences in classification performance between different classifiers, making it a stable fusion method. Assuming there are n independent classifiers, applying each classifier to the training set yields a c×c confusion matrix CM. j :

[0129]

[0130] Where the subscript c represents the total number of categories in the sample, D represents the j-th classifier. j w in the training samples k Class samples are divided into w s Number of classes, CM j The sum of the sth column For classifier D j Classify the samples into categories w s The total number. There are CMs.j The probability matrix LM can then be obtained. j , of which elements D represents j Classifier w k Class samples are divided into w s The probability of a class.

[0131] Right now Because of the independence between classifiers, the true class w is obtained. k The probability of:

[0132]

[0133] Ultimately, the sample's class is determined by taking the largest possible true class probability.

[0134] The effects of this invention can be further illustrated through experiments:

[0135] 1. Experimental conditions

[0136] This invention is based on real-world radar data recorded at the test site, which is authentic, reliable, and valuable. The hardware platform for this invention is an Intel Core i7 CPU with 8GB of RAM, and the software platform is a Windows 10 operating system and a PyCharm editor (Python 3.8).

[0137] 2. Explanation of Experimental Results

[0138] Reference Figure 5 The figures show the curves of the recognition accuracy and loss value of the one-dimensional and two-dimensional convolutional neural networks designed in this invention as a function of the number of iterations. It can be seen from the training process of the one-dimensional convolutional neural network that there are certain errors in the training set and the test set during the iteration process, and the loss value of the test set oscillates to some extent in the later stages as the number of iterations increases. In contrast, the training set and the test set of the two-dimensional convolutional neural network perform well, but the classification and recognition accuracy decreases.

[0139] Reference Figure 6 The radar target multimodal data is input into convolutional neural networks of different dimensions to obtain classification and recognition results. Finally, the natural Bayes algorithm is used to fuse the results at the decision level, which improves the accuracy of classification and recognition.

[0140] The above description is merely a specific example of the present invention and does not constitute any limitation on the present invention. Obviously, those skilled in the art, after understanding the content and principles of the present invention, may make various modifications and changes in form and detail without departing from the principles and structure of the present invention; however, these modifications and changes based on the spirit of the present invention are still within the scope of protection of the claims of the present invention.

Claims

1. A classification and recognition method based on convolutional neural networks and multimodal fusion, characterized in that, Includes the following steps: (1) Acquisition and processing of raw echo signals from radar targets; (2) Target Doppler data fusion and acquisition of target distance-Doppler image; (3) Construct a multimodal fusion model based on convolutional neural networks, wherein a one-dimensional convolutional neural network is used to extract features from the target Doppler data; and a two-dimensional convolutional neural network is used to extract features from the distance-Doppler image. (4) Train and test the network model to obtain stable model and target category results; (5) The classification results obtained from the two classification and recognition modules are fused at the decision level using the Natural Bayes algorithm to obtain the comprehensive result of the final classification and recognition of the target; The construction of a multimodal fusion model based on a convolutional neural network is described in step (3). The specific process is as follows: The structure of the one-dimensional convolutional neural network and the two-dimensional convolutional neural network includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer. The convolutional layer extracts local features by sliding the convolutional kernel. In order to improve the nonlinear expression capability of the model, the local features output by the convolutional layer are processed by the activation function. The pooling layer is used to reduce the feature dimension and increase the receptive field range. Finally, the fully connected layer is used to achieve feature fusion and output the target category probability. (3a) A convolutional layer is represented as follows: the input network data size is m×m, the convolutional layer includes K convolutional kernels, the kernel size is F×F, and the padding size is... If the stride is denoted by S, then the size of the output after convolution is... For discrete sequences, the convolution process is represented as: For a two-dimensional image, the convolution process is represented as: Where x represents the input sequence, h represents the convolution kernel, k and j represent the length of the data, and * represents the convolution operation; (3b) The activation function is a non-linear σ = ReLU() function, and the output is expressed as: σ(G(m,n))=σ(x[m,n]*h[m,n])=max(0,x[m,n]*h[m,n]), (3c) The pooling layer is represented as follows: each submatrix of the input tensor is compressed, the pooling region size is set to k×k, and the pooling standard is max pooling. If the input is m×m dimensional, then the output is represented as follows: (3d) A fully connected layer is represented as follows: each input neuron is fully connected to the next layer of neurons, and the output neurons represent the probability of each class. Setting the activation function and the number of neurons L in each fully connected layer, the activation function is typically σ = Sigmoid(). The output of the connected layer is then: H = σ(G(m,n)), The Sigmoid function is:

2. The classification and recognition method based on convolutional neural networks and multimodal fusion according to claim 1, characterized in that, The original target echo signal received by the radar in step (1) is described as follows: Where s r (t) represents the received signal, a is the amplitude of the received signal, f0 is the carrier frequency of the transmitted signal, and f D It's a Doppler shift. This is due to the phase shift of the received signal caused by the target's motion, which is achieved by mixing it with the transmitted signal according to the following formula: s t (t)=acos(2πf0t), After passing through a synchronous detector and a low-pass filter, a complex Doppler signal is generated as follows: The data is recalculated according to the magnitude, I-path, and Q-path arrangements to obtain the range-Doppler-amplitude data of each frame of the target echo; the data of the four range cells before and after the target and the 16 Doppler channels to the left and right of each frame are extracted to form a dataset.

3. The classification and recognition method based on convolutional neural networks and multimodal fusion according to claim 2, characterized in that, The target Doppler data fusion described in step (2) involves directly concatenating different modal Doppler data within each range cell into a single data line, fusing it into one-dimensional target data. The result is as follows: f(x)=concat[dis1, dis2, dis3,…, disn], Where, disn refers to all one-dimensional Doppler channel data of the nth distance cell, which are fused from different modes by the concat concatenation function; The concatenated one-dimensional Doppler data of the target constitutes the dataset for inputting a one-dimensional convolutional neural network, while the truncated 4×16-dimensional distance-Doppler image of the target constitutes the dataset for inputting a two-dimensional convolutional neural network.

4. The classification and recognition method based on convolutional neural networks and multimodal fusion according to claim 1, characterized in that, The target classification and recognition training and testing based on the convolutional neural network in step (4) specifically includes the following steps: (4a) Train the constructed convolutional neural network, including the forward propagation process and the back propagation process; Forward propagation process: During the forward propagation process, the parameters of each layer of the convolutional neural network are convolved with the data of the same layer, and a bias is added. This process is then passed forward step by step to obtain the processing result of the entire network. Backpropagation process: After the input data is forward propagated, the difference between it and the true label is calculated through the loss function, which is called the residual. The residual is backpropagated through gradient descent. According to the chain rule, the trainable weights and biases of each layer of the convolutional neural network are updated layer by layer. The learning rate is used to control the strength of the residual backpropagation. (4b) The network parameters are continuously updated iteratively according to the loss function, and finally the trained network model is obtained; The goal of training a convolutional neural network is to minimize the loss function, and a commonly used loss function is the cross-entropy loss function: Where y i Indicates sample x i The real label, h θ (x i () represents the probability that the sample is in the positive class; Convolutional neural network training uses the gradient descent algorithm for iterative processing. The gradient vector formula is as follows: Training ends when the loss function converges to a sufficiently small value or the number of training iterations reaches the maximum, resulting in a convolutional neural network model that can be used for object classification and recognition. (4c) Save the trained network model parameters and input test data to obtain the target classification and recognition results; When saving the parameters of a convolutional neural network (CNN) model as a PowerBuilder (PB) file, the first step is to specify the session to be saved and a specific tag. Then, the save is performed using the `build` method in the `Saver` class of the Tensorflow deep learning framework, and the restore is performed using the `load` method. The `variables` folder contains all the variables and names of the CNN model, and the `saved model.pb` file contains information such as the structure of the CNN model. The saved CNN model is then called to process the test data and obtain the target classification and recognition results.

5. The classification and recognition method based on convolutional neural networks and multimodal fusion according to claim 1, characterized in that, The Natural Bayesian fusion algorithm in step (5) assumes that there are n independent classifiers. Each classifier is applied to the training set to obtain a c×c confusion matrix CM. j : Where the subscript c represents the total number of categories in the sample, D represents the j-th classifier. j w in the training samples k Class samples are divided into w s Number of classes, CM j The sum of the sth column For classifier D j Classify the samples into categories w s The total number; there are CMs j The probability matrix LM can then be obtained. j , of which elements D represents j Classifier w k Class samples are divided into w s The probability of a class; Right now Because of the independence between classifiers, the true class w is obtained. k The probability of: Ultimately, the sample's class is determined by taking the largest possible true class probability.

Images