Small rotor unmanned aerial vehicle target motion recognition method based on 4D radar

Abstract

The present application relates to the field of radar monitoring and signal processing, in particular to a small rotor unmanned aerial vehicle target motion recognition method based on a 4D radar, comprising: monitoring the motion of a small rotor unmanned aerial vehicle as a target through a 4D radar; calculating the energy signal of the 4D radar; determining whether the energy signal of the 4D radar exceeds a preset threshold; if yes, extracting the radar measurement information corresponding to the current frame energy signal; otherwise, returning to step S2; realizing target continuous tracking according to the extracted multi-frame radar measurement information through a probability data association algorithm, and generating a tracking 3D trajectory; inputting the tracking 3D trajectory into a trained long short-term memory-connectionist temporal classification network model based on 3D projection, and outputting a target continuous motion recognition result. The present application can ensure the accuracy of small rotor unmanned aerial vehicle target detection and tracking in a clutter environment, and can recognize the motion of a small rotor unmanned aerial vehicle target without pre-segmentation.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of radar monitoring and signal processing, in particular to a small rotor unmanned aerial vehicle target motion recognition method based on 4D radar. BACKGROUND

[0002] At present, various types of sensors are used for small rotor unmanned aerial vehicle perception, mainly including acoustic, optical and radar sensors. Acoustic sensors achieve small rotor unmanned aerial vehicle detection by monitoring the noise generated by the rotation of the rotor blades, and have good performance within a short distance. However, due to the interference of other sound sources and the lack of distance information, it is difficult to accurately locate and track this type of sensor. Optical sensors, such as high-definition digital cameras, obtain the motion trajectory of small rotor unmanned aerial vehicles through computer vision algorithms based on continuous image sequences. However, they are easily affected by light and visibility, especially in bad weather conditions. Radar achieves small rotor unmanned aerial vehicle detection and tracking by transmitting and receiving high-frequency electromagnetic wave signals, has the advantages of long detection distance, high resolution, strong anti-interference ability, and can directly measure the relative distance and speed information of the target under all-weather conditions.

[0003] However, most of the current small rotor unmanned aerial vehicles fly in low-altitude airspace, which has the problems of small radar cross section (RCS), weak radar echo signal, target signal easily submerged by noise and clutter or covered by strong RCS targets, etc., making detection and tracking difficult, resulting in poor effectiveness of target motion recognition. In addition, due to the highly flexible maneuverability of small rotor unmanned aerial vehicles, the length of the input tracking trajectory and the output motion pattern sequence may be different, and it is difficult to accurately align, resulting in weak segmentation characteristics of the input trajectory stream. For existing methods that require explicit pre-segmentation, it is difficult to map the input tracking trajectory to the corresponding output motion pattern sequence, further resulting in poor accuracy of small rotor unmanned aerial vehicle target motion recognition. Therefore, how to improve the effectiveness and accuracy of small rotor unmanned aerial vehicle target motion recognition is a technical problem that needs to be solved. SUMMARY

[0004] In view of the deficiencies of the prior art, the technical problem to be solved by the present application is to provide a small rotor unmanned aerial vehicle target motion recognition method based on 4D radar, which can ensure the accuracy of small rotor unmanned aerial vehicle target detection and tracking in cluttered environments, and can recognize the motion of small rotor unmanned aerial vehicles without pre-segmentation, thereby improving the effectiveness and accuracy of small rotor unmanned aerial vehicle target motion recognition.

[0005] In order to solve the above technical problems, the technical scheme adopted by the present application is as follows:

[0006] The method for target motion recognition of small rotor unmanned aerial vehicle based on 4D radar comprises:

[0007] S1: monitoring the motion of the small rotor unmanned aerial vehicle as a target through the 4D radar;

[0008] S2: calculating the energy signal of the 4D radar;

[0009] S3: judging whether the energy signal of the 4D radar exceeds a preset threshold: if yes, extracting the corresponding radar measurement information from the energy signal; otherwise, returning to step S2;

[0010] S4: realizing the continuous tracking of the target through a probability data association algorithm according to the extracted multi-frame radar measurement information, and generating a tracking 3D trajectory;

[0011] S5: inputting the tracking 3D trajectory into a trained 3D projection-based long short-term memory-connectionist temporal classification network model, and outputting a target continuous motion recognition result.

[0012] Preferably, the energy signal of the 4D radar is calculated through the following steps:

[0013] S201: obtaining the target echo signal of the small rotor unmanned aerial vehicle received by each antenna array of the 4D radar;

[0014] S202: mixing the target echo signal with the transmission signal generated by the synthesizer of the 4D radar to generate a beat signal;

[0015] S203: performing four-dimensional fast Fourier transform processing on the beat signal to generate an energy signal containing the distance information, velocity information, azimuth angle information and pitch angle information of the small rotor unmanned aerial vehicle.

[0016] Preferably, the target echo signal is represented by the following formula:

[0017] ;

[0018] In the formula: represents the target echo signal; represents the fast time, represents the fast time sampling serial number, represents the total number of samples, represents the sampling interval, represents the pulse duration; represents the Doppler dimension sampling time, represents the pulse accumulation number, represents the number of pulses; represents the signal amplitude; represents the initial frequency; represents the signal time delay, denotes the speed of light, denotes the target initial range, denotes the target velocity; denotes the pulse signal slope.

[0019] Preferably, the transmitted signal is represented by the following equation:

[0020] ;

[0021] where: denotes the transmitted signal.

[0022] Preferably, the beat signal is represented by the following equation:

[0023] ;

[0024] where: denotes the beat signal; denotes the index of the horizontal antenna used for azimuth angle estimation; denotes the number of horizontal antennas used for azimuth angle estimation; denotes the index of the vertical antenna used for elevation angle estimation, denotes the number of vertical antennas used for elevation angle estimation; denotes the virtual receiving antenna spacing; denotes the target azimuth angle; denotes the target elevation angle.

[0025] Preferably, the energy signal is represented by the following equation:

[0026] ;

[0027] where: denotes the energy signal of the th frame; denotes a complex constant; denotes the target range of the th frame; denotes the range frequency variable; denotes the target velocity of the th frame; denotes the Doppler frequency variable; denotes the target azimuth angle of the th frame; denotes the azimuth angle frequency variable; denotes the target elevation angle of the th frame; denotes the elevation angle frequency variable.

[0028] Preferably, the processing steps of the probabilistic data association algorithm are as follows:

[0029] S401: initialize the target state and the corresponding probability distribution according to the initial measurement information;

[0030] S402: perform state prediction according to the dynamic model of the target, and obtain the predicted position and velocity at the next time;

[0031] S403: associate the radar measurement information at the current time with the predicted position, and update the probability distribution of the target state by calculating the association probability;

[0032] Among them, The probability that the measurement at time t comes from the target is represented as:

[0033]

[0034] The probability that there is no measurement at time t from the target is represented as:

[0035]

[0036] In the formula: The probability that the measurement at time t comes from the target is represented as: The probability that the measurement at time t comes from the target is represented as: The probability that there is no measurement at time t from the target is represented as: The probability that there is no measurement at time t from the target is represented as: , The measurement value at time t is represented as: The measurement innovation sequence is represented as: The covariance of the measurement innovation is represented as: The number of confirmed measurements at time t is represented as: , The spatial density of interference clutter is represented as: The target detection probability is represented as: The probability that the correct measurement falls into the tracking gate is represented as: S404: filter the measurement by the filtering algorithm and further update the probability distribution of the target state; S405: estimate the trajectory of the target according to the probability distribution of the target state, and then fuse the estimated target position and velocity to generate a tracking 3D trajectory.

[0037] The processing steps of the preferred 3D projection-based long short-term memory-connectionist temporal classification network model are as follows:

[0038] S501: feature extraction is performed on the input tracking 3D trajectory to generate a 3D trajectory feature;

[0039]

[0040] S501: feature extraction is performed on the input tracking 3D trajectory to generate a 3D trajectory feature;

[0041] ​​​​​​S502: coordinate decomposition is performed on the 3D trajectory feature through a 3D projection layer to generate a 3D coordinate decomposition feature;

[0042] S503: local feature extraction is performed on the 3D coordinate decomposition feature through a CNN network layer to generate a local spatio-temporal feature;

[0043] S504: global time modeling is performed on the local spatio-temporal feature through an LSTM network layer to generate a hidden state vector;

[0044] S505: the hidden state vector is input into a softmax layer to estimate a class conditional motion probability;

[0045] S506: a CTC network layer performs decoding operation on the class conditional motion probability through a greedy search algorithm to generate a continuous motion recognition result of the target small rotor unmanned aerial vehicle.

[0046] Preferably, the 3D coordinate decomposition of the 3D projection layer is represented as:

[0047] ;

[0048] In the formula: represents the 3D projection layer; represents the 3D coordinate decomposition feature; represents a target position coordinate vector at a time step ; represents the dimension of the 3D coordinate decomposition feature .

[0049] Preferably, the CTC network layer obtains a label sequence probability from a forward variable and a backward variable through a dynamic programming algorithm to calculate a CTC loss, and uses a back propagation algorithm for parameter updating;

[0050] The CTC loss is calculated through the following formula:

[0051] ;

[0052] In the formula: represents the CTC loss; represents the obtained label sequence probability; represents a continuous tracking 3D trajectory used as a training sample; represents a real continuous motion recognition result, i.e., a class label sequence.

[0053] Compared with the prior art, the small rotor unmanned aerial vehicle target motion recognition method based on the 4D radar in the application has the following beneficial effects:

[0054] The present application monitors the movement of a small rotor unmanned aerial vehicle through a 4D radar, extracts radar measurement information to generate a tracking 3D trajectory when the energy signal of the 4D radar exceeds a preset threshold, and then outputs the continuous movement recognition result of the small rotor unmanned aerial vehicle target through a long short-term memory-connectionist temporal classification network model based on 3D projection. On the one hand, the present application calculates the energy signal of the 4D radar and extracts radar measurement information based on threshold decision processing to solve the problems of small radar scattering cross section of the small rotor unmanned aerial vehicle target, difficulty in target detection and tracking in a low signal-to-noise ratio environment, etc., so that radar measurement information in high energy signals exceeding the threshold can be extracted to generate a tracking 3D trajectory (the higher the signal energy, the lower the energy of noise and clutter signals), thereby suppressing the interference of noise and clutter and ensuring the accuracy of small rotor unmanned aerial vehicle target detection and tracking in a clutter environment, thereby improving the effectiveness of small rotor unmanned aerial vehicle target movement recognition. On the other hand, the present application realizes the generation of a tracking 3D trajectory and the recognition of continuous movement through a probability data association algorithm and a long short-term memory-connectionist temporal classification network model based on 3D projection, wherein the probability data association algorithm has the advantages of fusion measurement and prediction, processing incomplete and uncertain data, effective maintenance of target information, adaptability to dynamic environment, and strong scalability in tracking 3D trajectory, which can improve the accuracy and robustness of small rotor unmanned aerial vehicle target movement recognition; at the same time, the long short-term memory-connectionist temporal classification network model based on 3D projection has the advantages of realizing three-dimensional information fusion, high data operation efficiency, strong generalization ability, and good real-time performance in small rotor unmanned aerial vehicle target continuous movement recognition, which can recognize the movement of small rotor unmanned aerial vehicle targets without pre-segmentation, and also improve the accuracy and robustness of small rotor unmanned aerial vehicle target continuous movement recognition. BRIEF DESCRIPTION OF DRAWINGS

[0055] In order to make the purpose, technical scheme and advantages of the application clearer, the application will be further described in detail below with reference to the drawings, in which:

[0056] Figure 1 The logic block diagram of the small rotor unmanned aerial vehicle target movement recognition method based on the 4D radar;

[0057] Figure 2 The principle flowchart of the small rotor unmanned aerial vehicle target movement recognition method based on the 4D radar;

[0058] Figure 3 The principle flowchart of the probability data association algorithm;

[0059] Figure 4 The principle flowchart of the long short-term memory-connectionist temporal classification network model based on 3D projection. DETAILED DESCRIPTION

[0060] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will be combined with the accompanying drawings of the embodiments of the present application to make a clear and complete description of the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0061] It should be noted that: similar reference numerals and letters represent similar items in the following drawings, therefore, once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. In the description of the present application, it needs to be explained that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship when the product of the present application is usually placed, which is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the indicated device or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second", "third" and the like are only used for differentiation in description, and cannot be understood as indicating or implying relative importance. In addition, the terms "horizontal", "vertical" and the like do not mean that the components must be absolutely horizontal or vertical, but can be slightly inclined. For example, "horizontal" only means that its direction is relatively more horizontal than "vertical", and does not mean that the structure must be completely horizontal, but can be slightly inclined. In the description of the present application, it also needs to be explained that, unless otherwise explicitly specified and limited, the terms "arrangement", "installation", "connection", "connection" should be understood broadly, for example, it can be fixedly connected, or detachably connected, or integrally connected; it can be mechanically connected, or electrically connected; it can be directly connected, or indirectly connected through an intermediate medium, or the communication inside two elements. For those of ordinary skill in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0062] The following will be further described in detail through specific embodiments:

[0063] Embodiment:

[0064] A small rotor unmanned aerial vehicle target motion recognition method based on 4D radar is disclosed in the embodiment.

[0065] AsFigure 1 and Figure 2 As shown in the figure, the small rotor unmanned aerial vehicle target motion recognition method based on 4D radar comprises:

[0066] S1: monitoring the motion of the small rotor unmanned aerial vehicle as a target through the 4D radar;

[0067] S2: calculating the energy signal of the 4D radar;

[0068] S3: judging whether the energy signal of the 4D radar exceeds a preset threshold: if yes, extracting the corresponding radar measurement information from the energy signal; otherwise, returning to step S2;

[0069] In this embodiment, the radar measurement information comprises distance information, speed information, azimuth angle information and pitch angle information.

[0070] S4: realizing target continuous tracking according to the extracted multi-frame radar measurement information through a probability data association algorithm, and generating a tracking 3D (Three Dimensional, 3D) trajectory;

[0071] S5: inputting the tracking 3D trajectory into a trained 3D projection-based long short-term memory-connectionist temporal classification network model, and outputting a target continuous motion recognition result.

[0072] In this embodiment, the continuous motion recognition result (i.e. continuous motion mode) comprises motion information of the small rotor unmanned aerial vehicle in the front, back, left, right, up and down directions.

[0073] The present application monitors the movement of a small rotor unmanned aerial vehicle through a 4D radar, extracts radar measurement information to generate a tracking 3D trajectory when the energy signal of the 4D radar exceeds a preset threshold, and then outputs the continuous movement recognition result of the small rotor unmanned aerial vehicle target through a 3D projection-based long short-term memory-connectionist temporal classification network model. On the one hand, the present application calculates the energy signal of the 4D radar and extracts radar measurement information based on threshold decision processing to solve the problems of small radar scattering cross section of the small rotor unmanned aerial vehicle target, difficulty in target detection and tracking in a low signal-to-noise ratio environment, etc., so that radar measurement information in high energy signals exceeding the threshold can be extracted to generate a tracking 3D trajectory (the higher the signal energy, the lower the energy of noise and clutter signals), thereby suppressing the interference of noise and clutter and ensuring the accuracy of small rotor unmanned aerial vehicle target detection and tracking in a clutter environment, thereby improving the effectiveness of small rotor unmanned aerial vehicle target movement recognition. On the other hand, the present application realizes the generation of a tracking 3D trajectory and the recognition of continuous movement through a probabilistic data association algorithm and a 3D projection-based long short-term memory-connectionist temporal classification network model. The probabilistic data association algorithm has the advantages of fusing measurement and prediction, processing incomplete and uncertain data, effectively maintaining target information, adapting to dynamic environments, and being highly scalable in tracking 3D trajectories, which can improve the accuracy and robustness of small rotor unmanned aerial vehicle target movement recognition. At the same time, the 3D projection-based long short-term memory-connectionist temporal classification network model has the advantages of realizing three-dimensional information fusion, high data operation efficiency, strong generalization ability, and good real-time performance in small rotor unmanned aerial vehicle target continuous movement recognition, which can recognize the movement of small rotor unmanned aerial vehicle targets without pre-segmentation and also improve the accuracy and robustness of small rotor unmanned aerial vehicle target continuous movement recognition.

[0074] In the implementation process, the energy signal of the 4D radar is calculated by the following steps:

[0075] S201: Obtain the target echo signal of the target small rotor unmanned aerial vehicle received by each antenna array of the 4D radar;

[0076] S202: Mix the target echo signal with the transmission signal generated by the synthesizer of the 4D radar (mixing in the mixer) to generate a beat signal;

[0077] S203: Perform four-dimensional fast Fourier transform (4D-FFT) processing on the beat signal to generate an energy signal containing distance information, velocity information, azimuth angle information, and pitch angle information of the target small rotor unmanned aerial vehicle.

[0078] In this embodiment, the radar measurement information extracted from the energy signal refers to radar measurement information containing distance information, velocity information, azimuth angle information and elevation angle information extracted from the signal peak of the energy signal.

[0079] 1) 4D radar

[0080] For a 4D radar with 3 transmitting antennas with interval and 4 receiving antennas with interval , a "L" shaped virtual antenna array with 12 elements with interval can be synthesized, where represents the wavelength.

[0081] The 4D radar adopts a multiple-input multiple-output technology, and the radar configuration mode is an interleaved mode. By selecting and , the following virtual array is obtained:

[0082] ;

[0083] In the formula: represents the position of the th transmitting antenna, represents the position of the th receiving antenna, represents the number of transmitting antennas, represents the number of receiving antennas. This is a uniform array with an interval of , and the interval is selected as to avoid spatial aliasing.

[0084] In this application, the three transmitting antennas of the 4D radar transmit mutually orthogonal frequency-modulated continuous wave signals, and the four receiving antennas extract the signals of each transmitting antenna by utilizing waveform orthogonality, to synthesize a "L" shaped virtual antenna array with twelve apertures. The horizontally arranged antennas in the array are used for azimuth angle estimation, and the vertically arranged antennas are used for elevation angle estimation, which is conducive to better obtaining the 3D motion trajectory of a small rotor unmanned aerial vehicle target.

[0085] 2) The target echo signal is represented by the following formula:

[0086] ;

[0087] In the formula: represents the target echo signal; represents fast time, represents the fast time sampling number, represents the total number of samples, represents the sampling interval, represents the pulse duration; represents the Doppler dimension sampling time, Indicates the cumulative number of pulses. Indicates the number of pulses; Indicates signal amplitude; Indicates the initial frequency; Indicates signal delay. Represents the speed of light. Indicates the initial distance to the target. Indicates the target speed; Indicates the slope of the pulse signal; It represents the imaginary unit.

[0088] In this invention, the 4D radar transmitter sends linear frequency modulated signals in a periodic manner. The transmitted signals are transmitted back after passing through a small rotary-wing UAV target. The target echo signal received by the radar receiver is the time delay of the transmitted signal.

[0089] 3) The transmit signal generated by the synthesizer can be represented by the following formula:

[0090] ;

[0091] In the formula: This indicates that a signal has been transmitted.

[0092] 4) The beat signal is represented by the following formula:

[0093] ;

[0094] In the formula: Indicates beat signal; Indicates the serial number of the horizontal antenna used for azimuth estimation; This indicates the number of horizontal antennas used for azimuth estimation; This indicates the number of the vertical antenna used for elevation angle estimation. This indicates the number of vertical antennas used for elevation angle estimation; Indicates the virtual receiving antenna spacing; Indicates the target azimuth angle; Indicates the target pitch angle.

[0095] In this invention, the beat signal of the 4D radar contains information on the target's range, speed, azimuth, and pitch angle. Subsequent algorithms can perform joint parameter estimation on the range, speed, azimuth, and pitch angle information of the small rotary-wing UAV target, thereby further improving the accuracy of target motion recognition for the small rotary-wing UAV.

[0096] 5) The energy signal is represented by the following formula:

[0097] ;

[0098] In the formula: represents the first frame energy signal; represents a complex constant; represents the first frame target distance; represents a distance frequency variable; represents the first frame target speed; represents a Doppler frequency variable; represents the first frame target azimuth angle; represents an azimuth angle frequency variable; represents the first frame target elevation angle; represents an elevation angle frequency variable.

[0099] Specifically:

[0100] After one-dimensional fast Fourier transform processing of the beat signal, the following is obtained:

[0101] ;

[0102] After two-dimensional fast Fourier transform processing of the beat signal, the following is obtained:

[0103] ;

[0104] After three-dimensional fast Fourier transform processing of the beat signal, the following is obtained:

[0105] ;

[0106] In the application, the 4D-FFT is used for coherent accumulation to accumulate target signal energy and improve the output signal-to-noise ratio, which is conducive to better judging the strength of the energy signal, so as to ensure the target tracking accuracy of the small rotor unmanned aerial vehicle in the clutter environment.

[0107] In the specific implementation process, the probabilistic data association algorithm (Probabilistic Data Association Algorithm, PDA) can be used for target continuous tracking, and target association and tracking are performed in combination with multi-frame radar measurement information. The PDA algorithm is based on the theory of Bayesian filtering, and the posterior probability of the target is calculated by fusing the radar measurement information and the target prediction model, so as to realize the tracking of the target.

[0108] As shown in Figure 3 , the processing steps of the probabilistic data association algorithm are as follows:

[0109] S401: Initialize the target state and the corresponding probability distribution according to the initial measurement information;

[0110] S402: According to the dynamic model of the target, the state prediction is performed to obtain the predicted position and velocity at the next time;

[0111] S403: The radar measurement information at the current time is associated with the predicted position, and the probability distribution of the target state is updated by calculating the association probability; this step is mainly used to determine the matching degree of the target and the measurement, and to update the state of the target by updating the weight.

[0112] Wherein, at The probability that the th measurement at the time originates from the target is represented as:

[0113] ;

[0114] The probability that there is no measurement at the time originating from the target is represented as:

[0115] ;

[0116] In the formula: represents the probability that the th measurement at the time originates from the target; represents the probability that there is no measurement at the time originating from the target; , represents the measurement value at the time; represents the measurement innovation sequence, represents the covariance of the measurement innovation; represents the number of confirmed measurements at the time; , represents the spatial density of interference clutter, represents the target detection probability, represents the probability that the correct measurement falls into the tracking gate; S404: The measurement is filtered by a filtering algorithm (such as Kalman filtering) and the probability distribution of the target state is further updated; this step is mainly used to eliminate measurements with large errors and improve the accuracy of tracking. S405: The trajectory of the target is estimated according to the probability distribution of the target state, and the estimated position and velocity of the target are fused to generate a 3D tracking trajectory.

[0117] S405: The trajectory of the target is estimated according to the probability distribution of the target state, and the estimated position and velocity of the target are fused to generate a 3D tracking trajectory.

[0118] S405: The trajectory of the target is estimated according to the probability distribution of the target state, and the estimated position and velocity of the target are fused to generate a 3D tracking trajectory.

[0119] ​​​In the present application, the probability data association algorithm considers that all measurement information may be derived from the tracked target, and only the probability of each measurement information derived from the target is different. In the case of not changing the radar hardware system, the target tracking performance in the clutter environment is significantly improved. At the same time, the probability data association algorithm has the advantages of fusing measurement and prediction, processing incomplete and uncertain data, effectively maintaining target information, adapting to dynamic environment and strong scalability in tracking 3D trajectory, which can improve the accuracy and robustness of small rotor unmanned aerial vehicle target motion recognition.

[0120] In the implementation process, in order to obtain sufficient information for small rotor unmanned aerial vehicle motion recognition, the 3D projection layer is proposed to expand the tracking trajectory features to higher dimensions. On this basis, the long short-term memory (LSTM) architecture is adopted, combined with convolutional neural network (CNN), which integrates the characteristics of CNN spatial expansion and LSTM time expansion, realizes local spatio-temporal feature extraction and global time modeling. In addition, the connectionist temporal classification (CTC) is used for continuous motion recognition of weak segmented small rotor unmanned aerial vehicle tracking trajectory stream.

[0121] In combination Figure 4 As shown in the figure, the processing steps of the long short-term memory-connectionist temporal classification network model based on 3D projection are as follows:

[0122] S501: feature extraction is performed on the input tracking 3D trajectory to generate 3D trajectory features;

[0123] S502: 3D coordinate decomposition features are generated by performing coordinate decomposition on the 3D trajectory features through the 3D projection layer;

[0124] S503: local spatio-temporal features are generated by performing local feature extraction on the 3D coordinate decomposition features through the CNN network layer;

[0125] S504: hidden state vectors are generated by performing global time modeling on the local spatio-temporal features through the LSTM network layer;

[0126] S505: the hidden state vectors are input into the softmax layer to estimate the class conditional motion probability;

[0127] S506: the CTC network layer performs decoding operation on the class conditional motion probability through the greedy search algorithm to generate the continuous motion recognition result of the target small rotor unmanned aerial vehicle.

[0128] 1) The 3D coordinate decomposition of the 3D projection layer is represented as:

[0129] ;

[0130] wherein: represents the 3D projection layer; represents the 3D coordinate decomposition feature; represents the target position coordinate vector at time step ; represents the dimension of the 3D coordinate decomposition feature ;

[0131] 2) The feature extraction of the CNN network layer is represented as:

[0132] ;

[0133] wherein: represents the CNN network layer; represents the local spatio-temporal feature; represents the 3D coordinate decomposition feature; represents the number of convolution kernels;

[0134] 3) The global time modeling of the LSTM network layer is represented as:

[0135] ;

[0136] wherein: represents the hidden state vector with hidden states; represents the local spatio-temporal feature; represents the hidden state vector of the previous time step; represents the linear rectified activation function; and represent the weight matrix.

[0137] 4) Greedy search algorithm

[0138] The greedy search algorithm is a commonly used decoding method, which is widely used in decoding CTC (Connectionist Temporal Classification) networks. CTC network is a neural network model used for sequence labeling tasks, whose output is a sequence of probability distributions after softmax operation, representing the probability distribution of labels at each time step. In decoding CTC network, the basic idea of greedy search algorithm is to select the label with the highest probability in the probability distribution at each time step as the output of the current time step, and then perform post-processing according to the requirements of CTC decoding. The specific steps are as follows:

[0139] 1. According to the output of the CTC network, obtain the label probability distribution at each time step.

[0140] 2. For each time step, select the label with the maximum label probability as the output of the current time step.

[0141] 3. Post-processing of CTC decoding, including operations such as de-duplication, merging consecutive identical labels, etc., to obtain the final decoding result.

[0142] 5) The CTC network layer calculates the CTC loss by obtaining the label sequence probability from the forward variable and the backward variable using the dynamic programming algorithm, and updates the parameters using the backpropagation algorithm; the training process of the CTC network layer iterates multiple batches to maximize the probability of the correct label sequence, and gradually learns to recognize continuous motion recognition results (continuous motion patterns) from continuous tracking 3D trajectories (i.e. tracking trajectory stream);

[0143] The CTC loss is calculated by the following formula:

[0144] ;

[0145] In the formula: represents the CTC loss; represents the obtained label sequence probability; represents the continuous tracking 3D trajectory of the small rotor unmanned aerial vehicle used as a training sample (i.e. training tracking trajectory stream); represents the true continuous motion recognition result of the small rotor unmanned aerial vehicle, i.e. the class label sequence.

[0146] Wherein, the label sequence probability can be obtained from the forward variable and the backward variable using the Viterbi algorithm.

[0147] First, define the forward variable alpha. For a given label sequence and observation sequence, alpha[t][i] represents the probability of the partial sequence ending with label i at time t.

[0148] 1. Initialize alpha:

[0149] For time step t=0, set alpha[0][i] = initial state probability * emission probability;

[0150] For other time steps t>0, set alpha[t][i] = sum(alpha[t-1][j] * transition probability[j][i] * emission probability[i][observation sequence[t]]), where j represents the index of the previous label.

[0151] Define the backward variable beta. For a given label sequence and observation sequence, beta[t][i] represents the probability of the partial sequence starting with label i at time t.

[0152] 2. Initialize beta:

[0153] For time step t = T (length of the observation sequence), set beta[T][i] = 1, where i represents the index of the label.

[0154] For other time steps t < T, set beta[t][i] = sum(beta[t+1][j] * transition probability[i][j] * emission probability[j][observation sequence[t+1]]), where j represents the index of the next label.

[0155] 3. Calculate the probability of the label sequence:

[0156] For a given observation sequence, the probability of the label sequence can be calculated by the joint probability distribution, i.e.:

[0157] P(observation sequence) = sum(alpha[T][i]), where i represents the index of the label.

[0158] It should be noted that, in order to avoid numerical underflow, the calculation can be performed using the logarithmic probability.

[0159] In the present application, the obtained tracking 3D trajectory features are expanded to higher dimensions through the 3D projection layer, the characteristics of CNN spatial expansion and LSTM time expansion are fused, local spatio-temporal feature extraction and global time modeling are realized, and the CTG loss function is used to recognize the continuous motion of the small rotor unmanned aerial vehicle under the condition of no pre-segmentation. At the same time, the model has the advantages of realizing three-dimensional information fusion, high data operation efficiency, strong generalization ability, good real-time performance and the like in the continuous motion recognition of small rotor unmanned aerial vehicles, and can recognize the motion of small rotor unmanned aerial vehicle targets without pre-segmentation, which can also improve the accuracy and robustness of the continuous motion recognition of small rotor unmanned aerial vehicles.

[0160] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit the technical solutions, and those of ordinary skill in the art should understand that those who modify or equivalently replace the technical solutions of the present application without departing from the purpose and scope of the technical solutions should be covered in the scope of the claims of the present application.

Claims

1. A target motion recognition method for small rotary-wing UAVs based on 4D radar, characterized in that, include: S1: Motion monitoring of small rotary-wing drones as targets using 4D radar; S2: Calculate the energy signal of the 4D radar; S3: Determine whether the energy signal of the 4D radar exceeds the preset threshold: if so, extract the corresponding radar measurement information from the energy signal; Otherwise, return to step S2; S4: Continuous target tracking is achieved by using a probabilistic data association algorithm based on the extracted multi-frame radar measurement information, generating a 3D tracking trajectory; S5: Input the tracked 3D trajectory into the trained 3D projection-based long short-term memory-connectionist temporal classification network model, and output the target continuous motion recognition result.

2. The target motion recognition method for small rotary-wing UAVs based on 4D radar as described in claim 1, characterized in that, In step S2, the energy signal of the 4D radar is calculated through the following steps: S201: Acquire the target echo signal of the small rotary-wing UAV received by each antenna array of the 4D radar; S202: Mix the target echo signal with the transmitted signal generated by the synthesizer of the 4D radar to generate a beat signal; S203: Performs four-dimensional fast Fourier transform processing on the beat signal to generate an energy signal containing distance, speed, azimuth, and pitch information of the target small rotary-wing UAV.

3. The target motion recognition method for small rotary-wing UAVs based on 4D radar as described in claim 2, characterized in that, In step S201, the target echo signal is represented by the following formula: ； In the formula: Indicates the target echo signal; Indicates a fast time. Indicates the fast time sampling sequence number. Indicates the total number of samples. Indicates the sampling interval. Indicates the pulse duration; Indicates the Doppler dimension sampling time. Indicates the cumulative number of pulses. Indicates the number of pulses; Indicates signal amplitude; Indicates the initial frequency; Indicates signal delay. Represents the speed of light. Indicates the initial distance to the target. Indicates the target speed; This represents the slope of the pulse signal.

4. The target motion recognition method for small rotary-wing UAVs based on 4D radar as described in claim 3, characterized in that, In step S202, the transmitted signal is represented by the following formula: ； In the formula: This indicates that a signal has been transmitted.

5. The target motion recognition method for a small rotary-wing UAV based on 4D radar as described in claim 4, characterized in that, In step S202, the beat signal is represented by the following formula: ； In the formula: Indicates beat signal; Indicates the serial number of the horizontal antenna used for azimuth estimation; This indicates the number of horizontal antennas used for azimuth estimation; This indicates the number of the vertical antenna used for elevation angle estimation. This indicates the number of vertical antennas used for elevation angle estimation; Indicates the virtual receiving antenna spacing; Indicates the target azimuth angle; Indicates the target pitch angle.

6. The target motion recognition method for a small rotary-wing UAV based on 4D radar as described in claim 5, characterized in that, In step S203, the energy signal is represented by the following formula: ； In the formula: Indicates the first Frame energy signal; Represents a complex constant; Indicates the first Frame target distance; Represents distance-frequency variables; Indicates the first Frame target speed; Indicates the Doppler frequency variable; Indicates the first Frame target azimuth angle; Indicates the azimuth frequency variable; Indicates the first Frame target pitch angle; This represents the pitch angle frequency variable.

7. The target motion recognition method for a small rotary-wing UAV based on 4D radar as described in claim 1, characterized in that: In step S4, the processing steps of the probabilistic data association algorithm are as follows: S401: Initialize the target state and corresponding probability distribution based on the initial measurement information; S402: Predict the state based on the target's dynamic model and obtain the predicted position and velocity at the next moment; S403: Associate the radar measurement information at the current moment with the predicted position, and update the probability distribution of the target state by calculating the association probability; Among them, Time of the first The probability that a measurement originates from a target is expressed as: ； exist The probability that no measurement is made at any given time due to the target is expressed as: ； In the formula: express Time of the first Each measurement originates from the probability of the target; express The probability of originating from the target is not measured at any given time; , express The measured value at a given time; Indicates the sequence of measured information. This represents the covariance of the measured information; express Always confirm the number of measurements; , This represents the spatial density of interfering clutter. This represents the probability of target detection. This indicates the probability of correctly measuring the value falling into the tracking gate; S404: The probability distribution of the target state is updated by filtering measurements using a filtering algorithm. S405: Estimate the target's trajectory based on the probability distribution of the target's state, and then fuse the estimated target position and velocity to generate a tracking 3D trajectory.

8. The target motion recognition method for a small rotary-wing UAV based on 4D radar as described in claim 1, characterized in that: In step S5, the processing steps of the 3D projection-based Long Short-Term Memory-Connectivist Temporal Classification Network model are as follows: S501: Extract features from the input tracking 3D trajectory to generate 3D trajectory features; S502: Perform coordinate decomposition on 3D trajectory features through a 3D projection layer to generate 3D coordinate decomposition features; S503: Local features are extracted from 3D coordinate decomposition features through CNN network layers to generate local spatiotemporal features; S504: The local spatiotemporal features are modeled globally in time using LSTM network layers to generate hidden state vectors; S505: Input the hidden state vector into the softmax layer to estimate the class-conditional motion probability; S506: The CTC network layer decodes the class-conditional motion probabilities using a greedy search algorithm to generate continuous motion recognition results for the target small rotary-wing UAV.

9. The target motion recognition method for a small rotary-wing UAV based on 4D radar as described in claim 8, characterized in that: In step S502, the 3D coordinate decomposition of the 3D projection layer is represented as follows: ； In the formula: Indicates a 3D projection layer; Represents 3D coordinate decomposition features; Indicates at time step The target position coordinate vector at that time; Representing 3D coordinate decomposition features Dimensions.

10. The target motion recognition method for a small rotary-wing UAV based on 4D radar as described in claim 9, characterized in that: In step S506, the CTC network layer obtains the label sequence probability from the forward and backward variables using a dynamic programming algorithm to calculate the CTC loss, and uses the backpropagation algorithm to update the parameters. CTC loss is calculated using the following formula: ； In the formula: Indicates CTC loss; This represents the probability of the obtained label sequence; This represents a continuously tracked 3D trajectory used as a training sample; This represents the true continuous motion recognition result, i.e., the class label sequence.

Images