A complex wing beat pattern recognition method based on multi-scale feature fusion
By employing a deep learning architecture that integrates multi-scale features, and utilizing short-time Fourier transform and neural networks to identify the wing flapping patterns of aerial organisms, this approach addresses the problem of insufficient feature extraction in traditional methods and achieves high-accuracy species identification.
Patent Information
- Application Number
- CN202211194079.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-09-28
- Publication Date
- 2026-02-13
- Estimated Expiration
- 2042-09-28
AI Technical Summary
Traditional methods for identifying aerial organisms rely on manually extracted features from time-domain echoes, which results in insufficient feature separability, making it impossible to effectively identify aerial organisms and also leads to the loss of phase information.
A deep learning architecture based on multi-scale feature fusion is adopted. The time-frequency analysis result map is obtained through short-time Fourier transform. After standardization, three target datasets of wing flapping patterns are selected, and a multi-scale fusion neural network structure is constructed for training and recognition.
It improved the recognition rate of wing flapping patterns of aerial biological targets, addressed the problem of insufficient feature extraction, and improved the accuracy of species identification.
Smart Images
Figure CN115565041B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to a wing vibration mode recognition method based on multi-scale feature fusion, in particular to an aerial biological wing vibration mode recognition method based on multi-scale feature fusion, and belongs to the technical field of insect radar. BACKGROUND
[0002] Radar is a powerful tool for studying aerial organisms (insects, birds, bats, etc.) due to its all-weather and all-day advantages. Radar has been applied to the study of insect, bird, and bat migration, bird and bat flow near airports and wind power plants, and aerial organism species identification. An important direction of radar aerial organism research is to quantify the number and species of targets in the airspace, but insects, birds, bats, and other flying objects and weather phenomena also provide a significant proportion of echo signals. It is not easy to distinguish a single species signal from a large number of echo signals. Obtaining single-species biological flux and flight trajectory parameters, predicting the flight direction of biological targets, and making early warnings are of great significance for preventing aviation bird and bat collision incidents and studying aerial organism migration theory.
[0003] Micro-Doppler is one of the important radar target characteristics. As a non-rigid target, the movement of different parts of the aerial organism target during its movement causes the radar echo to contain Doppler information representing the movement of these components. Studies have shown that the echo of an aerial organism target during flight will exhibit different fluctuation characteristics, and different wing vibration modes can be used to preliminarily identify the species.
[0004] Traditional aerial organism species identification research is mostly based on artificial feature extraction and machine learning methods to recognize the species based on time-domain echoes. Time-domain echoes lose phase information on the one hand, and the extraction of artificial features depends on the researchers' knowledge and understanding, which is insufficient for distinguishing between species and cannot achieve good species identification of aerial organisms. SUMMARY
[0005] The technical problem to be solved by the present application is to overcome the shortcomings of the prior art and provide a wing vibration mode recognition method based on multi-scale feature fusion. This method is based on a deep learning architecture with multi-scale feature fusion, which can recognize complex wing vibration modes of aerial organism targets, solve the problem of insufficient feature extraction, improve the recognition rate of aerial organism target wing vibration modes, and help to study the wing vibration modes of aerial organisms and further identify different species based on their flight characteristics.
[0006] A wing vibration mode recognition method based on multi-scale feature fusion, the steps of which include:
[0007] Step 1: Perform short-time Fourier transform on the time-domain echo to obtain a time-frequency analysis result graph.
[0008] Step two, the standardization processing is carried out to the time-frequency analysis result graph obtained in step one, and a standardization processing graph is obtained, and the obtained standardization processing graph is screened, and three wing vibration mode target data sets are obtained, and the three wing vibration mode target data sets are a sustained wing vibration mode target data set, a beat-stop wing vibration mode target data set and a random wing vibration mode target data set;
[0009] Step three, a multi-scale fusion neural network structure is constructed according to the three wing vibration mode target data sets obtained in step two;
[0010] Step four, the multi-scale fusion neural network structure obtained in step three is trained by using the three wing vibration mode target data sets obtained in step two;
[0011] Step five, the test data is imported into the neural network structure trained in step four, and a wing vibration mode recognition result is obtained.
[0012] In step one, the specific method for performing short-time Fourier transform on the time-domain echo is:
[0013] Supposing that the time-domain echo of the target at t time is s(t), it is assumed that the aerial biological signal is stable in the intercepted window, the signal in the window is analyzed by using Fourier transform to determine the frequency components of the signal in the window, then the window function is moved along the time direction of the signal, and finally the transformation relationship of the signal frequency with time is obtained, that is, the required time-frequency analysis result graph, and the mathematical expression form of the short-time Fourier transform (STFT) is:
[0014]
[0015] Wherein, s(t+τ) is the time translation of the echo signal, ω(·) is the window function, and τ is the length of the intercepted window;
[0016] In step two, the standardization processing of the time-frequency analysis result graph refers to the normalization processing of the slow time dimension of the time-frequency analysis result graph, and the processing result is:
[0017]
[0018] In step two, the screening method is that the sustained wing vibration mode refers to that the target has an overall Doppler frequency vibration rule, the beat-stop wing vibration mode refers to that the target has two flight modes of wing vibration and gliding, and the beat-stop wing vibration mode is obvious in the result, and the random wing vibration mode refers to that the target has a vibration characteristic, but has no obvious rule.
[0019] The step three, the constructed multi-scale fusion neural network structure includes a feature extraction part and a feature fusion part, the feature extraction part includes four residual blocks, the four residual blocks are sequentially connected, each residual block is composed of two convolution blocks, the convolution block is composed of a first convolution layer, an activation function and a second convolution layer, the output of the convolution block is represented as:
[0020] O(x) = I(x) + x
[0021] Where x is the input of the convolution block, I(x) is the output of the second convolution layer, and O(x) is the output of the convolution block;
[0022] The method for constructing the multi-scale fusion neural network structure, the steps include:
[0023] First, calculate the feature extraction results of the four residual blocks, the four residual blocks are the first residual block, the second residual block, the third residual block and the fourth residual block, the feature extraction result of the first residual block is F1, the feature extraction result of the second residual block is F2, the feature extraction result of the third residual block is F3, and the feature extraction result of the fourth residual block is F4;
[0024] Wherein, the two convolution blocks included in the first residual block are the first convolution block and the second convolution block, and the feature extraction result F1 of the first residual block is obtained by the following method:
[0025] (11) using the first convolution layer of the first convolution block to convolve the input x1 of the first convolution block to obtain the feature Wherein, L represents the number of channels of the feature, F 11 (i) represents the i-th channel in F, i = 1, 2,..., L, F 11 (i) is represented by the following formula:
[0026]
[0027] Wherein, Represents the convolution operation, Kernel(i) represents the i-th convolution kernel, b Li Represents the offset of the i-th convolution kernel;
[0028] (12) using a nonlinear activation function to perform nonlinear mapping on the feature F 11 Obtained by the first convolution layer, to obtain the feature F 12 , and input the obtained feature F 12 Into the second convolution layer, using the second convolution layer to convolve the feature F 12 , to obtain the feature F 13 ;
[0029] (13) the feature F 13The output of the first convolutional block is obtained by adding the first convolutional block input x1.
[0030] Similarly, the method for obtaining the output of the second convolutional block in the first residual block is as follows:
[0031] (14) Convolution is performed on the output of the first convolutional block using the first convolutional layer of the second convolutional block to obtain a feature F wherein L represents the number of channels of the feature, F 14 (i) represents the i-th channel in F, i = 1, 2,..., L, F 14 (i) is represented by the following formula:
[0032]
[0033] wherein, represents a convolution operation, Kernel(i) represents the i-th convolution kernel, b Li represents the offset of the i-th convolution kernel;
[0034] (15) The feature F 14 obtained by the first convolutional layer of the second convolutional block is subjected to nonlinear mapping using a nonlinear activation function to obtain a feature F 15 The obtained feature F 15 is input into the second convolutional layer of the second convolutional block, and the feature F 15 is subjected to convolution using the second convolutional layer of the second convolutional block to obtain a feature F 16 .
[0035] (16) The feature F 16 obtained in step (15) is added to the output of the first convolutional block to obtain the feature extraction result of the first residual block as F1.
[0036] The feature extraction result of the first residual block is taken as F1 as the input of the first convolutional block in the second residual block, and the method of steps (11)-(13) is adopted to obtain the output of the first convolutional block in the second residual block. The output of the first convolutional block in the second residual block is taken as the input of the second convolutional block in the second residual block, and the method of steps (14)-(16) is adopted to obtain the feature extraction result of the second residual block as F2.
[0037] The feature extraction result of the second residual block is taken as F2 as the input of the first convolutional block in the third residual block, and the method of steps (11)-(13) is adopted to obtain the output of the first convolutional block in the third residual block. The output of the first convolutional block in the third residual block is taken as the input of the second convolutional block in the third residual block, and the method of steps (14)-(16) is adopted to obtain the feature extraction result of the third residual block as F3.
[0038] Taking the feature extraction result of the third residual block F3 as the input of the first convolutional block in the fourth residual block, taking steps (11)-(13), the output of the first convolutional block in the fourth residual block is obtained, and then taking the output of the first convolutional block in the fourth residual block as the input of the second convolutional block in the fourth residual block, taking steps (14)-(16), the feature extraction result F4 of the fourth residual block is obtained.
[0039] Secondly, the feature extraction results F1, F2 and F3 of the first, second and third residual blocks are adjusted to the same number of channels as the number of channels of the feature extraction result F4 of the fourth residual block through a 1x1 convolutional block, and then the adjusted feature extraction results F'1, F'2, F'3 and F4 of the first, second, third and fourth residual blocks are respectively input into the adaptive sliding window pooling layer, and the feature map sizes of the feature extraction results F'1, F'2, F'3 and F4 of the first, second, third and fourth residual blocks are adjusted to be consistent.
[0040] Thirdly, the multi-scale features of the feature extraction result F'1 of the first residual block after the feature map size is adjusted to be consistent are stretched into a one-dimensional vector g1 by full connection, the multi-scale features of the feature extraction result F'2 of the second residual block after the feature map size is adjusted to be consistent are stretched into a one-dimensional vector g2 by full connection, the multi-scale features of the feature extraction result F'3 of the third residual block after the feature map size is adjusted to be consistent are stretched into a one-dimensional vector g3 by full connection, and the multi-scale features of the feature extraction result F'4 of the fourth residual block after the feature map size is adjusted to be consistent are stretched into a one-dimensional vector g4 by full connection.
[0041] Fourthly, a multi-scale attention mechanism is adopted to add and fuse the one-dimensional vectors g1, g2, g3 and g4 obtained in the third step to obtain the fused feature F g which is represented by the following formula:
[0042]
[0043] wherein N=1, 2, 3, 4, and a j represents the parameter of the network attention mechanism for learning the importance of the scale features;
[0044] Fifthly, the fused feature F g is linearly mapped through a full connection layer to obtain a vector C g of the total number of categories. gEnter the Softmax classification layer, get the recognition result, the i-th target in the sample belongs to each class probability as follows:
[0045]
[0046] Wherein, c represents the class serial number, C represents the total number of classes, after calculating the probability P(i|x) of the i-th target belonging to each class in turn, the maximum value is taken as the target class, and the construction of the multi-scale fusion neural network structure is completed.
[0047] The present application has the following beneficial effects:
[0048] (1) The present application is a multi-scale time-frequency feature extraction and fusion based aerial biological target wing beat pattern recognition method, which provides an effective means for distinguishing different wing beat pattern types.
[0049] (2) The present application first uses short-time Fourier transform to solve the time-frequency analysis graph of the target, then pre-processes the image, and finally designs a multi-scale feature fusion neural network structure to realize wing beat pattern recognition. Compared with existing recognition methods, the present method has higher accuracy.
[0050] (3) The present application discloses a multi-scale feature fusion based deep learning architecture, which realizes the recognition of complex wing beat patterns of aerial biological targets. Using the radar measured time domain echo of aerial biological targets, the short-time Fourier transform is used to solve the time-frequency analysis result of the target, then the picture is standardized, and a multi-scale feature extraction and fusion convolutional neural network is used for type recognition.
[0051] (4) The present application can realize the wing beat pattern recognition of aerial biological targets, improve the problem of insufficient feature extraction in traditional methods, and improve the target type recognition accuracy. This has important significance for further studying the radar characteristics and type recognition of aerial biological targets. BRIEF DESCRIPTION OF DRAWINGS
[0052] Figure 1 The time-frequency analysis result graph of the Doppler frequency of the continuous wing beat pattern target changing with time;
[0053] Figure 2 The time-frequency analysis result graph of the Doppler frequency of the beat wing beat pattern target changing with time;
[0054] Figure 3 The time-frequency analysis result graph of the Doppler frequency of the irregular pattern target changing with time;
[0055] Figure 4 The residual convolution block schematic diagram;
[0056] Figure 5A schematic diagram of a multi-scale feature extraction and fusion neural network structure;
[0057] Figure 6 The accuracy of the wing vibration mode network training and testing. DETAILED DESCRIPTION
[0058] The application provides a deep learning architecture based on multi-scale feature fusion, realizes identification of complex wing vibration modes of aerial biological targets, and has the following basic ideas: firstly, time-frequency analysis result images of the targets are obtained by performing short-time Fourier transform on biological target time domain echoes; secondly, standardized inputs are generated by preprocessing the images; and finally, a multi-scale feature extraction and fusion neural network structure is designed to realize high-precision identification of different wing vibration mode targets.
[0059] A wing vibration mode identification method based on multi-scale feature fusion, and steps of the method include:
[0060] Step one: performing short-time Fourier transform on time domain echoes to obtain time-frequency analysis result images;
[0061] Step two: performing standardization processing on the time-frequency analysis result images obtained in step one to obtain a standardized processing image, and performing screening on the obtained standardized processing image to obtain three wing vibration mode target data sets, wherein the three wing vibration mode target data sets are a continuous wing vibration mode target data set, a beat-stop wing vibration mode target data set and a random wing vibration mode target data set;
[0062] Step three: constructing a multi-scale fusion neural network structure according to the three wing vibration mode target data sets obtained in step two;
[0063] Step four: training the multi-scale fusion neural network structure obtained in step three by using the three wing vibration mode target data sets obtained in step two;
[0064] Step five: importing to-be-tested data into the neural network structure trained in step four to obtain wing vibration mode identification results.
[0065] In step one, the specific method for performing short-time Fourier transform on time domain echoes is as follows:
[0066] Supposing that the time domain echo of a target at t time is s(t), it is assumed that the aerial biological signal is stable in the intercepted window, the signal in the window is analyzed by using Fourier transform to determine the frequency components of the signal in the window, then the window function is moved along the time direction of the signal, and finally the transformation relationship of the signal frequency with time, i.e., the required time-frequency analysis result image, is obtained, and the mathematical expression form of the short-time Fourier transform (STFT) is as follows:
[0067]
[0068] Wherein, s(t+τ) is the time shift of echo signal, ω(·) is window function, τ is the length of the window;
[0069] The step two, the standardization processing of the time-frequency analysis result image is to process and normalize the slow time dimension of the time-frequency analysis result image, and the processing result is:
[0070]
[0071] The step two, the screening method is: the continuous wing vibration mode means that the target has a whole Doppler frequency vibration rule, the wing vibration and gliding mode means that the target has wing vibration and gliding two stages, and the wing vibration mode without rule means that the target has vibration characteristics, but there is no obvious rule;
[0072] The step three, the multi-scale fusion neural network structure includes a feature extraction part and a feature fusion part, the feature extraction part includes four residual blocks, the four residual blocks are sequentially connected, each residual block is composed of two convolution blocks, the convolution block is composed of a first convolution layer, an activation function and a second convolution layer, and the output of the convolution block is represented as:
[0073] O(x)=I(x)+x
[0074] Wherein x is the input of the convolution block, I(x) is the output of the second convolution layer, and O(x) is the output of the convolution block;
[0075] The method for constructing the multi-scale fusion neural network structure, comprising the following steps:
[0076] Firstly, the feature extraction results of the four residual blocks are calculated, the four residual blocks are a first residual block, a second residual block, a third residual block and a fourth residual block, the feature extraction result of the first residual block is F1, the feature extraction result of the second residual block is F2, the feature extraction result of the third residual block is F3, and the feature extraction result of the fourth residual block is F4;
[0077] Wherein, the two convolution blocks included in the first residual block are a first convolution block and a second convolution block, and the feature extraction result F1 of the first residual block is obtained by:
[0078] (11) using the first convolution layer of the first convolution block to convolve the input x1 of the first convolution block to obtain the feature Wherein, L represents the channel number of the feature, F 11 (i) represents the i-th channel in F, i=1, 2,..., L, F 11 (i) is represented by the following formula:
[0079]
[0080] wherein, represents a convolution operation, Kernel(i) represents the ith convolution kernel, b Li represents the offset of the ith convolution kernel;
[0081] (12) using a nonlinear activation function to perform nonlinear mapping on the feature F 11 obtained by the first convolution layer, to obtain a feature F 12 , and inputting the obtained feature F 12 into the second convolution layer, using the second convolution layer to perform convolution on the feature F 12 , to obtain a feature F 13 ;
[0082] (13) adding the feature F 13 obtained in step (12) to the first convolution block input x1, to obtain the output of the first convolution block;
[0083] Similarly, the method for obtaining the output of the second convolution block in the first residual block is as follows:
[0084] (14) using the first convolution layer of the second convolution block to perform convolution on the output of the first convolution block, to obtain a feature wherein, L represents the number of channels of the feature, F 14 (i) represents the ith channel in F, i = 1, 2, …, L, F 14 (i) is represented by the following formula:
[0085]
[0086] wherein, represents a convolution operation, Kernel(i) represents the ith convolution kernel, b Li represents the offset of the ith convolution kernel;
[0087] (15) using a nonlinear activation function to perform nonlinear mapping on the feature F 14 obtained by the first convolution layer of the second convolution block, to obtain a feature F 15 , and inputting the obtained feature F 15 into the second convolution layer of the second convolution block, using the second convolution layer of the second convolution block to perform convolution on the feature F 15 , to obtain a feature F 16 ;
[0088] (16) adding the feature F 16 obtained in step (15) to the output of the first convolution block, to obtain the feature extraction result of the first residual block as F1;
[0089] The feature extraction result of the first residual block is taken as F1 as the input of the first convolutional block in the second residual block, the method of steps (11)-(13) is taken, the output of the first convolutional block in the second residual block is obtained, and then the output of the first convolutional block in the second residual block is taken as the input of the second convolutional block in the second residual block, the method of steps (14)-(16) is taken, and the feature extraction result of the second residual block is obtained as F2;
[0090] The feature extraction result of the second residual block is taken as F2 as the input of the first convolutional block in the third residual block, the method of steps (11)-(13) is taken, the output of the first convolutional block in the third residual block is obtained, and then the output of the first convolutional block in the third residual block is taken as the input of the second convolutional block in the third residual block, the method of steps (14)-(16) is taken, and the feature extraction result of the third residual block is obtained as F3;
[0091] The feature extraction result of the third residual block is taken as F3 as the input of the first convolutional block in the fourth residual block, the method of steps (11)-(13) is taken, the output of the first convolutional block in the fourth residual block is obtained, and then the output of the first convolutional block in the fourth residual block is taken as the input of the second convolutional block in the fourth residual block, the method of steps (14)-(16) is taken, and the feature extraction result of the fourth residual block is obtained as F4;
[0092] Secondly, the feature extraction result F1 of the first residual block, the feature extraction result F2 of the second residual block and the feature extraction result F3 of the third residual block are adjusted to the same channel number as the channel number of the feature extraction result F4 of the fourth residual block through a 1x1 convolutional block, and then the adjusted feature extraction result F'1 of the first residual block, the feature extraction result F'2 of the second residual block, the feature extraction result F'3 of the third residual block and the feature extraction result F4 of the fourth residual block are respectively input into an adaptive sliding window pooling layer, and the feature map sizes of the feature extraction result F'1 of the first residual block, the feature extraction result F'2 of the second residual block, the feature extraction result F'3 of the third residual block and the feature extraction result F4 of the fourth residual block are adjusted to be consistent;
[0093] Thirdly, the multi-scale features of the feature extraction result F'1 of the first residual block after the feature map size is adjusted to be consistent are stretched into a one-dimensional vector g1 by full connection, the multi-scale features of the feature extraction result F'2 of the second residual block after the feature map size is adjusted to be consistent are stretched into a one-dimensional vector g2 by full connection, the multi-scale features of the feature extraction result F'3 of the third residual block after the feature map size is adjusted to be consistent are stretched into a one-dimensional vector g3 by full connection, and the multi-scale features of the feature extraction result F'4 of the fourth residual block after the feature map size is adjusted to be consistent are stretched into a one-dimensional vector g4 by full connection;
[0094] Fourthly, the one-dimensional vector g1, the one-dimensional vector g2, the one-dimensional vector g3 and the one-dimensional vector g4 obtained in the third step are added and fused by using a multi-scale attention mechanism to obtain a fused feature F g is represented by the following formula:
[0095]
[0096] wherein N = 1, 2, 3, 4, a j represents a parameter of importance learning of the network attention mechanism for the scale feature;
[0097] Fifthly, the fused feature F g is linearly mapped by a fully connected layer to obtain a vector C of the total number of categories g The vector C g enters a Softmax classification layer to obtain a recognition result, and the probability of the i-th target belonging to each category is as follows:
[0098]
[0099] wherein c represents a category serial number, C represents the total number of categories, the maximum value of the probability P(i|x) of the i-th target belonging to each category is taken after the probability P(i|x) of the i-th target belonging to each category is calculated in turn, and the construction of the multi-scale fusion neural network structure is completed.
[0100] The application will be described in detail below with reference to the accompanying drawings and embodiments.
[0101] The specific method for performing short-time Fourier transform on the time-domain echo in the step one is as follows:
[0102] Suppose that the time-domain echo of the target at the t time is s(t), and the air biological signal is assumed to be stationary in the intercepted window, then the Fourier transform is used to analyze the signal in the window to determine the frequency components of the signal in the window, and then the window function is moved along the time direction of the signal, and finally the transformation relationship of the signal frequency with time is obtained, that is, the required time-frequency analysis result graph, and the mathematical expression form of the short-time Fourier transform (STFT) is as follows:
[0103]
[0104] wherein s(t+τ) is the time translation of the echo signal, ω(·) is the window function, and τ is the length of the intercepted window;
[0105] The standardization processing of the time-frequency analysis result graph in the step two refers to the processing and normalization of the slow time dimension of the time-frequency analysis result graph, and the processing result is as follows:
[0106]
[0107] wherein STFT raw (t) is the Doppler vector at time t, STFT normalization (t) is the normalized time-frequency analysis result.
[0108] The method for screening in step two is as follows: the continuous wing vibration mode refers to the target having an overall Doppler frequency vibration rule, the wing vibration and gliding mode refers to the target having two flight stages of wing vibration and gliding, and the irregular wing vibration mode refers to the target having vibration characteristics but no obvious rule. The normalized time-frequency analysis result graphs of the three wing vibration modes are shown in Figure 1 、 Figure 2 and Figure 3 respectively.
[0109] The method for constructing the multi-scale fusion neural network structure in step three is as follows: the feature extraction part is connected by four residual convolution blocks, each residual block is composed of two residual convolution blocks, and the residual convolution block is composed of a convolution layer-activation function-convolution layer, as shown in Figure 4 The output of the residual convolution block can be expressed as:
[0110] O(x) = I(x) + x
[0111] wherein x is the input of the convolution block, I(x) is the output of the last convolution layer in the convolution block, and O(x) is the output of the convolution block. The convolution layer convolves the input to obtain features wherein L represents the number of channels of the features, and F(i) represents the ith channel in F. F(i) can be expressed as:
[0112]
[0113] wherein represents a convolution operation, Kernel(i) represents the ith convolution kernel, and b Li represents the offset of the ith convolution kernel.
[0114] For the feature vector F j extracted by each deep convolution block, the outputs of the first three convolution blocks are first adjusted to the same number of channels as that of the last convolution block through a 1x1 convolution module, and then respectively enter an adaptive sliding window pooling layer to adjust the feature map size to be consistent, and then the multi-scale features are respectively stretched into one-dimensional vectors g j by full connection, a multi-scale attention mechanism is introduced to fuse the features of each scale, and the fused features F g are expressed as:
[0115]
[0116] wherein, N represents the total number of scales, a represents the scale weight, and x represents the input feature vector. j represents the parameter of the network attention mechanism for learning the importance of the scale feature.
[0117] Finally, the fused features are linearly mapped by a fully connected layer to obtain a vector C of the total number of classes g , which is input into a Softmax classification layer to obtain the recognition result. The probability that the i-th target in the sample belongs to each class is as follows:
[0118]
[0119] wherein, c represents the class number, C represents the total number of classes, and the maximum value of the probability P(i|x) that the i-th target belongs to each class is taken as the class to which the target belongs. The overall architecture of the network is as shown in Figure 5 .
[0120] In order to verify the target wingbeat pattern recognition method described above, 1800 groups of target echo data sets (600 for each class) based on experimental measurement are used to complete the pattern recognition by using the air biological target wingbeat pattern recognition method based on time-frequency feature extraction and fusion network, and the accuracy rate of the training set and the test set with the training number is calculated.
[0121] The accuracy rate of the air biological target wingbeat pattern recognition method based on the multi-scale feature extraction and fusion network is as shown in Figure 6 . The recognition accuracy of each target type of the method is above 98.
[0122] The method is applicable to the radar of the measurable air biological target echo, realizes the wingbeat pattern recognition of the air biological target, improves the problem of insufficient feature extraction in the traditional method, and improves the wingbeat pattern recognition accuracy of the target.
[0123] In summary, the above is only an embodiment of the present application based on 1800 air biological target data measured by the present application, and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.
Claims
1. A wingbeat pattern recognition method based on multi-scale feature fusion, characterized in that The steps of the method include: Step one, short-time Fourier transform is performed on the time domain echo to obtain a time-frequency analysis result graph; Step two, the time-frequency analysis result graph obtained in step one is standardized to obtain a standardized processing graph, and the obtained standardized processing graph is screened to obtain three wing vibration mode target data sets, which are a sustained wing vibration mode target data set, a beat-stop wing vibration mode target data set, and a random wing vibration mode target data set; Step three, a multi-scale fusion neural network structure is constructed according to the three wing vibration mode target data sets obtained in step two; Step four, the multi-scale fusion neural network structure obtained in step three is trained using the three wing vibration mode target data sets obtained in step two; Step five, the test data is imported into the neural network structure trained in step four to obtain the recognition result of the wing vibration mode; In step three, the multi-scale fusion neural network structure includes a feature extraction part and a feature fusion part, the feature extraction part includes four residual blocks, the four residual blocks are sequentially connected, each residual block is composed of two convolution blocks, the convolution block is composed of a first convolution layer, an activation function and a second convolution layer, and the output of the convolution block is represented as: wherein x is the output of the convolutional block, is the output of the second convolutional layer, is the output of the convolutional block; The method for constructing a multi-scale fusion neural network structure includes: First step, calculate the feature extraction results of the four residual blocks, the four residual blocks are the first residual block, the second residual block, the third residual block and the fourth residual block, the feature extraction result of the first residual block is F1, the feature extraction result of the second residual block is F2, the feature extraction result of the third residual block is F3, and the feature extraction result of the fourth residual block is F4; Second step, adjust the feature extraction results F1, F2 and F3 of the first, second and third residual blocks to the same channel number as the channel number of the feature extraction result F4 of the fourth residual block through a 1x1 convolution block, then adjust the feature extraction results F'1, F'2 and F'3 of the first, second and third residual blocks to the same feature map size as the feature extraction result F4 of the fourth residual block, and then respectively input the adjusted feature extraction results F'1, F'2, F'3 and F4 of the first, second, third and fourth residual blocks into an adaptive sliding window pooling layer; In the third step, the multi-scale features of the feature extraction result F'1 of the first residual block after the feature maps are adjusted to have consistent sizes are stretched into one-dimensional vectors by full connection In the third step, the multi-scale features of the feature extraction result F'1 of the first residual block after the feature maps are adjusted to have consistent sizes are stretched into one-dimensional vectors by full connection In the third step, the multi-scale features of the feature extraction result F'1 of the first residual block after the feature maps are adjusted to have consistent sizes are stretched into one-dimensional vectors by full connection In the third step, the multi-scale features of the feature extraction result F'1 of the first residual block after the feature maps are adjusted to have consistent sizes are stretched into one-dimensional vectors by full connection ; In the fourth step, a multi-scale attention mechanism is used to add and fuse the one-dimensional vectors obtained in the third step one-dimensional vector one-dimensional vector one-dimensional vector to obtain the fused features which are represented by the following formula: wherein N = 1, 2, 3, 4, represent parameters of the network attention mechanism for learning the importance of the scale features. Fifth step, the fused features obtained in the fourth step Linear mapping through the fully connected layer to obtain a vector of the total number of categories , vector Enter the Softmax classification layer to obtain the recognition result, vector The probability that the nth node in vector belongs to each category is as follows: Wherein, c represents the category number, C represents the total number of categories, c = 1, 2, …, C, the probability of the output value of the nth node (n = 1, 2, …, C) belonging to each category is calculated in turn After that, the maximum value in it is the target belonging category, and the construction of the multi-scale fusion neural network structure is completed.
2. The wing vibration mode recognition method based on multi-scale feature fusion according to claim 1, wherein: In step one, the specific method for performing short-time Fourier transform on the time domain echo is: Targeted t The time domain echo is The short-time Fourier transform is a time-frequency analysis method, which uses Fourier transform to analyze the signal in the window, then moves the window function along the time direction of the signal, and finally obtains the transform relationship of the signal frequency with time, that is, the required time-frequency analysis result graph. The mathematical expression of short-time Fourier transform is: wherein is a time shift of the echo signal, is a window function of length is the length of the window, is the length of the window, is the signal angular frequency.
3. The wing vibration mode recognition method based on multi-scale feature fusion according to claim 1 or 2, wherein: In step two, the standardization processing of the time-frequency analysis result graph refers to the processing and normalization of the slow time dimension of the time-frequency analysis result graph, and the processing result is: ； wherein is the original short-time Fourier transform result vector at time t.
4. The wing vibration mode recognition method based on multi-scale feature fusion according to claim 3, wherein: The method for screening in the second step is that the continuous wing vibration mode refers to the target having an overall Doppler frequency vibration rule, the wing vibration and gliding mode refers to the target having two stages of wing vibration and gliding, and the irregular wing vibration mode refers to the target having a vibration characteristic.
5. The wing vibration mode recognition method based on multi-scale feature fusion according to claim 4, wherein: The method for obtaining the feature extraction result F1 of the first residual block comprises two convolution blocks, namely a first convolution block and a second convolution block, and the steps are as follows: (11) performing convolution on the input x1 of the first convolution block using a first convolution layer of the first convolution block to obtain a feature where L represents a number of channels of the feature, represents an i-th channel in F, i = 1, 2, …, L, is represented by the following formula: wherein, represents a convolution operation, represents the i-th convolution kernel, represents an offset for the i-th convolution kernel; (12) using a nonlinear activation function on the features F obtained by the first convolutional layer 11 performing nonlinear mapping to obtain features F 12 , and inputting the obtained features F 12 to the second convolutional layer, using the second convolutional layer to perform convolution on the features F 12 to obtain features F 13 ; (13) the feature F obtained in step (12) is input into a second convolutional block 13 is added to the first convolutional block input x1 to obtain the output of the first convolutional block; (14) performing convolution on the output of the first convolution block using a first convolution layer of a second convolution block to obtain features where L represents a number of channels of the features, represents an i-th channel in F, i = 1, 2, …, L, which is represented by the following formula: wherein, represents a convolution operation, represents the i-th convolution kernel, represents an offset for the i-th convolution kernel; (15) using a nonlinear activation function on the features F obtained by the first convolutional layer of the second convolutional block 14 performing nonlinear mapping to obtain features F 15 , and inputting the obtained features F 15 to the second convolutional layer of the second convolutional block, and using the second convolutional layer of the second convolutional block to perform convolution on the features F 15 to obtain features F 16 ; (16) add the output of the first convolutional block to the feature F 16 The feature extraction result of the first residual block is F1.
6. The wing vibration mode recognition method based on multi-scale feature fusion according to claim 5, wherein: The method for obtaining the feature extraction result F2 of the second residual block comprises: The feature extraction result F1 of the first residual block is taken as the input of the first convolution block in the second residual block, the steps (11)-(13) are taken to obtain the output of the first convolution block in the second residual block, and then the output of the first convolution block in the second residual block is taken as the input of the second convolution block in the second residual block, the steps (14)-(16) are taken to obtain the feature extraction result F2 of the second residual block.
7. The wing vibration mode recognition method based on multi-scale feature fusion according to claim 6, wherein: The method for obtaining the feature extraction result F3 of the third residual block comprises: The feature extraction result F2 of the second residual block is taken as the input of the first convolution block in the third residual block, the steps (11)-(13) are taken to obtain the output of the first convolution block in the third residual block, and then the output of the first convolution block in the third residual block is taken as the input of the second convolution block in the third residual block, the steps (14)-(16) are taken to obtain the feature extraction result F3 of the third residual block.
8. The wing vibration mode recognition method based on multi-scale feature fusion according to claim 7, wherein: The method for obtaining the feature extraction result F4 of the fourth residual block comprises: The feature extraction result F3 of the third residual block is taken as the input of the first convolution block in the fourth residual block, the steps (11)-(13) are taken to obtain the output of the first convolution block in the fourth residual block, and then the output of the first convolution block in the fourth residual block is taken as the input of the second convolution block in the fourth residual block, the steps (14)-(16) are taken to obtain the feature extraction result F4 of the fourth residual block.