A firearm shooting recognition method based on BiLSTM long short-term memory recurrent neural network

Abstract

The present invention relates to a method for identifying firearm shootings based on a BiLSTM long short-term memory recurrent neural network. It includes: Step S1, collecting the original acceleration data generated during the entire shooting process; Step S2, preprocessing the collected original acceleration data; screening data frames, performing forward differences on the three-axis acceleration signals respectively, and determining candidate shooting signal frames by setting thresholds; selecting an appropriate time window length T, intercepting the firearm shooting instant signals and non-firearm shooting instant signals with a fixed number of sampling points, labeling them with corresponding labels respectively, as a data set, and dividing it into a training set and a test set; Step S3, using the BiLSTM long short-term memory recurrent neural network to learn the training set, and adjusting the parameters of the network with the Adam backpropagation algorithm; Step S4, using the suspected shooting acceleration data to be predicted in the test set as the input of the model, and finally obtaining the model with the highest recognition accuracy. The present invention can significantly improve the accuracy and reliability of firearm shooting identification.

Description

Technical Field

[0001] The present invention belongs to the technical field of firearm shooting recognition, and relates to a firearm shooting recognition method based on a BiLSTM long short-term memory recurrent neural network. Background Art

[0002] Currently, for the firearm shooting recognition technology in the shooting training process, it mainly includes: using a pressure sensor to detect the shooting shock wave, using an ultraviolet detector to detect the muzzle flame of shooting, using a photoelectric sensor to count the number of bullets fired, using a Hall sensor to detect the movement of firearm components, and using a sound sensor to extract the gunfire signal from the sound information. Most of the above solutions have disadvantages such as difficult deployment, narrow applicable range, and the need for firearm modification. Especially in terms of firearm modification, ordinary people do not have the authority to modify it. Compared with other types of sensors, the acceleration sensor has the advantages of small size, easy to carry, does not damage the original structure of the firearm, and strong function extensibility. Therefore, the acceleration sensor is used as the acquisition device for the gunfire signal.

[0003] Currently, when using an acceleration sensor for detection, classification and other tasks, it mainly uses feature engineering in combination with traditional machine learning models for research. Among them, the feature engineering method mainly extracts features from the time domain and frequency domain. In terms of the selection of machine learning models, the classification algorithms used mainly include the logistic regression algorithm, support vector machine algorithm, naive Bayes algorithm, decision tree algorithm, and there is also the use of K-means clustering analysis. However, these traditional machine learning methods can only achieve good accuracy under specific features selected artificially, and have disadvantages such as low recognition accuracy, long time-consuming for selecting appropriate features, and feature leakage.

[0004] In the application of deep learning to detect acceleration signals, it mainly includes data feature map processing and using a neural network for recognition. In terms of feature map processing, it includes using signal processing methods such as Fourier transform, wavelet transform, and discrete cosine transform to extract energy features, spectrograms, amplitudes, etc. of the acceleration signal; the neural network models mainly include traditional deep models such as multi-layer perceptron, convolutional neural network, and recurrent neural network.

[0005] However, the technology of using a neural network for firearm shooting recognition based on acceleration signals has not yet emerged. For the firearm shooting acceleration signal, the BiLSTM long short-term memory recurrent neural network not only considers the influence of the current input acceleration data and historical acceleration data on recognition, but also stacks the ordinary LSTM structure from back to front, and encodes the gunfire acceleration sequence from back to front, which can effectively improve the recognition accuracy and achieve more fine-grained classification. And BiLSTM is a kind of recurrent neural network, which can better capture the dependencies with a large time step distance in the time series than the general recurrent neural network, and eliminate the influence of the intercepted length of different time segments on the recognition accuracy under real shooting conditions.

[0006] Currently, there is no research in the publicly published literature and patents on the use of BiLSTM long short-term memory recurrent neural networks for firearm shooting. Summary of the Invention

[0007] The purpose of the present invention is to provide a firearm shooting recognition method based on BiLSTM long short-term memory recurrent neural networks, so as to overcome the defects of low recognition accuracy in existing related technologies, thereby improving the accuracy and reliability of shooting recognition.

[0008] To achieve the above object, the technical solution of the present invention is: a firearm shooting recognition method based on BiLSTM long short-term memory recurrent neural networks, including the following steps:

[0009] Step S1: Deploy a triaxial acceleration sensor with a set sampling frequency of fs on the firearm to collect the original acceleration data generated during the entire shooting process;

[0010] Step S2: Preprocess the collected original acceleration data. First, perform data cleaning, delete outliers and missing values to avoid data confusion; secondly, use smoothing filtering to remove noise; then perform data frame screening, perform forward differences on the triaxial acceleration signals respectively, and determine the candidate gunshot signal frames by setting thresholds; finally, select the time window length T, intercept the firearm shooting instant signals and non-firearm shooting instant signals with a fixed number of sampling points, label them respectively, use them as a data set, and divide it into a training set and a test set;

[0011] Step S3: Use the BiLSTM long short-term memory recurrent neural network to learn the training set, and use the Adam backpropagation algorithm to adjust the parameters of the network;

[0012] Step S4: Use the acceleration data of the suspected gunshot to be predicted in the test set as the input of the model, obtain the category of the acceleration signal, and finally obtain the model with the highest recognition accuracy.

[0013] Compared with the prior art, the present invention has the following beneficial effects: A firearm shooting recognition method based on the BiLSTM long short-term memory recurrent neural network of the present invention. The BiLSTM structure can effectively process the changes in the gunshot acceleration signal, solve the problem of gradient disappearance. Through experiments, it is proved that the recognition accuracy can reach a high level, and it has a good generalization effect for different types of firearms. This technology expands the monitoring means of firearm shooting recognition based on acceleration sensors, and for the first time uses a neural network model for shooting recognition and classification, which can be applied to different types of firearms, providing reliable technical assistance for military organs, public security organs, shooting ranges and other firearm-holding institutions in aspects such as projectile assessment and bullet management. And for the existing shooting recognition technology, the present invention greatly improves the accuracy and reliability of shooting recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a flowchart of the firearm shooting recognition method based on the BiLSTM long short-term memory recurrent neural network in the present invention.

[0015] Figure 2 It is a schematic diagram of the three-axis acceleration at the moment of firearm shooting extracted after preprocessing the signal collected from the acceleration sensor in an embodiment of the present invention.

[0016] Figure 3 It is a schematic diagram of the three-axis acceleration of the signal at the moment of non-firearm shooting extracted after preprocessing the signal collected from the acceleration sensor in an embodiment of the present invention.

[0017] Figure 4 It is a diagram of the BiLSTM long short-term memory recurrent neural network model in an embodiment of the present invention.

[0018] Figure 5 It is the forward calculation process in the forward propagation in an embodiment of the present invention.

[0019] Figure 6 It is the backward calculation process in the forward propagation in an embodiment of the present invention.

[0020] Figure 7 It is a schematic diagram of the descent curve of the Adam backpropagation algorithm in an embodiment of the present invention.

[0021] Figure 8 It is a graph showing the change of accuracy with the number of training times during the training of the BiLSTM long short-term memory recurrent neural network in an embodiment of the present invention.

[0022] Figure 9 It is a graph showing the change of loss with the number of training times during the training of the BiLSTM long short-term memory recurrent neural network in an embodiment of the present invention.

[0023] Figure 10This is the confusion matrix result graph for classifying 190 samples in the test set using a BiLSTM long short-term memory recurrent neural network in an embodiment of the present invention.

[0024] Figure 11 This is the Lift curve of the BiLSTM long short-term memory recurrent neural network for the classification result in an embodiment of the present invention. Detailed implementation manners

[0025] The technical solution of the present invention will be specifically described below with reference to the accompanying drawings.

[0026] A firearm shooting recognition method based on a BiLSTM long short-term memory recurrent neural network according to the present invention has a specific process block diagram as Figure 1 shown, and the steps are as follows:

[0027] Step S1: Deploy a three-axis acceleration sensor with a set sampling frequency of f s on the firearm to collect the original acceleration data generated during the entire shooting process;

[0028] Step S2: Preprocess the collected original acceleration data. First, perform data cleaning to delete outliers and missing values to avoid data chaos; secondly, use smoothing filtering to remove noise; then perform data frame screening, perform forward differences on the three-axis acceleration signals respectively, and determine candidate shooting signal frames by setting thresholds; finally, select an appropriate time window length T, intercept the firearm shooting instant signals and non-firearm shooting instant signals with a fixed number of sampling points, label them respectively, use them as a data set, and divide it into a training set and a test set;

[0029] Step S3: Use a BiLSTM long short-term memory recurrent neural network to learn the training data set, and use the Adam backpropagation algorithm to adjust the parameters of the network;

[0030] Step S4: Use the acceleration data of the suspected shooting to be predicted in the test data set as the input of the model, obtain the category of the acceleration signal, and finally obtain the model with the highest recognition accuracy.

[0031] In an embodiment of the present invention, in the step S1, the sampling rate f s , and the Nyquist sampling theorem should be satisfied, that is, the sampling frequency should be greater than twice the highest frequency in the signal (f s ≥2f max), the sampled acceleration data can completely retain the information of the original signal. It is recommended that the sampling frequency be higher than 1.6 kHz; the acceleration sensor can be deployed at different positions such as above the barrel, the buttstock, and the aiming device. Considering that the firearm itself is a rigid body and there is a buffering effect, the sampling rate can be appropriately increased at positions with obvious buffering such as the buttstock; the acceleration sensor records the shooting process from the start of the aiming action to the end of the firing, which may include action signals such as bullet loading, bolt pulling, and firearm handling.

[0032] In an embodiment of the present invention, in the step S2, the smoothing filter sums the adjacent time series of the accelerations of each axis and then takes the average value to obtain a smoothed acceleration signal. Using the difference algorithm, the original signals of the three-axis accelerations after filtering are respectively subjected to forward difference processing. By setting a threshold, the positions of the sampling points where the difference results exceed the threshold are recorded, and the number of sampling points in the time window is intercepted as T*f s of the three-axis acceleration signal as the candidate gunshot signal frame. The firearm shooting instant signals and non-firearm shooting instant signals extracted according to this idea are as Figure 2 and Figure 3 shown. In this embodiment, for the acceleration values in the sample, the maximum sampling amplitude of the acceleration sensor can be directly used to normalize the maximum and minimum values to the interval [0, 1], and then the corresponding firearm shooting instant signals and non-firearm shooting instant signal labels are added as the dataset for the next operation.

[0033] In an embodiment of the present invention, in the step S3, a BiLSTM long short-term memory recurrent neural network is constructed. The model structure diagram is as Figure 4 shown, where module C represents the acceleration signal input module, module L represents the LSTM module, module FC represents the fully connected layer module, and the output module represents the classification output result. And the parameters of the network are adjusted in the following way:

[0034] Step S31, the forward propagation includes the forward calculation and the backward calculation of the input signal. The forward calculation method is: f t = sigmoid(W f ·[h t-1 , x t +b f ), i t = sigmoid(W i ·[h t-1 , x t +b i ), o t = sigmoid(W o ·[h t-1 , x t +b o ), ht = o t ·tanh(C t ), where x t represents the input of the LSTM network at time t, h t-1 represents the input of the hidden node at time t in the forward calculation, f t represents the output of the forget gate in the forward calculation, i t represents the output of the input gate in the forward calculation, represents the update value of the cell state in the forward calculation, C t represents the hidden state value at time t in the forward calculation (output of the LSTM network), C t-1 represents the hidden state value at time t - 1 in the forward calculation, o t represents the output of the output gate in the forward calculation, h t represents the output of the hidden node at time t in the forward calculation, W f , W i , W C , W o are the weights for the forward calculation, b f , b i , b C , b o are the biases for the forward calculation. The forward calculation process is as Figure 5 shown. The backward calculation method is: f' t = sigmoid(W' f ·[h' t+1 , x t + b' f ), i' t = sigmoid(W' i ·[h' t+1 , x t + b' i ), o' t = sigmoid(W' o ·[h' t+1 , x t + b' o ), h' t = o' t ·tanh(C' t ), where x t represents the input of the LSTM network at time t, h' t+1 represents the input of the hidden node at time t + 1 in the backward calculation, f' t represents the output of the forget gate in the backward calculation, i' t represents the output of the input gate in the backward calculation, represents the update value of the cell state in the backward calculation, C' tDenote the hidden state value (output of the LSTM network) at time t calculated in reverse, C′ t+1 Denote the hidden state value at time t+1 calculated in reverse, o′ t Denote the output of the output gate calculated in reverse, h′ t Denote the output of the hidden node at time t calculated in reverse. W′ f 、W′ i 、W′ C 、W′ o Are the weights for reverse calculation, b′ f 、b′ i 、b′ C 、b′ o Are the biases for reverse calculation. The reverse calculation process is as Figure 6 shown. sigmoid and tanh represent the sigmoid function and the tanh function respectively, where x represents the input.

[0035] Step S32, Backpropagation, uses the Adam backpropagation to update the loss function. The method is as follows:

[0036] First, define an objective function where, represents the objective function, i represents the sample class, y i represents the label of sample i, with the positive class being 1 and the negative class being 0. represents the probability that the model predicts as class i, p represents the parameter to be adjusted, i.e., the weights and biases.

[0037] Then use the Adam optimization algorithm's backpropagation algorithm to adjust the parameters. First, calculate the biased first-order moment estimate and the biased second-order moment estimate, where t is the training time, s t is the biased first-order moment estimate at time t, s t-1 is the biased first-order moment estimate at time t-1, r t is the biased second-order moment estimate at time t, r t-1 is the biased second-order moment estimate at time t-1, is the objective function, p is the parameter to be adjusted, p t-1 is the parameter at time t-1, ρ1, ρ2 are the decay rates in the interval [0,1); secondly, calculate the corrected first-order moment deviation and the second-order moment deviation, where, is the first-order moment deviation at time t, is the second-order moment deviation at time t, ρ1, ρ2 are the decay rates in the interval [0,1); finally, update the parameters, where p t is the parameter to be adjusted at time t, p t-1is the parameter at time t-1, ε is the global learning rate, and δ is a constant, usually set to 10 -6 . The schematic diagram of the Adam backpropagation descent curve is as Figure 7 shown.

[0038] In this example, the signal dataset collected by the gun shooting acceleration sensor open-sourced from http: / / dx.doi.org / 10.7910 / DVN / 25918 is used as the experimental data source. According to the file description, a total of 359 gun shooting instant signal samples and 981 non-gun shooting instant signal samples are extracted. 70% of the data in the original dataset is selected as the training set, and 30% of the data in the original dataset is selected as the test set. The changes in the accuracy rate and loss function during the training process are as Figure 8 and Figure 9 shown. It can be seen that after the training round reaches 15 rounds, the accuracy rate curve and the loss function curve tend to be stable. Throughout the process, the training set accuracy rate is always lower than that of the test set, and the test set loss function value is always lower than that of the test set, indicating that the model does not have an overfitting phenomenon. The classification accuracy rate of the model in the test set reaches 100%, fully reflecting the accuracy of the present invention, as Figure 10 shown in the confusion matrix. In order to evaluate the generalization performance of the model, the Lift curve is used to evaluate the recognition ability of the model for shooting signals. The slope of the Lift curve of the model is much higher than that of the baseline, indicating that the generalization performance of the model is excellent, as Figure 11 shown. According to the practical results, it can be shown that the method proposed by the present invention can perform relatively accurate shooting recognition, fully reflecting that the present invention can provide a better gun shooting recognition method.

[0039] The above are the preferred embodiments of the present invention. All changes made according to the technical solution of the present invention, when the functions and effects produced do not exceed the scope of the technical solution of the present invention, shall fall within the protection scope of the present invention.

Claims

1. A firearm shooting recognition method based on a BiLSTM long short-term memory recurrent neural network, characterized in that, It includes the following steps: Step S1: Deploy a triaxial acceleration sensor with a set sampling frequency of fs on the firearm, and collect the original acceleration data generated during the entire shooting process; Step S2: Preprocess the collected original acceleration data. First, perform data cleaning to delete outliers and missing values; secondly, use smoothing filtering to remove noise; Then, perform data frame screening. Perform forward differences on the triaxial acceleration signals respectively, and determine the candidate gunshot signal frames by setting thresholds; finally, select the time window length T, intercept the firearm shooting instant signals and non-firearm shooting instant signals with a fixed number of sampling points, label them respectively, use them as the data set, and divide it into a training set and a test set; Step S3: Use a BiLSTM long short-term memory recurrent neural network to learn the training set, and use the Adam backpropagation algorithm to adjust the network parameters; Step S4: Use the acceleration data of the suspected gunshot to be predicted in the test set as the input of the model, obtain the category of the acceleration signal, and finally obtain the model with the highest recognition accuracy; The sampling frequency f described in step S1 s , must satisfy the Nyquist sampling theorem, that is, the sampling frequency is greater than or equal to twice the highest frequency in the signal, that is, f s ≥2f max , the acceleration data after sampling can completely retain the information of the original signal; the acceleration sensors are deployed at positions including above the barrel, the buttstock, and the aiming device; the acceleration sensors record the action signals during the shooting process, including the process from the start of the aiming action to the end of the firing; the sampling frequency f s is higher than 1.6 kHz.

2. The method for identifying firearm shootings based on a BiLSTM long short-term memory recurrent neural network according to claim 1, characterized in that The smoothing filtering described in step S2 is to sum the adjacent time series of the accelerations of each axis and then take the average value to obtain a smoothed acceleration signal.

3. A firearm shooting recognition method based on a BiLSTM long short-term memory recurrent neural network according to claim 1, characterized in that, The forward difference on the triaxial acceleration signals described in step S2 and determining the candidate gunshot signal frames by setting thresholds means using the difference algorithm to perform forward difference processing on the triaxial acceleration signals after smoothing filtering respectively; By setting a threshold, record the positions of the sampling points where the differential result exceeds the threshold, and intercept the triaxial acceleration signal with the number of sampling points in the time window being T*f s as a candidate gunshot signal frame.

4. A firearm shooting recognition method based on a BiLSTM long short-term memory recurrent neural network according to claim 1, characterized in that, In step S3, a BiLSTM long short-term memory recurrent neural network is constructed in the following way, and the Adam backpropagation algorithm is used to adjust the network parameters: Step S31. Forward propagation includes forward calculation and backward calculation of input signals. The forward calculation method is: f t = sigmoid(W f ·[h t-1 , x t + b f ), i t = sigmoid(W i ·[h t-1 , x t + b i ), o t = sigmoid(W o ·[h t-1 , x t + b o ), h t = o t ·tanh(C t ), where x t represents the input of the LSTM network at time t, h t-1 represents the input of the hidden node at time t - 1 in forward calculation, f t represents the output of the forget gate in forward calculation, i t represents the output of the input gate in forward calculation, represents the update value of the cell state in forward calculation, C t represents the hidden state value at time t in forward calculation, C t-1 represents the hidden state value at time t - 1 in forward calculation, o t represents the output of the output gate in forward calculation, h t represents the output of the hidden node at time t in forward calculation, W f , W i , W C , W o are the weights in forward calculation, b f , b i , b C , b o are the bias values in forward calculation; The backward calculation method is: f t ' = sigmoid(W' f ·[h' t+1 , x t + b f '), i t ' = sigmoid(W i '·[h t ' +1 , x t + b i '), o' t = sigmoid(W' o · [h' t+1 , x t + b' o ), h' t = o' t · tanh(C' t ), where x t represents the input of the LSTM network at time t, h' t+1 represents the input of the hidden node at time t+1 in the reverse calculation, f t ' represents the output of the forget gate in the reverse calculation, i t ' represents the output of the input gate in the reverse calculation, represents the updated value of the cell state in the reverse calculation, C' t represents the hidden state value at time t in the reverse calculation, C' t+1 represents the hidden state value at time t+1 in the reverse calculation, o' t represents the output of the output gate in the reverse calculation, h' t represents the output of the hidden node at time t in the reverse calculation, W' f , W i ', W' C , W o ' are the weights in the reverse calculation, b f ', b i ', b' C , b' o are the biases in the reverse calculation; sigmoid and tanh represent the sigmoid function and the tanh function respectively, where x represents the input; Step S32: Backpropagation, using the Adam backpropagation to update the loss function, and its method is: First, define an objective function where represents the objective function, i represents the sample category, and y i represents the label of sample category i, with the positive class being 1 and the negative class being 0; represents the probability that the model predicts for sample category i, and p represents the parameter to be adjusted, i.e., the weight and bias values; Then, the Adam optimization algorithm is used to adjust the parameters through backpropagation. First, the biased first moment estimate and the biased second moment estimate are calculated. where t is the training time, s t is the biased first moment estimate at time t, s t-1 is the biased first moment estimate at time t - 1, r t is the biased second moment estimate at time t, r t-1 is the biased second moment estimate at time t - 1. is the objective function, p t-1 is the parameter at time t - 1, ρ1 and ρ2 are the decay rates, and their value ranges are in the interval [0, 1); secondly, the corrected first moment deviation and the second moment deviation are calculated. where is the first moment deviation at time t, is the second moment deviation at time t; finally, the parameters are updated. ε is the global learning rate, and δ is a constant.

Images