A firearm shooting recognition method based on a multi-scale convolutional neural network

Abstract

The present invention relates to a method for identifying firearm shootings based on a multi-scale convolutional neural network. It includes: Step S1, collecting the original acceleration data generated during the entire shooting process; Step S2, preprocessing the collected original acceleration data, then performing data frame screening, respectively performing forward differences on the three-axis acceleration signals, determining candidate shooting signal frames by setting thresholds, and finally selecting an appropriate time window length T, intercepting shooting signals and non-shooting signals with a fixed number of sampling points, respectively labeling them, using them as a data set, and dividing it into a training set and a test set; Step S3, using a multi-scale convolutional neural network to learn the training data set and adjusting the parameters of the network with the Adam backpropagation algorithm; Step S4, using the acceleration data of suspected shootings to be predicted in the test data set as the input of the model, and finally obtaining the model with the highest recognition accuracy. The technical solution of the present invention can significantly improve the accuracy and reliability of firearm shooting identification.

Description

Technical Field

[0001] The present invention belongs to the technical field of firearm shooting recognition, and particularly relates to a firearm shooting recognition method based on a multi-scale convolutional neural network. Background Art

[0002] To prevent the serious threat posed by the abuse of firearms to the personal and property safety of the people, China implements a strict bullet and firearm control policy. Therefore, it is undoubtedly necessary to improve the supervision of the production, transportation of firearms and ammunition, as well as the use and storage links of firearms and ammunition where omissions are most likely to occur. In addition, when military organizations conduct daily shooting training and battlefield drills, and public security organs conduct police entry training and crime scene drills, firearm shooting is often a special skill that requires practical assessment, and the number of fired bullets is one of the assessment criteria. It not only relates to the shooting training effect of personnel, but also provides a reference for the material storage and logistics support of relevant units.

[0003] Currently, for the firearm shooting recognition technology during the shooting training process, it mainly includes: using a pressure sensor to detect the shooting shock wave, using an ultraviolet detector to detect the muzzle flame of shooting, using an optoelectronic sensor to count the number of fired bullets, using a Hall sensor to detect the movement of firearm components, and using an acoustic sensor to extract the gunshot signal from the sound information. Most of the above solutions have disadvantages such as difficult deployment, narrow applicable range, and the need for firearm modification. Especially in terms of firearm modification, ordinary people do not have the authority to modify it. Compared with other types of sensors, the acceleration sensor has the advantages of small size, easy to carry, does not damage the original structure of the firearm, and strong function extensibility. Therefore, the acceleration sensor is used as the acquisition device for the gunshot signal.

[0004] Currently, when using an acceleration sensor for detection, classification and other tasks, it mainly uses feature engineering in combination with traditional machine learning models for research. Among them, the feature engineering method mainly extracts features from the time domain and frequency domain. In terms of the selection of machine learning models, the classification algorithms used mainly include the logistic regression algorithm, support vector machine algorithm, naive Bayes algorithm, decision tree algorithm, and in addition, K-means clustering analysis is also used. However, these traditional machine learning methods can only achieve good accuracy under specific features selected by humans, and have disadvantages such as low recognition accuracy, long time-consuming for selecting appropriate features, and feature leakage.

[0005] In the application of deep learning to detect acceleration signals, it mainly includes data feature map processing and using a neural network for recognition. In terms of feature map processing, it includes using signal processing methods such as Fourier transform, wavelet transform, and discrete cosine transform to extract energy features, spectrograms, amplitudes, etc. of the acceleration signal; the neural network models mainly include traditional deep models such as multi-layer perceptron, convolutional neural network, and recurrent neural network.

[0006] However, the technology of using neural networks for gunshot recognition based on acceleration signals has not yet emerged. For gunshot acceleration signals, the multi-scale convolutional neural network uses multiple parallel convolutional kernels to process acceleration signals, enabling the model to automatically select an appropriate convolutional kernel size and effectively improving the recognition accuracy. Moreover, through this multi-scale design, while enhancing the performance of the neural network, the utilization efficiency of computing resources is ensured, making it more advantageous than ordinary convolutional neural networks in extracting features of gunshot acceleration signals.

[0007] Currently, there is no research on applying multi-scale convolutional neural networks to gunshots in the publicly published literature and patents. Summary of the Invention

[0008] The purpose of the present invention is to provide a gunshot recognition method based on a multi-scale convolutional neural network to overcome the defects such as low recognition accuracy in existing related technologies, thereby improving the accuracy and reliability of gunshot recognition.

[0009] To achieve the above purpose, the technical solution of the present invention is: a gunshot recognition method based on a multi-scale convolutional neural network, including the following steps:

[0010] Step S1: Deploy a triaxial acceleration sensor with a set sampling frequency of f s on the gun to collect the original acceleration data generated during the entire shooting process;

[0011] Step S2: Preprocess the collected original acceleration data. First, perform data cleaning, delete outliers and missing values to avoid data chaos; secondly, use smoothing filtering to remove noise; then perform data frame screening, perform forward difference on the triaxial acceleration signals respectively, and determine candidate gunshot signal frames by setting thresholds; finally, select the time window length T, intercept the instantaneous signals of gunshots and non-gunshots with a fixed number of sampling points, label them respectively, use them as a data set, and divide it into a training set and a test set;

[0012] Step S3: Use the multi-scale convolutional neural network to learn the training set and adjust the network parameters with the Adam backpropagation algorithm;

[0013] Step S4: Use the suspected gunshot acceleration data to be predicted in the test set as the input of the model, obtain the category of the acceleration signal, and finally obtain the model with the highest recognition accuracy.

[0014] Compared with the prior art, the present invention has the following beneficial effects: A method for identifying gunshots based on a multi-scale convolutional neural network according to the present invention. The multi-scale convolutional structure can effectively process the changes in gunshot acceleration signals, solve the problem of difficulty in selecting convolutional kernels suitable for different types of gunshot acceleration signals, and through experiments, it is proved that the recognition accuracy can reach a high level, and it has a good generalization effect for different types of guns. This technology expands the monitoring means for gunshot recognition based on acceleration sensors, and for the first time, a neural network model is used for the recognition and classification of gunshots, which can be applied to different types of firearms, providing reliable technical assistance for military organs, public security organs, shooting ranges and other gun-holding institutions in aspects such as projectile assessment and bullet management. And for the existing shooting recognition technology, the present invention greatly improves the accuracy and reliability of shooting recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flowchart of a method for identifying gunshots based on a multi-scale convolutional neural network in the present invention.

[0016] Figure 2 It is a schematic diagram of the three-axis acceleration at the moment of gunshot extraction after preprocessing the signals collected from the acceleration sensor in an embodiment of the present invention.

[0017] Figure 3 It is a schematic diagram of the three-axis acceleration of the non-gunshot moment signal extraction after preprocessing the signals collected from the acceleration sensor in an embodiment of the present invention.

[0018] Figure 4 It is a multi-scale convolutional neural network model diagram in an embodiment of the present invention.

[0019] Figure 5 It is a schematic diagram of the descent curve of the Adam backpropagation algorithm in an embodiment of the present invention.

[0020] Figure 6 It is a graph showing the change of accuracy with the number of training times during the training of the multi-scale convolutional neural network in an embodiment of the present invention.

[0021] Figure 7 It is a graph showing the change of loss with the number of training times during the training of the multi-scale convolutional neural network in an embodiment of the present invention.

[0022] Figure 8 It is a confusion matrix result graph of classifying 190 samples in the test set by the multi-scale convolutional neural network in an embodiment of the present invention.

[0023] Figure 9 It is a KS graph of the classification results of the multi-scale convolutional neural network in an embodiment of the present invention.

[0024] Figure 10This is the Lift graph of the classification results of the multi-scale convolutional neural network in an embodiment of the present invention. Detailed implementation manners

[0025] The technical solution of the present invention will be specifically described below with reference to the accompanying drawings.

[0026] The present invention provides a firearm shooting recognition method based on a multi-scale convolutional neural network, and its specific flow block diagram is as Figure 1 shown, and the steps are as follows:

[0027] Step S1: Deploy a triaxial acceleration sensor with a set sampling frequency of f s on the firearm to collect the original acceleration data generated during the entire shooting process;

[0028] Step S2: Preprocess the collected original acceleration data. First, perform data cleaning, delete outliers and missing values to avoid data chaos; secondly, use smoothing filtering to remove noise; then perform data frame screening, perform forward differences on the triaxial acceleration signals respectively, and determine candidate shooting signal frames by setting thresholds; finally, select an appropriate time window length T, intercept the shooting signals and non-shooting signals with a fixed number of sampling points, label them respectively, use them as a data set, and divide it into a training set and a test set;

[0029] Step S3: Use a multi-scale convolutional neural network to learn the training data set, and use the Adam backpropagation algorithm to adjust the network parameters;

[0030] Step S4: Use the acceleration data of the suspected shooting to be predicted in the test data set as the input of the model, obtain the category of the acceleration signal, and finally obtain the model with the highest recognition accuracy.

[0031] In an embodiment of the present invention, in the step S1, the sampling rate f s , it is necessary to satisfy the Nyquist sampling theorem, that is, the sampling frequency should be greater than 2 times the highest frequency in the signal (f s ≥2f max ), and the sampled acceleration data can completely retain the information of the original signal. It is recommended that the sampling frequency be higher than 1.6 kHz; the acceleration sensor can be deployed at different positions such as above the barrel, the buttstock, and the aiming device. Considering that the firearm itself is a rigid body and there is a buffering effect, the sampling rate can be appropriately increased at positions with obvious buffering such as the buttstock; the acceleration sensor records the shooting process from the start of the aiming action to the end of the firing, and may include action signals such as bullet loading, bolt pulling, and holding the firearm in the middle.

[0032] In an embodiment of the present invention, in the step S2, the smoothing filter sums the adjacent time series of the accelerations of each axis and then takes the average value to obtain a smoothed acceleration signal. Using the difference algorithm, the original signals of the three-axis accelerations after filtering are respectively subjected to forward difference processing. By setting a threshold, the positions of the sampling points where the difference results exceed the threshold are recorded, and the number of sampling points in the time window is intercepted as T*f s The three-axis acceleration signals are used as candidate shooting signal frames. The firearm shooting instant signals and non-firearm shooting instant signals extracted according to this idea are as shown in Figure 2 and Figure 3 In this embodiment, for the acceleration values in the samples, the maximum sampling amplitude of the acceleration sensor can be directly used to normalize the maximum and minimum values to the interval [0,1], and then the corresponding shooting and non-shooting signal labels are added as the data set for the next operation.

[0033] In an embodiment of the present invention, in the step S3, a multi-scale convolutional neural network is constructed. The model structure diagram is as shown in Figure 4 where the module Input layer represents the acceleration signal input module, the module 40×3Conv1d represents one-dimensional convolution with 3 convolution kernels of size 40, the module 20×3Conv1d represents one-dimensional convolution with 3 convolution kernels of size 20, the module 10×3Conv1d represents one-dimensional convolution with 3 convolution kernels of size 10, the module 3×3max pooling represents the maximum pooling layer with a downsampling size of 3, the module 1×1Conv1d represents one-dimensional convolution with 1 convolution kernel of size 1, the module Filter concatenation represents concatenating the feature maps along the third dimension, the BN module represents the batch normalization layer, the GAP module represents the global average pooling layer, and the Output layer module represents the fully connected layer to output the classification result. And the parameters of the network are adjusted in the following way:

[0034] Step S31, forward propagation, using multi-scale convolution, the method is: where l represents the number of layers of the neural network, represents the output feature map, u represents the row size of the feature map, v represents the column size of the feature map, represents the input feature map, represents the weight obtained by rotating the convolution kernel by 180°, b (l) represents the bias value, i represents the i-th element in the row direction of the convolution kernel, j represents the j-th element in the column direction of the convolution kernel, n represents the row (or column) size of the convolution kernel, relu represents the activation function,

[0035] Step S32, the pooling layer downsamples the feature map, where l represents the number of layers of the neural network, Denote the output feature map, i represents the row size of the output feature map, j represents the column size of the output feature map, and β (l+1) denotes the weight of each convolutional kernel, denote the input feature map, u represents the row size of the input feature map, v represents the column size of the input feature map, r represents the downsampling step, and b (l+1) denotes the bias term.

[0036] Step S33, backpropagation, using the Adam backpropagation to update the loss function, and the method is as follows:

[0037] First, define an objective function where, denotes the objective function, i represents the sample category, and y i denotes the label of sample i, with the positive class being 1 and the negative class being 0. denotes the probability that the model predicts as class i, and p represents the parameter to be adjusted, that is, the weight and bias values.

[0038] Then use the Adam optimization algorithm's backpropagation algorithm to adjust the parameters. First, calculate the biased first moment estimate and the biased second moment estimate, where t is the training time, and s t is the biased first moment estimate at time t, and s t-1 is the biased first moment estimate at time t - 1, and r t is the biased second moment estimate at time t, and r t-1 is the biased second moment estimate at time t - 1, is the objective function, p is the parameter to be adjusted, and p t-1 is the parameter at time t - 1, ρ1 and ρ2 are the decay rates in the interval [0, 1); secondly, calculate the corrected first moment deviation and the second moment deviation, where, is the first moment deviation at time t, is the second moment deviation at time t, and ρ1 and ρ2 are the decay rates in the interval [0, 1); finally, update the parameters, where p t is the parameter to be adjusted at time t, and p t-1 is the parameter at time t - 1, ε is the global learning rate, and δ is a constant, usually set to 10 -6 . The schematic diagram of the Adam backpropagation descent curve is as shown in Figure 5 .

[0039] In this example, the signal dataset collected by a gun shooting acceleration sensor open-sourced from http: / / dx.doi.org / 10.7910 / DVN / 25918 was used as the experimental data source. A total of 359 gun shooting instant signal samples and 981 non-gun shooting instant signal samples were extracted according to the file description. 70% of the data in the original dataset was selected as the training set, and 30% of the data in the original dataset was selected as the test set. The changes in the loss function and accuracy during the training process are as shown in Figure 6 and Figure 7 . It can be seen that after 10 training rounds, the accuracy curve and the loss function curve tend to be stable. After the curves tend to be stable, the loss function value of the training set is higher than that of the test set, and the loss function value of the test set is lower than that of the test set, indicating that the model does not have an overfitting phenomenon. The classification accuracy of the model in the test set reached 100%, fully demonstrating the accuracy of the present invention, as shown in Figure 8 the confusion matrix. To evaluate the performance of the model, according to the KS curve, it shows that the model is less affected by sample imbalance. The KS value of 0.306 indicates that the model has strong discrimination ability, as shown in Figure 9 . And the Lift curve was used to evaluate the recognition ability of the model for shooting signals. The slope of the Lift curve of the model is much higher than that of the baseline, indicating that the generalization performance of the model is excellent, as shown in Figure 10 . According to the practical results, it can be shown that the method proposed by the present invention can perform relatively accurate shooting recognition, fully demonstrating that the present invention can provide a better gun shooting recognition method.

[0040] The above are the preferred embodiments of the present invention. Any changes made to the technical solution of the present invention that do not exceed the scope of the technical solution of the present invention in terms of the functions and effects produced shall fall within the protection scope of the present invention.

Claims

1. A firearm shooting recognition method based on a multi-scale convolutional neural network, characterized in that, It includes the following steps: S1. Set the sampling frequency to f s of the three-axis acceleration sensor on the firearm to collect the original acceleration data generated during the entire shooting process; f s should be greater than twice the highest frequency in the signal, that is, f s ≥2f max , and the sampled acceleration data can completely retain the information of the original signal; the acceleration sensor is deployed at positions including above the barrel, the buttstock, and the aiming device; the acceleration sensor records the action signals during the shooting process from the start of the aiming action to the end of the firing S2. Preprocess the collected original acceleration data. First, perform data cleaning to delete outliers and missing values. Second, use smoothing filtering to remove noise; Then, perform data frame screening. Perform forward difference on the three-axis acceleration signals respectively, and determine the candidate shooting signal frames by setting thresholds. Finally, select the time window length T, intercept the signals at the moment of gun shooting and non-gun shooting with a fixed number of sampling points, label them with corresponding labels respectively, use them as a data set, and divide it into a training set and a test set; S3. Use a multi-scale convolutional neural network to learn the training set, and use the Adam backpropagation algorithm to adjust the parameters of the network; The multi-scale convolutional neural network is divided into two core modules, namely: an input parallel module and a high-level feature extraction module; The elements that make up the input parallel module include one-dimensional convolutions with convolution kernel sizes of 40, 20, and 10 in 3 parallel paths, and one-dimensional convolution with a convolution kernel size of 1 after passing through a max pooling layer first; The high-level feature extraction module adds a one-dimensional convolution with a convolution kernel size of 1 on the basis of the input parallel module for dimensionality reduction; Finally, output through a global average pooling layer and a fully connected layer; The multi-scale convolutional neural network adjusts the parameters of the network in the following way: S31. Forward propagation, using multi-scale convolution, the method is: l represents the number of neural network layers, represents the output feature map, u represents the row size of the feature map, v represents the column size of the feature map, represents the input feature map, represents the weight obtained by rotating the convolution kernel by 180°, b (l) represents the bias value, i represents the i-th element in the row direction of the convolution kernel, j represents the j-th element in the column direction of the convolution kernel, n represents the row or column size of the convolution kernel, relu represents the activation function, S32. The pooling layer downsamples the feature map. l represents the number of neural network layers. represents the output feature map, i represents the row size of the output feature map, j represents the column size of the output feature map, and β (l+1) represents the weight of each convolutional kernel. represents the input feature map, u represents the row size of the input feature map, v represents the column size of the input feature map, r represents the downsampling step, and b (l+1) represents the bias term. S33. Backward propagation, using the Adam backpropagation to update the loss function, the method is: Define an objective function represents the objective function, i represents the sample category, and y i represents the label of sample category i, with the positive class being 1 and the negative class being 0; represents the probability that the model predicts for sample category i, and p represents the parameter to be adjusted, i.e., the weight and bias values; Use the Adam optimization algorithm's backpropagation algorithm to adjust the parameters. First, calculate the biased first moment estimate and the biased second moment estimate. t is the training time, s t is the biased first moment estimate at time t, s t-1 is the biased first moment estimate at time t - 1, r t is the biased second moment estimate at time t, r t-1 is the biased second moment estimate at time t - 1, is the objective function, p is the parameter to be adjusted, p t-1 is the parameter at time t - 1, ρ1 and ρ2 are the decay rates, and their value ranges are in the interval [0, 1); secondly, calculate the corrected first moment deviation and the second moment deviation. is the first moment deviation at time t, is the second moment deviation at time t; finally, update the parameters. ε is the global learning rate, and δ is a constant; S4. Use the acceleration data suspected of shooting to be predicted in the test set as the input of the model, obtain the category of the acceleration signal, and finally obtain the model with the highest recognition accuracy.

2. The method for identifying firearm shootings based on a multi-scale convolutional neural network according to claim 1, wherein Sampling frequency f s Higher than 1.6 kHz.

3. A firearm shooting recognition method based on a multi-scale convolutional neural network according to claim 1, characterized in that, The acceleration sensor is deployed at a position with obvious buffering including the gunstock.

4. A firearm shooting recognition method based on a multi-scale convolutional neural network according to claim 1, characterized in that, The smoothing filtering in S2 is to sum the adjacent time series of the acceleration of each axis and then take the average value to obtain a smoothed acceleration signal.

5. A method for identifying firearm shootings based on a multi-scale convolutional neural network according to claim 1, characterized in that The forward difference on the three-axis acceleration signals in S2, and determine the candidate shooting signal frames by setting thresholds, that is, use the difference algorithm to perform forward difference processing on the three-axis acceleration signals after smoothing filtering respectively; By setting a threshold, record the positions of the sampling points where the differential results exceed the threshold, and intercept the triaxial acceleration signal with the number of sampling points in the time window being T*f s as a candidate gunshot signal frame.

6. The method for identifying firearm shootings based on a multi-scale convolutional neural network according to claim 1, wherein The interception of the signals at the moment of gun shooting and non-gun shooting with a fixed number of sampling points in S2, label them with corresponding labels respectively, use them as a data set, and divide it into a training set and a test set, that is, for the three-axis acceleration signals after smoothing filtering, perform minimum-maximum normalization to the interval [0,1] according to the maximum sampling amplitude of the three-axis acceleration sensor, then add the corresponding signals and labels at the moment of gun shooting and non-gun shooting, divide the training set and the test set, and use the training set and the test set as the input of the multi-scale convolutional neural network.

Images