Gunshot source identification system and method based on Mel spectrogram and YOLOv8 model
By combining Mel spectrograms with the YOLOv8 model, the problems of distinguishing between friendly and enemy gunshots and environmental noise interference in gunshot detection are solved, achieving high accuracy and real-time performance in gunshot recognition, which is suitable for scenarios such as individual soldier operations and security monitoring.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-20
- Publication Date
- 2026-03-13
AI Technical Summary
Existing technologies struggle to accurately distinguish between "friendly fire" and "enemy fire" in gunshot detection, and their accuracy is insufficient in complex, noisy environments, failing to meet real-time requirements.
By combining Mel spectrograms with the YOLOv8 model, a Mel spectrogram dataset with high distinguishability of gunshot features is constructed through data acquisition, preprocessing, feature extraction, and model training. Then, the image feature extraction and classification capabilities of YOLOv8 are used to achieve rapid and accurate identification of gunshot sources.
It significantly improves the distinguishability of gunshot sources, enhances identification accuracy and environmental robustness, meets the needs of real-time early warning and decision-making, and is suitable for diverse scenarios such as individual combat and security monitoring.
Smart Images
Figure CN121662078A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of acoustic signal processing and artificial intelligence, and in particular to a gunshot source identification system and method based on Mel spectrograms and the YOLOv8 model. Background Technology
[0002] Gunshot detection and recognition are key technologies in security monitoring and military fields. Current mainstream gunshot detection technologies focus on two main areas: determining the presence of gunshots and locating their direction. These technologies largely rely on time-domain feature detection (such as energy analysis and short-time zero-crossing rate calculation) or spectrum analysis. While these methods can effectively identify gunshot events in the environment, in specific scenarios such as military confrontations and law enforcement operations, it is necessary to distinguish between friendly gunshots and other sounds (such as enemy gunshots and ambient noise). Existing technologies still have the following shortcomings in differentiating the source of gunshots: Weak feature differentiation: Although traditional acoustic features (such as pitch and timbre) can provide a basic description of sound, they are difficult to capture the subtle differences in features when faced with gunshots from the same type of firearm in different environments, resulting in insufficient accuracy in source differentiation. Insufficient model robustness: Schemes based on pattern matching or traditional machine learning (such as SVM support vector machine, GMM Gaussian mixture model) are prone to recognition bias under the interference of complex environmental noise (such as explosion sound), and the robustness and accuracy are difficult to meet the needs of actual combat. Lack of real-time adaptability: Tactical scenarios have extremely high requirements for recognition response speed. Some existing algorithms are difficult to achieve millisecond-level fast judgment due to their complex structure and large amount of computation, and cannot match the real-time decision-making needs in actual combat.
[0003] Therefore, there is an urgent need for a new technical solution that can accurately and quickly distinguish one's own gunshots from those of others. Summary of the Invention
[0004] To address the problem of existing technologies failing to effectively distinguish the source of gunshots, this invention provides a gunshot source identification system and method based on Mel spectrograms and a YOLOv8 model. The system acquires WAV files of gunshot audio through audio data acquisition and performs standardized preprocessing. The preprocessed target audio signal is then converted into a two-dimensional Mel spectrogram. Mel spectrogram datasets are constructed for two main categories: "own gunshot" and "other sounds." The YOLOv8 classification model is trained to distinguish subtle feature differences between the two types of gunshot spectrograms. Finally, the Mel spectrogram of an unknown real-time gunshot is compared, allowing the model to output a classification result and confidence level indicating whether the gunshot belongs to "own gunshot" or "someone else's gunshot," effectively distinguishing between them.
[0005] To achieve the above objectives, the present invention provides the following technical solution: A gunshot source identification method based on Mel spectrograms and the YOLOv8 model includes: Step 1: Data Acquisition and Preprocessing: Acquire gunshot audio signals and save them as WAV format files. Perform preprocessing operations on the WAV format files to obtain the preprocessed target audio signals. Step 2, Feature Extraction: Perform frame windowing, short-time Fourier transform, power spectrum calculation, Mel filter bank filtering, and logarithmic transform on the target audio signal to convert the one-dimensional audio signal into a two-dimensional Mel spectrum. Step 3, Model Deployment and Inference Recognition: After processing the real-time gunshot audio in Steps 1 and 2, the data is input into the target YOLOv8 classification model. The model outputs two probability vectors. When the highest probability value exceeds the confidence threshold of 0.8, the corresponding category is determined as the gunshot source recognition result. If the value is lower than the threshold, the data is re-acquired and recognized.
[0006] As an improvement, the data acquisition operation includes setting the sampling rate, bit depth, and channel mode. The sampling rate and bit depth are set to the adaptation range of basic general scenarios, and the channel mode is set to mono mode. The preprocessing operation includes noise reduction, silence removal, and amplitude normalization.
[0007] As an improvement, the frame-by-frame windowing includes setting the frame length and frame shift for a general scenario, and using a window function to suppress spectral leakage.
[0008] As an improvement, the target YOLOv8 classification model needs to be converted to ONNX or NCNN format before deployment to adapt to the hardware platforms of cloud servers, embedded devices, or mobile terminals.
[0009] As an improvement, the present invention also includes YOLOv8 classification model training, which includes: constructing a Mel spectrogram dataset containing two categories of labels, "self-gunshot" and "other sounds"; performing data augmentation on the dataset; inputting the augmented dataset into the initial YOLOv8 classification model; inputting the constructed Mel spectrogram dataset into the model for multiple rounds of iterative training; continuously optimizing the model weights until the classification accuracy on the validation set reaches the expected level, thereby obtaining the trained target YOLOv8 classification model.
[0010] As an improvement, during the data collection process for the "my_gunshot" tag, audio samples were recorded using a specific type of firearm under various environments and converted into Mel spectrograms, which were then tagged with "my_gunshot". The dataset labeled "Other Sounds" includes: audio recordings of others firing the same specific type of firearm in various environments, as well as ambient noise. These samples are then converted into Mel spectrograms and labeled "other_gunshot".
[0011] As an improvement, the initial YOLOv8 classification model uses the yolov8s-cls.pt version, the loss function is the cross-entropy loss function, and the optimizer is either Adam or SGD.
[0012] As an improvement, the data augmentation process includes one or more combinations of Gaussian noise addition, random occlusion, and contrast adjustment.
[0013] As an improvement, the method further includes storing the probability vector, recognition result, and corresponding original audio data of each recognition in a database for subsequent source tracing and model iteration optimization.
[0014] Based on the same inventive concept, this invention also provides a system for executing the gunshot source identification method based on Mel spectrogram and YOLOv8, for executing a gunshot source identification method based on Mel spectrogram and YOLOv8, comprising: Audio acquisition and preprocessing module: used to acquire gunshot audio and save it as WAV format, perform noise reduction, silence removal and amplitude normalization operations, and output the target audio signal; Spectrum conversion module: Performs frame-by-frame windowing, short-time Fourier transform, power spectrum calculation, Mel filter bank filtering, and logarithmic transform on the target audio signal, converting it into a two-dimensional Mel spectrum. Model training module: Constructs and enhances the Mel spectrogram dataset with two labels, loads pre-trained weights to train the initial YOLOv8 classification model, and generates the target YOLOv8 classification model; Recognition output module: Input the real-time audio image to be recognized into the target model, output the recognition result based on the probability vector and confidence threshold, and store the relevant data.
[0015] The beneficial effects of this invention are as follows: (1) The present invention uses Mel spectrogram to accurately depict the energy distribution pattern of gunshots in the time and frequency domain. Compared with traditional acoustic features (such as pitch and timbre), it significantly improves the feature differentiation of gunshots from different sources. Combined with the powerful image feature extraction and classification capabilities of the YOLOv8 model, it can keenly capture the subtle "soundprint" differences of different gunshots, greatly reducing the misjudgment rate from a technical perspective and ensuring the accuracy of the recognition results.
[0016] (2) By utilizing the excellent anti-interference ability of the YOLOv8 model itself against image translation and noise interference, the present invention transforms the one-dimensional audio signal into a two-dimensional Mel spectrogram, so that the scheme can still stably identify under complex environmental noise (such as wind and rain) or small changes in gunshot signals (such as different shooting distances and angles), effectively avoiding the identification deviation caused by environmental interference.
[0017] (3) The YOLOv8 model in this invention has lightweight architecture and efficient inference performance as its core advantages. It can quickly complete the feature analysis and category judgment of the Mel spectrogram without complicated calculation process. It can easily meet the real-time early warning of security monitoring and the instant decision-making needs of tactical scenarios, and avoid the risks caused by recognition delay.
[0018] (4) The solution of the present invention can be deeply adapted to various scenarios such as individual combat system (assisting in distinguishing friendly and enemy gun sounds and improving battlefield situation awareness), intelligent security monitoring (accurately identifying the source of abnormal gun sounds and quickly linking early warning), and personnel training evaluation (judging the standardization of gun sounds during training and optimizing training effects). It can not only enhance the user's safety protection capabilities, but also provide data support for scenario-based decision-making, and has outstanding practical value.
[0019] In summary, this invention improves the feature differentiation of gunshots from different sources by using Mel spectrograms. Combined with YOLOv8's image extraction and classification capabilities, anti-interference capabilities, lightweight architecture, and efficient inference performance, it significantly improves recognition accuracy and environmental robustness, while meeting the needs of real-time early warning and decision-making. Furthermore, it can be deeply adapted to diverse scenarios such as individual soldier operations, security monitoring, and training evaluation, possessing both security and practical value. Attached Figure Description
[0020] Figure 1 This is a flowchart of the gunshot source identification method of the present invention; Figure 2 This is a schematic diagram of the gunshot audio signal of the present invention; Figure 3 This is a schematic diagram of the Mel spectrum of the present invention. Detailed Implementation
[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0022] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.
[0023] Example 1 like Figure 1-2 As shown, this invention provides a gunshot source identification method based on Mel spectrograms and the YOLOv8 model, comprising: S1. Data Acquisition and Preprocessing: Acquire gunshot audio signals and save them as WAV format files. Perform preprocessing operations on the WAV format files to obtain the preprocessed target audio signals. S2. Feature extraction: Perform frame windowing, short-time Fourier transform, power spectrum calculation, Mel filter bank filtering, and logarithmic transform on the target audio signal to convert the one-dimensional audio signal into a two-dimensional Mel spectrum. S3. Model Deployment and Inference Recognition: After processing the real-time collected gunshot audio through steps one and two, it is input into the target YOLOv8 classification model. The model outputs two probability vectors. When the highest probability value exceeds the confidence threshold of 0.8, the corresponding category is determined as the gunshot source recognition result. If it is lower than the threshold, the data is collected and re-identified.
[0024] Preferably, the data acquisition operation includes setting the sampling rate, bit depth, and channel mode. The sampling rate and bit depth are set to the adaptation range of basic general scenarios. For example, setting the sampling rate to 44.1kHz and the bit depth to 16-bit covers the range of human hearing (20Hz-20kHz) and can completely preserve the high-frequency details of the gunshot. The 16-bit bit depth can provide 65,536 dynamic range levels, avoiding signal distortion caused by differences in the intensity of the gunshot. The channel mode is set to mono mode, which can reduce channel redundancy and reduce the computational load of data storage and subsequent processing. The preprocessing operation includes noise reduction, silence removal, and amplitude normalization.
[0025] Preferably, the frame segmentation and windowing includes setting the frame length and frame shift for general scenarios, such as: setting the frame length to 25 milliseconds to balance time resolution and frequency resolution; setting the frame shift to 10 milliseconds to avoid signal information loss and ensure the continuity of time and frequency characteristics; and using window functions (such as Hamming windows) to suppress spectral leakage and improve the accuracy of the spectrogram.
[0026] Optionally, before deploying the target YOLOv8 classification model, it needs to be converted to ONNX or NCNN format to adapt to the hardware platform of cloud servers, embedded devices or mobile terminals, thereby improving the applicability and implementation capability of the invention.
[0027] Example 2 like Figure 2-3 As shown, components that are the same as or corresponding to those in Embodiment 1 are referred to using the same reference numerals as in Embodiment 1. For simplicity, only the differences from Embodiment 1 are described below. The difference between Embodiment 2 and Embodiment 1 is as follows: In this embodiment, the gunshot source identification method based on Mel spectrogram and YOLOv8 model according to Embodiment 1 further includes YOLOv8 classification model training. The YOLOv8 classification model training includes: constructing a Mel spectrogram dataset containing two categories of labels: "self-gunshot" and "other sounds"; performing data augmentation on the dataset; inputting the augmented dataset into the initial YOLOv8 classification model; inputting the constructed Mel spectrogram dataset into the model for multiple rounds of iterative training; continuously optimizing the model weights until the classification accuracy on the validation set reaches the expected level, thereby obtaining the trained target YOLOv8 classification model.
[0028] Preferably, during the data collection process for the “my_gunshot” tag, audio samples are recorded using a specific type of firearm under various environments and converted into Mel spectrograms, which are then tagged with “my_gunshot”. The dataset labeled "Other Sounds" includes: audio recordings of others firing the same specific type of firearm in various environments, as well as ambient noise. These samples are then converted into Mel spectrograms and labeled "other_gunshot," significantly improving the reliability of identification in complex scenarios during actual combat.
[0029] Optionally, the initial YOLOv8 classification model uses the lightweight classification model yolov8s-cls.pt, which balances inference speed and accuracy and is suitable for deployment scenarios with limited resources; the loss function uses the cross-entropy loss function to effectively measure the probability difference between the two classes; the optimizer can flexibly select the fast-converging Adam optimizer or the highly generalizable SGD optimizer according to the size of the dataset.
[0030] Preferably, the data augmentation process includes one or more of the following methods: Gaussian noise addition, random occlusion, and contrast adjustment. These three methods correspond to three core scenarios: random noise interference, local information loss, and environmental variable differences. When used in combination, they can simulate more complex real-world application environments (such as audio with simultaneous noise, frequency band loss, and amplitude fluctuations, and images with simultaneous noise, occlusion, and lighting changes), allowing the model to output stable results even under multiple interferences and significantly reducing errors in actual implementation.
[0031] Preferably, the method further includes storing the probability vector, recognition result, and corresponding original audio data of each recognition in a database to completely preserve the recognition process data, realize the full-link traceability of "recognition result - original signal - probability basis", provide new data in real-world scenarios for model iteration, facilitate subsequent investigation of the cause of misjudgment (such as a certain type of environmental noise being misjudged as gunshot), and optimize the model by continuously accumulating data, so that the recognition accuracy can be continuously improved as the application scenarios expand.
[0032] Example 3 like Figure 1 As shown, components that are the same as or corresponding to those in Embodiment 1 are referred to using the same reference numerals as in Embodiment 1. For simplicity, only the differences from Embodiment 1 are described below. The difference between Embodiment 3 and Embodiment 1 is as follows: In this embodiment, a system for executing the gunshot source identification method based on Mel spectrogram and YOLOv8 is also provided. This system is used to execute a gunshot source identification method based on Mel spectrogram and YOLOv8, including: Audio acquisition and preprocessing module: used to acquire gunshot audio and save it as WAV format, perform noise reduction, silence removal and amplitude normalization operations, and output the target audio signal; Spectrum conversion module: Performs frame-by-frame windowing, short-time Fourier transform, power spectrum calculation, Mel filter bank filtering, and logarithmic transform on the target audio signal, converting it into a two-dimensional Mel spectrum. Model training module: Constructs and enhances the Mel spectrogram dataset with two labels, loads pre-trained weights to train the initial YOLOv8 classification model, and generates the target YOLOv8 classification model; Recognition output module: Input the real-time audio image to be recognized into the target model, output the recognition result based on the probability vector and confidence threshold, and store the relevant data.
[0033] Work process (1) Optimize and convert the target YOLOv8 classification model into a format suitable for deployment, such as ONNX and NCNN, and deploy it to the target hardware platform (such as cloud server, embedded device, mobile terminal, etc.). (2) When the system acquires a new gunshot audio, it performs the same preprocessing and feature extraction process as S1 and S2 to obtain its Mel spectrum. (3) Use the Mel spectrogram of the new gunshot audio as input to deploy the target YOLOv8 classification model; (4) The target YOLOv8 classification model performs a forward inference and outputs a vector containing the probabilities of two categories, such as [P(my_gunshot), P(other_sound)], which is the gunshot of the target and the other sound.
[0034] (5) Compare the two probability values and take the category with the higher probability value as the final recognition result. At the same time, set a confidence threshold of 0.8. If the highest probability exceeds the threshold, output the determined classification result; if the highest probability does not exceed the threshold, automatically re-acquire the audio signal in the current scene, repeat the preprocessing, Mel spectrogram conversion and model inference process, and eliminate the low confidence problem caused by single signal interference (such as instantaneous strong noise, incomplete signal acquisition).
[0035] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for gunshot source identification based on Mel spectrograms and YOLOv8, characterized in that, include: Step 1: Data Acquisition and Preprocessing: Acquire gunshot audio signals and save them as WAV format files. Perform preprocessing operations on the WAV format files to obtain the preprocessed target audio signals. Step 2, Feature Extraction: Perform frame windowing, short-time Fourier transform, power spectrum calculation, Mel filter bank filtering, and logarithmic transform on the target audio signal to convert the one-dimensional audio signal into a two-dimensional Mel spectrum. Step 3, Model Deployment and Inference Recognition: After processing the real-time gunshot audio in Steps 1 and 2, the data is input into the target YOLOv8 classification model. The model outputs two probability vectors. When the highest probability value exceeds the confidence threshold of 0.8, the corresponding category is determined as the gunshot source recognition result. If the value is lower than the threshold, the data is re-acquired and recognized.
2. The gunshot source identification method based on Mel spectrogram and YOLOv8 according to claim 1, characterized in that, The data acquisition operation includes setting the sampling rate, bit depth, and channel mode. The sampling rate and bit depth are set to the adaptation range of basic general scenarios, and the channel mode is set to mono mode. The preprocessing operation includes noise reduction, silence removal, and amplitude normalization.
3. The gunshot source identification method based on Mel spectrogram and YOLOv8 according to claim 1, characterized in that, The frame-by-frame windowing includes setting the frame length and frame shift for general scenarios, and using a window function to suppress spectral leakage.
4. The gunshot source identification method based on Mel spectrogram and YOLOv8 according to claim 1, characterized in that, Before deploying the target YOLOv8 classification model, it needs to be converted to ONNX or NCNN format to adapt to the hardware platform of cloud servers, embedded devices, or mobile terminals.
5. A gunshot source identification method based on Mel spectrogram and YOLOv8 according to any one of claims 1-4, characterized in that, The YOLOv8 classification model training includes: constructing a Mel spectrogram dataset containing two categories of labels, "self-gunshot" and "other sounds"; performing data augmentation on the dataset; inputting the augmented dataset into the initial YOLOv8 classification model; inputting the constructed Mel spectrogram dataset into the model for multiple rounds of iterative training; continuously optimizing the model weights until the classification accuracy on the validation set reaches the expected level, thus obtaining the trained target YOLOv8 classification model.
6. The gunshot source identification method based on Mel spectrogram and YOLOv8 according to claim 5, characterized in that, During the data collection process for the "my_gunshot" tag, audio samples were recorded using a specific type of firearm under various environments and converted into Mel spectrograms, which were then tagged with "my_gunshot". The dataset labeled "Other Sounds" includes: audio recordings of others firing the same specific type of firearm in various environments, as well as ambient noise. These samples are then converted into Mel spectrograms and labeled "other_gunshot".
7. The gunshot source identification method based on Mel spectrogram and YOLOv8 according to claim 5, characterized in that, The initial YOLOv8 classification model uses the yolov8s-cls.pt version, the loss function is the cross-entropy loss function, and the optimizer is either Adam or SGD.
8. The gunshot source identification method based on Mel spectrogram and YOLOv8 according to claim 5, characterized in that, The data augmentation process includes one or more combined processing methods such as Gaussian noise addition, random occlusion, and contrast adjustment.
9. The gunshot source identification method based on Mel spectrogram and YOLOv8 according to claim 1, characterized in that, The method also includes storing the probability vector, recognition result, and corresponding original audio data of each recognition into a database for subsequent source tracing and model iteration optimization.
10. A gunshot source identification system based on Mel spectrograms and a YOLOv8 model, used to execute a gunshot source identification method based on Mel spectrograms and YOLOv8 as described in any one of claims 1-9, characterized in that, include: Audio acquisition and preprocessing module: used to acquire gunshot audio and save it as WAV format, perform noise reduction, silence removal and amplitude normalization operations, and output the target audio signal; Spectrum conversion module: Performs frame-by-frame windowing, short-time Fourier transform, power spectrum calculation, Mel filter bank filtering, and logarithmic transform on the target audio signal, converting it into a two-dimensional Mel spectrum. Model training module: Constructs and enhances the Mel spectrogram dataset with two labels, loads pre-trained weights to train the initial YOLOv8 classification model, and generates the target YOLOv8 classification model; Recognition output module: Input the real-time audio image to be recognized into the target model, output the recognition result based on the probability vector and confidence threshold, and store the relevant data.