Cross-site aircraft audio classification method based on data enhancement and convolutional neural network
By enhancing data and extracting features of aircraft audio signals and training with convolutional neural networks, the problem of insufficient audio data recognition capabilities at different locations in the prior art is solved, and high-accuracy aircraft audio signal classification is achieved.
Patent Information
- Application Number
- CN202411272239.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-11
- Publication Date
- 2025-01-14
AI Technical Summary
The existing audio classification network has limited generalization capabilities in different environments, especially poor performance on audio data at different locations, resulting in a degradation in model performance.
Using a method based on data augmentation and convolutional neural network, the aircraft audio signal is subjected to speed perturbation, volume perturbation, frequency mask and time mask processing, and the Mel spectrum features are extracted, and the convolutional neural network model is used for training and Identify.
The recognition rate of the model for audio data at different locations is improved, the robustness and generalization capabilities of the model are enhanced, and high-accurate aircraft audio signal classification is achieved.
Smart Images
Figure CN119314515A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of audio classification, and in particular to a cross-location aircraft audio classification method based on data enhancement and convolutional neural network. Background Art
[0002] In the field of aviation monitoring and security, the detection and analysis of aircraft audio signals plays a vital role. Traditionally, manual detection and analysis of aircraft audio signals is a common method, but this method is limited by human resources, time cost and expertise, and is easily affected by subjective factors, so it is not efficient and reliable enough.
[0003] In recent years, with the rapid development of artificial intelligence and machine learning technologies, automated aircraft audio signal recognition methods have received increasing attention. Among them, deep learning-based methods have made significant progress in audio signal processing and pattern recognition. As a powerful deep learning model, convolutional neural networks have achieved great success in fields such as image recognition and speech recognition.
[0004] However, there are still challenges in identifying aircraft audio signals. Existing audio classification networks are usually trained and tested on audio datasets from a single environment. However, the generalization ability of this method in different environments may be limited. In particular, when audio data comes from different locations, differences in environmental noise and recording equipment can lead to a significant decrease in model performance. Therefore, how to improve the generalization ability of the model on audio data from different locations has become a key issue. Summary of the invention
[0005] The technical problem to be solved by the present invention is how to improve the generalization ability of the model on audio data from different locations; in view of this, the present invention provides a cross-location aircraft audio classification method based on data enhancement and convolutional neural network.
[0006] The technical solution adopted by the present invention is a method for classifying cross-location aircraft audio based on data enhancement and convolutional neural network, comprising:
[0007] Step S1, performing data enhancement processing on the original aircraft audio signal;
[0008] Step S2, extracting Melton spectrum features from the current aircraft audio signal;
[0009] Step S3, using the configured convolutional neural network model to train the extracted Mel spectrum features;
[0010] Step S4, input the audio signal to be identified into the trained convolutional neural network model, process the signal through a forward propagation process, output a recognition result and a corresponding probability value, so as to determine the category of the audio signal to be identified.
[0011] In one embodiment, step S1 includes:
[0012] Perform speech rate disturbance enhancement on the current aircraft audio signal;
[0013] Perform volume disturbance enhancement on the current aircraft audio signal;
[0014] Perform frequency masking on the current aircraft audio signal;
[0015] Perform time mask processing on the current aircraft audio signal.
[0016] In one embodiment, step S2 includes:
[0017] Perform pre-processing operations on the current aircraft audio signal;
[0018] Perform Fourier transform on the preprocessed aircraft audio signal to obtain the corresponding amplitude spectrum, and further take the modulus square to obtain the corresponding power spectrum;
[0019] The power spectrum is input into a Mel filter to complete feature extraction.
[0020] In one embodiment, step S3 includes:
[0021] The mel spectrum features are trained using a convolutional neural network model; wherein the convolutional neural network consists of 4 convolutional blocks, each of which includes two 3*3 convolutional layers;
[0022] After each of the convolutional layers, batch normalization and ReLU activation function are applied to improve the training efficiency and stability of the model;
[0023] After the last convolutional layer, global average pooling is applied, and Dropout is added after the pooling operation to prevent overfitting and convert the feature map into a fixed-length vector;
[0024] A fully connected layer is added to the fixed-length vector to extract embedded features for further classification.
[0025] Another aspect of the present invention further provides a device for classifying aircraft audio across locations based on data enhancement and convolutional neural network, comprising:
[0026] an enhancement unit configured to perform data enhancement processing on the original aircraft audio signal;
[0027] A Melton feature extraction unit configured to extract Melton spectrum features from the current aircraft audio signal;
[0028] A neural network unit, configured to train the extracted Mel frequency spectrum features using a configured convolutional neural network model;
[0029] The classification unit is configured to input the audio signal to be identified into the trained convolutional neural network model, process the signal through a forward propagation process, output a recognition result and a corresponding probability value, so as to determine the category of the audio signal to be identified.
[0030] In one embodiment, the enhancement unit is specifically configured as follows:
[0031] Perform speech rate disturbance enhancement on the current aircraft audio signal;
[0032] Perform volume disturbance enhancement on the current aircraft audio signal;
[0033] Perform frequency masking on the current aircraft audio signal;
[0034] Perform time mask processing on the current aircraft audio signal.
[0035] In one embodiment, the Melton feature extraction unit is specifically configured as follows:
[0036] Perform pre-processing operations on the current aircraft audio signal;
[0037] Perform Fourier transform on the preprocessed aircraft audio signal to obtain the corresponding amplitude spectrum, and further take the modulus square to obtain the corresponding power spectrum;
[0038] The power spectrum is input into a Mel filter to complete feature extraction.
[0039] In one embodiment, the neural network unit is specifically configured as follows:
[0040] The mel spectrum features are trained using a convolutional neural network model; wherein the convolutional neural network consists of 4 convolutional blocks, each of which includes two 3*3 convolutional layers;
[0041] After each of the convolutional layers, batch normalization and ReLU activation function are applied to improve the training efficiency and stability of the model;
[0042] After the last convolutional layer, global average pooling is applied, and Dropout is added after the pooling operation to prevent overfitting and convert the feature map into a fixed-length vector;
[0043] A fully connected layer is added to the fixed-length vector to extract embedded features for further classification.
[0044] Another aspect of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the cross-location aircraft audio classification method based on data enhancement and convolutional neural network as described in any one of the above items.
[0045] Another aspect of the present invention further provides a computer storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the cross-location aircraft audio classification method based on data enhancement and convolutional neural network as described in any one of the above items are implemented.
[0046] Compared with the prior art, the present invention has at least the following advantages:
[0047] The technical solution of the present invention utilizes a convolutional neural network model and a convolutional neural network structure to extract and classify the features of aircraft audio signals; specifically, the method first performs speech rate disturbance enhancement and voice volume disturbance enhancement processing on the aircraft audio signal, and then converts the processed audio signal into a Mel spectrum feature representation; after obtaining the spectrum features, frequency masks and time masks are used to mask random frequency segments and time segments on the spectrum matrix, which enhances the model's ability to resist specific information loss, thereby improving the robustness of the model; finally, the convolutional neural network model is used to train these features to learn the classification pattern of aircraft audio signals. In the recognition process, by inputting the aircraft audio signal to be recognized into the trained convolutional neural network model, the signal is classified and recognized, and the aircraft type information is output. This method has a high recognition rate for aircraft audio data across locations. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 A schematic diagram of a flow chart of a method for cross-location aircraft audio classification based on data enhancement and convolutional neural network according to an embodiment of the present invention;
[0049] Figure 2 is a flow chart of an identification method according to an embodiment of the present invention;
[0050] Figure 3 A flowchart of Mel spectrum feature extraction according to an embodiment of the present invention;
[0051] Figure 4 is a recognition accuracy curve diagram according to an embodiment of the present invention;
[0052] Figure 5 A schematic diagram of a device for classifying cross-location aircraft audio based on data enhancement and convolutional neural network according to an embodiment of the present invention;
[0053] Figure 6 The figure is a schematic diagram of the composition of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0054] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined purpose, the present invention is described in detail below in conjunction with the accompanying drawings and preferred embodiments.
[0055] Unless otherwise defined, all terms (including technical terms and scientific terms) used in this article have the same meaning as commonly understood by ordinary technicians in the field to which this application belongs. It should also be understood that terms (such as terms defined in commonly used dictionaries) should be interpreted as having the same meaning as their meaning in the context of the relevant technology, and will not be interpreted in an idealized or overly formal sense unless explicitly defined in this article.
[0056] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0057] In an embodiment of the present invention, a method for classifying cross-location aircraft audio based on data enhancement and convolutional neural network is provided. Figure 1 As shown, including:
[0058] Step S1, performing data enhancement processing on the original aircraft audio signal;
[0059] Step S2, extracting Melton spectrum features from the current aircraft audio signal;
[0060] Step S3, using the configured convolutional neural network model to train the extracted Mel spectrum features;
[0061] Step S4, input the audio signal to be identified into the trained convolutional neural network model, process the signal through a forward propagation process, output a recognition result and a corresponding probability value, so as to determine the category of the audio signal to be identified.
[0062] refer to Figure 2 , the method provided in this embodiment will be described in detail step by step below.
[0063] Step S1, performing data enhancement processing on the original aircraft audio signal;
[0064] In this embodiment, speech rate perturbation and volume perturbation techniques are applied to the original aircraft audio signal. These techniques simulate different environmental sounds by changing the playback speed and volume of the audio. This processing helps the model learn more robust features and improve the generalization ability of the model. The following is an explanation of speech rate perturbation enhancement and volume perturbation enhancement.
[0065] 1) Speech rate disturbance enhancement
[0066] Speech rate perturbation enhancement refers to a method of processing audio by changing the playback speed of the audio signal without changing the audio content. Changing the playback speed will affect the duration of the audio, but will not change the frequency of the audio.
[0067] Given the original audio signal x(t), the audio signal after changing the speech rate is x'(t), then its expression is:
[0068]
[0069] Where α is the speech speed factor. If α>1, it means the playback is accelerated and the audio length is shortened; if α<1, it means the playback is decelerated and the audio length is lengthened.
[0070] 2) Volume disturbance enhancement
[0071] Volume perturbations are achieved by adjusting the amplitude of the audio signal. A change in gain results in a change in volume, but does not change other characteristics of the audio.
[0072] Given the original audio signal x(t), the audio signal after volume perturbation is x'(t), then its expression is:
[0073] x'(t)=x(t)+β
[0074] where β is a randomly selected gain value from a specified range, usually expressed in decibels (dB).
[0075] 3) Frequency Mask
[0076] Frequency masking simulates removing information from certain frequencies by randomly selecting some frequency bins in the spectrum and setting their amplitudes to zero. This can help the model combat frequency loss or frequency selective interference.
[0077] For a given spectrum matrix X, whose dimensions are (batch, time, frequency), given a random mask length L f , random mask position P f , then the expression of the masked matrix X' is:
[0078] X′[:,P f :P f +L f ,:]=0
[0079] 4) Time Mask
[0080] Temporal masking simulates removing information at certain times by randomly selecting some time segments in the spectrum and setting their amplitude to zero. This can help the model combat frequency dropouts or frequency selective interference.
[0081] For a given spectrum matrix X, whose dimensions are (batch, time, frequency), given a random mask length L t , random mask position P t , then the expression of the masked matrix X' is:
[0082] X′[:,P t :P t +L t ,:]=0
[0083] Step S2, extracting Melton spectrum features from the current aircraft audio signal;
[0084] Mel spectrum features are widely used acoustic features in audio recognition. The feature extraction steps are as follows: Figure 2 As shown, the specific steps are as follows:
[0085] (1) Pre-processing of aircraft audio signals by pre-emphasis, framing and windowing
[0086] (2) The preprocessed aircraft audio signal x i Perform Fourier transform to obtain the amplitude spectrum x of the aircraft audio signal i , and then square the modulus to get the power spectrum P i , whose expression is as follows:
[0087]
[0088] Among them, FFT refers to fast Fourier transform, and N is the signal length.
[0089] (3) Input the power spectrum into the Mel filter. The Mel filter bank is a set of triangular filters used to map the spectrum to the Mel scale. The Mel scale is a scale of human auditory perception. It is different from the linear frequency scale. The center frequency of the Mel filter increases with the increase of frequency, and there is overlap between filters. Usually, the number of Mel filters is 20-40.
[0090] Step S3, using the configured convolutional neural network model to train the extracted Mel spectrum features;
[0091] The convolutional neural network consists of 4 convolutional blocks, each of which includes two 3*3 convolutional layers.
[0092] After each convolutional layer, batch normalization and ReLU activation function are applied to improve the training efficiency and stability of the model.
[0093] After the last convolutional layer, global average pooling is applied, and Dropout is added after the pooling operation to prevent overfitting. In this step, the feature map is converted into a fixed-length vector.
[0094] Finally, a fully connected layer is added on the fixed-length vector to extract embedded features for further classification.
[0095] Step S4, input the audio signal to be identified into the trained convolutional neural network model, process the signal through a forward propagation process, output a recognition result and a corresponding probability value, so as to determine the category of the audio signal to be identified.
[0096] Combine the following Figure 3 Specific experiments further illustrate the present invention:
[0097] The method has been experimented on drone, fighter, helicopter and noise datasets. These datasets contain different numbers of one-second audio segments, as shown in the table below. In order to verify the generalization performance of the model, the training set and test set were collected from different locations, so the ratio of the two does not strictly follow a specific rule.
[0098] type Drones noise fighter Helicopters Number of training sets 220 1159 110 990 Number of test sets 200 832 210 220
[0099] The recognition accuracy of this method on the test set is shown in the following table, and the recognition accuracy curve is as follows Figure 3 As shown in the figure, the recognition method has achieved high recognition accuracy on drone, fighter, helicopter and noise datasets.
[0100] type Drones noise fighter Helicopters Recognition accuracy 96.00％ 100％ 98.51％ 99.54％
[0101] The above is a typical application of the present invention, and the application of the present invention is not limited thereto.
[0102] The method of the present invention can be applied not only to aircraft identification in military and civil aviation, but also to the monitoring and analysis of various other aircraft and noise sources. For example, the noise monitoring system of an airport can use this method to identify and classify the types of aircraft in real time, thereby helping the airport management department to perform noise control and aircraft scheduling. In the management and monitoring system of drones, this method can also identify and track the activities of drones, thereby improving safety monitoring capabilities.
[0103] Compared with the prior art, this embodiment has at least the following effects:
[0104] 1) In an embodiment of the present invention, the aircraft audio signal to be analyzed is subjected to speech rate disturbance enhancement and volume disturbance enhancement processing, and Mel spectrum features are extracted. Then, frequency masks and time masks are used to mask random frequency segments and time segments on the spectrum matrix to obtain enhanced audio feature representations. Finally, the enhanced audio feature representations are used as input to train a convolutional neural network model.
[0105] 2) The embodiments of the present invention are verified through experiments, wherein the training set and the test set in the experiments are from different locations, and the high accuracy of the method on the test set illustrates the high generalization of the model.
[0106] The second embodiment of the present invention corresponds to the first embodiment, and can be referred to Figure 5 This embodiment introduces a cross-location aircraft audio classification device based on data enhancement and convolutional neural network, including the following components:
[0107] an enhancement unit configured to perform data enhancement processing on the original aircraft audio signal;
[0108] A Melton feature extraction unit configured to extract Melton spectrum features from the current aircraft audio signal;
[0109] A neural network unit, configured to train the extracted Mel frequency spectrum features using a configured convolutional neural network model;
[0110] The classification unit is configured to input the audio signal to be identified into the trained convolutional neural network model, process the signal through a forward propagation process, output a recognition result and a corresponding probability value, so as to determine the category of the audio signal to be identified.
[0111] In one embodiment, the enhancement unit is specifically configured as follows:
[0112] Perform speech rate disturbance enhancement on the current aircraft audio signal;
[0113] Perform volume disturbance enhancement on the current aircraft audio signal;
[0114] Perform frequency masking on the current aircraft audio signal;
[0115] Perform time mask processing on the current aircraft audio signal.
[0116] In one embodiment, the Melton feature extraction unit is specifically configured as follows:
[0117] Perform pre-processing operations on the current aircraft audio signal;
[0118] Perform Fourier transform on the preprocessed aircraft audio signal to obtain the corresponding amplitude spectrum, and further take the modulus square to obtain the corresponding power spectrum;
[0119] The power spectrum is input into a Mel filter to complete feature extraction.
[0120] In one embodiment, the neural network unit is specifically configured as follows:
[0121] The mel spectrum features are trained using a convolutional neural network model; wherein the convolutional neural network consists of 4 convolutional blocks, each of which includes two 3*3 convolutional layers;
[0122] After each of the convolutional layers, batch normalization and ReLU activation function are applied to improve the training efficiency and stability of the model;
[0123] After the last convolutional layer, global average pooling is applied, and Dropout is added after the pooling operation to prevent overfitting and convert the feature map into a fixed-length vector;
[0124] A fully connected layer is added to the fixed-length vector to extract embedded features for further classification.
[0125] A third embodiment of the present invention is an electronic device, such as Figure 6 As shown, it can be understood as a physical device, including a processor and a memory storing instructions executable by the processor. When the instructions are executed by the processor, the following operations are performed:
[0126] Step S1, performing data enhancement processing on the original aircraft audio signal;
[0127] Step S2, extracting Melton spectrum features from the current aircraft audio signal;
[0128] Step S3, using the configured convolutional neural network model to train the extracted Mel spectrum features;
[0129] Step S4, input the audio signal to be identified into the trained convolutional neural network model, process the signal through a forward propagation process, output a recognition result and a corresponding probability value, so as to determine the category of the audio signal to be identified.
[0130] The fourth embodiment of the present invention, the process of the cross-location aircraft audio classification method based on data enhancement and convolutional neural network in this embodiment is the same as that of the first, second or third embodiment, the difference is that in engineering implementation, this embodiment can be implemented by means of software plus the necessary general hardware platform, of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the method of the present invention can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), including a number of instructions for a device to execute the method described in the embodiment of the present invention.
[0131] Through the description of the specific implementation methods, a deeper and more specific understanding of the technical means and effects adopted by the present invention to achieve the predetermined purpose should be obtained. However, the accompanying drawings are only for reference and illustration purposes and are not intended to limit the present invention.
Claims
1. A method for cross-location aircraft audio classification based on data augmentation and convolutional neural network, characterized in that: include: Step S1, performing data enhancement processing on the original aircraft audio signal; Step S2, extracting Melton spectrum features from the current aircraft audio signal; Step S3, using the configured convolutional neural network model to train the extracted Mel spectrum features; Step S4, input the audio signal to be identified into the trained convolutional neural network model, process the signal through a forward propagation process, output a recognition result and a corresponding probability value, so as to determine the category of the audio signal to be identified.
2. The method for cross-location aircraft audio classification based on data enhancement and convolutional neural network according to claim 1, characterized in that: The step S1 comprises: Perform speech rate disturbance enhancement on the current aircraft audio signal; Perform volume disturbance enhancement on the current aircraft audio signal; Perform frequency masking on the current aircraft audio signal; Perform time mask processing on the current aircraft audio signal.
3. The method for cross-location aircraft audio classification based on data enhancement and convolutional neural network according to claim 2 is characterized in that: The step S2 comprises: Perform pre-processing operations on the current aircraft audio signal; Perform Fourier transform on the preprocessed aircraft audio signal to obtain the corresponding amplitude spectrum, and further take the modulus square to obtain the corresponding power spectrum; The power spectrum is input into a Mel filter to complete feature extraction.
4. The method for cross-location aircraft audio classification based on data enhancement and convolutional neural network according to claim 3 is characterized in that: The step S3 comprises: The mel spectrum features are trained using a convolutional neural network model; wherein the convolutional neural network consists of 4 convolutional blocks, each of which includes two 3*3 convolutional layers; After each of the convolutional layers, batch normalization and ReLU activation function are applied to improve the training efficiency and stability of the model; After the last convolutional layer, global average pooling is applied, and Dropout is added after the pooling operation to prevent overfitting and convert the feature map into a fixed-length vector; A fully connected layer is added to the fixed-length vector to extract embedded features for further classification.
5. A cross-location aircraft audio classification device based on data enhancement and convolutional neural network, characterized in that: include: an enhancement unit configured to perform data enhancement processing on the original aircraft audio signal; A Melton feature extraction unit configured to extract Melton spectrum features from the current aircraft audio signal; A neural network unit, configured to train the extracted Mel spectrum features using a configured convolutional neural network model; The classification unit is configured to input the audio signal to be identified into the trained convolutional neural network model, process the signal through a forward propagation process, output a recognition result and a corresponding probability value, so as to determine the category of the audio signal to be identified.
6. The cross-location aircraft audio classification device based on data enhancement and convolutional neural network according to claim 5, characterized in that: The enhancement unit is specifically configured as follows: Perform speech rate disturbance enhancement on the current aircraft audio signal; Perform volume disturbance enhancement on the current aircraft audio signal; Perform frequency masking on the current aircraft audio signal; Perform time mask processing on the current aircraft audio signal.
7. The cross-location aircraft audio classification device based on data enhancement and convolutional neural network according to claim 6, characterized in that: The Melton feature extraction unit is specifically configured as follows: Perform pre-processing operations on the current aircraft audio signal; Perform Fourier transform on the preprocessed aircraft audio signal to obtain the corresponding amplitude spectrum, and further take the modulus square to obtain the corresponding power spectrum; The power spectrum is input into a Mel filter to complete feature extraction.
8. The cross-location aircraft audio classification device based on data enhancement and convolutional neural network according to claim 7, characterized in that: The neural network unit is specifically configured as follows: The mel spectrum features are trained using a convolutional neural network model; wherein the convolutional neural network consists of 4 convolutional blocks, each of which includes two 3*3 convolutional layers; After each of the convolutional layers, batch normalization and ReLU activation function are applied to improve the training efficiency and stability of the model; After the last convolutional layer, global average pooling is applied, and Dropout is added after the pooling operation to prevent overfitting and convert the feature map into a fixed-length vector; A fully connected layer is added to the fixed-length vector to extract embedded features for further classification.
9. An electronic device, characterized in that: The electronic device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the method for cross-location aircraft audio classification based on data enhancement and convolutional neural network as claimed in any one of claims 1 to 4 are implemented.
10. A computer storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the method for cross-location aircraft audio classification based on data enhancement and convolutional neural network as claimed in any one of claims 1 to 4 are implemented.