Audio scene recognition method and device based on neural network, and medium
Through neural network technology, the use of quaternion filtering and deep hashing modules to extract the high-dimensional features of the audio signal, solving the difficulty of identifying traditional methods under dynamic changes in noise and time, and achieving more efficient audio scene recognition.
Patent Information
- Application Number
- CN202411454116.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-17
- Publication Date
- 2025-01-10
AI Technical Summary
Traditional audio feature extraction methods are difficult to capture and distinguish subtle feature changes in complex and variable audio signals, and are not well identified in environments of noise interference and dynamic temporal changes.
Using a neural network-based method, the audio segments are converted into spectral graph data of different scales through quaternary filtering, and high-dimensional features are extracted using the backbone network, and probability sampling is performed in combination with the deep hash module to obtain representative feature vectors.
More comprehensively captures the complex structure and subtle changes of audio signals, suppresses noise interference, improves identification accuracy and efficiency, and adapts to the time evolution characteristics of non-stationary signals.
Smart Images
Figure CN119296583A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of audio recognition technology, and in particular to a method, device and medium for audio scene recognition based on a neural network. Background Art
[0002] In the field of audio processing and analysis, audio scene recognition, as a key technology, aims to automatically identify specific environments or events, such as parks, streets, concerts, or baby crying, from complex and changing audio signals.
[0003] Although traditional audio feature extraction methods have promoted the development of audio recognition technology to a certain extent, their inherent limitations are becoming increasingly prominent and it is difficult to meet the growing demand for accurate recognition and real-time processing. The essence of audio signals lies in their high randomness and complexity. Audio events in different scenarios are often accompanied by the simultaneous existence and interaction of multiple sound sources, making it difficult for traditional methods to effectively capture and distinguish these subtle changes in audio features. In addition, audio signals in real environments are often accompanied by various noises, such as background noise, echoes, etc. These noises will seriously interfere with the extraction and recognition process of audio features. Traditional methods show great limitations when processing non-stationary signals, and it is difficult to effectively filter out noise interference, thereby affecting recognition accuracy. In addition, the dynamic changes of audio signals in time are another challenge. Traditional extraction methods based on fixed features often ignore the time evolution characteristics of audio signals, resulting in poor recognition results in rapidly changing audio scenes. Summary of the invention
[0004] In order to solve the above problems, this application proposes an audio scene recognition method based on a neural network, comprising:
[0005] Acquire an audio segment to be recognized, and process the audio segment to obtain a processed audio segment;
[0006] Through quaternion filtering, the processed audio clips are converted into spectrogram data at different scales;
[0007] Inputting the spectrum graph data into a backbone network, and extracting high-dimensional features from the spectrum graph data through the backbone network;
[0008] Inputting the high-dimensional features into a deep hash module connected to the backbone network, and performing probability sampling on the high-dimensional features through the deep hash module to obtain corresponding feature vectors;
[0009] The feature vector is mapped to a predefined audio scene category to obtain an audio scene corresponding to the audio clip.
[0010] In one implementation of the present application, processing the audio segment to obtain a processed audio segment specifically includes:
[0011] Performing frame processing on the audio segment to divide the audio segment into a plurality of audio signals of fixed duration;
[0012] For a plurality of frames of audio signals obtained after the frame processing, a windowing operation is performed on the audio signals based on a preset window function;
[0013] The audio signal obtained after the windowing operation is overlapped to reduce the edge effect between the audio signals of adjacent frames.
[0014] In one implementation of the present application, the window function is specifically expressed as:
[0015]
[0016] Where w(n) is the point of the nth sample value in the window function, and N is the total length of the window function.
[0017] In one implementation of the present application, the processed audio clips are converted into spectrogram data at different scales through quaternion filtering, specifically including:
[0018] Generate real convolution kernel and imaginary convolution kernel through quaternion filtering;
[0019] Decomposing the audio clip into initial spectrogram data at three different scales, using the initial spectrogram data as components, respectively convolving with the real convolution kernel and cross-multiplying with the imaginary convolution kernel to obtain corresponding first spectrogram data and second spectrogram data;
[0020] The first spectrum graph data and the second spectrum graph data are superimposed to obtain spectrum graph data at different scales.
[0021] In one implementation of the present application, before performing a windowing operation on the audio signal based on a preset window function, the method further includes:
[0022] Performing time-frequency analysis on the audio segment to determine whether the statistical characteristics corresponding to the audio segment change over time, so as to obtain the signal type corresponding to the audio segment; wherein the signal type includes a stationary signal and a non-stationary signal;
[0023] According to the signal type, a total length corresponding to a preset window function is determined; wherein the total length is positively correlated with the smoothness of the signal type.
[0024] In one implementation of the present application, probabilistic sampling of the high-dimensional features is performed through the deep hash module, specifically including:
[0025] By means of the deep hash module, a number of sampling points contained in the high-dimensional features are classified into types to obtain corresponding sampling point types; wherein the sampling point types include anchor points and difficult sampling points;
[0026] For the difficult sampling points, probability sampling is performed on the difficult sampling points according to a preset probability function, so as to extract target sampling points that can reflect high-dimensional distribution characteristics from the difficult sampling points and obtain feature vectors corresponding to the target sampling points.
[0027] In one implementation of the present application, the probability function is specifically expressed as:
[0028]
[0029] Among them, x i and x j represents a point in n-dimensional Euclidean space, D L Represents x i and x j The distance between.
[0030] In one implementation of the present application, before performing frame processing on the audio segment to divide the audio segment into a plurality of audio signals of fixed duration, the method further includes:
[0031] The phonemes of the audio segment are determined, and based on the phonemes, a frame length used when the audio segment is divided into frames is determined; wherein the frame length is smaller than the length of the phonemes.
[0032] The embodiment of the present application provides an audio scene recognition device based on a neural network, the device comprising:
[0033] at least one processor;
[0034] and, a memory communicatively coupled to the at least one processor;
[0035] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the neural network-based audio scene recognition method as described in any one of the above items.
[0036] The embodiment of the present application provides a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured as follows:
[0037] A neural network-based audio scene recognition method as described in any of the above items.
[0038] The neural network-based audio scene recognition method proposed in this application can bring the following results:
[0039] Beneficial effects:
[0040] By converting audio clips into spectrogram data at different scales and using the backbone network to extract high-dimensional features, the complex structure and subtle changes in the audio signal can be captured more comprehensively. Compared with traditional methods based only on fixed features, spectrogram data can retain more time-frequency information, which helps to better characterize the essential characteristics of audio signals.
[0041] The audio clips are converted into spectrogram data at different scales through quaternion filtering. This step can more comprehensively capture the frequency domain characteristics of the audio signal, analyze the changes of the audio signal at different time scales, suppress the background information of the spectrogram, and overcome the data redundancy problem caused by the traditional direct transmission of the spectrogram.
[0042] The deep hashing module further performs probabilistic sampling on high-dimensional features to obtain more representative feature vectors, which not only reduces the dimension of the features and the amount of computation, but also retains key information through probabilistic sampling, thereby improving the accuracy and effectiveness of the feature vectors. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0044] Figure 1 A flowchart of a neural network-based audio scene recognition method provided in an embodiment of the present application;
[0045] Figure 2 A schematic diagram of sampling point distribution provided in an embodiment of the present application;
[0046] Figure 3 A feature extraction comparison schematic diagram provided in an embodiment of the present application;
[0047] Figure 4 A schematic diagram of the structure of a neural network-based audio scene recognition device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0048] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.
[0049] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.
[0050] like Figure 1 As shown, an audio scene recognition method based on a neural network provided in an embodiment of the present application includes:
[0051] S101: Acquire an audio segment to be recognized, and process the audio segment to obtain a processed audio segment.
[0052] Audio is composed of sound waves of different frequencies. To recognize audio, we need to analyze the characteristics of each sound wave. However, a complete audio segment is usually non-stationary, that is, its characteristics change over time. Therefore, the audio segment needs to be processed to obtain a processed audio segment.
[0053] In order to facilitate the processing and analysis of audio, it needs to be divided into several frames of fixed length. The selection of frame length needs to meet multiple conditions. First, the factors of the audio segment are determined. The frame length used when the audio segment is framed must be less than the length of the phoneme to ensure that the sound vibration frequency in each frame does not change significantly. Secondly, each frame should contain multiple vibration cycles for Fourier transform and frequency analysis. Usually, the length of a vibration cycle is 10 milliseconds or 5 milliseconds, so the frame length is usually set to 20 milliseconds. After the audio segment is divided into several audio signals of fixed length, it is necessary to perform a windowing operation on the audio signal based on a preset window function for the several frames of audio signals obtained after the frame processing, that is, multiply the initial audio signal with a fixed-length window function, and reduce the spectrum leakage by reducing the signal amplitude at both ends. When performing spectrum analysis, the signal usually needs to be divided into multiple short time periods for processing. Directly performing Fourier transform on these segments will cause spectrum leakage, because the signal will be discontinuous at the boundary of the segment. Windowing can reduce this discontinuity and reduce spectrum leakage. The window function can be specifically expressed as:
[0054]
[0055] Where w(n) is the point of the nth sample value in the window function, and N is the total length of the window function.
[0056] It should be noted that the window length determines the length of each frame. A longer window length provides higher frequency resolution but reduces time resolution. A shorter window length provides higher time resolution but reduces frequency resolution. Different window lengths are required to adapt to different scales, and the size of the window length is related to the stability of the audio signal.
[0057] Specifically, the purpose of performing time-frequency analysis on an audio clip is to determine whether its statistical characteristics change significantly over time, which can be achieved by calculating various statistics of the audio signal (such as mean, variance, autocorrelation function, etc.) and observing their changes over time. If the statistics remain relatively stable over a long period of time, the audio clip can be regarded as a stationary signal; if the statistics change significantly over time, the audio clip is regarded as a non-stationary signal. Based on the results of the time-frequency analysis, the audio clip is classified as a stationary signal or a non-stationary signal. The total length of the preset window function is determined based on the stationarity of the signal type. Generally, for stationary signals, a longer window can be used to reduce spectral leakage and increase frequency resolution; while for non-stationary signals, a shorter window is required to better track the time-varying characteristics of the signal. Therefore, the total length of the window function is positively correlated with the stationarity of the signal type: a longer window corresponds to a stationary signal, and a shorter window corresponds to a non-stationary signal.
[0058] After windowing the audio signal, the audio signals of adjacent frames will have mutations at the beginning and end. These mutations will introduce high-frequency components and affect the accuracy of subsequent audio scene recognition. Therefore, it is also necessary to overlap the audio signals obtained after the windowing operation to reduce the edge effect between the audio signals of adjacent frames and ensure that the audio signal transitions more smoothly between frames.
[0059] S102: Convert the processed audio clips into spectrogram data at different scales through quaternion filtering.
[0060] The processed audio clips are converted into spectrogram data at different scales in two orthogonal dimensions through the two components of quaternion filtering, the real part and the imaginary part, where the real part is a 35 35 convolution kernel generated by the real part of the quaternion Gabor, and the imaginary part is a 35 35 convolution kernel generated by the quaternion Gabor.
[0061] In one embodiment, a real convolution kernel and an imaginary convolution kernel are generated by quaternion filtering, wherein the real convolution kernel can be expressed as:
[0062]
[0063] The imaginary convolution kernel can be expressed as:
[0064]
[0065] x′=xcosθ+yθ
[0066] y′=-xsinθ+ycosθ
[0067] Among them, λ represents the wavelength, θ represents the orientation of the filter, ψ represents the phase shift, σ is used to determine the size of the filter, γ represents the spatial aspect ratio, and x and y represent the orthogonal dimensions of the filter.
[0068] For the spectrum data of each scale, two complex convolution kernels are applied respectively, and the initial spectrum data is used as a component, and convolved with the real convolution kernel and cross-multiplied with the imaginary convolution kernel respectively to obtain the corresponding first spectrum data and second spectrum data. The result obtained by the real convolution at each scale and the result obtained by the imaginary cross product are superimposed to obtain a new spectrum data, which integrates the interaction information of the original spectrum data and the two complex convolution kernels.
[0069] S103: Input the spectrum graph data into the backbone network, and extract high-dimensional features from the spectrum graph data through the backbone network.
[0070] After obtaining the converted spectrum graph data through quaternion filtering, it is necessary to select a suitable backbone network to perform feature extraction on the spectrum graph data. In the embodiment of the present application, a DenseNet network structure is used as the backbone network, and each layer is connected to the remaining layers through a feedforward connection, so that the input of each layer uses the output of all previous layers, and the feature map of the layer is input into all subsequent layers, and finally the high-dimensional features in the spectrum graph data are extracted.
[0071] S104: Input the high-dimensional features into the deep hash module connected to the backbone network, and perform probability sampling on the high-dimensional features through the deep hash module to obtain the corresponding feature vector.
[0072] After extracting the high-dimensional features, they need to be input into the deep hash module connected to the backbone network. The deep hash module includes three fully connected layers and an activation function. Among them, the fully connected layer is used to reduce the high-dimensional features from 1024 dimensions to a suitable k dimension. At the same time, in the process of dimensionality reduction, the selection of samples will also affect the final recognition effect. Therefore, the deep hash module captures the distribution characteristics of high-dimensional features by probabilistic sampling of high-dimensional features, so as to improve the accuracy and efficiency of subsequent audio scene recognition tasks.
[0073] In one embodiment, a deep hash module is used to classify several sampling points contained in the high-dimensional features to obtain corresponding sampling point types; wherein the sampling point types include anchor points and difficult sampling points, and the difficult sampling points include difficult negative sampling points and difficult positive sampling points. Anchor points are also called standard points or benchmarks, and all standardization processes are based on this. Since the distance formula between difficult negative sampling points and anchor points cannot directly determine their category, their classification is more difficult for the model. Therefore, during the training process of the model, difficult negative sampling points are marked as negative samples, that is, their categories are inconsistent with the categories to which the anchor points belong. Because of their long distance, it becomes uncertain whether the difficult positive sampling points belong to the same category as the anchor points. Although the difficult positive sampling points belong to the same category as the anchor points, due to the influence of distance, their classification task is also challenging for the model, so they are marked as positive sample points.
[0074] There are two main types of difficult sampling points: Figure 2 As shown in the figure, negative difficult sampling points (such as point B) and positive difficult sampling points (such as point C). Difficult sampling points usually contain richer information. Compared with easy sampling points, they can provide more valuable signals in the calculation of the loss function, thereby promoting faster convergence of the model. By focusing on these difficult sampling points, the model can better learn complex category boundaries and improve the overall classification performance.
[0075] In n-dimensional space, the distance distribution of randomly selected points approaches the normal distribution. The characteristics of this distribution mean that the distances between most point pairs are concentrated near the mean. In order to more comprehensively describe the distribution of points in high-dimensional space, it is necessary to perform probabilistic sampling of difficult sampling points based on a preset probability function to avoid the problem of distances being too concentrated near the mean. This method aims to extract target sampling points that can reflect the characteristics of high-dimensional distribution from difficult sampling points, retain key features by reducing noise and redundant information, and then obtain the feature vectors corresponding to the target sampling points through feature extraction. These feature vectors not only contain the information of the target sampling points themselves, but also imply some characteristics of the high-dimensional distribution, which can better reflect the true distribution characteristics of points in high-dimensional space, thereby improving the accuracy of subsequent recognition tasks.
[0076] Among them, the probability function can be expressed as:
[0077]
[0078] Among them, x i and x j represents a point in n-dimensional Euclidean space, D L Represents x i and x j The distance between.
[0079] S105: Map the feature vector to a predefined audio scene category to obtain an audio scene corresponding to the audio clip.
[0080] After extracting the feature vector through the deep hash module, it is necessary to map the feature vector to the pre-defined audio scene category based on the pre-trained classification model, so as to obtain the audio scene corresponding to the audio clip. When training the classification model, a data set containing multiple audio clips and corresponding scene categories is required for training. The training set includes at least environmental sound, indoor sound, outdoor sound, human voice, animal sound and instrument sound.
[0081] In one embodiment, Figure 3 A feature extraction comparison diagram provided in the embodiment of the present application. Figure 3 As shown in the figure, the left figure shows the result of clustering analysis of feature vectors without adding deep hashing. The distance between any two points in the figure represents the distance between two samples in an N-dimensional space. The middle figure shows the effect without adding probability sampling. It can be seen that the distinction between three music styles is not good, namely jazz, metal and reggae. In particular, metal and reggae have a high similarity and are difficult to distinguish. The right figure shows the effect after adding probability sampling. Except for metal and reggae, the classification effect of other music styles is significantly improved, and even metal and reggae can be basically distinguished. Both music styles have strong rhythms, so they are similar in style.
[0082] The embodiment of the present application suppresses the background information of the spectrum graph through quaternion filtering, thereby overcoming the data redundancy problem caused by the traditional direct transmission of the spectrum graph. In order to pay attention to the sounds of different vibration frequencies at the same time, a multi-scale fusion method is proposed, which adopts different window lengths and frame shifts to enhance the connection between data in time and frequency, thereby improving the final classification ability of the model. Combining deep hashing with metric learning, the problem of incomplete data caused by randomly selected negative samples is solved. Since randomly selected negative samples often conform to the normal distribution, most of the sampling points will be located near the reference point if they are not constrained, resulting in incomplete mapping data of negative samples. To solve this problem, the embodiment of the present application introduces a probabilistic sampling strategy, which suppresses sampling near the reference point by sampling at positions close to and far from the reference point, thereby obtaining samples uniformly distributed in N-dimensional space.
[0083] The above are embodiments of the method proposed in this application. Based on the same idea, some embodiments of this application also provide devices and non-volatile computer storage media corresponding to the above methods.
[0084] Figure 4 The following is a schematic diagram of the structure of a neural network-based audio scene recognition device provided in an embodiment of the present application. Figure 4 As shown, including:
[0085] at least one processor; and,
[0086] at least one processor is communicatively connected to a memory; wherein,
[0087] The memory stores instructions executable by at least one processor, the instructions being executed by at least one processor to enable the at least one processor to:
[0088] Acquire an audio segment to be recognized, process the audio segment, and obtain a processed audio segment;
[0089] Through quaternion filtering, the processed audio clips are converted into spectrogram data at different scales;
[0090] Input the spectrum graph data into the backbone network, and extract the high-dimensional features in the spectrum graph data through the backbone network;
[0091] The high-dimensional features are input into the deep hash module connected to the backbone network. Through the deep hash module, the high-dimensional features are probabilistically sampled to obtain the corresponding feature vectors.
[0092] The feature vector is mapped to a predefined audio scene category to obtain the audio scene corresponding to the audio clip.
[0093] The embodiment of the present application provides a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured as follows:
[0094] Acquire an audio segment to be recognized, process the audio segment, and obtain a processed audio segment;
[0095] Through quaternion filtering, the processed audio clips are converted into spectrogram data at different scales;
[0096] Input the spectrum graph data into the backbone network, and extract the high-dimensional features in the spectrum graph data through the backbone network;
[0097] The high-dimensional features are input into the deep hash module connected to the backbone network. Through the deep hash module, the high-dimensional features are probabilistically sampled to obtain the corresponding feature vectors.
[0098] The feature vector is mapped to a predefined audio scene category to obtain the audio scene corresponding to the audio clip.
[0099] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.
[0100] The devices and media provided in the embodiments of the present application correspond one-to-one to the methods. Therefore, the devices and media also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.
[0101] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0102] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0103] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0104] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0105] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0106] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0107] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0108] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0109] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. A method for audio scene recognition based on neural network, characterized in that: The method comprises: Acquire an audio segment to be recognized, and process the audio segment to obtain a processed audio segment; Through quaternion filtering, the processed audio clips are converted into spectrogram data at different scales; Inputting the spectrum graph data into a backbone network, and extracting high-dimensional features from the spectrum graph data through the backbone network; Inputting the high-dimensional features into a deep hash module connected to the backbone network, and performing probability sampling on the high-dimensional features through the deep hash module to obtain corresponding feature vectors; The feature vector is mapped to a predefined audio scene category to obtain an audio scene corresponding to the audio clip.
2. The method for audio scene recognition based on a neural network according to claim 1, characterized in that: Processing the audio segment to obtain a processed audio segment specifically includes: Performing frame processing on the audio segment to divide the audio segment into a plurality of audio signals of fixed duration; For a plurality of frames of audio signals obtained after the frame processing, a windowing operation is performed on the audio signals based on a preset window function; The audio signal obtained after the windowing operation is overlapped to reduce the edge effect between the audio signals of adjacent frames.
3. The method for audio scene recognition based on a neural network according to claim 2, characterized in that: The window function is specifically expressed as: Where w(n) is the point of the nth sample value in the window function, and N is the total length of the window function.
4. The method for audio scene recognition based on a neural network according to claim 1, characterized in that: Through quaternion filtering, the processed audio clips are converted into spectrogram data at different scales, including: Generate real convolution kernel and imaginary convolution kernel through quaternion filtering; Decomposing the audio clip into initial spectrogram data at three different scales, using the initial spectrogram data as components, respectively convolving with the real convolution kernel and cross-multiplying with the imaginary convolution kernel to obtain corresponding first spectrogram data and second spectrogram data; The first spectrum graph data and the second spectrum graph data are superimposed to obtain spectrum graph data at different scales.
5. The method for audio scene recognition based on neural network according to claim 2, characterized in that: Before performing a windowing operation on the audio signal based on a preset window function, the method further includes: Performing time-frequency analysis on the audio segment to determine whether the statistical characteristics corresponding to the audio segment change over time, so as to obtain the signal type corresponding to the audio segment; wherein the signal type includes a stationary signal and a non-stationary signal; According to the signal type, a total length corresponding to a preset window function is determined; wherein the total length is positively correlated with the smoothness of the signal type.
6. The method for audio scene recognition based on neural network according to claim 1, characterized in that: The high-dimensional features are subjected to probability sampling by the deep hash module, specifically including: By means of the deep hash module, a number of sampling points contained in the high-dimensional features are classified into types to obtain corresponding sampling point types; wherein the sampling point types include anchor points and difficult sampling points; For the difficult sampling points, probability sampling is performed on the difficult sampling points according to a preset probability function, so as to extract target sampling points that can reflect high-dimensional distribution characteristics from the difficult sampling points and obtain feature vectors corresponding to the target sampling points.
7. The method for audio scene recognition based on neural network according to claim 6, characterized in that: The probability function is specifically expressed as: Among them, x i and x j represents a point in n-dimensional Euclidean space, D L Represents x i and x j The distance between.
8. The method for audio scene recognition based on a neural network according to claim 2, characterized in that: Before performing frame processing on the audio segment to divide the audio segment into a plurality of audio signals of fixed duration, the method further includes: The phonemes of the audio segment are determined, and based on the phonemes, a frame length used when the audio segment is divided into frames is determined; wherein the frame length is smaller than the length of the phonemes.
9. An audio scene recognition device based on a neural network, characterized in that: The device comprises: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the neural network-based audio scene recognition method as described in any one of claims 1-8.
10. A non-volatile computer storage medium storing computer executable instructions, characterized in that: The computer executable instructions are configured to: A method for audio scene recognition based on a neural network as described in any one of claims 1 to 8.