Audio recognition method, hearing aid and hearing aid scene recognition method

Abstract

The invention relates to an audio recognition method, a hearing aid and a hearing aid scene recognition method, and the method comprises the steps: carrying out the sampling of an audio, and obtaining the frequency spectrum information of the audio in each sampling window; extracting features of the frequency spectrum information to obtain a plurality of short-time features of the audio; and obtaining a long-time feature of the audio according to the plurality of short-time features, and identifying a scene corresponding to the audio based on the long-time feature. By adopting the method, the problems of low accuracy and large memory demand of a scene recognition method can be solved.

Description

Technical Field

[0001] The present application relates to the field of audio recognition, and in particular to an audio recognition method, a hearing aid, and a hearing aid scene recognition method. Background Art

[0002] Audio scene recognition analyzes sound signals to determine the environment or scene in which the sound is located. It is mostly used in scenarios or devices such as smart security systems, assistive hearing devices, and smart homes.

[0003] In the scene recognition methods in the related art, the recognition of the scene is often achieved based on artificially designed features. For example, when scene recognition is performed based on machine learning methods such as support vector machines and hidden Markov models, it is very dependent on prior knowledge such as feature design, which will lead to limited applicable scenarios for audio recognition and inability to accurately recognize complex real-life scenes. Deep learning methods rely on the feature extraction capabilities of CNN (Convolutional Neural Networks) or Self-Attention (self-attention mechanism). This method has a large spatial complexity and is often not supported by products with limited memory. Although the recurrent neural network has low spatial complexity, its memory capacity is limited. When processing longer time series, it is easy to forget historical information, and its recognition accuracy is low during application.

[0004] Currently, no effective solution has been proposed to address the problems of low accuracy and large memory requirements of scene recognition methods in related technologies. Summary of the invention

[0005] Based on this, it is necessary to provide an audio recognition method, a hearing aid, and a hearing aid scene recognition method that can solve the problems of low accuracy and large memory requirements of scene recognition methods in response to the above technical problems.

[0006] In a first aspect, an audio recognition method is provided in this embodiment, the method comprising:

[0007] Sampling the audio to obtain frequency spectrum information of the audio in each sampling window;

[0008] Extracting features of the frequency spectrum information to obtain multiple short-term features of the audio;

[0009] A long-term feature of the audio is obtained according to the multiple short-term features, and a scene corresponding to the audio is identified based on the long-term feature.

[0010] In some embodiments, extracting the features of the spectrum information to obtain a plurality of short-term features of the audio includes:

[0011] Inputting the spectrum information into a first neural network;

[0012] The spectrum information is dimensionally upgraded based on the first neural network, and the dependency relationship in the spectrum information is learned to output the short-time feature.

[0013] In some of the embodiments, the first neural network includes: a projection layer and a first recurrent neural network layer, the output of the projection layer is connected to the input of the first recurrent neural network layer, the projection layer receives the spectral information, and the first recurrent neural network layer outputs the short-time features.

[0014] In some embodiments, obtaining the long-term features of the audio according to the multiple short-term features, and identifying the scene corresponding to the audio based on the long-term features, includes:

[0015] Inputting a plurality of the short-term features into a second neural network;

[0016] Based on the second neural network, multiple short-term features are modeled to obtain long-term features of the audio, and scenes corresponding to the audio are classified based on the long-term features, and classification results are output.

[0017] In some embodiments, the second neural network includes: a second recurrent neural network layer and a classifier, the output of the second recurrent neural network layer is connected to the input of the classifier, the second recurrent neural network layer is used to receive the short-time features and output the long-time features; the classifier is used to receive the long-time features and output the scene corresponding to the audio.

[0018] In some embodiments, extracting the features of the spectrum information to obtain a plurality of short-term features of the audio includes:

[0019] Feature extraction is performed on the spectrum information in each sampling window based on the sampling order to obtain short-term features corresponding to each sampling window.

[0020] In some embodiments, obtaining the long-term feature of the audio according to the short-term feature includes:

[0021] Based on the generation order of the short-term features, sequentially acquiring a plurality of the short-term features;

[0022] Extracting the short-term features obtained at the current moment to obtain the long-term features;

[0023] The long-term feature is updated according to the feature extraction result of the short-term feature obtained at the next moment, until the number of updates of the long-term feature reaches a preset number, and the updating of the long-term feature is stopped.

[0024] In a second aspect, a hearing aid is provided in this embodiment, including a scene recognition module and a parameter adjustment module;

[0025] The scene recognition module is used to implement the audio recognition method described in the first aspect above;

[0026] The parameter adjustment module is used to obtain a corresponding hearing aid adjustment strategy according to the scene recognition result output by the scene recognition module, and adjust the hearing aid based on the hearing aid adjustment strategy.

[0027] In a third aspect, a hearing aid scene recognition method is provided in this embodiment, including:

[0028] Sampling the audio picked up by the hearing aid to obtain frequency spectrum information of the audio in each sampling window;

[0029] The spectral information is input into the scene recognition model, and the scene corresponding to the audio is output; wherein the hearing aid recognition model is used to extract features of the spectral information to obtain multiple short-time features of the audio; multiple short-time features are modeled to obtain long-time features of the audio, and the scene corresponding to the audio is obtained based on the long-time feature recognition.

[0030] In some embodiments, the hearing aid recognition model includes: a first neural network and a second neural network, wherein the first neural network is connected to the second neural network;

[0031] The first neural network is used to extract features of the spectrum information to obtain short-term features of the audio;

[0032] The recurrent neural network layer in the second neural network is used to model the multiple short-time features to obtain the long-time features of the audio, and the classifier in the second neural network is used to identify the scene corresponding to the audio based on the long-time features.

[0033] The above-mentioned audio recognition method, hearing aid and hearing aid scene recognition method first extract the short-time features of the audio, and then obtain the long-time features of the entire audio based on the short-time features, avoiding the situation of previous historical features when extracting features, and have low spatial complexity and small memory requirements, thereby solving the problems of low accuracy and large memory requirements of audio scene recognition methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A diagram of an application environment of an audio recognition method in an embodiment;

[0035] Figure 2 is a flowchart of an audio recognition method in one embodiment;

[0036] Figure 3is a schematic diagram of a process of hearing aid scene recognition in one embodiment;

[0037] Figure 4 is a schematic diagram of a hearing aid scene recognition method in another embodiment;

[0038] Figure 5 is a structural block diagram of a feature network in an embodiment;

[0039] Figure 6 is a structural block diagram of a hearing aid in one embodiment;

[0040] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0042] The audio recognition method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The terminal or a system including the terminal or the server samples and extracts features of the audio, performs long-term feature extraction based on the short-term feature extraction results, obtains the overall features of the audio, and realizes the scene recognition of the audio based on the overall features of the audio. Among them, the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0043] In one embodiment, Figure 2 As shown, an audio recognition method is provided, which is applied to Figure 1 The terminal 102 in the example is used as an example to illustrate, and the following steps are included:

[0044] Step S202: Sample the audio to obtain frequency spectrum information of the audio in each sampling window.

[0045] Among them, the audio can be obtained by real-time sound pickup, or it can be a pre-acquired sound to be recognized. The window length corresponding to each sampling window can be set according to the application requirements. There may be overlapping parts between the sampling time periods obtained based on the sampling windows, or there may be no overlapping. The spectrum information can be the logarithmic amplitude spectrum, logarithmic Mel spectrum, etc. of the audio. Optionally, the audio is obtained by real-time pickup, and the audio is sampled based on a specified frequency, and the analog signal is converted into a data signal. The logarithmic amplitude spectrum of the audio within the specified window length is obtained by sliding the sampling window.

[0046] Step S204: extracting features of the spectrum information to obtain multiple short-term features of the audio.

[0047] The characteristics of the spectrum information may be one or more characteristics of the frequency component, energy distribution, structure, etc. of the audio, such as the zero crossing rate, sub-band energy ratio, fundamental frequency, etc. Short-term characteristics refer to the characteristics of the audio in a shorter time.

[0048] Optionally, the spectrum information in a sampling window is used as input data, or multiple spectrum information in multiple consecutive sampling windows is used as input data, and is input into a neural network for audio feature extraction, and the neural network outputs short-term features corresponding to the input data. Repeat the above data input steps to obtain multiple short-term features.

[0049] Step S206, obtaining long-term features of the audio according to the multiple short-term features, and identifying the scene corresponding to the audio based on the long-term features.

[0050] Among them, the long-term feature refers to the overall feature of the audio, which can be one or more features of the audio such as frequency components, energy distribution, structure, etc. Optionally, multiple short-term features are input into another neural network for audio feature extraction, and the neural network extracts the overall features from the multiple short-term features and outputs the long-term feature. Optionally, the audio scene corresponding to the long-term feature is identified by a machine learning method such as a neural network or a classification model.

[0051] In the above audio recognition method, multiple short time sequences in the long audio sequence are obtained by sampling through a sampling window, and the spectrum information of the multiple short time sequences is obtained, and the short-time features corresponding to the short time sequences are obtained through the characteristics of the spectrum information; then the overall features of these short-time features are extracted to obtain the long-time features of the audio, thereby realizing audio scene recognition based on the long-time features. The spatial complexity in scene recognition is low and the memory requirement is small. In addition, by splitting the audio, the problem of easy forgetting of historical information in the process of extracting long-time series features is avoided, thereby achieving the effect of improving accuracy. The problems of low accuracy and large memory requirement of the scene recognition method are solved.

[0052] In one embodiment, features of spectral information are extracted to obtain multiple short-time features of audio, including: inputting the spectral information into a first neural network; upgrading the spectral information based on the first neural network, learning dependencies in the spectral information, and outputting short-time features.

[0053] Among them, the first neural network is used to realize audio feature extraction. By upgrading the dimension of the spectrum information, the expression ability of the first neural network can be increased, so that the first neural network is capable of learning more complex and higher-level features. Optionally, the dimension of the spectrum information is upgraded by a fully connected layer, adding a convolution channel, etc., and feature extraction is realized by a recurrent neural network, a long short-term memory network, and other networks. It is understandable that if the spectrum information is not upgraded and the features of the spectrum information are directly extracted, short-term features can also be obtained.

[0054] Furthermore, the first neural network includes: a projection layer and a first recurrent neural network layer, the output of the projection layer is connected to the input of the first recurrent neural network layer, the projection layer receives the spectrum information, and the first recurrent neural network layer outputs the short-time features. After receiving the spectrum information, the projection layer performs dimension upgrade on the spectrum information, and the first recurrent neural network layer is used to extract features from the spectrum information after dimension upgrade, thereby outputting short-time features. Optionally, the first recurrent neural network layer is composed of stacked GRU (Gated Recurrent Unit) units. Alternatively, the first recurrent neural network can also use RNNs (recurrent neural networks) and LSTMs (long short-term memory networks). In this embodiment, the feature extraction of the spectrum information is achieved by the first neural network, the spatial complexity is low and no artificial design of features is required, so that the audio recognition method can achieve a high recognition accuracy in various scenarios.

[0055] In one embodiment, long-time features of audio are obtained based on multiple short-time features, and scenes corresponding to the audio are identified based on the long-time features, including: inputting multiple short-time features into a second neural network; modeling multiple short-time features based on the second neural network to obtain long-time features of audio, and classifying scenes corresponding to the audio based on the long-time features, and outputting classification results.

[0056] The second neural network is also used to realize audio feature extraction. Optionally, multiple short-time features are input into the second neural network synchronously or asynchronously, and the second neural network learns the connection between multiple short-time features at different time points to obtain the overall features of multiple short-time features, that is, long-time features. The long-time features can be classified by existing classifiers with classification functions, such as support vector machines, random forests, etc. to realize scene recognition, or they can be various types of neural networks with feature recognition functions obtained by training.

[0057] Furthermore, the second neural network includes: a second recurrent neural network layer and a classifier, the output of the second recurrent neural network layer is connected to the input of the classifier, the second recurrent neural network layer is used to receive short-time features and output long-time features; the classifier is used to receive long-time features and output the scene corresponding to the audio. Among them, the second recurrent neural network layer is used to extract features from short-time features, thereby outputting long-time features. The second recurrent neural network can also use RNNs, LSTMs, and GRU. Optionally, the second recurrent neural network is composed of stacked GRU units, and the classifier is FNN (Feedforward Neural Network).

[0058] In this embodiment, the feature extraction of the spectrum information is realized by the second neural network, which has low spatial complexity and occupies less memory. In addition, there is no need to manually design features to extract long-term features, so that the audio recognition method can be applied to various scenarios to achieve the effect of improving the accuracy of scene recognition.

[0059] In one embodiment, extracting features of the spectrum information to obtain multiple short-time features of the audio includes: extracting features of the spectrum information in each sampling window based on the sampling order to obtain multiple short-time features.

[0060] Among them, according to the sampling windows at different moments, multiple short-time features corresponding to each other can be obtained. Optionally, based on the audio playback order or the audio pickup order, feature extraction is performed on the spectrum information of each sampling window. The sampling window length is fixed, and spectrum information corresponding to multiple time periods is obtained according to the sampling windows at different moments. When extracting features from the frequency information in the time period corresponding to a certain sampling window, the method includes: after extracting features from the spectrum information of the first sampling frame in the time period, a first short-time feature is obtained; each time the features of the spectrum information of a new sampling frame are extracted subsequently, the first short-time feature is updated according to the newly extracted features, until the feature extraction of the spectrum information in the time period corresponding to the sampling window is completed, and the first short-time feature obtained is the short-time feature corresponding to the sampling window.

[0061] Optionally, the spectrum information is input to the first neural network in a streaming input manner. In the time period corresponding to the first sampling window, each time the first neural network receives a piece of spectrum information, it extracts features for it, and updates the previous short-time feature based on the feature of the extraction result of the latter feature. After the corresponding time of the sampling window ends, the updated feature is used as the short-time feature corresponding to the first sampling window. In the time period corresponding to the second sampling window, the first neural network receives new spectrum information, and based on the features of the subsequent spectrum information received in the time period corresponding to the second sampling window, updates the features of the new spectrum information to obtain the short-time features corresponding to the second sampling window. Based on the same principle, multiple short-time features are obtained according to sampling windows at different times. After obtaining the long-time features based on the multiple short-time features, stop acquiring the audio, no longer extract the short-time features, and identify the scene corresponding to the audio based on the long-time features.

[0062] Further, in one embodiment, extracting the long-term features of the audio according to the short-term features includes: obtaining multiple short-term features in sequence based on the generation order of the short-term features; extracting the features of the short-term features obtained at the current moment to obtain the long-term features; updating the long-term features according to the feature extraction results of the short-term features obtained at the next moment, until the number of updates of the long-term features reaches a preset number, and then stopping the updating of the long-term features. The preset number can be set according to the requirements.

[0063] Optionally, the short-term features are input to the second neural network in a streaming input manner. The second neural network continuously models the short-term features based on the newly acquired short-term features, thereby extracting information of the entire time series, updating the long-term features, and obtaining the final recognition result.

[0064] In this embodiment, audio features are extracted by real-time updating, and feature extraction can be achieved without complete audio, thereby improving the real-time performance of audio scene recognition.

[0065] Based on the same inventive concept, in one embodiment, Figure 3 As shown, the embodiment of the present application also provides a hearing aid scene recognition method. A hearing aid scene recognition method is provided, such as Figure 3 As shown, including:

[0066] Step S302, sampling the audio picked up by the hearing aid to obtain the frequency spectrum information of the audio in each sampling window. Optionally, after the hearing aid starts to operate, it receives external audio in real time to obtain the audio picked up by the hearing aid.

[0067] Step S304, input the spectrum information into the scene recognition model, and output the scene corresponding to the audio; wherein the hearing aid recognition model is used to extract the features of the spectrum information to obtain multiple short-time features of the audio; multiple short-time features are modeled to obtain long-time features of the audio, and the scene corresponding to the audio is obtained based on the long-time feature recognition.

[0068] Among them, a hearing aid is a device that helps hearing-impaired patients improve their hearing. In different usage scenarios, the wearer has different demands. For example, in a noisy scene, the wearer wants to suppress the noise; in a conversation scene, the wearer wants to hear the voice clearly. Due to hearing loss, the wearer often cannot accurately judge the scene they are in. Therefore, this embodiment automatically identifies the sound picked up by the hearing aid microphone to judge the current scene, and optimizes the sound in a targeted manner, which can achieve the effect of improving the wearer experience and reducing operating costs.

[0069] Optionally, the hearing aid recognition model includes: a first neural network and a second neural network, the first neural network is connected to the second neural network; wherein the first neural network is used to extract features of spectral information to obtain short-term features of the audio; the recurrent neural network layer in the second neural network is used to model multiple short-term features to obtain long-term features of the audio, and the classifier in the second neural network is used to obtain the scene corresponding to the audio based on the long-term feature recognition. Optionally, the audio samples and the corresponding scene types are used as training data, and the hearing aid recognition model including the first neural network and the second neural network is iterated to implement the training of the hearing aid recognition model. The hearing aid recognition model is trained as a whole so that the hearing aid recognition model can perform well in scene recognition.

[0070] In one embodiment, Figure 4 A schematic diagram of another hearing aid scene recognition method is provided, such as Figure 4 As shown, the hearing aid recognition model includes a short-time feature network and a long-time feature network. Among them, the short-time feature network refers to the first neural network in the above embodiment, and the long-time feature network refers to the second neural network in the above embodiment. In order to reduce memory consumption, RNN can be used as the skeleton of the short-time feature network and the long-time neural network. Furthermore, considering that the traditional RNN is prone to numerical instability during training, there is a probability of gradient disappearance or gradient explosion, so a gated neural unit (GRU) with smaller parameters and higher performance can be selected. The specific structures of the short-time feature network and the long-time feature network are shown in the figure. Figure 5 As shown. Among them, Figure 5 (a) is a schematic diagram of the short-term feature network. The first layer of the short-term feature network is a projection layer, which maps the input to a high-dimensional feature space, followed by a stack of several layers of GRU to complete the modeling of short-term time series. Figure 5(b) is a schematic diagram of the long-term feature network. The long-term feature network first models the data of the short-term neural network by stacking GRU layers to extract the information of the entire time series. A feedforward neural network is connected to the GRU layer to complete the classification of the scene.

[0071] like Figure 4 As shown, the logarithmic amplitude spectrum of the audio is selected as the input of the model. The logarithmic amplitude spectrum data of the audio at different times is streamed into the hearing aid recognition model. The short-term features are extracted by the short-term feature network, so that several short-time series are obtained according to the long-time series sampling, and the features are extracted respectively by using the short-time series. The long-time feature network receives the short-time features extracted by the short-time feature network, extracts its overall features, and obtains the long-time features. The long-time features are sent to a classifier to obtain the final recognition result.

[0072] The hearing aid scene recognition method in this embodiment has low spatial complexity and small memory requirements, and can be applied to hearing aids with limited memory capacity. By splitting the modeling process into two parts, the problem of weak feature extraction capability of the RNN network and limited memory capability for long time series is made up, and the effect of improving the accuracy of scene recognition is achieved based on a smaller memory.

[0073] Based on the same inventive concept, the embodiment of the present application also provides a hearing aid for implementing the audio recognition method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations of one or more hearing aid embodiments provided below can refer to the limitations of the audio recognition method above, and will not be repeated here.

[0074] In one embodiment, Figure 6 As shown, a hearing aid 600 is provided, comprising: a scene recognition module 601 and a parameter adjustment module 602, wherein: the scene recognition module 601 is used to implement the steps in any one or more of the above-mentioned audio recognition method embodiments. The parameter adjustment module 602 is used to obtain a corresponding hearing aid adjustment strategy according to the scene recognition result output by the scene recognition module, and adjust the hearing aid based on the hearing aid adjustment strategy. Optionally, different scenes have corresponding hearing aid adjustment strategies, and the hearing aid adjustment strategy is used to adjust the sound playback parameters of the hearing aid. The sound playback parameters include but are not limited to volume, frequency, noise suppression, etc.

[0075] In one embodiment, the scene recognition module is used to sample the audio to obtain the spectrum information of the audio in each sampling window; extract the features of the spectrum information to obtain multiple short-time features of the audio; obtain the long-time features of the audio based on the multiple short-time features, and identify the scene corresponding to the audio based on the long-time features.

[0076] Furthermore, the scene recognition module extracts features of the spectrum information to obtain multiple short-term features of the audio, including: inputting the spectrum information into a first neural network; upgrading the spectrum information based on the first neural network, learning dependencies in the spectrum information, and outputting short-term features. Optionally, the first neural network includes: a projection layer and a first recurrent neural network layer, the output of the projection layer is connected to the input of the first recurrent neural network layer, the projection layer receives the spectrum information, and the first recurrent neural network layer outputs the short-term features.

[0077] Furthermore, the scene recognition module obtains the long-term features of the audio based on the multiple short-term features, and identifies the scene corresponding to the audio based on the long-term features, including: inputting the multiple short-term features into the second neural network; modeling the multiple short-term features based on the second neural network to obtain the long-term features of the audio, and classifying the scene corresponding to the audio based on the long-term features, and outputting the classification result. Optionally, the second neural network includes: a second recurrent neural network layer and a classifier, the output of the second recurrent neural network layer is connected to the input of the classifier, the second recurrent neural network layer is used to receive short-term features and output long-term features; the classifier is used to receive long-term features and output the scene corresponding to the audio.

[0078] In one embodiment, the scene recognition module extracts features of the spectrum information to obtain multiple short-term features of the audio, including: extracting features of the spectrum information in each sampling window based on the sampling order to obtain short-term features corresponding to each sampling window. Further, the scene recognition module obtains long-term features of the audio based on the short-term features, including: obtaining multiple short-term features in sequence based on the generation order of the short-term features; extracting features of the short-term features obtained at the current moment to obtain long-term features; updating the long-term features based on the feature extraction results of the short-term features obtained at the next moment, until the number of updates of the long-term features reaches a preset number, and then stopping the updating of the long-term features.

[0079] Each module in the above-mentioned hearing aid can be implemented in whole or in part by software, hardware or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0080] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store audio data and recognition results. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an audio recognition method is implemented.

[0081] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0082] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above audio recognition method or hearing aid scene recognition method when executing the computer program.

[0083] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned audio recognition method or hearing aid scene recognition method are implemented.

[0084] In one embodiment, a computer program product is provided, comprising a computer program, which implements the steps in the above audio recognition method or hearing aid scene recognition method when executed by a processor.

[0085] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0086] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0087] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An audio recognition method, characterized in that: The method comprises: Sampling the audio to obtain frequency spectrum information of the audio in each sampling window; Extracting features of the frequency spectrum information to obtain multiple short-term features of the audio; A long-term feature of the audio is obtained according to the multiple short-term features, and a scene corresponding to the audio is identified based on the long-term feature.

2. The method according to claim 1, characterized in that The extracting the features of the spectrum information to obtain a plurality of short-term features of the audio includes: Inputting the spectrum information into a first neural network; The spectrum information is dimensionally upgraded based on the first neural network, and the dependency relationship in the spectrum information is learned to output the short-time feature.

3. The method according to claim 2, characterized in that The first neural network includes: a projection layer and a first recurrent neural network layer, the output of the projection layer is connected to the input of the first recurrent neural network layer, the projection layer receives the spectrum information, and the first recurrent neural network layer outputs the short-time features.

4. The method according to claim 1, characterized in that: The obtaining of the long-term features of the audio according to the plurality of short-term features, and identifying the scene corresponding to the audio based on the long-term features, includes: Inputting a plurality of the short-term features into a second neural network; Based on the second neural network, multiple short-term features are modeled to obtain long-term features of the audio, and scenes corresponding to the audio are classified based on the long-term features, and classification results are output.

5. The method according to claim 4, characterized in that The second neural network includes: a second recurrent neural network layer and a classifier, the output of the second recurrent neural network layer is connected to the input of the classifier, the second recurrent neural network layer is used to receive the short-time features and output the long-time features; the classifier is used to receive the long-time features and output the scene corresponding to the audio.

6. The audio recognition method according to claim 1, characterized in that: The step of extracting the features of the spectrum information to obtain a plurality of short-term features of the audio includes: Feature extraction is performed on the spectrum information in each sampling window based on the sampling order to obtain short-term features corresponding to each sampling window.

7. The method according to claim 1 or 6, characterized in that: The obtaining the long-term feature of the audio according to the short-term feature includes: Based on the generation order of the short-term features, sequentially acquiring a plurality of the short-term features; Extracting the short-term features obtained at the current moment to obtain the long-term features; The long-term feature is updated according to the feature extraction result of the short-term feature obtained at the next moment, until the number of updates of the long-term feature reaches a preset number, and the updating of the long-term feature is stopped.

8. A hearing aid, characterized in that: Including scene recognition module and parameter adjustment module; The scene recognition module is used to implement the method according to any one of claims 1 to 7; The parameter adjustment module is used to obtain a corresponding hearing aid adjustment strategy according to the scene recognition result output by the scene recognition module, and adjust the hearing aid based on the hearing aid adjustment strategy.

9. A hearing aid scene recognition method, characterized in that: include: Sampling the audio picked up by the hearing aid to obtain frequency spectrum information of the audio in each sampling window; The spectral information is input into a hearing aid recognition model, and the scene corresponding to the audio is output; wherein the hearing aid recognition model is used to extract features of the spectral information to obtain multiple short-time features of the audio; multiple short-time features are modeled to obtain long-time features of the audio, and the scene corresponding to the audio is obtained based on the long-time feature recognition.

10. The hearing aid scene recognition method according to claim 9, characterized in that: The hearing aid recognition model includes: a first neural network and a second neural network, wherein the first neural network is connected to the second neural network; wherein The first neural network is used to extract features of the spectrum information to obtain short-term features of the audio; The recurrent neural network layer in the second neural network is used to model the multiple short-time features to obtain the long-time features of the audio, and the classifier in the second neural network is used to identify the scene corresponding to the audio based on the long-time features.

Images