Audio recognition method and device, electronic equipment and storage medium

Abstract

The invention discloses an audio recognition method and device, electronic equipment and a storage medium, and relates to the field of software and platforms or other related technical fields, and the method comprises the steps: collecting audio data, and carrying out the preprocessing of the audio data; calculating a weighted decibel value of the preprocessed audio data, and comparing the weighted decibel value with a decibel threshold to obtain a comparison result; under the condition that the comparison result indicates that the weighted decibel value of the audio data is greater than the decibel threshold value, performing preliminary verification on the audio data to obtain a verification result; and under the condition that the verification result indicates that the audio data passes the preliminary verification, entering an audio identification stage through a double-cache switching mechanism, inputting the audio data into an audio identification model, and outputting an audio identification result. According to the method and the device, the technical problems of low efficiency and high hysteresis of an audio recognition mode based on a fixed high-bit-width precision calculation strategy and serial processing in related technologies are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of software and platform or other related technical fields, in particular, to an audio recognition method and device, an electronic device and a storage medium. BACKGROUND

[0002] As an important branch of artificial intelligence, sound signal recognition technology has shown broad application prospects in various application scenarios such as smart home, industrial Internet of Things, security monitoring, etc. Through real-time sensing and intelligent analysis of the sound in the environment, functions such as abnormal event alarm, voiceprint recognition, device state recognition can be realized, which has important significance in improving the intelligent level of the system and the user interaction experience.

[0003] In related technologies, the sound recognition process usually adopts a serial processing mode of "first acquisition, then processing". That is, the system needs to wait for the completion of the input of complete audio data before starting the inference and decision-making process. This serial structure not only leads to a significant response delay in the end-to-end recognition process, making it difficult to meet the needs of application scenarios with high real-time requirements, but also easily produces response lag when processing sudden acoustic events, affecting the immediacy and accuracy of the overall system. In addition, existing acoustic recognition models generally use fixed high-bit-width precision calculation strategies in the inference process, which makes it difficult to achieve a better balance between calculation efficiency and calculation accuracy.

[0004] At present, there is no effective solution to the above problems. SUMMARY

[0005] The embodiments of the present application provide an audio recognition method and device, an electronic device and a storage medium to at least solve the technical problems of low efficiency and strong hysteresis in the related art audio recognition method based on fixed high-bit-width precision calculation strategy and serial processing.

[0006] According to an aspect of an embodiment of the present application, an audio recognition method is provided, comprising: collecting audio data and pre-processing the audio data; calculating a weighted decibel value of the pre-processed audio data and comparing the weighted decibel value with a decibel threshold to obtain a comparison result, wherein the decibel threshold is updated in real time based on environmental noise; in the case that the comparison result indicates that the weighted decibel value of the audio data is greater than the decibel threshold, performing preliminary verification on the audio data to obtain a verification result; in the case that the verification result indicates that the audio data passes the preliminary verification, entering an audio recognition stage through a double buffer switching mechanism, inputting the audio data into an audio recognition model, and outputting an audio recognition result, wherein the audio recognition model is a machine learning model that is pre-constructed and quantized in layers.

[0007] Further, the step of updating the decibel threshold in real time comprises: collecting environmental audio in a target time period, and performing frame processing on the environmental audio to obtain N environmental audio frames, wherein N is a positive integer; calculating the time domain energy of each environmental audio frame, and converting the time domain energy into a decibel value; calculating the noise mean and noise standard deviation of the environmental audio based on the decibel value of each environmental audio frame; and performing weighted calculation on the noise mean and the noise standard deviation to obtain the decibel threshold.

[0008] Further, the step of pre-processing the audio data comprises: performing frame processing on the audio data to obtain M audio frames, wherein M is a positive integer; calculating the decibel value of each audio frame, and calculating the median of the decibel values based on the decibel value of each audio frame; for each audio frame, calculating the deviation value of the decibel value of the audio frame from the median of the decibel values; identifying transient abnormal signals based on the deviation value, and processing the transient abnormal signals.

[0009] Further, the step of calculating the weighted decibel value of the pre-processed audio data comprises: filtering the audio data based on a target frequency band where the to-be-identified sound source signal is located to obtain target frequency band audio data; configuring weight values for the target frequency band audio data and the audio data of the full frequency band, respectively; and calculating the weighted decibel value of the audio data based on the decibel value of the target frequency band audio data, the weight value of the target frequency band audio data, the decibel value of the audio data of the full frequency band, and the weight value of the audio data of the full frequency band.

[0010] Further, the step of performing preliminary verification on the audio data to obtain a verification result comprises: performing frame processing on the audio data to obtain K audio frames, wherein K is a positive integer; performing Fourier transform on each audio frame to convert the audio frame into a frequency domain signal; applying a Mel frequency filter on the frequency domain signal to extract Mel spectrum features through the Mel frequency filter; and verifying whether a target sound source signal exists in the audio data according to the Mel spectrum features to obtain the verification result.

[0011] Further, the step of entering the audio recognition stage through the double buffer switching mechanism comprises: determining a current process of collecting the audio data; controlling the current process to perform an audio recognition task, and controlling other processes to perform a frequency collection task.

[0012] Further, the step of inputting the audio data into the audio recognition model and outputting an audio recognition result comprises: vector encoding the audio data and inputting the audio data into the audio recognition model; performing feature extraction on the audio data by a feature extraction layer of the audio recognition model to obtain audio features, wherein the feature extraction layer adopts a differential quantizer for hierarchical quantization in a training process; calculating a probability distribution of signal classification by a recurrent neural network of the audio recognition model, and generating and outputting the audio recognition result based on the probability distribution of signal classification, wherein the recurrent neural network adopts the differential quantizer for hierarchical quantization in the training process, and a quantization bit width of the feature extraction layer is smaller than a quantization bit width of the recurrent neural network.

[0013] According to another aspect of the embodiments of the present application, an audio recognition device is further provided, comprising: a collection unit configured to collect audio data and pre-process the audio data; a comparison unit configured to calculate a weighted decibel value of the pre-processed audio data, and compare the weighted decibel value with a decibel threshold to obtain a comparison result, wherein the decibel threshold is updated in real time based on environmental noise; a verification unit configured to, in a case where the comparison result indicates that the weighted decibel value of the audio data is greater than the decibel threshold, perform preliminary verification on the audio data to obtain a verification result; and a recognition unit configured to, in a case where the verification result indicates that the audio data passes the preliminary verification, enter an audio recognition stage by a double-buffer switching mechanism, input the audio data into an audio recognition model, and output an audio recognition result, wherein the audio recognition model is a machine learning model that is pre-constructed and subjected to hierarchical quantization.

[0014] Further, the audio recognition device comprises: a first collection module configured to collect environmental audio in a target time period and perform frame processing on the environmental audio to obtain N environmental audio frames, wherein N is a positive integer; a first calculation module configured to calculate a time domain energy of each of the environmental audio frames and convert the time domain energy into a decibel value; a second calculation module configured to calculate a noise mean value and a noise standard deviation of the environmental audio based on the decibel values of the environmental audio frames; and a third calculation module configured to perform weighted calculation on the noise mean value and the noise standard deviation to obtain the decibel threshold.

[0015] Further, the collecting unit comprises: a first processing module, configured to perform frame processing on the audio data to obtain M audio frames, wherein M is a positive integer; a fourth calculating module, configured to calculate a decibel value of each of the audio frames and calculate a median of the decibel values based on the decibel value of each of the audio frames; a fifth calculating module, configured to calculate, for each of the audio frames, a deviation value of the decibel value of the audio frame from the median of the decibel values; and a first identifying module, configured to identify an instantaneous abnormal signal based on the deviation value and to process the instantaneous abnormal signal.

[0016] Further, the comparing unit comprises: a first filtering module, configured to filter the audio data based on a target frequency band in which a sound source signal to be identified is located to obtain target frequency band audio data; a first configuring module, configured to configure a weight value for the target frequency band audio data and the audio data in a full frequency band, respectively; and a sixth calculating module, configured to calculate a weighted decibel value of the audio data based on a decibel value of the target frequency band audio data, the weight value of the target frequency band audio data, a decibel value of the audio data in the full frequency band, and a weight value of the audio data in the full frequency band.

[0017] Further, the verifying unit comprises: a second processing module, configured to perform frame processing on the audio data to obtain K audio frames, wherein K is a positive integer; a first transforming module, configured to perform Fourier transform on each of the audio frames to convert the audio frame into a frequency domain signal; a first extracting module, configured to apply a Mel frequency filter on the frequency domain signal to extract a Mel frequency spectrum feature through the Mel frequency filter; and a first verifying module, configured to verify whether a target sound source signal exists in the audio data according to the Mel frequency spectrum feature to obtain the verification result.

[0018] Further, the identifying unit comprises: a first determining module, configured to determine a current process of collecting the audio data; and a first control module, configured to control the current process to perform an audio identification task and control other processes to perform a frequency collecting task.

[0019] Further, the recognition unit further includes: a first encoding module, configured to perform vector encoding on the audio data, and input the audio data to the audio recognition model; a second extraction module, configured to perform feature extraction on the audio data through a feature extraction layer of the audio recognition model, to obtain audio features, wherein the feature extraction layer adopts a differential quantizer to perform hierarchical quantization in a training process; and a first generation module, configured to calculate a probability distribution of signal classification through a recurrent neural network of the audio recognition model, and generate and output the audio recognition result based on the probability distribution of the signal classification, wherein the recurrent neural network adopts the differential quantizer to perform hierarchical quantization in the training process, and a quantization bit width of the feature extraction layer is smaller than a quantization bit width of the recurrent neural network.

[0020] According to another aspect of the embodiments of the present application, a computer readable storage medium is also provided, which includes a stored computer program, wherein the computer readable storage medium controls a device where the computer readable storage medium is located to perform any of the above audio recognition methods when the computer program is running.

[0021] According to another aspect of the embodiments of the present application, an electronic device is also provided, which includes one or more processors and a memory, the memory being configured to store one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement any of the above audio recognition methods.

[0022] In the present application, by the following steps: collecting audio data, and pre-processing the audio data, calculating a weighted decibel value of the pre-processed audio data, and comparing the weighted decibel value with a decibel threshold to obtain a comparison result, wherein the decibel threshold is updated in real time based on environmental noise, then in the case that the comparison result indicates that the weighted decibel value of the audio data is greater than the decibel threshold, performing preliminary verification on the audio data to obtain a verification result, and finally in the case that the verification result indicates that the audio data passes the preliminary verification, entering an audio recognition stage through a double buffer switching mechanism, inputting the audio data to an audio recognition model, and outputting an audio recognition result, wherein the audio recognition model is a machine learning model that is pre-constructed and subjected to hierarchical quantization.

[0023] In the present application, the double buffer switching mechanism is adopted to realize the parallel processing mode of audio acquisition and audio recognition. In the case where the target audio is detected, the audio can be identified in time, and the real-time acquisition of audio data is ensured, the lag of audio processing is reduced, the real-time of audio processing is ensured, and the different bit width accuracy is configured for different processing networks of the model by using the hierarchical quantization model training mode, the model calculation and identification efficiency is improved, and the calculation accuracy is ensured, thereby solving the technical problems of low efficiency and strong lag in the related art. BRIEF DESCRIPTION OF DRAWINGS

[0024] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the present application and together with the description serve to explain the present application. In the drawings:

[0025] Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing an audio recognition method is shown;

[0026] Figure 2 A flowchart of an optional audio recognition method according to an embodiment of the present application is shown;

[0027] Figure 3 A schematic diagram of an optional audio recognition flow according to an embodiment of the present application is shown;

[0028] Figure 4 A principle diagram of an optional double buffer switching mechanism according to an embodiment of the present application is shown;

[0029] Figure 5 A schematic diagram of an optional audio recognition device according to an embodiment of the present application is shown;

[0030] Figure 6 A hardware structure block diagram of an electronic device (or mobile device) for implementing an audio recognition method according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0031] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative labor should fall within the scope of protection of the present application.

[0032] It is to be noted that the terms "first", "second", and the like in the description and in the claims of the present application and the above-described drawings are intended to distinguish similar objects and not necessarily describe a particular sequential or chronological order. It should be understood that the terms so used are interchangeable under appropriate circumstances and that the embodiments of the application described herein are capable of functioning in other sequences than the one described or illustrated herein. Furthermore, the terms "comprise", "comprising", "include", "including", and any variations thereof are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a list of steps or elements but not necessarily limited to those recited, but can include additional steps or elements not expressly listed or inherent to such process, method, product, or apparatus.

[0033] To facilitate the understanding of the present application for those skilled in the art, the following explanations are made for some terms or names involved in the embodiments of the present application:

[0034] Flash memory, i.e. flash, is a non-volatile storage technology that can keep data from being lost after power failure.

[0035] It should be noted that the audio recognition method and device in the present application can be used in the field of software and platform for recognizing audio based on parallel processing and artificial intelligence, and can also be used in any field other than the field of software and platform for recognizing audio based on parallel processing and artificial intelligence. The application field of the audio recognition method and device in the present application is not limited.

[0036] It should be noted that the information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for display, analyzed data, etc.) involved in the present application are information and data authorized by the user or authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of related data comply with relevant laws, regulations and standards, necessary security measures are taken, do not violate public order and good customs, and provide corresponding operation portal for user to choose authorization or refusal. For example, the system and the related users or institutions are provided with an interface, which provides a corresponding operation portal for the user to choose to agree or refuse the automatic decision result; if the user chooses to refuse, the expert decision process is entered.

[0037] The following embodiments of the present application can be applied to various audio recognition systems / applications / devices. The present application introduces an audio data stream parallel inference mechanism to realize parallel processing of sound collection and sound recognition, thereby greatly reducing the response delay; at the same time, a hybrid quantization strategy is adopted to quantize different bit widths and precision of different layers of the model, to realize the optimal balance between calculation precision and resource utilization.

[0038] The application will be described in detail below with reference to various embodiments.

[0039] Embodiment one

[0040] According to the embodiments of the present application, an embodiment of an audio recognition method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0041] The method embodiment provided by the embodiment one of the present application can be executed in a mobile terminal, a computer terminal or similar computing device. Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing the audio recognition method is shown. As shown in Figure 1 , the computer terminal 10 (or mobile device) can include one or more processors 102 (the processor 102 can include but not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it can also include a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which can be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. Those skilled in the art can understand that Figure 1 The structure shown is only schematic, which does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 can include more or less components than those shown in Figure 1 , or have a different configuration than that shown in Figure 1 .

[0042] It should be noted that the one or more processors 102 and / or other data processing circuits described above can be referred to herein as "data processing circuits" in general. The data processing circuit can be embodied in whole or in part as software, hardware, firmware or any other combination. In addition, the data processing circuit can be a single independent processing module, or any one of the other elements combined into the computer terminal 10 (or mobile device) in whole or in part. As referred to in the embodiments of the present application, the data processing circuit as a kind of processor control (for example, the selection of the variable resistance terminal path connected with the interface).

[0043] The memory 104 can be used to store software programs of application software and modules, such as program instructions / data storage means corresponding to the audio recognition method in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, i.e. implements the audio recognition method described above. The memory 104 can include a high-speed random access memory, and can further include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 104 can further include memories remotely arranged with respect to the processor 102, which can be connected to the computer terminal 10 through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0044] The transmission device 106 is used to receive or send data via a network. The specific examples of the network can include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network interface controller (NIC) which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module which is used to communicate with the Internet in a wireless manner.

[0045] The display can be, for example, a touch screen type liquid crystal display (LCD) which can enable a user to interact with the user interface of the computer terminal 10 (or a mobile device).

[0046] Under the above operating environment, the present application provides an audio recognition method as shown in Figure 2 The implementation subject of the method is an audio recognition system.

[0047] Figure 2 is a flowchart of an optional audio recognition method according to an embodiment of the present application, as shown in Figure 2 The method includes the following steps:

[0048] In step S201, audio data is collected and preprocessed.

[0049] In the step S201, the sound data in the environment is collected through the microphone or the audio sensor, and the collected audio data is the basis for the system to perform sound recognition. The collected audio is pre-processed, specifically including frame processing, band-pass filtering, noise reduction processing, instantaneous abnormal signal identification and processing, etc. Through the pre-processing, high-quality audio data is obtained, the signal-to-noise ratio is improved, the influence of background noise on signal recognition is significantly reduced, and clear and accurate data basis is provided for subsequent sound recognition and feature analysis.

[0050] Further, the step of pre-processing the audio data includes: performing frame processing on the audio data to obtain M audio frames, where M is a positive integer; calculating a decibel value of each audio frame, and calculating a median of the decibel values based on the decibel value of each audio frame; for each audio frame, calculating a deviation value of the decibel value of the audio frame from the median of the decibel values; identifying an instantaneous abnormal signal based on the deviation value, and processing the instantaneous abnormal signal.

[0051] Specifically, when pre-processing the audio data, first, the continuous audio signal stream is divided into a plurality of continuous audio frames according to a fixed time window length. It is assumed that the total number of audio frames obtained after the division is M, where M is a positive integer. The purpose of this step is to cut the continuous audio signal into short-time stable analysis units, facilitating subsequent feature extraction and signal processing. Each audio frame contains audio information within a certain time length, for example, an audio frame can contain 25 milliseconds of signal. For each audio frame, its energy or amplitude is calculated to convert it into a decibel value. The decibel value is a logarithmic scale representation that can intuitively reflect the relative intensity of the audio signal, and is an important parameter in sound signal preprocessing.

[0052] Based on the decibel values of the M audio frames obtained above, the median of the decibel values of the entire audio frame sequence is calculated. The median, as a central tendency indicator of the data set, can effectively resist the interference of extreme values and provide a reference for judging the range of normal audio frames. For each audio frame, the decibel value of the frame is compared with the median of the decibel values calculated above to calculate its deviation value. The deviation value reflects the difference between the energy of the current audio frame and the overall median value, and is a key basis for identifying instantaneous abnormal signals.

[0053] Based on the deviation value of each audio frame, the median absolute deviation (MAD) method is used to identify the instantaneous abnormal signal. According to the deviation value of the calculated decibel value and the median of the decibel value, those audio frames that exceed the normal range of change, i.e. the instantaneous abnormal signal, are identified. The instantaneous abnormal signal may be caused by sudden noise of non-target sound sources (such as thunder sound, vehicle passing sound, etc.), which poses a potential threat to the stability and recognition performance of the system. For the instantaneous abnormal signal, signal smoothing processing, noise reduction processing, etc. can be taken to reduce its influence on target audio recognition.

[0054] Step S202, the weighted decibel value of the preprocessed audio data is calculated, and the weighted decibel value is compared with the decibel threshold to obtain a comparison result.

[0055] In the above step S202, on the basis of the preprocessed audio frame, first, the time domain energy decibel value in the full frequency range is calculated, so as to obtain the total energy representation of the audio frame at all frequencies as the basis for subsequent comparison and fusion. Then, the time domain energy decibel value of the audio frame in a specific target frequency band (for example, 1-4 kHz) is calculated. The setting of the target frequency band is based on the typical frequency distribution of the target sound source, which helps to more accurately capture the characteristics of the target sound source.

[0056] A weighting strategy based on the target frequency band and the full frequency band decibel value is adopted, and the fusion ratio is dynamically adjusted by comparing the relative energy levels of the two. When the energy of the target frequency band is not much different from the energy of the full frequency band, it means that the current signal may contain rich target information, and more decibel values of the target frequency band are used. On the contrary, if the energy of the target frequency band is much lower than that of the full frequency band, it may be interfered by a large amount of non-target noise, and the decibel value of the full frequency band is used to retain the effective information of the signal as much as possible. The goal of weighted fusion is to reduce the influence of persistent background noise such as wind noise, while maintaining the integrity of the signal and the clarity of the target sound source.

[0057] The decibel threshold is updated in real time based on the environmental noise, which is used for preliminary judgment of the audio. When the decibel value of the audio exceeds the preset decibel threshold, it indicates that there may be a target sound source, and the decibel threshold is dynamically adjusted so that the system can adapt to the fluctuations of the environmental noise, thereby realizing stable target sound source detection.

[0058] In the process of continuous audio acquisition, the weighted decibel value of each audio frame is calculated in real time, and compared with the dynamically updated decibel threshold. If the weighted decibel value exceeds the threshold, it is determined that the audio data may contain a target sound source, thereby triggering the subsequent sound recognition process. The above comparison step is the core of the system response mechanism, which helps the system to distinguish between background noise and target sound source signal, reduces the processing of invalid data, and improves the efficiency and accuracy of the entire recognition process.

[0059] Further, the step of updating the decibel threshold in real time comprises: collecting environmental audio in a target time period, and performing frame processing on the environmental audio to obtain N environmental audio frames, wherein N is a positive integer; calculating the time domain energy of each environmental audio frame and converting the time domain energy into a decibel value; calculating the noise mean and noise standard deviation of the environmental audio based on the decibel values of the environmental audio frames; and performing weighted calculation on the noise mean and noise standard deviation to obtain the decibel threshold.

[0060] Specifically, when updating the decibel threshold in real time, the system continuously monitors the surrounding environment and collects uninterrupted audio data within a specified time. The length of the specified time should be long enough to reflect the dynamic changes of environmental noise, but not too long to miss immediate acoustic events. For example, the target time period can be set to a few seconds to tens of seconds, depending on the specific application scenario.

[0061] The collected environmental audio needs to be segmented into a series of consecutive audio frames, denoted as N environmental audio frames, where N is a positive integer. The framing process is based on the principle of short-term stationarity of audio signals, dividing the audio signal into a series of short-time stable segments for subsequent feature extraction and analysis. Each environmental audio frame covers acoustic information within a specific time window, usually with a time window length of 20 to 50 milliseconds.

[0062] For each environmental audio frame, its time-domain energy is calculated, which reflects the total intensity of the audio signal within that frame time. Then, the time-domain energy is converted to a decibel value, which is a logarithmic scale of energy representation, which can more intuitively show the intensity change of the audio signal, facilitating comparison and analysis across time frames.

[0063] Based on the decibel values of all the environmental audio frames obtained above, the noise mean and noise standard deviation are calculated. The noise mean reflects the average intensity of the background noise, while the noise standard deviation reflects the fluctuation degree of the background noise intensity. These two statistics are key to identifying target sound and environmental noise, used to set a reasonable decibel threshold to distinguish between signal and noise.

[0064] Finally, the noise mean and noise standard deviation are used to calculate the real-time decibel threshold. The weighted calculation takes into account the specific noise level of the current environment and the fluctuation of the noise intensity, ensuring that the threshold setting can effectively respond to instantaneous acoustic changes while avoiding frequent false triggers. This dynamic updating mechanism of the threshold makes the system adapt to the natural fluctuations of environmental noise, ensuring stable and accurate identification of target sound in different noise environments.

[0065] Further, the step of calculating the weighted decibel value of the preprocessed audio data includes: filtering the audio data based on the target frequency band where the sound source signal to be identified is located to obtain target frequency band audio data; configuring weight values for the target frequency band audio data and the full frequency band audio data respectively; calculating the weighted decibel value of the audio data based on the decibel value of the target frequency band audio data, the weight value of the target frequency band audio data, the decibel value of the full frequency band audio data, and the weight value of the full frequency band audio data.

[0066] Specifically, the calculation of the weighted decibel value can improve the recognition ability and stability of the audio recognition system in a complex noise environment. When calculating the weighted decibel value, the audio data after preprocessing is extracted through a band-pass filter to obtain audio data in the target frequency band. The setting of the target frequency band is based on the analysis of the characteristics of the target sound source signal, ensuring that the filtered signal contains the most abundant target sound source information while removing noise unrelated to the target. For example, if the target sound source is human speech, the target frequency band may be set between 1 kHz and 4 kHz, as the main frequency components of human voice are within this range.

[0067] The filtered target frequency band audio data and the unfiltered full frequency band audio data are respectively assigned weight values. The setting of the weight values depends on the target recognition requirements and the characteristics of the noise environment. Generally, the weight value of the target frequency band is greater than that of the full frequency band, to emphasize the priority of target signal recognition. The weight allocation strategy aims to balance the signal-to-noise ratio and integrity, ensuring that the target signal characteristics are not excessively weakened while reducing the impact of noise.

[0068] The time-domain energy of the target frequency band audio data and the full frequency band audio data is calculated respectively, and the energy is converted into decibel values. The decibel value calculation is based on the logarithmic energy ratio, which can intuitively reflect the change in audio signal strength and is an effective indicator for signal strength comparison and anomaly detection.

[0069] The decibel values of the target frequency band audio data and the full frequency band audio data are combined and weighted according to the respective configured weight values. The weighted decibel value integrates the signal strength in the target frequency band and the full frequency band signal strength, more comprehensively reflecting the actual strength of the audio data in the target recognition context. This calculation method helps the recognition system to consider the target signal characteristics while taking into account the impact of environmental noise, improving the accuracy and stability of recognition.

[0070] Step S203, in the case where the comparison result indicates that the weighted decibel value of the audio data is greater than the decibel threshold value, the audio data is preliminarily verified to obtain a verification result.

[0071] In the above step S203, when the weighted decibel value calculation result shows that the intensity of the preprocessed audio data exceeds the decibel threshold value dynamically adjusted by the system, the verification mechanism is started. The decibel threshold value is updated in real time based on the statistical quantity of environmental noise in the target time period, ensuring that it can accurately reflect the current environmental noise level and avoid false triggering due to environmental changes. Further feature extraction is performed on the audio data, including Mel frequency spectrum features, which can effectively capture the spectral characteristics of the audio signal.

[0072] Subsequently, a pre-trained lightweight linear classifier, such as logistic regression or support vector machine, is used to quickly classify the audio data based on the Mel-spectral features, thus completing the preliminary verification of the audio data. The verification result provides a preliminary verification of whether the audio data contains the target sound source signal. If the classifier confirms that the features in the audio data match the features of the target sound source, the system will enter the subsequent detailed identification process; otherwise, if the verification result indicates that the features in the audio data do not match or have low matching degree with the target features, the system will ignore the audio data and continue to monitor the next audio frame until the audio data that may contain the target sound source signal is found.

[0073] Further, the step of preliminarily verifying the audio data to obtain a verification result includes: performing frame processing on the audio data to obtain K audio frames, where K is a positive integer; performing Fourier transform on each audio frame to convert the audio frame into a frequency domain signal; applying a Mel frequency filter on the frequency domain signal to extract Mel-spectral features through the Mel frequency filter; and verifying whether the target sound source signal exists in the audio data according to the Mel-spectral features to obtain the verification result.

[0074] Specifically, during the preliminary verification, the audio data is subjected to frame processing to divide it into K continuous audio frames, where K is a positive integer. The purpose of frame processing is to decompose long-time audio signals into a series of shorter, relatively stationary segments, facilitating subsequent feature extraction and analysis. Each audio frame contains acoustic information within a target time period, and the typical time window length is about 20 to 50 milliseconds, which ensures that the extraction of features can reflect the transient changes of acoustic events while maintaining the feasibility of calculation.

[0075] Fourier transform is performed on each audio frame to convert the time domain signal into a frequency domain signal. Fourier transform is a basic tool in signal processing that can reveal the frequency composition of an audio signal and separate different frequency components. A Mel frequency filter is applied on the frequency domain signal to extract Mel-spectral features. The Mel frequency filter is designed based on the characteristics of the human auditory system and can simulate the perceptual differences of different frequency signals by the human ear. Through Mel-spectral feature extraction, the frequency components related to the target sound source in the audio signal can be highlighted, while irrelevant noise frequencies are suppressed, improving the recognition rate of the target signal.

[0076] Based on the extracted Mel-spectral features, it is evaluated whether the target sound source signal exists in the audio data. The features can be distinguished by a pre-trained classifier or compared with the feature template of the known target sound source. The verification process aims to confirm that the audio data with a weighted decibel value exceeding the threshold indeed carries the features of the target sound source, avoiding false positives caused by transient noise or other non-target abnormal signals.

[0077] Step S204, in the case where the verification result indicates that the audio data passes the preliminary verification, the audio recognition phase is entered through the double buffer switching mechanism, the audio data is input to the audio recognition model, and the audio recognition result is output.

[0078] In the above step S204, in the case where the target sound source signal may exist in the preliminary verification of the audio data, the system will seamlessly transfer the audio data from the collection buffer to the recognition buffer through the double buffer switching mechanism to enter the audio recognition phase, while continuously collecting the audio stream through other processes to ensure the real-time processing of the audio stream. Through the double buffer switching, the audio data is transmitted to the audio recognition model, which is a machine learning model that is pre-constructed and has undergone hierarchical quantization processing. After the audio recognition model processes the audio data, the audio recognition result is output, which contains information such as classification results and recognition conclusions of the audio data.

[0079] Further, the step of entering the audio recognition phase through the double buffer switching mechanism includes: determining the current process of collecting the audio data; controlling the current process to perform the audio recognition task, and controlling other processes to perform the audio collection task.

[0080] Specifically, the system will run multiple processes simultaneously during the real-time monitoring phase, each process responsible for collecting audio data in different time periods. When a certain audio data passes the preliminary verification, i.e., it is confirmed that it may contain the target sound source signal, the current process responsible for collecting the audio data needs to be determined. The process is instructed to stop audio data collection and start the audio recognition task instead. This means that the audio data will be input to the pre-constructed audio recognition model for analysis and recognition. By controlling the current process to switch from collection mode to recognition mode, the system can seamlessly send the target audio data to the recognition model, avoiding data waiting and transmission delay, and speeding up the entire process from data collection to recognition result output.

[0081] At the same time that the current process enters the recognition phase, the system will automatically schedule other processes that are not involved in recognition to continue performing the audio data collection task, ensuring the continuous collection of subsequent audio data. Parallel processing of audio data collection and recognition is achieved, and through dynamic allocation and switching of tasks of each process, the system can collect new data while recognizing the target audio data, avoiding the gap between data collection and model inference, and improving the overall throughput and response speed of the system.

[0082] Further, the step of inputting the audio data into the audio recognition model and outputting an audio recognition result comprises: vector encoding the audio data and inputting the audio data into the audio recognition model; performing feature extraction on the audio data through a feature extraction layer of the audio recognition model to obtain audio features, wherein the feature extraction layer adopts a differential quantizer for hierarchical quantization in a training process; calculating a probability distribution of signal classification through a recurrent neural network of the audio recognition model, and generating and outputting an audio recognition result based on the probability distribution of signal classification, wherein the recurrent neural network adopts the differential quantizer for hierarchical quantization in the training process, and a quantization bit width of the feature extraction layer is less than a quantization bit width of the recurrent neural network.

[0083] Specifically, in the embodiment of the present application, the audio recognition model pre-trained is used for audio recognition. First, the audio data is vector encoded to convert the audio signal into a form that can be processed by the machine learning model to obtain feature vectors. Then, the feature vectors are input into the audio recognition model pre-constructed and subjected to hierarchical quantization.

[0084] The feature extraction layer of the audio recognition model receives the encoded audio data and starts feature extraction. The feature extraction layer adopts a differential quantizer for hierarchical quantization, which means that the quantization bit width is adjusted according to the layer. The role of the feature extraction layer is to extract meaningful acoustic features from the audio data for subsequent signal classification. The hierarchical quantization strategy enables the model to maintain high recognition accuracy while reducing model size and computational complexity. Lower quantization bit width is used for processing low-frequency feature extraction layers, which helps to retain sufficient information details under limited computing resources.

[0085] The audio features extracted are passed to the recurrent neural network (RNN) part of the audio recognition model, which also adopts a differential quantizer for hierarchical quantization, but with a higher quantization bit width than the feature extraction layer. The recurrent neural network is used to capture the time sequence dependency in the audio signal, which is particularly important for recognizing sounds with time sequence properties such as speech and animal calls. Higher quantization bit width is used in the recurrent neural network layer to ensure the accuracy of time-sensitive calculations, thereby improving the accuracy of signal classification.

[0086] The feature extraction layer and the recurrent neural network layer use different bit width quantization strategies, which not only ensure computational efficiency but also maintain recognition accuracy, improving the computational efficiency and resource utilization of the audio recognition model, while ensuring high-quality recognition of audio signals.

[0087] The recurrent neural network calculates the probability distribution of signal classification by analyzing the audio features, and then generates an audio recognition result based on the probability distribution. Finally, the recognition result is output.

[0088] Through the above steps, the audio data is collected and preprocessed, the weighted decibel value of the preprocessed audio data is calculated, and the weighted decibel value is compared with a decibel threshold to obtain a comparison result, wherein the decibel threshold is updated in real time based on environmental noise. Then, in the case that the comparison result indicates that the weighted decibel value of the audio data is greater than the decibel threshold, the audio data is preliminarily verified to obtain a verification result. Finally, in the case that the verification result indicates that the audio data passes the preliminary verification, the double-buffer switching mechanism is entered into an audio recognition stage, the audio data is input into an audio recognition model, and an audio recognition result is output, wherein the audio recognition model is a machine learning model that is pre-constructed and quantized in layers.

[0089] In this embodiment, the double-buffer switching mechanism is used to realize the parallel processing mode of audio collection and audio recognition. In the case that the target audio possibly exists is detected, the audio can be recognized in time, and the real-time collection of the audio data is ensured, so that the hysteresis of the audio processing is reduced, and the real-time performance of the audio processing is ensured. Meanwhile, the model training mode of layer quantization is used, different bit width precisions are configured for different processing networks of the model, the model calculation and recognition efficiency are improved, and the calculation precision is ensured, thereby solving the technical problems of the low efficiency and strong hysteresis of the audio recognition mode based on the fixed high bit width precision calculation strategy and the serial processing in the related art.

[0090] The following will be described in detail in combination with another optional specific embodiment.

[0091] Figure 3 is a schematic diagram of an optional audio recognition flow according to an embodiment of the present application, as Figure 3 shown, the audio recognition flow specifically includes:

[0092] Step one, after the audio stream is pre-emphasized, framed and windowed, the invalid frames are filtered based on the noise energy and zero-crossing rate detection, and the parallelization data stream management of the collection and inference threads is realized through the double-buffer mechanism;

[0093] Step two, whether the sound is collected, if yes, step three is executed, if not, the cache space is released, and the real-time sound collection is continued;

[0094] After the audio recognition system is started, the environmental audio data is collected and framed, the decibel value of the time domain energy is calculated, and the trigger threshold is set based on the initial noise statistics to control the false trigger probability.

[0095] The environmental audio data refers to the environmental sound signals continuously collected after the system starts, and is used to estimate the initial noise characteristics. The frame processing refers to dividing the audio data into fixed time length analysis units, so as to perform frame-by-frame feature calculation. The initial noise characteristics are obtained by calculating the time domain energy decibel value of each frame of the environmental audio, and then calculating the noise mean value and standard deviation, so as to calculate the decibel threshold value of the environmental noise. The decibel threshold value is a judgment standard set based on the noise mean value and the standard deviation, and is used to distinguish the target sound source from the environmental noise, so as to reduce the false triggering probability. The false triggering probability refers to the probability that the system is activated by mistake in the case that there is no target sound source, and is used to evaluate the stability of the system.

[0096] After entering the real-time monitoring stage, the system dynamically updates the noise statistics, and adopts a weighted strategy to fuse the decibel values of the full frequency band and the target frequency band, so as to reduce the wind noise interference. The weighted strategy refers to calculating the weighted decibel value according to the proportion of the energy of the target frequency band and the full frequency band, so as to optimize the signal detection effect. The target frequency band refers to the frequency range reserved after the band-pass filtering (1-4 kHz), and this frequency band contains the main target sound source information. The wind noise interference refers to the low-frequency noise generated by air flow, and the influence of the wind noise interference on the detection result can be reduced through the weighted strategy.

[0097] The MAD method is used to detect outliers, identify transient interference, and pause the statistics update, otherwise the noise statistics are iteratively calculated according to the smoothing factor, and the triggering threshold value is dynamically adjusted to adapt to the environmental changes. The MAD method is based on the median of the last 5 frames to calculate outliers, and is used to identify noise mutations. The transient interference refers to short-time high-intensity noise such as thunderstorm, which may affect the stability of the noise statistics of the system.

[0098] Step three, the front end of the collection process is blocked, and when the valid audio frames of the buffer reach three seconds continuously, the pointer switching is triggered;

[0099] When the weighted decibel value of a frame exceeds the decibel threshold value, a preliminary verification mechanism is triggered, and the preliminary verification mechanism determines whether there is a possible target sound source signal in the audio stream based on the rapid classification of the MFCC features. The MFCC feature is the Mel frequency cepstral coefficient extracted from the audio signal, and is used to represent the frequency spectrum characteristics of the sound. The lightweight linear classifier is preset in the Flash memory, and can complete the rapid discrimination within 3 ms, so as to improve the response speed of the system. When the audio stream passes the preliminary verification, a double-buffer switching mechanism is triggered, and the audio recognition is performed.

[0100] Step four, the audio frames are read from the filled buffer and the feature extraction and real-time recognition are performed, and the collection from another process is started at the same time;

[0101] Figure 4 It is a principle diagram of an optional double-buffer switching mechanism according to an embodiment of the present application, as shown in Figure 4As shown, the embodiment of the present application is configured with process A and process B, both of which have cache areas, process A performs real-time audio collection, and when the audio collected by process A detects an audio signal that may contain target audio, process A is switched to audio recognition, and process B is started to perform real-time audio collection.

[0102] Step five, using a hierarchical quantization strategy to perform reasoning, combining a sliding window attention mechanism to fuse multi-frame time sequence context features;

[0103] Step six, dynamically switching high-precision / balanced / power-saving modes according to device temperature, memory load and noise type, optimizing algorithm power distribution to balance delay, precision and energy consumption.

[0104] The embodiment of the present application realizes parallel processing of sound collection and sound recognition by introducing an audio data stream parallel reasoning mechanism, thereby greatly reducing response delay; at the same time, a hybrid quantization strategy is used to quantize different layers of the model with different bit widths and precision, achieving an optimal balance between calculation precision and resource utilization.

[0105] The following will be described in detail in conjunction with another embodiment.

[0106] Embodiment two

[0107] The audio recognition device provided in the embodiment includes a plurality of implementation units, each implementation unit corresponding to each implementation step in the above-mentioned embodiment one, and the specific implementation manner and beneficial effects can refer to the foregoing method embodiment, which will not be described here again.

[0108] Figure 5 is a schematic diagram of an optional audio recognition device according to the embodiment of the present application, as Figure 5 shown, the audio recognition device can include a collection unit 51, a comparison unit 52, a verification unit 53, and an identification unit 54, wherein,

[0109] The collection unit 51 is configured to collect audio data and pre-process the audio data.

[0110] The comparison unit 52 is configured to calculate a weighted decibel value of the pre-processed audio data, and compare the weighted decibel value with a decibel threshold value to obtain a comparison result, wherein the decibel threshold value is updated in real time based on environmental noise.

[0111] The verification unit 53 is configured to preliminarily verify the audio data to obtain a verification result in a case where the comparison result indicates that the weighted decibel value of the audio data is greater than the decibel threshold value.

[0112] The recognition unit 54 is configured to, in a case where the verification result indicates that the audio data passes the preliminary verification, enter an audio recognition stage through a double-buffer switching mechanism, input the audio data into an audio recognition model, and output an audio recognition result, wherein the audio recognition model is a machine learning model that is pre-constructed and subjected to hierarchical quantization.

[0113] The audio recognition device is configured to: collect audio data through the collection unit 51 and pre-process the audio data; calculate a weighted decibel value of the pre-processed audio data through the comparison unit 52, and compare the weighted decibel value with a decibel threshold to obtain a comparison result, wherein the decibel threshold is updated in real time based on environmental noise; perform preliminary verification on the audio data through the verification unit 53 in a case where the comparison result indicates that the weighted decibel value of the audio data is greater than the decibel threshold, and obtain a verification result; in a case where the verification result indicates that the audio data passes the preliminary verification, enter an audio recognition stage through a double-buffer switching mechanism through the recognition unit 54, input the audio data into an audio recognition model, and output an audio recognition result, wherein the audio recognition model is a machine learning model that is pre-constructed and subjected to hierarchical quantization.

[0114] In this embodiment, the double-buffer switching mechanism is used to implement a parallel processing mode of audio collection and audio recognition. In a case where a target audio is detected, the target audio can be identified in a timely manner, while ensuring real-time collection of audio data, reducing the lag of audio processing, and ensuring the real-time performance of audio processing. In addition, the hierarchical quantization model training mode is used to configure different bit width precisions for different processing networks of the model, thereby improving the model calculation and recognition efficiency, ensuring the calculation precision, and solving the technical problems of low efficiency and strong lag in the related art caused by the fixed high bit width precision calculation strategy and the serial processing mode of audio recognition.

[0115] Further, the audio recognition device comprises: a first collection module configured to collect environmental audio in a target time period and perform frame processing on the environmental audio to obtain N environmental audio frames, wherein N is a positive integer; a first calculation module configured to calculate a time domain energy of each environmental audio frame and convert the time domain energy into a decibel value; a second calculation module configured to calculate a noise mean value and a noise standard deviation of the environmental audio based on the decibel value of each environmental audio frame; and a third calculation module configured to perform weighted calculation on the noise mean value and the noise standard deviation to obtain a decibel threshold.

[0116] Further, the collecting unit comprises: a first processing module, configured to perform frame processing on the audio data to obtain M audio frames, wherein M is a positive integer; a fourth calculation module, configured to calculate a decibel value of each audio frame, and calculate a median of the decibel values based on the decibel value of each audio frame; a fifth calculation module, configured to calculate, for each audio frame, a deviation value of the decibel value of the audio frame from the median of the decibel values; and a first identification module, configured to identify the transient abnormal signal based on the deviation value, and to process the transient abnormal signal.

[0117] Further, the comparison unit comprises: a first filtering module, configured to filter the audio data based on a target frequency band in which the to-be-identified sound source signal is located, to obtain target frequency band audio data; a first configuration module, configured to respectively configure a weight value for the target frequency band audio data and the full frequency band audio data; and a sixth calculation module, configured to calculate a weighted decibel value of the audio data based on the decibel value of the target frequency band audio data, the weight value of the target frequency band audio data, the decibel value of the full frequency band audio data, and the weight value of the full frequency band audio data.

[0118] Further, the verification unit comprises: a second processing module, configured to perform frame processing on the audio data to obtain K audio frames, wherein K is a positive integer; a first transformation module, configured to perform Fourier transform on each audio frame to convert the audio frame into a frequency domain signal; a first extraction module, configured to apply a Mel frequency filter on the frequency domain signal to extract a Mel spectrum feature through the Mel frequency filter; and a first verification module, configured to verify whether the target sound source signal exists in the audio data according to the Mel spectrum feature, to obtain a verification result.

[0119] Further, the identification unit comprises: a first determination module, configured to determine a current process of collecting the audio data; and a first control module, configured to control the current process to perform an audio identification task, and to control other processes to perform a frequency collecting task.

[0120] Further, the identification unit further comprises: a first encoding module, configured to perform vector encoding on the audio data, and to input the audio data into the audio identification model; a second extraction module, configured to perform feature extraction on the audio data through a feature extraction layer of the audio identification model, to obtain an audio feature, wherein the feature extraction layer is subjected to hierarchical quantization by using a differential quantizer in a training process; and a first generation module, configured to calculate a probability distribution of signal classification through a recurrent neural network of the audio identification model, and to generate and output an audio identification result based on the probability distribution of signal classification, wherein the recurrent neural network is subjected to hierarchical quantization by using the differential quantizer in the training process, and a quantization bit width of the feature extraction layer is less than a quantization bit width of the recurrent neural network.

[0121] It should be noted that the above collection unit 51, comparison unit 52, verification unit 53, and identification unit 54 correspond to steps S201 to S204 in Embodiment One, and have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment One. It should be noted that the above modules or units can be hardware components or software components stored in a memory (for example, memory 104) and processed by one or more processors (for example, processors 102a, 102b, …, 102n), or can be a part of the device and can run in the computer terminal 10 provided in Embodiment One.

[0122] The application will be described in detail below in conjunction with another alternative embodiment.

[0123] Embodiment Three

[0124] The embodiments of the application can also provide an electronic device, Figure 6 is a hardware structure block diagram of an electronic device (or mobile device) according to an alternative embodiment of the application for performing an audio recognition method, as shown in Figure 6 The electronic device can include one or more (only one is shown in the figure) processors 602, a memory 604, a storage controller, and a peripheral interface, wherein the peripheral interface is connected with a radio frequency module, an audio module, and a display. Figure 6

[0125] The memory can be used to store software programs and modules, such as program instructions / modules corresponding to the methods and devices in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implements the above-mentioned method. The memory can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory can further include a memory remotely arranged with respect to the processor, which can be connected to the terminal through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0126] ​The processor can call information and application programs stored in the memory through the transmission device to perform the following steps: collecting audio data and pre-processing the audio data; calculating a weighted decibel value of the pre-processed audio data and comparing the weighted decibel value with a decibel threshold to obtain a comparison result, wherein the decibel threshold is updated in real time based on environmental noise; in the case that the comparison result indicates that the weighted decibel value of the audio data is greater than the decibel threshold, performing preliminary verification on the audio data to obtain a verification result; in the case that the verification result indicates that the audio data passes the preliminary verification, entering an audio recognition stage through a double-buffer switching mechanism, inputting the audio data into an audio recognition model, and outputting an audio recognition result, wherein the audio recognition model is a machine learning model that is pre-constructed and quantized in layers.

[0127] The processor can call information and application programs stored in the memory through the transmission device to perform the following steps: collecting environmental audio in a target time period and performing frame processing on the environmental audio to obtain N environmental audio frames, wherein N is a positive integer; calculating a time domain energy of each environmental audio frame and converting the time domain energy into a decibel value; calculating a noise mean and a noise standard deviation of the environmental audio based on the decibel values of the environmental audio frames; and performing weighted calculation on the noise mean and the noise standard deviation to obtain a decibel threshold.

[0128] The processor can call information and application programs stored in the memory through the transmission device to perform the following steps: performing frame processing on the audio data to obtain M audio frames, wherein M is a positive integer; calculating a decibel value of each audio frame and calculating a median of the decibel values based on the decibel value of each audio frame; for each audio frame, calculating a deviation value of the decibel value of the audio frame from the median of the decibel values; identifying an instantaneous abnormal signal based on the deviation value and processing the instantaneous abnormal signal.

[0129] The processor can call information and application programs stored in the memory through the transmission device to perform the following steps: filtering the audio data based on a target frequency band in which a to-be-recognized sound source signal is located to obtain target frequency band audio data; configuring a weight value for the target frequency band audio data and full-band audio data, respectively; and calculating a weighted decibel value of the audio data based on a decibel value of the target frequency band audio data, the weight value of the target frequency band audio data, a decibel value of the full-band audio data, and a weight value of the full-band audio data.

[0130] The processor can call information and application programs stored in the memory through the transmission device to perform the following steps: performing frame processing on the audio data to obtain K audio frames, wherein K is a positive integer; performing Fourier transform on each audio frame to convert the audio frame into a frequency domain signal; applying a Mel frequency filter to the frequency domain signal to extract a Mel spectrum feature through the Mel frequency filter; and verifying whether a target sound source signal exists in the audio data according to the Mel spectrum feature to obtain a verification result.

[0131] The processor can call information and application programs stored in the memory through the transmission device to perform the following steps: determining a current process of collecting audio data; controlling the current process to perform an audio recognition task, and controlling other processes to perform a frequency collection task.

[0132] The processor can call information and application programs stored in the memory through the transmission device to perform the following steps: vector coding the audio data, and inputting the audio data into an audio recognition model; performing feature extraction on the audio data through a feature extraction layer of the audio recognition model to obtain audio features, wherein the feature extraction layer is subjected to hierarchical quantization by using a differential quantizer during a training process; calculating a probability distribution of signal classification through a recurrent neural network of the audio recognition model, and generating and outputting an audio recognition result based on the probability distribution of signal classification, wherein the recurrent neural network is subjected to hierarchical quantization by using the differential quantizer during the training process, and a quantization bit width of the feature extraction layer is less than a quantization bit width of the recurrent neural network.

[0133] By adopting the embodiment of the present application, an audio recognition scheme is provided. The parallel processing mode of audio collection and audio recognition is realized by adopting the double-buffer switching mechanism, in the case that the target audio possibly exists is detected, the audio can be recognized in time, and the real-time collection of the audio data is guaranteed, the hysteresis of the audio processing is reduced, the real-time performance of the audio processing is guaranteed, and the model training mode of hierarchical quantization is adopted, different bit width precisions are configured for different processing networks of the model, the model calculation and recognition efficiency is improved, the calculation precision is guaranteed, and the technical problems that the audio recognition mode based on the fixed high bit width precision calculation strategy and the serial processing in the related art has low efficiency and strong hysteresis are solved.

[0134] Those skilled in the art can understand that Figure 6 The structure shown is only schematic, and the electronic device can also be a terminal device such as a smart phone, a tablet computer, a palm computer, a Mobile Internet Device (MID), a PAD, and the like. Figure 6 It does not limit the structure of the electronic device. For example, the electronic device can further include more or less components (such as a network interface, a display device, and the like) than those shown in the figure, or have a different configuration from that shown in the figure. Figure 6 For example, the electronic device can further include more or less components (such as a network interface, a display device, and the like) than those shown in the figure, or have a different configuration from that shown in the figure. Figure 6 For example, the electronic device can further include more or less components (such as a network interface, a display device, and the like) than those shown in the figure, or have a different configuration from that shown in the figure.

[0135] Those skilled in the art can understand that all or part of the steps of the various methods in the above embodiments can be completed by instructing the terminal device related hardware through a program, and the program can be stored in a computer readable storage medium, which can include a flash disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0136] The application will be described in detail below in combination with another alternative embodiment.

[0137] Embodiment Four

[0138] The embodiments of the application further provide a computer readable storage medium. Optionally, in the embodiments of the application, the computer readable storage medium can be used to save the program code executed by the audio recognition method provided in the embodiment one.

[0139] Optionally, in the embodiments of the application, the storage medium can be located in any one of the computer terminals in the computer terminal group in the computer network, or in any one of the mobile terminals in the mobile terminal group.

[0140] The embodiments of the application further provide a computer program product, when executed on a data processing device, is adapted to execute the steps of the audio recognition method: collecting audio data, and pre-processing the audio data; calculating a weighted decibel value of the pre-processed audio data, and comparing the weighted decibel value with a decibel threshold to obtain a comparison result, wherein the decibel threshold is updated in real time based on environmental noise; in the case that the comparison result indicates that the weighted decibel value of the audio data is greater than the decibel threshold, performing preliminary verification on the audio data to obtain a verification result; in the case that the verification result indicates that the audio data passes the preliminary verification, entering an audio recognition stage through a double buffer switching mechanism, inputting the audio data into an audio recognition model, and outputting an audio recognition result, wherein the audio recognition model is a machine learning model that is pre-constructed and quantized in layers.

[0141] The above embodiment numbers of the application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0142] In the above embodiments of the application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0143] In several embodiments provided in the present application, it should be understood that the disclosed technology can be implemented by other ways. Among them, the above-described device embodiments are only schematic, for example, the division of the units can be a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some interfaces, units or modules, and can be electrical or other forms.

[0144] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed to multiple units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0145] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0146] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0147] The above is only the preferred embodiment of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present application.

Claims

1. An audio recognition method, characterized in that, include: Acquire audio data and preprocess the audio data; A weighted decibel value is calculated for the preprocessed audio data, and the weighted decibel value is compared with a decibel threshold to obtain a comparison result. The decibel threshold is updated in real time based on the ambient noise. If the comparison result indicates that the weighted decibel value of the audio data is greater than the decibel threshold, the audio data is preliminarily verified to obtain the verification result; If the verification result indicates that the audio data has passed the initial verification, the audio recognition stage is entered through a dual-buffer switching mechanism. The audio data is input into the audio recognition model, and the audio recognition result is output. The audio recognition model is a pre-built and hierarchically quantized machine learning model.

2. The method according to claim 1, characterized in that, The steps for updating the decibel threshold in real time include: Collect ambient audio within the target time period, and perform frame segmentation on the ambient audio to obtain N ambient audio frames, where N is a positive integer; Calculate the temporal energy of each environmental audio frame and convert the temporal energy into a decibel value; The noise mean and noise standard deviation of the ambient audio are calculated based on the decibel values ​​of each ambient audio frame. The decibel threshold is obtained by weighting the noise mean and the noise standard deviation.

3. The method according to claim 1, characterized in that, The steps for preprocessing the audio data include: The audio data is divided into frames to obtain M audio frames, where M is a positive integer; Calculate the decibel value for each audio frame, and calculate the median decibel value based on the decibel value for each audio frame; For each audio frame, calculate the deviation between the decibel value of the audio frame and the median of the decibel values; Based on the deviation value, instantaneous abnormal signals are identified and processed.

4. The method according to claim 1, characterized in that, The steps for calculating the weighted decibel value of the preprocessed audio data include: The audio data is filtered based on the target frequency band where the sound source signal to be identified is located to obtain the target frequency band audio data. Configure weight values ​​for the target frequency band audio data and the full frequency band audio data, respectively; The weighted decibel value of the audio data is calculated based on the decibel value of the target frequency band audio data, the weight value of the target frequency band audio data, the decibel value of the audio data across the entire frequency band, and the weight value of the audio data across the entire frequency band.

5. The method according to claim 1, characterized in that, The steps for performing preliminary verification on the audio data and obtaining the verification results include: The audio data is divided into frames to obtain K audio frames, where K is a positive integer; Perform a Fourier transform on each audio frame to convert the audio frame into a frequency domain signal; A Mel frequency filter is applied to the frequency domain signal, and Mel spectral features are extracted through the Mel frequency filter. The presence of a target sound source signal in the audio data is verified based on the Mel spectrum characteristics, and the verification result is obtained.

6. The method according to claim 1, characterized in that, The steps for entering the audio recognition stage through the dual-buffer switching mechanism include: Determine the current process of acquiring the audio data; The system controls the current process to perform audio recognition tasks and controls other processes to perform frequency acquisition tasks.

7. The method according to claim 1, characterized in that, The steps of inputting the audio data into the audio recognition model and outputting the audio recognition result include: The audio data is vector-encoded and then input into the audio recognition model; The audio data is processed by the feature extraction layer of the audio recognition model to obtain audio features. The feature extraction layer uses a differential quantizer for hierarchical quantization during training. The recurrent neural network of the audio recognition model calculates the probability distribution of signal classification, and generates and outputs the audio recognition result based on the probability distribution of signal classification. During the training process, the recurrent neural network uses the differential quantizer for hierarchical quantization, and the quantization bit width of the feature extraction layer is smaller than the quantization bit width of the recurrent neural network.

8. An audio recognition device, characterized in that, include: The acquisition unit is used to acquire audio data and preprocess the audio data; The comparison unit is used to calculate a weighted decibel value for the preprocessed audio data and compare the weighted decibel value with a decibel threshold to obtain a comparison result, wherein the decibel threshold is updated in real time based on the ambient noise. The verification unit is configured to perform preliminary verification on the audio data and obtain a verification result when the comparison result indicates that the weighted decibel value of the audio data is greater than the decibel threshold. The recognition unit is used to enter the audio recognition stage through a dual-buffer switching mechanism when the verification result indicates that the audio data has passed the preliminary verification. The audio data is input into the audio recognition model and the audio recognition result is output. The audio recognition model is a pre-built and hierarchically quantized machine learning model.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the audio recognition method according to any one of claims 1 to 7.

10. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the audio recognition method according to any one of claims 1 to 7.

Images