Audio recognition method, device, electronic device, medium, and program product

Abstract

The present application relates to the field of audio processing technology, and provides audio recognition methods, devices, electronic devices, media, and program products. The method includes: quantizing the audio data to be recognized; inputting the quantized audio data to be recognized and the fixed-point historical audio data into the audio recognition model to obtain the audio recognition result data and the updated fixed-point historical audio data output by the audio recognition model; dequantizing the audio recognition result data to obtain the audio recognition result; the updated fixed-point historical audio data is obtained by the memory module of the audio recognition model using depthwise separable convolution. The audio recognition method provided in the embodiment of the present application can eliminate the redundant operations caused by the quantization and dequantization operations within the audio recognition model, and improve the reasoning speed of the audio recognition model; and the fixed-point historical audio data is obtained by the memory module using depthwise separable convolution, which can effectively reduce the number of operators, reduce the delay, and further improve the model reasoning efficiency.

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and in particular to audio recognition methods, devices, electronic devices, media, and program products. Background Art

[0002] Existing audio recognition methods mainly use deep learning models for audio recognition. The deep learning models used can be time series models, such as recurrent neural networks (RNNs). Deep learning models need to perform full integer quantization operations during the inference process of audio recognition to complete the quantization of weights and activation values. Specifically, when the model quantizes forward reasoning, the input data is of type float32 (floating-point data), and the model needs to perform quantization operations on it. In order to ensure the consistency of the types of input data and output data, the model also needs to perform dequantization operations on the output data to make it of type float32. However, the above quantization and dequantization operations will lead to model calculation redundancy. Summary of the Invention

[0003] This application aims to solve at least one of the technical problems existing in the prior art.

[0004] To this end, the present application proposes an audio recognition method, which sets the operations of quantizing the audio data to be recognized and dequantizing the audio recognition result data output by the audio recognition model to be performed outside the audio recognition model, thereby eliminating the redundant operations caused by the quantization and dequantization operations inside the audio recognition model and improving the reasoning speed of the audio recognition model; and the fixed-point historical audio data is obtained by the memory module of the audio recognition model using depthwise separable convolution, which can effectively improve the deformation degree, reduce the number of operators, reduce the delay, and improve the reasoning efficiency of the audio recognition model.

[0005] The present application proposes an audio recognition device.

[0006] The present application provides an electronic device.

[0007] The present application provides a non-transitory computer-readable storage medium.

[0008] The present application proposes a computer program product.

[0009] The audio recognition method according to the first embodiment of the present application includes:

[0010] Quantify the audio data to be recognized;

[0011] Inputting the quantized audio data to be recognized and the fixed-point historical audio data into the audio recognition model, obtaining the audio recognition result data output by the audio recognition model and the updated fixed-point historical audio data;

[0012] Dequantizing the audio recognition result data output by the audio recognition model to obtain an audio recognition result;

[0013] In which, the audio recognition model is trained based on audio data samples and audio recognition results corresponding to the audio data samples, and the updated fixed-point historical audio data is obtained by convolving the quantized audio data to be recognized and the fixed-point historical audio data through the memory module of the audio recognition model using depthwise separable convolution, and the updated fixed-point memory data is used for the next audio recognition of the audio recognition model.

[0014] According to the audio recognition method of the embodiment of the present application, the operations of quantizing the audio data to be recognized and dequantizing the audio recognition result data output by the audio recognition model are set to be performed outside the audio recognition model, which can eliminate the redundant calculations caused by the quantization and dequantization operations inside the audio recognition model and improve the reasoning speed of the audio recognition model; the audio recognition model directly uses the fixed-point historical audio data whose data type is fixed-point data during the audio recognition process without the need for quantization or dequantization operations, and the updated fixed-point historical audio data obtained by the audio recognition model through the memory module using depthwise separable convolution can be directly used for the next audio recognition of the audio recognition model, which can effectively improve the reasoning efficiency and accuracy of the audio recognition model.

[0015] According to one embodiment of the present application, the updated fixed-point historical audio data is obtained by convolving the quantized audio data to be recognized and the fixed-point historical audio data using depthwise separable convolution in the memory module of the audio recognition model, specifically:

[0016] The memory module of the audio recognition model uses a depth-wise separable convolution operator to replace the for loop operator, and convolves the quantized audio data to be recognized and the fixed-point historical audio data to obtain the updated fixed-point historical audio data.

[0017] According to the audio recognition method of the embodiment of the present application, the for loop operator of the memory module of the audio recognition model is equivalently replaced by a highly parallel depthwise separable convolution operator, which can reduce a large number of operators and for loops, reduce matrix dimensions, and reduce memory usage, thereby optimizing delays and improving the inference efficiency of the audio recognition model.

[0018] According to one embodiment of the present application, the for loop operator includes any one of the following or any combination thereof: a multidimensional slicing operator, a multiplication operator, and an addition operator.

[0019] According to the audio recognition method of the embodiment of the present application, the for loop operator of the memory module of the audio recognition model is equivalently replaced by the depthwise separable convolution operator to perform multiplication and addition operations, which can greatly reduce the use of operators, simplify the model calculation process, and improve the accuracy of the audio recognition model.

[0020] According to one embodiment of the present application, the memory module of the audio recognition model uses a depthwise separable convolution operator to replace the for loop operator, and convolves the quantized audio data to be recognized and the fixed-point historical audio data to obtain the updated fixed-point historical audio data, including:

[0021] Performing unified data format conversion on the input data and output data of the depthwise separable convolution of the memory module.

[0022] According to the audio recognition method of the embodiment of the present application, performing a unified data format conversion on the input data and output data of the depthwise separable convolution of the memory module is beneficial to improving the data processing efficiency of the memory module and reducing the data processing error rate caused by different data formats.

[0023] According to one embodiment of the present application, the unified data format conversion of the input data and output data of the depthwise separable convolution of the memory module is specifically performed as follows:

[0024] The data formats of the input data and output data of the depthwise separable convolution of the memory module are uniformly converted into NHWC data format.

[0025] According to the audio recognition method of the embodiment of the present application, the NHWC data format is a relatively commonly used data format in the field of deep learning. Converting the data format of the input data and output data of the depthwise separable convolution of the memory module into the NHWC data format can effectively improve the versatility of the memory module.

[0026] According to one embodiment of the present application, the audio recognition model is trained based on audio data samples and audio recognition results corresponding to the audio data samples, including:

[0027] Obtaining an audio data sample and an audio recognition result corresponding to the audio data sample;

[0028] An initial audio recognition model is trained according to the audio data sample and the audio recognition result corresponding to the audio data sample to obtain the audio recognition model.

[0029] According to the audio recognition method of the embodiment of the present application, an audio recognition model specifically used for audio recognition is trained based on audio data samples and the audio recognition results corresponding to the audio data samples, so that the audio recognition model can perform deep learning based on the audio data samples and the audio recognition results corresponding to the audio data samples, thereby improving the accuracy of the audio recognition model.

[0030] An audio recognition device according to an embodiment of the second aspect of the present application includes:

[0031] A quantization module is used to quantize the audio data to be recognized;

[0032] An audio recognition module is configured to input the quantized audio data to be recognized and the fixed-point historical audio data into an audio recognition model, and obtain the audio recognition result data and the updated fixed-point historical audio data output by the audio recognition model;

[0033] A dequantization module, configured to: dequantize the audio recognition result data output by the audio recognition model to obtain an audio recognition result;

[0034] In which, the audio recognition model is trained based on audio data samples and audio recognition results corresponding to the audio data samples, and the updated fixed-point historical audio data is obtained by convolving the quantized audio data to be recognized and the fixed-point historical audio data through the memory module of the audio recognition model using depthwise separable convolution, and the updated fixed-point historical audio data is used for the next audio recognition of the audio recognition model.

[0035] According to an embodiment of the third aspect of the present application, an electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the program, any of the above-mentioned audio recognition methods is implemented.

[0036] According to the non-transitory computer-readable storage medium of the fourth aspect embodiment of the present application, a computer program is stored thereon, and when the computer program is executed by a processor, it implements any of the above-mentioned audio recognition methods.

[0037] The computer program product according to the fifth aspect of the present application includes a computer program, which implements any of the above-mentioned audio recognition methods when executed by a processor.

[0038] The above-mentioned one or more technical solutions in the embodiments of the present application have at least one of the following technical effects: the operations of quantizing the audio data to be recognized and dequantizing the audio recognition result data output by the audio recognition model are set to be performed outside the audio recognition model, which can eliminate the redundant calculations caused by the quantization and dequantization operations inside the audio recognition model and improve the reasoning speed of the audio recognition model; the audio recognition model directly uses the fixed-point historical audio data of the fixed-point data type during the audio recognition process without quantizing or dequantizing it, and the updated fixed-point historical audio data obtained by the audio recognition model through the memory module using depthwise separable convolution can be directly used for the next audio recognition of the audio recognition model, which can effectively improve the reasoning efficiency and accuracy of the audio recognition model.

[0039] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become obvious from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0041] Figure 1 Schematic diagram of the flow of the audio recognition method provided in the embodiment of the present application;

[0042] Figure 2 A schematic diagram illustrating an application of an audio recognition model of an audio recognition method provided in an embodiment of the present application;

[0043] Figure 3 It is shown that the audio recognition method provided by the embodiment of the present application uses depthwise separable convolution to equivalently replace the for loop operator of the memory module in the audio recognition model to perform multiplication and addition operations;

[0044] Figure 4 Schematic diagram of the structure of the audio recognition device provided in the embodiment of the present application;

[0045] Figure 5 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] The following embodiments of the present invention are described in further detail with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0047] Figure 1It is a flowchart of the audio recognition method provided in an embodiment of the present application.

[0048] Reference Figure 1 , an audio recognition method provided by an embodiment of the present application includes:

[0049] S110, quantizing the audio data to be recognized;

[0050] S120: Input the quantized audio data to be recognized and the fixed-point historical audio data into the audio recognition model to obtain audio recognition result data output by the audio recognition model and updated fixed-point historical audio data;

[0051] S130, dequantizing the audio recognition result data output by the audio recognition model to obtain an audio recognition result;

[0052] In which, the audio recognition model is trained based on audio data samples and audio recognition results corresponding to the audio data samples, and the updated fixed-point historical audio data is obtained by convolving the quantized audio data to be recognized and the fixed-point historical audio data through the memory module of the audio recognition model using depthwise separable convolution, and the updated fixed-point historical audio data is used for the next audio recognition of the audio recognition model.

[0053] It should be noted that the execution entity of the audio recognition method provided in this application can be any terminal-side device, such as an audio recognition system, etc., wherein the audio recognition model can be preset in advance in the audio recognition system.

[0054] It should be noted that the audio recognition model can be pre-trained based on audio data samples and audio recognition results corresponding to the audio data samples. The specific training process may include:

[0055] Obtaining an audio data sample and an audio recognition result corresponding to the audio data sample;

[0056] An initial audio recognition model is trained according to the audio data sample and the audio recognition result corresponding to the audio data sample to obtain the audio recognition model.

[0057] During model training, the audio recognition model can perform deep learning based on audio data samples and the audio recognition results corresponding to the audio data samples, thereby improving the accuracy of the audio recognition model and the accuracy of model reasoning.

[0058] Alternatively, the audio recognition model can be any audio recognition model in the prior art that is used to implement audio recognition and has a memory module, which is not limited in this article.

[0059] In step S110, the terminal-side device quantizes the audio data to be recognized.

[0060] It should be noted that quantization means quantizing floating-point operations in the forward process of the neural network into integer operations to achieve the purpose of computing acceleration.

[0061] See also Figure 2 , Figure 2 An application diagram of the audio recognition model of the audio recognition method provided by an embodiment of the present application is shown, wherein x1 represents audio data to be recognized, x2 represents fixed-point historical audio data, y1 represents audio recognition result data, y2 represents updated fixed-point historical audio data, and t represents the number of audio recognitions. Specifically, the audio data to be recognized is usually floating-point data of type float32, and the audio data to be recognized is quantized, that is, the floating-point data of type float32 is converted into fixed-point data of type int8, and the floating-point data is mapped from a high dimension to a low dimension. The quantization operation can effectively improve the computational efficiency of the terminal side device in realizing audio recognition. If the quantization operation and the dequantization operation are performed in the audio recognition model, it is easy to cause redundant calculations of the model. Now the terminal side device first performs the quantization operation of the audio data to be recognized, and then inputs the quantized audio data to be recognized (the quantized audio data to be recognized is fixed-point data) into the audio recognition model, which can avoid redundant calculations of the audio recognition model and ensure the speed and efficiency of the terminal side device in realizing audio recognition.

[0062] In step S120, the terminal side device inputs the quantized audio data to be recognized and the fixed-point historical audio data into the audio recognition model to obtain the audio recognition result data output by the audio recognition model and the updated fixed-point historical audio data.

[0063] It should be noted that, see Figure 2 Before the first audio recognition of the audio recognition model, the terminal side device will initialize the initial historical audio data, so that the initial historical audio data becomes the fixed-point historical audio data corresponding to the floating-point data "0". After completing the quantization of the audio data to be recognized, the terminal side device can first splice the quantized audio data to be recognized and the initialized fixed-point historical audio data as the input of the memory module of the audio recognition model, and then use the depth-separable convolution to convolve the quantized audio data to be recognized and the fixed-point historical audio data through the memory module of the audio recognition model to extract the intermediate features of the audio data to be recognized and obtain the updated fixed-point historical audio data, and then perform audio recognition according to the intermediate features of the audio data to be recognized through other layers of the audio recognition model (such as the recognition layer). The specific audio recognition algorithm can be any one of the existing technologies and is not limited in this article.

[0064] Furthermore, the updated fixed-point historical audio data can be provided to the memory module of the audio recognition model for the next extraction of intermediate features of the audio data to be recognized. It should be noted that, in the prior art, the memory module will use a for loop to divide the input slices into (memory range (memory_range) + 1) groups of data with a size of (1*input frame number (frames) * input channel number (channels)), and then multiply the slice data with (memory range (memory_range) + 1) groups of parameters with a size of 1*128 and then add them together. In an embodiment of the present application, the memory module can use a highly parallel depth-separable convolution operator to equivalently replace the for loop operator of the memory module to perform multiplication and addition operations to extract the intermediate features of the audio data to be recognized for the subsequent reasoning of the audio recognition model.

[0065] In step S130, the terminal side device dequantizes the audio recognition result data output by the audio recognition model to obtain an audio recognition result.

[0066] It should be noted that the dequantization operation is the reverse process of the quantization operation. The terminal-side device dequantizes the audio recognition result data in the fixed-point data format output by the audio recognition model into floating-point data to keep it consistent with the data type of the audio data to be recognized.

[0067] It should be noted that the audio data to be identified may be the audio feature data to be identified, and the audio recognition result data may be the audio recognition probability data inferred by the audio recognition model through any existing audio recognition algorithm. For example, the audio recognition model may obtain an audio data sample that matches the audio feature data to be identified, and then obtain audio recognition probability data corresponding to the matching audio data sample. The terminal side device may use the existing audio recognition algorithm to obtain the audio recognition result based on the inverse quantized audio recognition probability data.

[0068] A general deep learning model will complete the conversion through a quantization tool within the model, but this method will cause computational redundancy for audio recognition models that require time series information, affecting the model's reasoning speed. Moreover, there is a memory module in the audio recognition model, which contains a for loop and a large number of operation operators. The operation of a large number of operators and the for loop will further aggravate the slowness of the model's reasoning operation, resulting in a waste of resources. The audio recognition method of the embodiment of the present application sets the operation of quantizing the audio data to be recognized and dequantizing the audio recognition result data output by the audio recognition model to be performed outside the audio recognition model, which can eliminate the redundant operations caused by the quantization and dequantization operations within the audio recognition model and improve the reasoning speed of the audio recognition model. The audio recognition model directly uses the fixed-point historical audio data with a fixed-point data type in the audio recognition process without quantizing or dequantizing it, and the updated fixed-point historical audio data obtained by the audio recognition model through the memory module using the depth-separable convolution can be directly used for the next audio recognition of the audio recognition model, which can effectively improve the reasoning efficiency and accuracy of the audio recognition model.

[0069] Furthermore, according to an audio recognition method provided in an embodiment of the present application, the updated fixed-point historical audio data is obtained by convolving the quantized audio data to be recognized and the fixed-point historical audio data using a depthwise separable convolution in the memory module of the audio recognition model, which can be specifically as follows:

[0070] The memory module of the audio recognition model uses a depth-wise separable convolution operator to replace the for loop operator, and convolves the quantized audio data to be recognized and the fixed-point historical audio data to obtain the updated fixed-point historical audio data.

[0071] It should be noted that the for loop operator may include any one of the following or any combination thereof: a multidimensional slicing operator, a multiplication operator, and an addition operator.

[0072] See also Figure 3 , Figure 3It is shown that the audio recognition method provided by the embodiment of the present application utilizes a depthwise separable convolution operator to equivalently replace the for loop operator of the memory module in the audio recognition model for multiplication and addition operations. Specifically, in the prior art, the memory module will utilize a for loop to divide the input slices into (memory range (memory_range) + 1) groups of data with a size of (1*input frame number (frames) * input channel number (channels)), and then multiply the slice data with (memory range (memory_range) + 1) groups of parameters with a size of 1*128 and then add them together. The audio recognition method provided by the embodiment of the present application utilizes a highly parallel depthwise separable convolution operator to equivalently replace the for loop operator of the memory module of the audio recognition model for multiplication and addition operations, which can reduce a large number of operators and for loops, reduce matrix dimensions, reduce memory usage, thereby optimizing delays and improving the inference efficiency of the audio recognition model.

[0073] Furthermore, in an audio recognition method provided according to an embodiment of the present application, the input shape (input_shape) of the depthwise separable convolution of the memory module is: 1*1*(memory range+number of input frames)*number of input channels, the output shape (output_shape) is: 1*1*number of output frames*number of output channels, the convolution kernel size (kernel size) is: 1*(memory range+1), and the number of channels of the convolution kernel is the same as the number of input channels.

[0074] Specifically, a large number of stridedslice operators (multidimensional slicing operators), mul operators (multiplication operators), and add operators (addition operators) in the for loop of the memory module can be replaced with a depth-wise separable convolution (such as depthwiseConv2D). The input_shape of depthwiseConv2D is 1*1*(memory range + number of input frames)*number of input channels. It is necessary to expand one dimension based on the original memory module input and reshape (adjust the data format) to the NHWC data format; the output_shape is 1*1*number of output frames*number of output channels. It is necessary to reduce one dimension to ensure that it is the same as the original module output. Among them, the kernel size of depthwiseConv2D is set to 1*(memory range + 1), and the number of channels of the convolution kernel is the same as the number of input channels.

[0075] When the memory module is applied in the audio recognition model, the terminal-side device will initialize the initial historical audio data before the audio recognition model performs audio recognition for the first time, so that the initial historical audio data becomes the fixed-point historical audio data corresponding to the floating-point data "0". After each audio recognition inference, the audio recognition model can directly use the updated fixed-point historical audio data obtained during the last audio recognition, without the need to quantize or dequantize the fixed-point historical audio data. More specifically, the terminal-side device or audio recognition model can first splice the fixed-point historical audio data (1*memory range*number of input channels) and the quantized audio data to be recognized (1*number of input frames*number of input channels) to obtain spliced ​​data (1*(memory range+number of input frames)*number of input channels) as the input of the memory module. After the operation of the memory module, the intermediate features of the audio data to be recognized can be obtained for subsequent audio recognition.

[0076] An audio recognition method provided in an embodiment of the present application sets the parameters of depthwise separable convolution according to the memory module of the audio recognition model, realizes the equivalent replacement of the for loop operator with the depthwise separable convolution operator, optimizes the latency without reducing the amount of calculation, reduces memory usage, and can effectively ensure the smooth operation of the audio recognition model and improve the audio recognition efficiency.

[0077] Furthermore, according to an audio recognition method provided by an embodiment of the present application, replacing the for loop operator with a depthwise separable convolution operator in the memory module of the audio recognition model, performing convolution on the quantized audio data to be recognized and the fixed-point historical audio data to obtain the updated fixed-point historical audio data may include:

[0078] Performing unified data format conversion on the input data and output data of the depthwise separable convolution of the memory module.

[0079] Performing a unified data format conversion on the input data and output data of the depthwise separable convolution of the memory module is beneficial to improving the data processing efficiency of the memory module and reducing the data processing error rate caused by different data formats.

[0080] It should be noted that the data formats of the input data and output data of the depthwise separable convolution of the memory module can be uniformly converted into the NHWC data format.

[0081] The NHWC data format is a commonly used data format in the field of deep learning. Converting the input and output data formats of the depthwise separable convolution of the memory module into the NHWC data format can effectively improve the versatility of the memory module.

[0082] The audio recognition device provided in an embodiment of the present application is described below. The audio recognition device described below and the audio recognition method described above can refer to each other.

[0083] Figure 4 A schematic diagram of the structure of an audio recognition device provided in an embodiment of the present application.

[0084] Reference Figure 4 , an audio recognition device provided by an embodiment of the present application may include:

[0085] Quantization module 210, used to: quantize the audio data to be recognized;

[0086] The audio recognition module 220 is configured to input the quantized audio data to be recognized and the fixed-point historical audio data into the audio recognition model, and obtain the audio recognition result data and the updated fixed-point historical audio data output by the audio recognition model;

[0087] The dequantization module 230 is used to: dequantize the audio recognition result data output by the audio recognition model to obtain an audio recognition result;

[0088] In which, the audio recognition model is trained based on audio data samples and audio recognition results corresponding to the audio data samples, and the updated fixed-point historical audio data is obtained by convolving the quantized audio data to be recognized and the fixed-point historical audio data through the memory module of the audio recognition model using depthwise separable convolution, and the updated fixed-point memory data is used for the next audio recognition of the audio recognition model.

[0089] It should be noted that the updated fixed-point historical audio data is obtained by convolving the quantized audio data to be recognized and the fixed-point historical audio data using depthwise separable convolution in the memory module of the audio recognition model, specifically:

[0090] The memory module of the audio recognition model uses a depth-wise separable convolution operator to replace the for loop operator, and convolves the quantized audio data to be recognized and the fixed-point historical audio data to obtain the updated fixed-point historical audio data.

[0091] It should be noted that the for loop operator includes any one of the following or any combination thereof: a multidimensional slicing operator, a multiplication operator, and an addition operator.

[0092] It should be noted that the memory module of the audio recognition model uses a depthwise separable convolution operator to replace the for loop operator, and convolves the quantized audio data to be recognized and the fixed-point historical audio data to obtain the updated fixed-point historical audio data, including:

[0093] Performing unified data format conversion on the input data and output data of the depthwise separable convolution of the memory module.

[0094] It should be noted that the unified data format conversion of the input data and output data of the depthwise separable convolution of the memory module is specifically as follows:

[0095] The data formats of the input data and output data of the depthwise separable convolution of the memory module are uniformly converted into NHWC data format.

[0096] It should be noted that the audio recognition model is trained based on audio data samples and audio recognition results corresponding to the audio data samples, including:

[0097] Obtaining an audio data sample and an audio recognition result corresponding to the audio data sample;

[0098] An initial audio recognition model is trained according to the audio data sample and the audio recognition result corresponding to the audio data sample to obtain the audio recognition model.

[0099] Figure 5 An example of a physical structure diagram of an electronic device is shown below. Figure 5 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the following method:

[0100] Quantify the audio data to be recognized;

[0101] Inputting the quantized audio data to be recognized and the fixed-point historical audio data into the audio recognition model, obtaining the audio recognition result data output by the audio recognition model and the updated fixed-point historical audio data;

[0102] Dequantizing the audio recognition result data output by the audio recognition model to obtain an audio recognition result;

[0103] In which, the audio recognition model is trained based on audio data samples and audio recognition results corresponding to the audio data samples, and the updated fixed-point historical audio data is obtained by convolving the quantized audio data to be recognized and the fixed-point historical audio data through the memory module of the audio recognition model using depthwise separable convolution, and the updated fixed-point memory data is used for the next audio recognition of the audio recognition model.

[0104] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0105] Furthermore, embodiments of the present application disclose a computer program product, comprising a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program comprises program instructions. When the program instructions are executed by a computer, the computer can perform the methods provided in the above-mentioned method embodiments, for example, including:

[0106] Quantify the audio data to be recognized;

[0107] Inputting the quantized audio data to be recognized and the fixed-point historical audio data into the audio recognition model, obtaining the audio recognition result data output by the audio recognition model and the updated fixed-point historical audio data;

[0108] Dequantizing the audio recognition result data output by the audio recognition model to obtain an audio recognition result;

[0109] In which, the audio recognition model is trained based on audio data samples and audio recognition results corresponding to the audio data samples, and the updated fixed-point historical audio data is obtained by convolving the quantized audio data to be recognized and the fixed-point historical audio data through the memory module of the audio recognition model using depthwise separable convolution, and the updated fixed-point memory data is used for the next audio recognition of the audio recognition model.

[0110] On the other hand, an embodiment of the present application further provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the transmission method provided in each of the above embodiments is implemented, for example, including:

[0111] Quantify the audio data to be recognized;

[0112] Inputting the quantized audio data to be recognized and the fixed-point historical audio data into the audio recognition model, obtaining the audio recognition result data output by the audio recognition model and the updated fixed-point historical audio data;

[0113] Dequantizing the audio recognition result data output by the audio recognition model to obtain an audio recognition result;

[0114] In which, the audio recognition model is trained based on audio data samples and audio recognition results corresponding to the audio data samples, and the updated fixed-point historical audio data is obtained by convolving the quantized audio data to be recognized and the fixed-point historical audio data through the memory module of the audio recognition model using depthwise separable convolution, and the updated fixed-point memory data is used for the next audio recognition of the audio recognition model.

[0115] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0116] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

[0118] The above embodiments are intended only to illustrate the present application and are not intended to limit the present application. Although the present application has been described in detail with reference to the embodiments, it should be understood by those skilled in the art that various combinations, modifications, or equivalent substitutions of the technical solutions of the present application do not depart from the spirit and scope of the technical solutions of the present application and should be encompassed by the claims of the present application.

Claims

1. An audio recognition method, characterized in that: include: Quantify the audio data to be recognized; Inputting the quantized audio data to be recognized and the fixed-point historical audio data into the audio recognition model, obtaining audio recognition result data output by the audio recognition model and updated fixed-point historical audio data; the data types of the fixed-point historical audio data and the audio recognition result data are both fixed-point data; Dequantizing the audio recognition result data output by the audio recognition model to obtain an audio recognition result; In which, the audio recognition model is trained based on audio data samples and audio recognition results corresponding to the audio data samples, and the updated fixed-point historical audio data is obtained by convolving the quantized audio data to be recognized and the fixed-point historical audio data through the memory module of the audio recognition model using depthwise separable convolution, and the updated fixed-point historical audio data is used for the next audio recognition of the audio recognition model.

2. The audio recognition method according to claim 1, wherein: The updated fixed-point historical audio data is obtained by convolving the quantized audio data to be recognized and the fixed-point historical audio data using depthwise separable convolution in the memory module of the audio recognition model, specifically: The memory module of the audio recognition model uses a depth-wise separable convolution operator to replace the loop operator, and convolves the quantized audio data to be recognized and the fixed-point historical audio data to obtain the updated fixed-point historical audio data.

3. The audio recognition method according to claim 2, characterized in that The loop operator includes any one of the following or any combination thereof: a multidimensional slicing operator, a multiplication operator, and an addition operator.

4. The audio recognition method according to claim 2, wherein: The memory module of the audio recognition model uses a depthwise separable convolution operator to replace a loop operator to convolve the quantized audio data to be recognized and the fixed-point historical audio data to obtain the updated fixed-point historical audio data, including: Performing unified data format conversion on the input data and output data of the depthwise separable convolution of the memory module.

5. The audio recognition method according to claim 4, characterized in that The unified data format conversion of the input data and output data of the depthwise separable convolution of the memory module is specifically performed as follows: The data formats of the input data and output data of the depthwise separable convolution of the memory module are uniformly converted into NHWC data format.

6. The audio recognition method according to any one of claims 1 to 5, characterized in that: The audio recognition model is trained based on audio data samples and audio recognition results corresponding to the audio data samples, including: Obtaining an audio data sample and an audio recognition result corresponding to the audio data sample; An initial audio recognition model is trained according to the audio data sample and the audio recognition result corresponding to the audio data sample to obtain the audio recognition model.

7. An audio recognition device, characterized in that: include: A quantization module is used to quantize the audio data to be recognized; An audio recognition module is configured to input the quantized audio data to be recognized and the fixed-point historical audio data into an audio recognition model, and obtain audio recognition result data and updated fixed-point historical audio data output by the audio recognition model; the data types of the fixed-point historical audio data and the audio recognition result data are both fixed-point data; A dequantization module, configured to: dequantize the audio recognition result data output by the audio recognition model to obtain an audio recognition result; In which, the audio recognition model is trained based on audio data samples and audio recognition results corresponding to the audio data samples, and the updated fixed-point historical audio data is obtained by convolving the quantized audio data to be recognized and the fixed-point historical audio data through the memory module of the audio recognition model using depthwise separable convolution, and the updated fixed-point historical audio data is used for the next audio recognition of the audio recognition model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the audio recognition method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the audio recognition method according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the audio recognition method according to any one of claims 1 to 6 is implemented.

Images