Audio recognition method and device

Abstract

The present invention discloses an audio recognition method and device, which includes the following steps: obtaining audio information to be processed; preprocessing the audio information to be processed to obtain first voice data feature information; and using an audio recognition model to perform recognition processing on the first voice data feature information to obtain target audio recognition information; the target audio recognition information includes M audio recognition result information, where M is a positive integer greater than or equal to 3. As can be seen, the present invention can obtain target audio recognition information by preprocessing the audio information to be processed and then performing further recognition processing using an audio recognition model. This facilitates the direct extraction of valid entity information from audio and audio classification, reduces intermediate data processing steps, avoids the impact of errors in converting audio data into text data, improves data processing efficiency, and saves maintenance costs.

Description

Technical Field

[0001] The present invention relates to the field of audio processing technology, and in particular to an audio recognition method and device. Background Art

[0002] Traditional multimodal knowledge graphs or entity recognition and classification methods often require converting audio data into corresponding text, then processing the text using commonly used entity recognition and classification models. These two steps require three models to process, making the data processing process time-consuming and inefficient. Therefore, it is particularly important to provide an audio recognition method and device that can directly extract valid entity information from audio and classify the audio, reducing intermediate data processing steps, avoiding the impact of errors in converting audio data into text data, improving data processing efficiency, and saving maintenance costs. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide an audio recognition method and device, which can pre-process the audio information to be processed and then use the audio recognition model to perform further recognition processing to obtain target audio recognition information, which is conducive to directly extracting effective entity information from the audio and classifying the audio, reducing the intermediate process of data processing, avoiding the influence of errors in converting audio data into text data, improving data processing efficiency and saving maintenance costs.

[0004] In order to solve the above technical problems, the first aspect of the embodiments of the present invention discloses an audio recognition method, the method comprising:

[0005] Get the audio information to be processed;

[0006] Preprocessing the audio information to be processed to obtain first voice data feature information;

[0007] The first speech data feature information is recognized and processed using an audio recognition model to obtain target audio recognition information; the target audio recognition information includes M audio recognition result information; and M is a positive integer greater than or equal to 3.

[0008] As an optional implementation manner, in the first aspect of the embodiment of the present invention, preprocessing the audio information to be processed to obtain first voice data feature information includes:

[0009] Performing denoising and separation processing on the audio information to be processed to obtain first audio processing information;

[0010] The first audio processing information is segmented and feature extracted to obtain first voice data feature information.

[0011] As an optional implementation manner, in the first aspect of the embodiment of the present invention, before using the audio recognition model to perform recognition processing on the first voice data feature information to obtain target audio recognition information, the method further includes:

[0012] Acquire an audio training set; the audio training set includes a plurality of audio training information;

[0013] The audio training set is used to train a preset first training model to obtain the audio recognition model.

[0014] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the using the audio training set to train a preset first training model to obtain the audio recognition model includes:

[0015] Determining target audio training information based on the audio training set;

[0016] Using the target audio training information to train a preset first training model to obtain a second training model;

[0017] Performing loss function calculation processing on the training parameter information corresponding to the second training model to obtain target loss function value information;

[0018] Determine whether the loss function value information meets the training termination condition, and obtain a first training judgment result;

[0019] When the first training judgment result is no, the first training model is updated using the second training model, and the step of determining target audio training information based on the audio training set is triggered;

[0020] When the first training judgment result is yes, the audio recognition model is determined according to the second training model.

[0021] As an optional implementation manner, in the first aspect of the embodiment of the present invention, performing loss function calculation processing on the training parameter information corresponding to the second training model to obtain target loss function value information includes:

[0022] Using a preset first loss function model to calculate the training parameter information corresponding to the second training model to obtain first loss function value information;

[0023] Using a preset second loss function model to calculate the training parameter information corresponding to the second training model to obtain second loss function value information;

[0024] Using a preset third loss function model to calculate the training parameter information corresponding to the second training model to obtain third loss function value information;

[0025] The first loss function value information, the second loss function value information and the third loss function value information are weightedly summed to obtain target loss function value information.

[0026] As an optional implementation manner, in the first aspect of the embodiment of the present invention, obtaining the audio training set includes:

[0027] Acquire an original video information set; the original video information set includes a plurality of original video information;

[0028] Extracting, identifying, and associating the original video information set to obtain target data set information;

[0029] The target data set information is classified and processed to obtain an audio training set.

[0030] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the extracting, identifying, and associating the original video information set to obtain target data set information includes:

[0031] Performing audio extraction processing on the original video information set to obtain a first audio information set; the first audio information set includes a plurality of first audio information;

[0032] Performing denoising and separation processing on the first audio information set to obtain a second audio information set; the second audio information set includes a plurality of second audio information;

[0033] Performing recognition processing on the second audio information set to obtain a third audio information set; the third audio information set includes a plurality of third audio information;

[0034] Correcting and labeling the third audio information set to obtain a fourth audio information set and a fifth audio information set; the fourth audio information set includes a plurality of fourth audio information; and the fifth audio information set includes a plurality of fifth audio information;

[0035] The second audio information set, the third audio information set, the fourth audio information set, and the fifth audio information set are associated to obtain an audio training set.

[0036] A second aspect of an embodiment of the present invention discloses an audio recognition device, comprising:

[0037] An acquisition module is used to obtain audio information to be processed;

[0038] A first processing module, configured to pre-process the audio information to be processed to obtain first voice data feature information;

[0039] The second processing module is used to use the audio recognition model to identify and process the feature information of the first voice data to obtain target audio recognition information; the target audio recognition information includes M audio recognition result information; M is a positive integer greater than or equal to 3.

[0040] As an optional implementation manner, in the second aspect of the embodiment of the present invention, the first processing module preprocesses the audio information to be processed to obtain the first voice data feature information in the following specific manner:

[0041] Performing denoising and separation processing on the audio information to be processed to obtain first audio processing information;

[0042] The first audio processing information is segmented and feature extracted to obtain first voice data feature information.

[0043] As an optional implementation, in the second aspect of the embodiment of the present invention, before the second processing module uses the audio recognition model to perform recognition processing on the first voice data feature information to obtain target audio recognition information, the acquisition module is further used to obtain an audio training set; the audio training set includes a plurality of audio training information;

[0044] The device further comprises:

[0045] The training module is used to train a preset first training model using the audio training set to obtain the audio recognition model.

[0046] As an optional implementation, in the second aspect of the embodiment of the present invention, the training module uses the audio training set to train the preset first training model, and the specific manner of obtaining the audio recognition model is:

[0047] Determining target audio training information based on the audio training set;

[0048] Using the target audio training information to train a preset first training model to obtain a second training model;

[0049] Performing loss function calculation processing on the training parameter information corresponding to the second training model to obtain target loss function value information;

[0050] Determine whether the loss function value information meets the training termination condition, and obtain a first training judgment result;

[0051] When the first training judgment result is no, the first training model is updated using the second training model, and the step of determining target audio training information based on the audio training set is triggered;

[0052] When the first training judgment result is yes, the audio recognition model is determined according to the second training model.

[0053] As an optional implementation, in the second aspect of the embodiment of the present invention, the training module performs loss function calculation processing on the training parameter information corresponding to the second training model, and obtains the target loss function value information in the following specific manner:

[0054] Using a preset first loss function model to calculate the training parameter information corresponding to the second training model to obtain first loss function value information;

[0055] Using a preset second loss function model to calculate the training parameter information corresponding to the second training model to obtain second loss function value information;

[0056] Using a preset third loss function model to calculate the training parameter information corresponding to the second training model to obtain third loss function value information;

[0057] The first loss function value information, the second loss function value information and the third loss function value information are weightedly summed to obtain target loss function value information.

[0058] As an optional implementation, in the second aspect of the embodiment of the present invention, the specific manner in which the training module obtains the audio training set is:

[0059] Acquire an original video information set; the original video information set includes a plurality of original video information;

[0060] Extracting, identifying, and associating the original video information set to obtain target data set information;

[0061] The target data set information is classified and processed to obtain an audio training set.

[0062] As an optional implementation, in the second aspect of the embodiment of the present invention, the training module extracts, identifies, and associates the original video information set to obtain the target data set information in the following manner:

[0063] Performing audio extraction processing on the original video information set to obtain a first audio information set; the first audio information set includes a plurality of first audio information;

[0064] Performing denoising and separation processing on the first audio information set to obtain a second audio information set; the second audio information set includes a plurality of second audio information;

[0065] Performing recognition processing on the second audio information set to obtain a third audio information set; the third audio information set includes a plurality of third audio information;

[0066] Correcting and labeling the third audio information set to obtain a fourth audio information set and a fifth audio information set; the fourth audio information set includes a plurality of fourth audio information; and the fifth audio information set includes a plurality of fifth audio information;

[0067] The second audio information set, the third audio information set, the fourth audio information set, and the fifth audio information set are associated to obtain an audio training set.

[0068] A third aspect of the present invention discloses another audio recognition device, comprising:

[0069] a memory storing executable program code;

[0070] a processor coupled to the memory;

[0071] The processor calls the executable program code stored in the memory to execute part or all of the steps in the audio recognition method disclosed in the first aspect of the embodiment of the present invention.

[0072] The fourth aspect of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute some or all of the steps in the audio recognition method disclosed in the first aspect of the embodiment of the present invention.

[0073] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0074] In an embodiment of the present invention, audio information to be processed is obtained; the audio information to be processed is preprocessed to obtain first voice data feature information; the first voice data feature information is recognized and processed using an audio recognition model to obtain target audio recognition information; the target audio recognition information includes M audio recognition result information; M is a positive integer greater than or equal to 3. It can be seen that the present invention can obtain target audio recognition information by preprocessing the audio information to be processed and then performing further recognition processing using an audio recognition model, which is beneficial for directly extracting effective entity information from audio and classifying the audio, reducing intermediate data processing processes, avoiding the influence of errors in converting audio data into text data, improving data processing efficiency, and saving maintenance costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0076] Figure 1 This is a flowchart of an audio recognition method disclosed in an embodiment of the present invention;

[0077] Figure 2 is a flowchart of another audio recognition method disclosed in an embodiment of the present invention;

[0078] Figure 3 is a structural diagram of an audio recognition device disclosed in an embodiment of the present invention;

[0079] Figure 4 is a structural diagram of another audio recognition device disclosed in an embodiment of the present invention;

[0080] Figure 5 A schematic structural diagram of another audio recognition device disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0081] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0082] The terms "first," "second," and so on, in the description and claims of the present invention and the accompanying drawings are used to distinguish between different objects, not to describe a specific order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or elements is not limited to the listed steps or elements but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or device.

[0083] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute a separate or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0084] The present invention discloses an audio recognition method and device that can preprocess pending audio information and then use an audio recognition model to perform further recognition processing to obtain target audio recognition information. This facilitates the direct extraction of effective entity information from audio and audio classification, reduces intermediate data processing steps, avoids the impact of errors in converting audio data into text data, improves data processing efficiency, and reduces maintenance costs. A detailed description of each is provided below.

[0085] Example 1

[0086] See also Figure 1 , Figure 1 This is a flow chart of an audio recognition method disclosed in an embodiment of the present invention. Figure 1 The described audio recognition method is applied to a data processing system, such as a local server or cloud server for audio recognition management, and the embodiments of the present invention do not limit this. Figure 1 As shown, the audio recognition method may include the following operations:

[0087] 101. Obtain audio information to be processed.

[0088] 102. Preprocess the audio information to be processed to obtain first voice data feature information.

[0089] 103. Use the audio recognition model to perform recognition processing on the feature information of the first voice data to obtain target audio recognition information.

[0090] In the embodiment of the present invention, the target audio recognition information includes M audio recognition result information.

[0091] In the embodiment of the present invention, the above-mentioned M is a positive integer greater than or equal to 3.

[0092] Optionally, the above-mentioned audio recognition result information includes audio category information, and / or audio entity information, and / or audio text result information, which is not limited in this embodiment of the present invention.

[0093] Optionally, the audio text result information represents text information corresponding to the audio information to be processed.

[0094] It can be seen that the audio recognition method described in the embodiment of the present invention can obtain target audio recognition information by preprocessing the audio information to be processed and then using the audio recognition model for further recognition processing, which is conducive to directly extracting effective entity information from the audio and classifying the audio, reducing the intermediate process of data processing, avoiding the impact of errors in converting audio data into text data, improving data processing efficiency and saving maintenance costs.

[0095] In an optional embodiment, preprocessing the audio information to be processed in step 102 to obtain first voice data feature information includes:

[0096] Performing denoising and separation processing on the audio information to be processed to obtain first audio processing information;

[0097] The first audio processing information is segmented and feature extracted to obtain first voice data feature information.

[0098] In this optional embodiment, as an optional implementation manner, the above-mentioned segmentation and feature extraction processing of the first audio processing information to obtain the first voice data feature information is specifically performed as follows:

[0099] Determine whether the length information corresponding to the first audio processing information meets a length threshold, and obtain a length determination result;

[0100] When the length determination result is negative, padding the first audio processing information with a [PAD] character and the length threshold to obtain first audio information to be used;

[0101] Segmenting the first audio information to be used to obtain a second audio information set to be used; the second audio information set to be used includes K pieces of second audio information to be used; the K represents the number of frames corresponding to the first audio information to be used;

[0102] Perform FBANK feature extraction on the second set of standby audio information to obtain first voice data feature information.

[0103] Optionally, the first voice data feature information includes K first voice data features.

[0104] Optionally, the first speech data feature is a 40-dimensional vector.

[0105] Optionally, the first voice data feature includes an audio time domain feature and / or an audio frequency domain feature, which is not limited in this embodiment of the present invention.

[0106] It can be seen that the implementation of the audio recognition method described in the embodiment of the present invention can obtain the first voice data feature information by performing denoising, separation, segmentation and feature extraction processing on the audio information to be processed, which is conducive to directly extracting effective entity information from the audio and classifying the audio, reducing the intermediate process of data processing, avoiding the impact of errors in converting audio data into text data, improving data processing efficiency and saving maintenance costs.

[0107] Example 2

[0108] See also Figure 2 , Figure 2 FIG. 1 is a flow chart of another audio recognition method disclosed in an embodiment of the present invention. Figure 2 The described audio recognition method is applied to a data processing system, such as a local server or cloud server for audio recognition management, and the embodiments of the present invention do not limit this. Figure 2 As shown, the audio recognition method may include the following operations:

[0109] 201. Obtain audio information to be processed.

[0110] 202. Preprocess the audio information to be processed to obtain first voice data feature information.

[0111] 203. Obtain an audio training set.

[0112] In the embodiment of the present invention, the audio training set includes a plurality of audio training information.

[0113] 204. Train a preset first training model using the audio training set to obtain an audio recognition model.

[0114] 205. Use the audio recognition model to perform recognition processing on the feature information of the first voice data to obtain target audio recognition information.

[0115] In the embodiment of the present invention, for the specific technical details and technical terminology of steps 201 and 202, reference can be made to the detailed description of steps 101 and 102 in the first embodiment, which will not be repeated in detail in the embodiment of the present invention.

[0116] It can be seen that the audio recognition method described in the embodiment of the present invention can pre-process the audio information to be processed, then train the first training model to obtain an audio recognition model, and finally use the audio recognition model to perform further recognition processing to obtain target audio recognition information. This is conducive to directly extracting effective entity information from the audio and classifying the audio, reducing the intermediate process of data processing, avoiding the impact of errors in converting audio data into text data, improving data processing efficiency and saving maintenance costs.

[0117] In an optional embodiment, the above-mentioned audio training set is used to train the preset first training model to obtain the audio recognition model, including:

[0118] Determining target audio training information based on the audio training set;

[0119] Using the target audio training information to train the preset first training model to obtain a second training model;

[0120] Performing loss function calculation processing on the training parameter information corresponding to the second training model to obtain target loss function value information;

[0121] Determine whether the loss function value information meets the training termination condition, and obtain a first training judgment result;

[0122] When the first training judgment result is no, the first training model is updated using the second training model, and the target audio training information is determined according to the audio training set;

[0123] When the first training judgment result is yes, an audio recognition model is determined according to the second training model.

[0124] Optionally, the above-mentioned training termination conditions include the number of iterations being equal to the iteration threshold, and / or the target loss function value information converges, which is not limited in the embodiment of the present invention.

[0125] It can be seen that the implementation of the audio recognition method described in the embodiment of the present invention can obtain an audio recognition model by training the first training model, which is more conducive to directly extracting effective entity information from the audio and classifying the audio, reducing the intermediate process of data processing, avoiding the impact of errors in converting audio data into text data, improving data processing efficiency and saving maintenance costs.

[0126] In another optional embodiment, performing loss function calculation processing on the training parameter information corresponding to the second training model to obtain target loss function value information includes:

[0127] Using a preset first loss function model to calculate the training parameter information corresponding to the second training model, to obtain first loss function value information;

[0128] Using a preset second loss function model to calculate the training parameter information corresponding to the second training model to obtain second loss function value information;

[0129] Using a preset third loss function model to calculate the training parameter information corresponding to the second training model to obtain third loss function value information;

[0130] The first loss function value information, the second loss function value information and the third loss function value information are weighted and summed to obtain the target loss function value information.

[0131] Optionally, the above-mentioned training parameter information includes sentence vector matrix information, and / or audio decoding matrix information, and / or target word label information, and / or target classification label information, and / or path parameter information, which is not limited in the embodiment of the present invention.

[0132] Optionally, the first loss function value information is obtained by calculating and processing the sentence vector matrix information and the audio decoding matrix information using the first loss function model.

[0133] Optionally, the first loss function value information represents information divergence.

[0134] Optionally, the above sentence vector matrix information is obtained through the following steps:

[0135] Perform word segmentation on the target audio training information to obtain word sequence information;

[0136] The word segmentation sequence information is transformed using a first loss function model to obtain word vector information; the word vector information includes a plurality of word vectors; the dimension of the word vector is 768;

[0137] The above word vector information is concatenated to obtain sentence vector information;

[0138] Obtain position encoding vector information; the dimension of the position encoding vector information is consistent with the dimension of the sentence vector information;

[0139] The above sentence vector information and position encoding vector information are associated to obtain sentence vector matrix information.

[0140] Optionally, the above audio decoding matrix information is obtained through the following steps:

[0141] Using the second training model, a two-layer convolution process is performed on the FBANK feature information corresponding to the target audio training information to obtain the first intermediate feature information;

[0142] Performing irrelevant information filtering on the first intermediate feature information to obtain second intermediate feature information;

[0143] Using multiple transformer encoders to perform feature extraction on the second intermediate feature information to obtain third intermediate feature information;

[0144] The third intermediate feature information is decoded using multiple serially connected decoders to obtain audio decoding matrix information.

[0145] Optionally, the above audio decoding matrix information includes a matrix with a dimension of L*768.

[0146] Optionally, the above L is equal to a length threshold.

[0147] Optionally, the target word label information includes word prediction label information and / or word real label information, which is not limited in this embodiment of the present invention.

[0148] Optionally, the target word label information is obtained through the following steps:

[0149] Convert the audio decoding matrix information to obtain first word vector information; the first word vector information includes L 7-dimensional word label vectors; and a label score corresponding to an element of each dimension in the word label vector;

[0150] The first word vector information is subjected to label score screening to obtain word prediction label information; the word prediction label information includes a plurality of word prediction labels; the word prediction label represents the maximum label score in each word label vector.

[0151] Optionally, the above path parameter information is obtained through the following steps:

[0152] Use N CRF layers to process the above word prediction label information to obtain path matrix information;

[0153] The first path matrix information is transformed to obtain path parameter information.

[0154] In this optional embodiment, as an optional implementation, the above-mentioned method of calculating and processing the training parameter information corresponding to the second training model using the preset second loss function model to obtain the second loss function value information is as follows:

[0155] Using the first loss function sub-model, a cross entropy comparison calculation is performed on the above-mentioned word predicted label information and the word true label information to obtain the first loss sub-function value information;

[0156] Utilizing the second loss function sub-model to perform optimal path loss calculation on the path parameter information, and obtain second loss sub-function value information;

[0157] The first loss sub-function value information and the second loss sub-function value information are weightedly summed to obtain the second loss function value information.

[0158] Optionally, the target classification label information includes predicted classification label information and / or real classification label information, which is not limited in this embodiment of the present invention.

[0159] Optionally, the target classification label information is obtained through the following steps:

[0160] Performing average calculation on the audio decoding matrix information to obtain first output vector information;

[0161] Performing a maximum value calculation process on the audio decoding matrix information to obtain second output vector information;

[0162] Performing splicing processing on the audio decoding matrix information, the first output vector information, and the second output vector information to obtain third output vector information;

[0163] Perform label classification on the third output vector information to obtain predicted classification label information.

[0164] Optionally, the third loss function value information is obtained by performing a cross entropy comparison calculation on the predicted classification label information and the true classification label information using a third loss function model.

[0165] It can be seen that the implementation of the audio recognition method described in the embodiment of the present invention can obtain the target loss function value information by calculating and processing the loss function, which is conducive to directly extracting effective entity information from the audio and classifying the audio, reducing the intermediate process of data processing, avoiding the impact of errors in converting audio data into text data, improving data processing efficiency and saving maintenance costs.

[0166] In yet another optional embodiment, the step of obtaining the audio training set includes:

[0167] Obtaining an original video information set; the original video information set includes a plurality of original video information;

[0168] Extract, identify and associate the original video information set to obtain the target data set information;

[0169] The target data set information is classified and processed to obtain the audio training set.

[0170] It can be seen that the implementation of the audio recognition method described in the embodiment of the present invention can obtain an audio training set by comprehensive processing of the original video information, which is more conducive to directly extracting effective entity information from the audio and classifying the audio, reducing the intermediate process of data processing, avoiding the impact of errors in converting audio data into text data, improving data processing efficiency and saving maintenance costs.

[0171] In yet another optional embodiment, the above extraction, recognition, and association processing of the original video information set to obtain target data set information includes:

[0172] Performing audio extraction processing on the original video information set to obtain a first audio information set; the first audio information set includes a plurality of first audio information;

[0173] Performing denoising and separation processing on the first audio information set to obtain a second audio information set; the second audio information set includes a plurality of second audio information;

[0174] Performing recognition processing on the second audio information set to obtain a third audio information set; the third audio information set includes a plurality of third audio information;

[0175] Correcting and labeling the third audio information set to obtain a fourth audio information set and a fifth audio information set; the fourth audio information set includes a plurality of fourth audio information; and the fifth audio information set includes a plurality of fifth audio information;

[0176] The second audio information set, the third audio information set, the fourth audio information set and the fifth audio information set are associated with each other to obtain an audio training set.

[0177] Optionally, the third audio information includes audio sequence number information, and / or audio name information, and / or audio text, which is not limited in this embodiment of the present invention.

[0178] Optionally, the fourth audio information includes audio sequence information, and / or audio tag information, and / or audio text information, which is not limited in this embodiment of the present invention.

[0179] Optionally, the fifth audio information includes audio sequence information, and / or audio category information, and / or audio text information, which is not limited in this embodiment of the present invention.

[0180] Optionally, the denoising and separation processing of the first audio information set includes separating human voice and background music from the first audio information set.

[0181] Optionally, the above-mentioned recognition processing of the second audio information set includes denoising the second audio information set, removing silent blank audio, and performing recognition conversion processing.

[0182] It can be seen that the implementation of the audio recognition method described in the embodiment of the present invention can obtain an audio training set by extracting, identifying and associating the original video information, which is more conducive to directly extracting effective entity information from the audio and classifying the audio, reducing the intermediate process of data processing, avoiding the impact of errors in converting audio data into text data, improving data processing efficiency and saving maintenance costs.

[0183] Example 3

[0184] See also Figure 3 , Figure 3 This is a schematic diagram of the structure of an audio recognition device disclosed in an embodiment of the present invention. Figure 3The described device can be applied to a data processing system, such as a local server or cloud server for audio recognition management, etc., and the embodiments of the present invention do not limit this. Figure 3 As shown, the device may include:

[0185] An acquisition module 301 is used to acquire audio information to be processed;

[0186] A first processing module 302 is configured to pre-process the audio information to be processed to obtain first voice data feature information;

[0187] The second processing module 303 is used to use the audio recognition model to perform recognition processing on the feature information of the first speech data to obtain target audio recognition information; the target audio recognition information includes M audio recognition result information; M is a positive integer greater than or equal to 3.

[0188] It can be seen that implementation Figure 3 The described audio recognition device can obtain target audio recognition information by preprocessing the audio information to be processed and then using the audio recognition model to perform further recognition processing. This is conducive to directly extracting effective entity information from the audio and classifying the audio, reducing the intermediate data processing process, avoiding the impact of errors in converting audio data into text data, improving data processing efficiency and saving maintenance costs.

[0189] In another optional embodiment, as Figure 4 As shown, the first processing module 302 pre-processes the audio information to be processed to obtain the first voice data feature information in the following manner:

[0190] Performing denoising and separation processing on the audio information to be processed to obtain first audio processing information;

[0191] The first audio processing information is segmented and feature extracted to obtain first voice data feature information.

[0192] It can be seen that implementation Figure 4 The described audio recognition device can obtain first voice data feature information by performing denoising, separation, segmentation and feature extraction on the audio information to be processed, which is conducive to directly extracting effective entity information from the audio and classifying the audio, reducing the intermediate data processing process, avoiding the impact of errors in converting audio data into text data, improving data processing efficiency and saving maintenance costs.

[0193] In another optional embodiment, Figure 4 As shown, before the second processing module 303 uses the audio recognition model to perform recognition processing on the first voice data feature information to obtain the target audio recognition information, the acquisition module 301 is further used to obtain an audio training set; the audio training set includes a plurality of audio training information;

[0194] The device also includes:

[0195] The training module 304 is configured to train a preset first training model using an audio training set to obtain an audio recognition model.

[0196] It can be seen that implementation Figure 4 The described audio recognition device can pre-process the audio information to be processed, then train the first training model to obtain an audio recognition model, and finally use the audio recognition model to perform further recognition processing to obtain target audio recognition information. This is conducive to directly extracting effective entity information from the audio and classifying the audio, reducing the intermediate data processing process, avoiding the impact of errors in converting audio data into text data, improving data processing efficiency and saving maintenance costs.

[0197] In another optional embodiment, Figure 4 As shown, the training module 304 uses the audio training set to train the preset first training model to obtain the audio recognition model in the following manner:

[0198] Determining target audio training information based on the audio training set;

[0199] Using the target audio training information to train the preset first training model to obtain a second training model;

[0200] Performing loss function calculation processing on the training parameter information corresponding to the second training model to obtain target loss function value information;

[0201] Determine whether the loss function value information meets the training termination condition, and obtain a first training judgment result;

[0202] When the first training judgment result is no, the first training model is updated using the second training model, and the target audio training information is determined according to the audio training set;

[0203] When the first training judgment result is yes, an audio recognition model is determined according to the second training model.

[0204] It can be seen that implementation Figure 4 The described audio recognition device can obtain an audio recognition model by training the first training model, which is more conducive to directly extracting effective entity information from the audio and classifying the audio, reducing the intermediate data processing process, avoiding the impact of errors in converting audio data into text data, improving data processing efficiency and saving maintenance costs.

[0205] In another optional embodiment, Figure 4As shown, the training module 304 performs loss function calculation processing on the training parameter information corresponding to the second training model, and obtains the target loss function value information in the following specific manner:

[0206] Using a preset first loss function model to calculate the training parameter information corresponding to the second training model, to obtain first loss function value information;

[0207] Using a preset second loss function model to calculate the training parameter information corresponding to the second training model to obtain second loss function value information;

[0208] Using a preset third loss function model to calculate the training parameter information corresponding to the second training model to obtain third loss function value information;

[0209] The first loss function value information, the second loss function value information and the third loss function value information are weighted and summed to obtain the target loss function value information.

[0210] It can be seen that implementation Figure 4 The described audio recognition device can obtain target loss function value information by calculating and processing the loss function, which is conducive to directly extracting effective entity information from audio and classifying the audio, reducing the intermediate data processing process, avoiding the impact of errors in converting audio data into text data, improving data processing efficiency and saving maintenance costs.

[0211] In another optional embodiment, Figure 4 As shown, the specific method for the training module 304 to obtain the audio training set is:

[0212] Obtaining an original video information set; the original video information set includes a plurality of original video information;

[0213] Extract, identify and associate the original video information set to obtain the target data set information;

[0214] The target data set information is classified and processed to obtain the audio training set.

[0215] It can be seen that implementation Figure 4 The described audio recognition device can obtain an audio training set through comprehensive processing of original video information, which is more conducive to directly extracting effective entity information from the audio and classifying the audio, reducing the intermediate data processing process, avoiding the influence of errors in converting audio data into text data, improving data processing efficiency and saving maintenance costs.

[0216] In another optional embodiment, Figure 4As shown, the training module 304 extracts, identifies and associates the original video information set to obtain the target data set information in the following manner:

[0217] Performing audio extraction processing on the original video information set to obtain a first audio information set; the first audio information set includes a plurality of first audio information;

[0218] Performing denoising and separation processing on the first audio information set to obtain a second audio information set; the second audio information set includes a plurality of second audio information;

[0219] Performing recognition processing on the second audio information set to obtain a third audio information set; the third audio information set includes a plurality of third audio information;

[0220] Correcting and labeling the third audio information set to obtain a fourth audio information set and a fifth audio information set; the fourth audio information set includes a plurality of fourth audio information; and the fifth audio information set includes a plurality of fifth audio information;

[0221] The second audio information set, the third audio information set, the fourth audio information set and the fifth audio information set are associated with each other to obtain an audio training set.

[0222] It can be seen that implementation Figure 4 The described audio recognition device can obtain an audio training set by extracting, identifying and associating original video information, which is more conducive to directly extracting effective entity information from audio and classifying the audio, reducing the intermediate data processing process, avoiding the impact of errors in converting audio data into text data, improving data processing efficiency and saving maintenance costs.

[0223] Example 4

[0224] See also Figure 5 , Figure 5 This is a structural diagram of another audio recognition device disclosed in an embodiment of the present invention. Figure 5 The described device can be applied to a data processing system, such as a local server or cloud server for audio recognition management, etc., and the embodiments of the present invention do not limit this. Figure 5 As shown, the device may include:

[0225] A memory 401 storing executable program code;

[0226] a processor 402 coupled to the memory 401;

[0227] The processor 402 calls the executable program code stored in the memory 401 to execute the steps of the audio recognition method described in the first or second embodiment.

[0228] Example 5

[0229] An embodiment of the present invention discloses a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps of the audio recognition method described in the first or second embodiment.

[0230] Example 6

[0231] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute the steps of the audio recognition method described in the first or second embodiment.

[0232] The device embodiments described above are merely illustrative. Modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical modules, i.e., they may be located in one place or distributed across multiple network modules. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0233] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus the necessary general hardware platform, or of course, by means of hardware. Based on this understanding, the above technical solution, in essence, or the portion that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0234] Finally, it should be noted that the audio recognition method and device disclosed in the embodiments of the present invention are only preferred embodiments of the present invention, and are only used to illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. An audio recognition method, characterized in that: The method comprises: Get the audio information to be processed; Preprocessing the audio information to be processed to obtain first voice data feature information; Using an audio recognition model to perform recognition processing on the first voice data feature information to obtain target audio recognition information; the target audio recognition information includes M audio recognition result information; where M is a positive integer greater than or equal to 3; The preprocessing of the audio information to be processed to obtain first voice data feature information includes: Performing denoising and separation processing on the audio information to be processed to obtain first audio processing information; Performing segmentation and feature extraction processing on the first audio processing information to obtain first voice data feature information; The segmenting and feature extraction processing of the first audio processing information to obtain first voice data feature information includes: Determine whether the length information corresponding to the first audio processing information meets a length threshold, and obtain a length determination result; When the length determination result is negative, padding the first audio processing information with a PAD character and the length threshold to obtain first standby audio information; Segmenting the first audio information to be used to obtain a second audio information set to be used; the second audio information set to be used includes K pieces of second audio information to be used; the K represents the number of frames corresponding to the first audio information to be used; Perform FBANK feature extraction on the second set of standby audio information to obtain first voice data feature information.

2. The audio recognition method according to claim 1, wherein: Before using the audio recognition model to perform recognition processing on the first voice data feature information to obtain target audio recognition information, the method further includes: Acquire an audio training set; the audio training set includes a plurality of audio training information; The audio training set is used to train a preset first training model to obtain the audio recognition model.

3. The audio recognition method according to claim 2, characterized in that The step of training a preset first training model using the audio training set to obtain the audio recognition model includes: Determining target audio training information based on the audio training set; Using the target audio training information to train a preset first training model to obtain a second training model; Performing loss function calculation processing on the training parameter information corresponding to the second training model to obtain target loss function value information; Determine whether the loss function value information meets the training termination condition, and obtain a first training judgment result; When the first training judgment result is no, the first training model is updated using the second training model, and the step of determining target audio training information based on the audio training set is triggered; When the first training judgment result is yes, the audio recognition model is determined according to the second training model.

4. The audio recognition method according to claim 3, characterized in that The performing loss function calculation processing on the training parameter information corresponding to the second training model to obtain target loss function value information includes: Using a preset first loss function model to calculate the training parameter information corresponding to the second training model to obtain first loss function value information; Using a preset second loss function model to calculate the training parameter information corresponding to the second training model to obtain second loss function value information; Using a preset third loss function model to calculate the training parameter information corresponding to the second training model to obtain third loss function value information; The first loss function value information, the second loss function value information and the third loss function value information are weightedly summed to obtain target loss function value information.

5. The audio recognition method according to claim 2, characterized in that The obtaining of the audio training set comprises: Acquire an original video information set; the original video information set includes a plurality of original video information; Extracting, identifying, and associating the original video information set to obtain target data set information; The target data set information is classified and processed to obtain an audio training set.

6. The audio recognition method according to claim 5, characterized in that The extracting, identifying and associating the original video information set to obtain target data set information includes: Performing audio extraction processing on the original video information set to obtain a first audio information set; the first audio information set includes a plurality of first audio information; Performing denoising and separation processing on the first audio information set to obtain a second audio information set; the second audio information set includes a plurality of second audio information; Performing recognition processing on the second audio information set to obtain a third audio information set; the third audio information set includes a plurality of third audio information; Correcting and labeling the third audio information set to obtain a fourth audio information set and a fifth audio information set; the fourth audio information set includes a plurality of fourth audio information; and the fifth audio information set includes a plurality of fifth audio information; The second audio information set, the third audio information set, the fourth audio information set, and the fifth audio information set are associated to obtain an audio training set.

7. An audio recognition device, characterized in that: The device comprises: An acquisition module is used to obtain audio information to be processed; A first processing module, configured to pre-process the audio information to be processed to obtain first voice data feature information; a second processing module, configured to perform recognition processing on the first voice data feature information using an audio recognition model to obtain target audio recognition information; the target audio recognition information includes M audio recognition result information; where M is a positive integer greater than or equal to 3; The first processing module pre-processes the audio information to be processed to obtain the first voice data feature information in the following manner: Performing denoising and separation processing on the audio information to be processed to obtain first audio processing information; Performing segmentation and feature extraction processing on the first audio processing information to obtain first voice data feature information; The first processing module performs segmentation and feature extraction on the first audio processing information to obtain the first voice data feature information in a specific manner including: Determine whether the length information corresponding to the first audio processing information meets a length threshold, and obtain a length determination result; When the length determination result is negative, padding the first audio processing information with a PAD character and the length threshold to obtain first standby audio information; Segmenting the first audio information to be used to obtain a second audio information set to be used; the second audio information set to be used includes K pieces of second audio information to be used; the K represents the number of frames corresponding to the first audio information to be used; Perform FBANK feature extraction on the second set of standby audio information to obtain first voice data feature information.

8. An audio recognition device, characterized in that: The device comprises: a memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the audio recognition method according to any one of claims 1 to 6.

9. A computer storage medium, characterized in that The computer storage medium stores computer instructions, and when the computer instructions are called, they are used to execute the audio recognition method according to any one of claims 1 to 6.

Images