Training method of audio recognition model and audio equipment

By cropping audio samples into audio clips and training the audio recognition model, the problem of high annotation cost in audio recognition model training is solved, and the audio length is consistent during model training and inference stages is achieved, reducing the annotation cost and improving the recognition accuracy .

CN118942464APending Publication Date: 2024-11-12ANKER INNOVATIONS TECH CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202310520497.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-09
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

During the current training process of audio recognition models, labeling costs are high because labeling is required for each sample audio, especially when identifying high-diversity sound events (such as dog barking), it is difficult to accurately distinguish, resulting in data. Improved quality of the collection requires a lot of manual effort.

Method used

By obtaining the target audio length and training sample set, crop the sample audio as an audio clip to ensure that the length of the audio clip is consistent with the target audio length. Use the audio clip as input data and use the sample label as output data to train and generate an audio recognition model, so as to achieve the model The training and inference stages keep the audio length consistent and reduce the annotation cost.

Benefits of technology

It effectively reduces the labeling cost, improves the training efficiency and accuracy of the audio recognition model, reduces the dependence on manual labeling, and improves the accuracy of the model when identifying various sound sources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118942464A_ABST
    Figure CN118942464A_ABST
Patent Text Reader

Abstract

The embodiment of the invention relates to a training method of an audio recognition model and audio equipment. The method comprises the following steps: acquiring a target audio length and a training sample set; according to the target audio length, cutting sample audios in the training sample set to obtain an audio clip set corresponding to the sample audios; and taking the audio clips in the audio clip set corresponding to the sample audio as input data, taking the sample tags corresponding to the sample audio as expected output data, and training to generate an audio recognition model. Therefore, under the condition that the length of the audio in the model training stage is greater than the length of the audio in the model reasoning stage, the labeled label can be used as the label of the cut audio clip, so that the labeling cost is reduced by ensuring that the lengths of the audio used in the model training stage are equal to the lengths of the audio used in the model reasoning stage.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and in particular to a training method and audio device for an audio recognition model. Background Art

[0002] The sounds of all things carry a lot of information about the daily environment and the physical events that occur in it, and SED (Sound Event Detect) can sense the sound scene we are in and identify each sound source. Through audio recognition technology, when the microphone picks up a sound, it can be identified by artificial intelligence, such as distinguishing between the sound of a pet barking, a baby crying, and the sound of glass breaking, so as to sense the current event and automatically trigger pre-set response measures.

[0003] In order to achieve good enough results, current artificial intelligence requires a large amount of labeled audio for model training in advance. The quality of data annotation is directly related to the accuracy of the later model application. In order to obtain a sufficiently accurate audio data set, it usually takes a lot of cost to label the audio data. For example, in the dog barking recognition task, a large amount of labeled audio is required during training to mark whether there is dog barking in each audio. Because the sounds of dog barking are diverse, there are dog barking sounds that are difficult to distinguish manually. Therefore, the same audio data often needs to be repeatedly corrected and confirmed to reduce the influence of the subjective feelings of the annotator and improve the quality of the data set.

[0004] In the current audio technology training process, in some scenarios, each audio sample is usually labeled, resulting in high labeling costs. Summary of the invention

[0005] In view of this, in order to solve the above technical problems, the embodiments of the present application provide a training method and an audio device for an audio recognition model.

[0006] In a first aspect, an embodiment of the present application provides a method for training an audio recognition model, the method comprising:

[0007] Obtaining a target audio length and a training sample set, wherein the target audio length is the length of the audio used to be input into the audio recognition model, the training samples in the training sample set include sample audio and sample labels corresponding to the sample audio, and the length of the sample audio is greater than the target audio length;

[0008] According to the target audio length, clipping the sample audio in the training sample set to obtain an audio segment set corresponding to the sample audio, wherein the length of the audio segments in the audio segment set is the target audio length;

[0009] The audio segment in the audio segment set corresponding to the sample audio is used as input data, and the sample label corresponding to the sample audio is used as expected output data to train and generate an audio recognition model.

[0010] In a possible implementation, the step of clipping the sample audio in the training sample set according to the target audio length to obtain a set of audio segments corresponding to the sample audio includes:

[0011] Determine the length of the sample audio in the training sample set to obtain the sample audio length;

[0012] Determining the number of clippings of the sample audio in the training sample set based on the target audio length and the sample audio length, wherein the number of clippings represents the number of audio segments obtained by clipping the sample audio;

[0013] According to the target audio length, the sample audio in the training sample set is cropped into the cropped number of audio segments to obtain an audio segment set corresponding to the sample audio.

[0014] In a possible implementation, determining the number of sample audios to be cropped in the training sample set based on the target audio length and the sample audio length includes:

[0015] Determining a quotient of the sample audio length and the target audio length;

[0016] Based on the quotient value, the number of clippings of the sample audio in the training sample set is determined.

[0017] In a possible implementation manner, determining the number of sample audios to be cropped in the training sample set based on the quotient value includes:

[0018] When the quotient value is an integer, the quotient value is determined as the number of clippings of the sample audio in the training sample set;

[0019] In the case that the quotient is a non-integer, a result of rounding up the quotient is determined as the number of sample audio clips in the training sample set.

[0020] In a possible implementation manner, when the quotient value is a non-integer, the step of cropping the sample audio in the training sample set into the cropped number of audio segments according to the target audio length to obtain the audio segment set corresponding to the sample audio includes:

[0021] Determine a result of rounding the quotient upwards;

[0022] Determine, based on the result, the sample audio length, and the target audio length, cropping position information of the sample audio in the training sample set, wherein the number of cropping positions indicated by the cropping position information is one less than the cropping number;

[0023] According to the target audio length, at the cropping position indicated by the cropping position information, the sample audio in the training sample set is cropped into the cropping number of audio segments to obtain an audio segment set corresponding to the sample audio.

[0024] In a possible implementation, the step of using an audio segment in an audio segment set corresponding to the sample audio as input data and a sample label corresponding to the sample audio as expected output data to train and generate an audio recognition model includes:

[0025] Obtain a first model set, wherein the number of first models in the first model set is the number of audio segments obtained by cropping the sample audio;

[0026] For each sample audio in the training sample set, each audio segment in the audio segment set corresponding to the sample audio is respectively determined as input data of each first model in the first model set, and actual output data of each first model is obtained, and the actual output data of each first model is used as input data of the second model, and the sample label corresponding to the sample audio is used as the expected output data of the second model, and each first model is trained respectively, wherein the first model is used to train to obtain an audio recognition model, and the actual output data of the first model represents the recognition result of the audio segment input to the first model;

[0027] The first model that has been trained is determined as the audio recognition model.

[0028] In a possible implementation manner, determining each audio segment in the audio segment set corresponding to the sample audio as input data of each first model in the first model set includes:

[0029] For each audio segment in the audio segment set corresponding to the sample audio, perform the following steps:

[0030] Randomly determine a target first model from the first model set, wherein the target first model is a first model of the input data corresponding to the sample audio that has not been determined;

[0031] The audio segment is determined as input data of the currently randomly determined target first model to determine input data corresponding to the sample audio of the currently randomly determined target first model.

[0032] In a possible implementation manner, the model structures of the first models in the first model set are consistent.

[0033] In a possible implementation, there are overlapping parts in the audio segments in the audio segment set corresponding to the cropped sample audio.

[0034] In a second aspect, an embodiment of the present application provides a training device for an audio recognition model, the device comprising:

[0035] an acquisition unit, configured to acquire a target audio length and a training sample set, wherein the target audio length is the length of the audio input to the audio recognition model, the training samples in the training sample set include sample audio and sample labels corresponding to the sample audio, and the length of the sample audio is greater than the target audio length;

[0036] a cropping unit, configured to crop the sample audio in the training sample set according to the target audio length to obtain an audio segment set corresponding to the sample audio, wherein the length of the audio segments in the audio segment set is the target audio length;

[0037] The training unit is used to train and generate an audio recognition model by taking an audio segment in an audio segment set corresponding to the sample audio as input data and a sample label corresponding to the sample audio as expected output data.

[0038] In a possible implementation, the step of clipping the sample audio in the training sample set according to the target audio length to obtain a set of audio segments corresponding to the sample audio includes:

[0039] Determine the length of the sample audio in the training sample set to obtain the sample audio length;

[0040] Determining the number of clippings of the sample audio in the training sample set based on the target audio length and the sample audio length, wherein the number of clippings represents the number of audio segments obtained by clipping the sample audio;

[0041] According to the target audio length, the sample audio in the training sample set is cropped into the cropped number of audio segments to obtain an audio segment set corresponding to the sample audio.

[0042] In a possible implementation, determining the number of sample audios to be cropped in the training sample set based on the target audio length and the sample audio length includes:

[0043] Determining a quotient of the sample audio length and the target audio length;

[0044] Based on the quotient value, the number of clippings of the sample audio in the training sample set is determined.

[0045] In a possible implementation manner, determining the number of sample audios to be cropped in the training sample set based on the quotient value includes:

[0046] When the quotient value is an integer, the quotient value is determined as the number of clippings of the sample audio in the training sample set;

[0047] In the case that the quotient is a non-integer, a result of rounding up the quotient is determined as the number of sample audio clips in the training sample set.

[0048] In a possible implementation manner, when the quotient value is a non-integer, the step of cropping the sample audio in the training sample set into the cropped number of audio segments according to the target audio length to obtain the audio segment set corresponding to the sample audio includes:

[0049] Determine a result of rounding the quotient upwards;

[0050] Determine, based on the result, the sample audio length, and the target audio length, cropping position information of the sample audio in the training sample set, wherein the number of cropping positions indicated by the cropping position information is one less than the cropping number;

[0051] According to the target audio length, at the cropping position indicated by the cropping position information, the sample audio in the training sample set is cropped into the cropping number of audio segments to obtain an audio segment set corresponding to the sample audio.

[0052] In a possible implementation, the step of using an audio segment in an audio segment set corresponding to the sample audio as input data and a sample label corresponding to the sample audio as expected output data to train and generate an audio recognition model includes:

[0053] Obtain a first model set, wherein the number of first models in the first model set is the number of audio segments obtained by cropping the sample audio;

[0054] For each sample audio in the training sample set, each audio segment in the audio segment set corresponding to the sample audio is respectively determined as input data of each first model in the first model set, and actual output data of each first model is obtained, and the actual output data of each first model is used as input data of the second model, and the sample label corresponding to the sample audio is used as the expected output data of the second model, and each first model is trained respectively, wherein the first model is used to train to obtain an audio recognition model, and the actual output data of the first model represents the recognition result of the audio segment input to the first model;

[0055] The first model that has been trained is determined as the audio recognition model.

[0056] In a possible implementation manner, determining each audio segment in the audio segment set corresponding to the sample audio as input data of each first model in the first model set includes:

[0057] For each audio segment in the audio segment set corresponding to the sample audio, perform the following steps:

[0058] Randomly determine a target first model from the first model set, wherein the target first model is a first model of the input data corresponding to the sample audio that has not been determined;

[0059] The audio segment is determined as input data of the currently randomly determined target first model to determine input data corresponding to the sample audio of the currently randomly determined target first model.

[0060] In a possible implementation manner, the model structures of the first models in the first model set are consistent.

[0061] In a possible implementation, there are overlapping parts in the audio segments in the audio segment set corresponding to the cropped sample audio.

[0062] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0063] Memory for storing computer programs;

[0064] The processor is used to execute the computer program stored in the memory, and when the computer program is executed, the method of any embodiment of the audio recognition model training method of the first aspect of the present application is implemented.

[0065] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method of any embodiment of the audio recognition model training method of the first aspect described above is implemented.

[0066] In a fifth aspect, an embodiment of the present application provides a computer program, which includes a computer-readable code. When the computer-readable code is executed on a device, the processor in the device implements a method as in any embodiment of the method for training an audio recognition model in the first aspect described above.

[0067] In a sixth aspect, an embodiment of the present application provides an audio device,

[0068] The audio device is used to implement the method of any embodiment of the method for training an audio recognition model according to the first aspect above;

[0069] or,

[0070] The audio device is used to perform audio recognition using an audio recognition model trained and generated by the method described in the first aspect above.

[0071] The training method of the audio recognition model provided in the embodiment of the present application can obtain the target audio length and the training sample set, wherein the target audio length is the length of the audio input to the audio recognition model, the training samples in the training sample set include the sample audio and the sample label corresponding to the sample audio, and the length of the sample audio is greater than the target audio length, and then, according to the target audio length, the sample audio in the training sample set is clipped to obtain the audio segment set corresponding to the sample audio, wherein the length of the audio segment in the audio segment set is the target audio length, and then, the audio segment in the audio segment set corresponding to the sample audio is used as input data, and the sample label corresponding to the sample audio is used as the expected output data to train and generate the audio recognition model. Thus, the sample audio in the training sample set can be clipped according to the target audio length to obtain the audio segment set corresponding to the sample audio, and the sample label corresponding to the sample audio is directly used as the audio segment in the audio segment set corresponding to the sample audio, so as to train the audio recognition model, so that when the length of the audio in the model training stage is greater than the length of the audio in the model inference stage, the labeled label can be used as the label of the clipped audio segment, thereby reducing the labeling cost by ensuring that the length of the audio used in the model training stage and the model inference stage is equal. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 A flowchart of a method for training an audio recognition model provided in an embodiment of the present application;

[0073] Figure 2 A flowchart of another method for training an audio recognition model provided in an embodiment of the present application;

[0074] Figure 3A A flowchart of another method for training an audio recognition model provided in an embodiment of the present application;

[0075] Figure 3B A schematic diagram of a training process of an audio recognition model involved in an embodiment of the present application;

[0076] Figure 3C A schematic diagram of the reasoning process of an audio recognition model involved in an embodiment of the present application;

[0077] Figure 4 A schematic diagram of the structure of a training device for an audio recognition model provided in an embodiment of the present application;

[0078] Figure 5 A schematic diagram of the structure of an audio device provided in an embodiment of the present application;

[0079] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0080] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present application.

[0081] Those skilled in the art will understand that the terms "first" and "second" in the embodiments of the present application are only used to distinguish between different steps, devices, modules and other objects, and do not represent any specific technical meanings, nor do they indicate the logical order between them.

[0082] It should also be understood that in this embodiment, “plurality” may refer to two or more than two, and “at least one” may refer to one, two or more than two.

[0083] It should also be understood that any component, data or structure mentioned in the embodiments of the present application can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0084] In addition, the term "and / or" in this application is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this application generally indicates that the associated objects before and after are in an "or" relationship.

[0085] It should also be understood that the description of the various embodiments in this application focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0086] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present application, its application, or uses.

[0087] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the above-mentioned technologies, methods, and equipment should be considered as part of the specification.

[0088] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0089] It should be noted that, in the absence of conflict, the embodiments in this application and the features in the embodiments can be combined with each other. To facilitate the understanding of the embodiments of the present application, the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0090] Figure 1 A flowchart of a method for training an audio recognition model provided in an embodiment of the present application is shown below. Figure 1 As shown, the method specifically includes:

[0091] Step 101, obtaining a target audio length and a training sample set, wherein the target audio length is the length of the audio used to be input into the audio recognition model, the training samples in the training sample set include sample audio and sample labels corresponding to the sample audio, and the length of the sample audio is greater than the target audio length.

[0092] In this embodiment, during the training phase and the inference phase of the audio recognition model, the length of each audio input to the audio recognition model is the target audio length.

[0093] The training sample set may include multiple training samples.

[0094] The sample audio may be the audio included in the training sample in the training sample set. The sample audio may be used to train and obtain an audio recognition model.

[0095] The sample label corresponding to the sample audio may indicate the category of the sample audio. For example, this label may indicate that the sample audio is a pet's cry, a baby's cry, or glass breaking.

[0096] The length of the audio (including the length of the sample audio) can indicate the duration of the audio.

[0097] Here, the length of each sample audio in the training sample set may be 10 seconds, and the target audio length may be 5 seconds.

[0098] Here, the lengths of any two audio samples in the training sample set may be equal or different.

[0099] Step 102: According to the target audio length, the sample audio in the training sample set is cropped to obtain an audio segment set corresponding to the sample audio, wherein the length of the audio segments in the audio segment set is the target audio length.

[0100] In this embodiment, the audio segments in the audio segment set corresponding to the sample audio may be one or more audio segments in the sample audio.

[0101] As an example, if the length of the sample audio is 10 seconds, and the length of the target audio is 5 seconds, then the audio segment set corresponding to the sample audio may include audio segment 1 and audio segment 2. Audio segment 1 may be the segment corresponding to the period from the start to 5 seconds of the sample audio. Audio segment 2 may be the segment corresponding to the period from 5 seconds to the end of the sample audio.

[0102] As another example, if the length of the sample audio is 10 seconds, and the length of the target audio is 5 seconds, then the audio segment set corresponding to the sample audio may include audio segment 1, audio segment 2, and audio segment 3. Among them, audio segment 1 may be the segment corresponding to the period from the beginning to 5 seconds of the sample audio. Audio segment 2 may be the segment corresponding to the period from 2.5 seconds to 7.5 seconds of the sample audio. Audio segment 3 may be the segment corresponding to the period from 5 seconds to the end of the sample audio.

[0103] Step 103: Use the audio segments in the audio segment set corresponding to the sample audio as input data, and use the sample labels corresponding to the sample audio as expected output data to train and generate an audio recognition model.

[0104] In this embodiment, the sample label corresponding to the sample audio can be used as the label of the audio segment in the audio segment set corresponding to the sample audio. Further, a machine learning algorithm is used to train and generate an audio recognition model by taking the audio segment in the audio segment set corresponding to the sample audio as input data and the sample label corresponding to the sample audio as expected output data.

[0105] As an example, if the training sample set includes training sample 1 and training sample 2. Training sample 1 includes sample audio 1 and sample label 1, and training sample 2 includes sample audio 2 and sample label 2. The set of audio segments corresponding to sample audio 1 obtained by cropping sample audio 1 includes audio segment 1 and audio segment 2, and the set of audio segments corresponding to sample audio 2 obtained by cropping sample audio 2 includes audio segment 3 and audio segment 4. Then, audio segment 1 can be used as input data and input into the first model used to train the audio recognition model to obtain the actual output data corresponding to audio segment 1. Audio segment 2 can be used as input data and input into the first model used to train the audio recognition model to obtain the actual output data corresponding to audio segment 2. Then, the actual output data corresponding to audio segment 1 and the actual output data corresponding to audio segment 2 are input into the second model to obtain the actual output data of the second model, and the model parameters of the first model are adjusted again by calculating the loss function value of the actual output data of the second model and sample label 1. Optionally, the model parameters of the second model can also be adjusted again by calculating the loss function value of the actual output data of the second model and sample label 1. Then, the audio segment 3 is used as input data and input into the first model used for training the audio recognition model to obtain the actual output data corresponding to the audio segment 3. The audio segment 4 is used as input data and input into the first model used for training the audio recognition model to obtain the actual output data corresponding to the audio segment 4. Thereafter, the actual output data corresponding to the audio segment 3 and the actual output data corresponding to the audio segment 4 are input into the second model to obtain the actual output data of the second model. The model parameters of the first model are adjusted by calculating the loss function value of the actual output data of the second model and the sample label 2. Optionally, the model parameters of the second model can be adjusted again by calculating the loss function value of the actual output data of the second model and the sample label 1. The first model after the model parameters are adjusted is determined as the audio recognition model.

[0106] It can be understood that the above example only describes the process of performing one round of training using a training sample set. In practice, the same training sample set can be used to perform multiple rounds of training to obtain an audio recognition model.

[0107] Optionally, other methods may be used to train the audio recognition model. Please refer to the following description for details, which will not be described here.

[0108] In some optional implementations of this embodiment, there are overlapping parts in the audio segments in the audio segment set corresponding to the cropped sample audio.

[0109] Here, for example, if the sample audio is 10 seconds, it can be split into 1 second to 6 seconds as one audio segment and 4 seconds to 10 seconds as another audio segment. In this way, each audio segment is 6 seconds long, and 4 seconds to 6 seconds in the sample audio are the overlapping parts of the two short audio segments.

[0110] It can be understood that in the above optional implementation, by adjusting the number of audio splits and the audio overlap width, it can be ensured that any sample audio can be segmented into any length to achieve the training of the audio recognition model.

[0111] The training method of the audio recognition model provided in the embodiment of the present application can obtain the target audio length and the training sample set, wherein the target audio length is the length of the audio input to the audio recognition model, the training samples in the training sample set include the sample audio and the sample label corresponding to the sample audio, and the length of the sample audio is greater than the target audio length, and then, according to the target audio length, the sample audio in the training sample set is clipped to obtain the audio segment set corresponding to the sample audio, wherein the length of the audio segment in the audio segment set is the target audio length, and then, the audio segment in the audio segment set corresponding to the sample audio is used as input data, and the sample label corresponding to the sample audio is used as the expected output data to train and generate the audio recognition model. Thus, the sample audio in the training sample set can be clipped according to the target audio length to obtain the audio segment set corresponding to the sample audio, and the sample label corresponding to the sample audio is directly used as the audio segment in the audio segment set corresponding to the sample audio, so as to train the audio recognition model, so that when the length of the audio in the model training stage is greater than the length of the audio in the model inference stage, the labeled label can be used as the label of the clipped audio segment, thereby reducing the labeling cost by ensuring that the length of the audio used in the model training stage and the model inference stage is equal.

[0112] Figure 2 A flowchart of another method for training an audio recognition model provided in an embodiment of the present application. Figure 2 As shown, the method specifically includes:

[0113] Step 201, obtain a target audio length and a training sample set, wherein the target audio length is the length of the audio used to input into the audio recognition model, the training samples in the training sample set include sample audio and sample labels corresponding to the sample audio, and the length of the sample audio is greater than the target audio length.

[0114] In this embodiment, step 201 and Figure 1 Step 101 in the corresponding embodiment is basically the same and will not be described again here.

[0115] Step 202: determine the length of the sample audio in the training sample set to obtain the sample audio length.

[0116] In this embodiment, the sample audio length may be the length of the sample audio in the training sample set.

[0117] Step 203: Determine the number of clippings of the sample audio in the training sample set based on the target audio length and the sample audio length, wherein the clipping number represents the number of audio segments obtained by clipping the sample audio.

[0118] In this embodiment, if the number of cropping is n, the sample audio in the training sample set may be cropped into n+1 audio segments, that is, the audio segment set corresponding to the sample audio includes n+1 elements.

[0119] In some optional implementations of this embodiment, the following method may be used to determine the number of sample audio clips in the training sample set based on the target audio length and the sample audio length:

[0120] First, a quotient value of the sample audio length and the target audio length is determined.

[0121] Then, the number of sample audios to be cropped in the training sample set is determined based on the quotient. For example, the number of sample audios to be cropped in the training sample set may be determined based on at least one of a result of rounding off, a result of rounding up, and a result of rounding down the quotient.

[0122] It can be understood that in the above optional implementation, the number of sample audios to be cropped in the training sample set can be determined based on the quotient of the sample audio length and the target audio length, which can improve the accuracy of determining the number of croppings.

[0123] In some application scenarios of the above optional implementations, the following method may be used to determine the number of sample audios to be cropped in the training sample set based on the quotient value:

[0124] When the quotient is an integer, the quotient is determined as the cropped number of sample audios in the training sample set; when the quotient is a non-integer, the result of rounding up the quotient is determined as the cropped number of sample audios in the training sample set.

[0125] It can be understood that in the above application scenario, when the quotient is an integer, the quotient can be directly determined as the number of sample audios to be cropped in the training sample set, so that the audio segments in the audio segment set corresponding to the cropped single sample audio are connected end to end and do not overlap each other, which not only makes full use of the sample audio, but also avoids the situation where the amount of calculation is too large due to data redundancy. In the case where the quotient is a non-integer, the result of rounding up the quotient is determined as the number of sample audios to be cropped in the training sample set, which can make more full use of the audio data in the sample audio.

[0126] In some cases of the above application scenarios, when the quotient value is a non-integer, the sample audio in the training sample set may be cropped into the cropped number of audio segments according to the target audio length in the following manner to obtain an audio segment set corresponding to the sample audio:

[0127] First, the result of rounding the quotient upward is determined.

[0128] Afterwards, based on the result, the sample audio length and the target audio length, the cropping position information of the sample audio in the training sample set is determined.

[0129] The number of cropping positions indicated by the cropping position information is one less than the cropping number.

[0130] As an example, the audio segment at the head of the sample audio and having a length equal to the target audio length may be firstly cut, then the audio segment at the tail of the sample audio and having a length equal to the target audio length may be cut, and then m-2 audio segments at the middle of the sample audio and having a length equal to the target audio length may be cut. Wherein, m is the difference between the above result and 2, and the overlapping parts of adjacent audio segments cut are equal in duration, and each part of the sample audio is included in the obtained audio segment.

[0131] Finally, according to the target audio length, at the cropping position indicated by the cropping position information, the sample audio in the training sample set is cropped into the cropping number of audio segments to obtain an audio segment set corresponding to the sample audio.

[0132] It can be understood that in the above case, through the results, the sample audio length and the target audio length, the cropping position information of the sample audio can be determined more accurately, and then the sample audio can be more fully utilized to avoid excessive calculation caused by data redundancy.

[0133] Step 204 : According to the target audio length, the sample audio in the training sample set is cropped into the cropped number of audio segments to obtain an audio segment set corresponding to the sample audio.

[0134] Step 205 , using the audio segments in the audio segment set corresponding to the sample audio as input data, and using the sample labels corresponding to the sample audio as expected output data, to train and generate an audio recognition model.

[0135] In this embodiment, step 205 and Figure 1 Step 103 in the corresponding embodiment is basically the same and will not be described again here.

[0136] It should be noted that, in addition to the above contents, this embodiment may also include Figure 1 The corresponding technical features described in the corresponding embodiments are realized Figure 1 For details, please refer to the technical effect of the training method of the audio recognition model shown in Figure 1 For the sake of brevity, the relevant description is not repeated here.

[0137] The training method of the audio recognition model provided in the embodiment of the present application determines the number of sample audio clips in the training sample set based on the target audio length and the sample audio length, and then clips the sample audio, and uses the clipped audio segments to train the audio recognition model, which can further improve the recognition accuracy of the audio recognition model.

[0138] Figure 3A A flow chart of another method for training an audio recognition model provided in an embodiment of the present application. This method can be applied to one or more electronic devices such as smart phones, laptops, desktop computers, portable computers, servers, etc. In addition, the execution subject of this method can be hardware or software. When the above-mentioned execution subject is hardware, the execution subject can be one or more of the above-mentioned electronic devices. For example, a single electronic device can execute this method, or multiple electronic devices can cooperate with each other to execute this method. When the above-mentioned execution subject is software, this method can be implemented as multiple software or software modules, or as a single software or software module. No specific limitation is made here.

[0139] Specifically, if Figure 3A As shown, the method specifically includes:

[0140] Step 301, obtain a target audio length and a training sample set, wherein the target audio length is the length of the audio used to input into the audio recognition model, the training samples in the training sample set include sample audio and sample labels corresponding to the sample audio, and the length of the sample audio is greater than the target audio length.

[0141] In this embodiment, step 301 and Figure 1 Step 101 in the corresponding embodiment is basically the same and will not be described again here.

[0142] Step 302: According to the target audio length, the sample audio in the training sample set is cropped to obtain an audio segment set corresponding to the sample audio, wherein the length of the audio segments in the audio segment set is the target audio length.

[0143] In this embodiment, step 302 and Figure 1 Step 102 in the corresponding embodiment is basically the same and will not be described again here.

[0144] Step 303: Obtain a first model set, wherein the number of first models in the first model set is the number of audio segments obtained by cropping the sample audio.

[0145] In this embodiment, each first model in the first model set may have the same model structure and first model parameters. The first model in the first model set is used to train and obtain an audio recognition model.

[0146] Step 304, for each sample audio in the training sample set, each audio segment in the audio segment set corresponding to the sample audio is respectively determined as the input data of each first model in the first model set, and the actual output data of each first model is obtained, and the actual output data of each first model is used as the input data of the second model, and the sample label corresponding to the sample audio is used as the expected output data of the second model, and each first model is trained respectively, wherein the first model is used to train to obtain an audio recognition model, and the actual output data of the first model represents the recognition result of the audio segment input to the first model.

[0147] In this embodiment, if the training sample set includes training sample 1 and training sample 2. Training sample 1 includes sample audio 1 and sample label 1, and training sample 2 includes sample audio 2 and sample label 2. The set of audio segments corresponding to sample audio 1 obtained by cropping sample audio 1 includes audio segment 1 and audio segment 2, and the set of audio segments corresponding to sample audio 2 obtained by cropping sample audio 2 includes audio segment 3 and audio segment 4. The first model set includes the first model 1 and the first model 2. Then, the audio segment 1 can be used as input data and input into the first model 1 used for training to obtain the audio recognition model to obtain the actual output data corresponding to the audio segment 1. The audio segment 2 can be used as input data and input into the first model 2 used for training to obtain the audio recognition model to obtain the actual output data corresponding to the audio segment 2. Afterwards, the actual output data corresponding to the audio segment 1 and the actual output data corresponding to the audio segment 2 are used as input data of the second model to obtain the actual output data of the second model, and the model parameters of the first model 1, the first model 2, and the second model are adjusted by calculating the loss function value of the actual output data of the second model and the sample label 1 corresponding to the sample audio 1. Then, the audio clip 3 is used as input data and input into the first model 1 for training the audio recognition model to obtain the actual output data corresponding to the audio clip 3. The audio clip 4 is used as input data and input into the first model 2 for training the audio recognition model to obtain the actual output data corresponding to the audio clip 4. After that, the actual output data corresponding to the audio clip 3 and the actual output data corresponding to the audio clip 4 are used as input data of the second model to obtain the actual output data of the second model, and the model parameters of the first model 1, the first model 2 and the second model are adjusted by calculating the loss function value of the actual output data of the second model and the sample label 2 corresponding to the sample audio 2. The first model 1 or the first model 2 after the model parameters are adjusted is determined as the audio recognition model.

[0148] It can be understood that the above example only describes the process of performing one round of training using a training sample set. In practice, the same training sample set can be used for multiple rounds of training to obtain an audio recognition model.

[0149] In some optional implementations of this embodiment, the following method may be used to respectively determine each audio segment in the audio segment set corresponding to the sample audio as input data of each first model in the first model set:

[0150] For each audio segment in the audio segment set corresponding to the sample audio, perform the following steps (including step 1 and step 2):

[0151] Step 1: randomly determine a target first model from the first model set.

[0152] The target first model is a first model of input data for which the sample audio is not determined.

[0153] Step 2: determine the audio segment as input data of the currently randomly determined target first model to determine the input data corresponding to the sample audio of the currently randomly determined target first model.

[0154] In this embodiment, if the training sample set includes training sample 1 and training sample 2. Training sample 1 includes sample audio 1 and sample label 1, and training sample 2 includes sample audio 2 and sample label 2. The audio segment set corresponding to sample audio 1 obtained by cropping sample audio 1 includes audio segment 1 and audio segment 2, and the audio segment set corresponding to sample audio 2 obtained by cropping sample audio 2 includes audio segment 3 and audio segment 4. The first model set includes first model 1 and first model 2.

[0155] Then, first, the first model can be randomly determined from the first model 1 and the first model 2. For example, the first model 1 can be used as the randomly determined first model. Afterwards, the audio segment 1 can be used as input data and input into the first model 1 for training the audio recognition model to obtain the actual output data corresponding to the audio segment 1. Afterwards, since the first model 1 is the first model that has determined the input data corresponding to the sample audio 1, the first model randomly determined for the second time is the first model 2. Next, the audio segment 2 is used as input data and input into the first model 2 for training the audio recognition model to obtain the actual output data corresponding to the audio segment 2. Subsequently, the actual output data corresponding to the audio segment 1 and the actual output data corresponding to the audio segment 2 are input into the second model to obtain the actual output data of the second model. By calculating the loss function value of the actual output data of the second model and the sample label 1, the model parameters of the first model 1 and the first model 2 are adjusted again. Optionally, the model parameters of the second model can also be adjusted by calculating the loss function value of the actual output data of the second model and the sample label 1.

[0156] At this point, the determination of the first model of the input data corresponding to the sample audio 1 has been completed, and then the determination of the first model of the input data corresponding to the sample audio 2 can be performed. Similarly, first determine to randomly determine the first model from the first model 1 and the first model 2, for example, the first model 1 can be used as the randomly determined first model. The audio segment 3 is used as input data and input into the first model 1 used for training to obtain the audio recognition model, and the actual output data corresponding to the audio segment 3 is obtained. Afterwards, since the first model 1 is the first model for which the input data corresponding to the sample audio 1 has been determined, the first model randomly determined for the second time is the first model 2. Next, the audio segment 4 is used as input data and input into the first model 2 used for training to obtain the audio recognition model, and the actual output data corresponding to the audio segment 4 is obtained. Subsequently, the actual output data corresponding to the audio segment 3 and the actual output data corresponding to the audio segment 4 are input into the second model to obtain the actual output data of the second model. The model parameters of the first model 1 and the first model 2 are adjusted by calculating the loss function value of the actual output data of the second model and the sample label 2. Optionally, the model parameters of the second model can also be adjusted by calculating the loss function value of the actual output data of the second model and the sample label 2. The first model 1 or the first model 2 after the model parameters are adjusted is determined as the audio recognition model.

[0157] Here, the above example only describes the process of performing one round of training using a training sample set. In practice, the same training sample set may be used to perform multiple rounds of training to obtain an audio recognition model.

[0158] It can be understood that in the above optional implementation, by randomly determining the target first model, the recognition accuracy of the audio recognition model finally trained can be further improved.

[0159] Step 305: determine the trained first model as the audio recognition model.

[0160] In this embodiment, any first model that has been trained can be determined as the audio recognition model, or the first model with the highest recognition accuracy among the first models that have been trained can be determined as the audio recognition model.

[0161] In some optional implementations of this embodiment, the model structures of the first models in the first model set are consistent.

[0162] It can be understood that in the above optional implementation, by ensuring that the model structures of the various first models in the first model set are consistent, it can be easier for the various first models to converge to a similar state, so as to improve the training efficiency.

[0163] The following is an illustrative description of the embodiments of the present application, but it should be noted that the embodiments of the present application may have the features described below, but the following description does not constitute a limitation on the scope of protection of the embodiments of the present application.

[0164] Here, we take the example that the audio length of the existing labeled database (that is, the length of the sample audio in the above training sample set) is 10s (seconds), and the target audio length to be detected is 5s, to explain how to train a model for 5s audio length input (that is, the above audio recognition model). In order to obtain 5s audio, the 10s audio data can be split into two 5s audio segments (that is, the above audio segments) for model training. This example is used for explanation. The schematic diagram of the whole process is shown below. Figure 3B and Figure 3C shown.

[0165] Step 1: In order to train the model with an input audio length of 5 seconds, all 10 seconds of audio in the database (that is, the above-mentioned sample audio set) can be cut into two segments, each with a length of 5 seconds.

[0166] In more general cases, if the target audio length is less than 5s, the audio can be cut into more than 2 segments, depending on specific needs. If the target audio length is greater than 5s, the audio can be split into two segments, and overlapping parts are allowed between the two audio segments. For example, a 10-second audio can be split into one segment from 1 to 6 seconds and another segment from 4 to 10 seconds. In this way, each audio segment is 6 seconds long, and the 4 to 6 seconds in the long audio are the overlapping parts of the two short audio segments. By adjusting the number of audio splits and the audio overlap width, it can be ensured that any long audio can be segmented into sub-models of any length requirements.

[0167] Step 2: Preprocess the audio data. Extract the main information features of the audio and remove the redundant information. This can reduce the amount of data that the model needs to process, thereby reducing the complexity of the algorithm. In the above example, two 5s audio clips are used as training input data for sub-model A and sub-model B respectively after feature extraction. This step 2 is not necessary. You can choose whether to perform audio preprocessing (feature extraction) based on the specific situation of the sub-model.

[0168] In order to make each sub-model receive the same audio data, the order of model A and model B can be repeatedly randomly shuffled during the entire training process. That is, the target first model is randomly determined from the first model set, wherein the target first model is the first model for which the input data corresponding to the sample audio has not been determined; the audio clip is determined as the input data of the currently randomly determined target first model, so as to determine the input data corresponding to the sample audio of the currently randomly determined target first model.

[0169] Step 3, based on the input data, sub-model A and sub-model B respectively calculate the probability of containing the target audio, that is, the confidence level. If it is a binary classification task, such as determining whether there is a dog barking in the audio, 0 represents 100% no dog barking, and 1 represents 100% dog barking, then each model outputs a confidence value in the range of 0 to 1, that is, the probability of the existence of dog barking. If it is a multi-classification task, such as determining whether the target audio contains dog barking, baby crying, human voice, etc., the model will output a vector composed of the probabilities of each target event. For example, if the sub-model analyzes that the input audio contains 11% probability of dog barking, 73% of baby crying, and 88% of human voice, the vector output by the sub-model is [0.11, 0.73, 0.88].

[0170] It is also worth noting that sub-model A (i.e., the first model) and sub-model B (i.e., the first model) are as consistent as possible in structure, mainly to make it easier for each sub-model to converge to a similar state in the end, and any sub-model can be selected for classification in the reasoning process. For example, sub-model A and sub-model B can include at least one of the following model structures: CNN (Convolutional Neural Network), RNN (Recurrent Neural Network).

[0171] Step 4: Input the confidence values ​​calculated by each sub-model into the information splicing model. The function of the information splicing model is to integrate the output information of each sub-model and judge whether the target audio exists in the entire long audio, that is, the audio before splitting.

[0172] The role of the information splicing model (that is, the second model mentioned above) is to integrate the information of all sub-models, because for long audio (corresponding to the 10s audio in this example), its audio label is known through manual annotation, and for the clipped audio (corresponding to the two 5s audio segments in this example), because the audio signal corresponding to the audio label may occur in any clipped audio segment, the specific labels of each audio segment after splitting cannot be determined. For example, it is known that there is target audio in the 10s long audio. After splitting, there are three situations: ① The first 5s audio segment contains the target event audio signal, but the second segment does not; ② The second 5s audio segment contains the target event audio signal, but the first 5s does not; ③ Both the first 5s and the second 5s contain the target event audio signal. Therefore, the information splicing model is needed to integrate the output information of the sub-model, and after analyzing the probability of the existence of the target in each audio segment given by the sub-model, the information splicing model makes the probability of the existence of the target audio in the entire audio segment.

[0173] Step 5: Compare the integrated information with the labels in the database, calculate the LF (loss function) based on the deviation of the comparison results, and then adjust each sub-model and the information splicing model according to the LF (loss function) to reduce the deviation between the output of the information splicing model and the database label.

[0174] Step 6, repeat steps 1 to 5, repeatedly traverse all audio data and labels in the database, and reduce LF (loss function) continuously. Finally, for each audio in the database, when the deviation between the output value of the second model and its corresponding label is small enough to satisfy the requirements, the loop can be stopped. At this point, the model training is completed.

[0175] After training, select any model from model A or model B. When training a sub-model, the length of its input audio is the length of the audio after editing, which is 5s for the above example. The output of the sub-model is the probability that the audio after editing contains the target event, so it can be directly used for reasoning in actual scenarios to obtain the probability that the input audio contains the target event. This method makes the length of the input audio in the actual reasoning process shorter than the long audio before editing, which reduces the amount of data that the model needs to process and reduces the amount of calculation.

[0176] According to this method, if the time length of the input audio needs to be adjusted during the training of the speech classification model, there is no need to re-label the audio. The existing labeled long audio data can be directly used to train the classification model with shorter input. In addition, the input audio in the actual reasoning process is shorter than the long audio for training, and the amount of data to be processed is reduced, thereby reducing the computational complexity accordingly.

[0177] It should be noted that, in addition to the contents recorded above, this embodiment may also include the technical features described in the above embodiments, so as to achieve the technical effects of the training method of the audio recognition model shown above. Please refer to the above description for details. For the sake of brevity, it will not be elaborated here.

[0178] The audio recognition model training method provided in the embodiment of the present application improves the speed of training to obtain the audio recognition model by training multiple first models and determining the audio recognition model therefrom.

[0179] Figure 4 A schematic diagram of the structure of a training device for an audio recognition model provided in an embodiment of the present application. Specifically comprising:

[0180] An acquisition unit 401 is used to acquire a target audio length and a training sample set, wherein the target audio length is the length of the audio input to the audio recognition model, the training samples in the training sample set include sample audio and sample labels corresponding to the sample audio, and the length of the sample audio is greater than the target audio length;

[0181] A cropping unit 402 is configured to crop the sample audio in the training sample set according to the target audio length to obtain an audio segment set corresponding to the sample audio, wherein the length of the audio segments in the audio segment set is the target audio length;

[0182] The training unit 403 is used to train and generate an audio recognition model by taking an audio segment in an audio segment set corresponding to the sample audio as input data and a sample label corresponding to the sample audio as expected output data.

[0183] In a possible implementation, the step of clipping the sample audio in the training sample set according to the target audio length to obtain a set of audio segments corresponding to the sample audio includes:

[0184] Determine the length of the sample audio in the training sample set to obtain the sample audio length;

[0185] Determining the number of clippings of the sample audio in the training sample set based on the target audio length and the sample audio length, wherein the number of clippings represents the number of audio segments obtained by clipping the sample audio;

[0186] According to the target audio length, the sample audio in the training sample set is cropped into the cropped number of audio segments to obtain an audio segment set corresponding to the sample audio.

[0187] In a possible implementation, determining the number of sample audios to be cropped in the training sample set based on the target audio length and the sample audio length includes:

[0188] Determining a quotient of the sample audio length and the target audio length;

[0189] Based on the quotient value, the number of clippings of the sample audio in the training sample set is determined.

[0190] In a possible implementation manner, determining the number of sample audios to be cropped in the training sample set based on the quotient value includes:

[0191] When the quotient value is an integer, the quotient value is determined as the number of clippings of the sample audio in the training sample set;

[0192] In the case that the quotient is a non-integer, a result of rounding up the quotient is determined as the number of sample audio clips in the training sample set.

[0193] In a possible implementation manner, when the quotient value is a non-integer, the step of cropping the sample audio in the training sample set into the cropped number of audio segments according to the target audio length to obtain the audio segment set corresponding to the sample audio includes:

[0194] Determine a result of rounding the quotient upwards;

[0195] Determine, based on the result, the sample audio length, and the target audio length, cropping position information of the sample audio in the training sample set, wherein the number of cropping positions indicated by the cropping position information is one less than the cropping number;

[0196] According to the target audio length, at the cropping position indicated by the cropping position information, the sample audio in the training sample set is cropped into the cropping number of audio segments to obtain an audio segment set corresponding to the sample audio.

[0197] In a possible implementation, the step of using an audio segment in an audio segment set corresponding to the sample audio as input data and a sample label corresponding to the sample audio as expected output data to train and generate an audio recognition model includes:

[0198] Obtain a first model set, wherein the number of first models in the first model set is the number of audio segments obtained by cropping the sample audio;

[0199] For each sample audio in the training sample set, each audio segment in the audio segment set corresponding to the sample audio is respectively determined as input data of each first model in the first model set, and actual output data of each first model is obtained, and the actual output data of each first model is used as input data of the second model, and the sample label corresponding to the sample audio is used as the expected output data of the second model, and each first model is trained respectively, wherein the first model is used to train to obtain an audio recognition model, and the actual output data of the first model represents the recognition result of the audio segment input to the first model;

[0200] The first model that has been trained is determined as the audio recognition model.

[0201] In a possible implementation manner, determining each audio segment in the audio segment set corresponding to the sample audio as input data of each first model in the first model set includes:

[0202] For each audio segment in the audio segment set corresponding to the sample audio, perform the following steps:

[0203] Randomly determine a target first model from the first model set, wherein the target first model is a first model of the input data corresponding to the sample audio that has not been determined;

[0204] The audio segment is determined as input data of the currently randomly determined target first model to determine input data corresponding to the sample audio of the currently randomly determined target first model.

[0205] In a possible implementation manner, the model structures of the first models in the first model set are consistent.

[0206] In a possible implementation, there are overlapping parts in the audio segments in the audio segment set corresponding to the cropped sample audio.

[0207] The training device for the audio recognition model provided in this embodiment can be as follows: Figure 4 The training device for the audio recognition model shown in can execute all the steps of the training methods for the audio recognition models described above, thereby achieving the technical effects of the training methods for the audio recognition models described above. For details, please refer to the relevant description above. For the sake of brevity, it will not be repeated here.

[0208] Figure 5 A schematic diagram of the structure of an audio device provided by an embodiment of the present invention, such as Figure 5 As shown, the audio device 1000 can implement any of the above-mentioned audio recognition model training methods;

[0209] or,

[0210] The audio device 1000 is used to perform audio recognition using an audio recognition model trained and generated by any of the above methods.

[0211] According to the audio device of an embodiment of the present invention, the training of the audio recognition model in the above embodiment is adopted. When the length of the audio in the model training stage is longer than the length of the audio in the model reasoning stage, the marked labels can be used as labels for the cropped audio segments, thereby reducing the labeling cost by ensuring that the lengths of the audio used in the model training stage and the model reasoning stage are equal.

[0212] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, Figure 6The electronic device 500 shown includes: at least one processor 501, a memory 502, at least one network interface 504 and other user interfaces 503. The various components in the electronic device 500 are coupled together via a bus system 505. It is understood that the bus system 505 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 505 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 505 is not shown in FIG. Figure 6 Various buses are labeled as bus system 505 .

[0213] The user interface 503 may include a display, a keyboard, or a pointing device (eg, a mouse, a trackball, a touch pad, or a touch screen).

[0214] It can be understood that the memory 502 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDRSDRAM), enhanced synchronous DRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus DRAM (DRRAM). The memory 502 described herein is intended to include, but is not limited to, these and any other suitable labeled memories.

[0215] In some implementations, the memory 502 stores the following elements, executable units or data structures, or a subset thereof, or an extended set thereof: an operating system 5021 and an application program 5022 .

[0216] Among them, the operating system 5021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application 5022 includes various application programs, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application services. The program for implementing the method of the embodiment of the present application can be included in the application 5022.

[0217] In this embodiment, by calling the program or instruction stored in the memory 502, specifically, the program or instruction stored in the application 5022, the processor 501 is used to execute the method steps provided by each method embodiment, for example, including:

[0218] Obtaining a target audio length and a training sample set, wherein the target audio length is the length of the audio used to be input into the audio recognition model, the training samples in the training sample set include sample audio and sample labels corresponding to the sample audio, and the length of the sample audio is greater than the target audio length;

[0219] According to the target audio length, clipping the sample audio in the training sample set to obtain an audio segment set corresponding to the sample audio, wherein the length of the audio segments in the audio segment set is the target audio length;

[0220] The audio segment in the audio segment set corresponding to the sample audio is used as input data, and the sample label corresponding to the sample audio is used as expected output data to train and generate an audio recognition model.

[0221] The method disclosed in the above embodiment of the present application can be applied to the processor 501, or implemented by the processor 501. The processor 501 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 501. The above processor 501 can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software units in the decoding processor can be executed. The software unit can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 502, and the processor 501 reads the information in the memory 502 and completes the steps of the above method in combination with its hardware.

[0222] It is understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPDevice, DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the above functions of the present application, or a combination thereof.

[0223] For software implementation, the technology described above in this article can be implemented by a unit that performs the functions described above in this article. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0224] The electronic device provided in this embodiment may be Figure 6 The electronic device shown in can execute all steps of the training methods of the audio recognition models described above, thereby achieving the technical effects of the training methods of the audio recognition models described above. Please refer to the above related description for details. For the sake of brevity, it will not be repeated here.

[0225] The embodiment of the present application also provides a storage medium (computer-readable storage medium). The storage medium here stores one or more programs. The storage medium may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a read-only memory, a flash memory, a hard disk or a solid-state drive; the memory may also include a combination of the above-mentioned types of memory.

[0226] When one or more programs in the storage medium can be executed by one or more processors, the training method of the audio recognition model executed on the electronic device side can be implemented.

[0227] The processor is used to execute the training program of the audio recognition model stored in the memory to implement the following steps of the training method of the audio recognition model executed on the electronic device side:

[0228] Obtaining a target audio length and a training sample set, wherein the target audio length is the length of the audio used to be input into the audio recognition model, the training samples in the training sample set include sample audio and sample labels corresponding to the sample audio, and the length of the sample audio is greater than the target audio length;

[0229] According to the target audio length, clipping the sample audio in the training sample set to obtain an audio segment set corresponding to the sample audio, wherein the length of the audio segments in the audio segment set is the target audio length;

[0230] The audio segment in the audio segment set corresponding to the sample audio is used as input data, and the sample label corresponding to the sample audio is used as expected output data to train and generate an audio recognition model.

[0231] The professionals should also be further aware that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0232] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0233] The specific implementation methods described above further describe the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only the specific implementation method of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. In addition, for the aforementioned embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present invention is not limited by the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present invention.

Claims

1. A method for training an audio recognition model, characterized in that: The method comprises: Obtaining a target audio length and a training sample set, wherein the target audio length is the length of the audio used to be input into the audio recognition model, the training samples in the training sample set include sample audio and sample labels corresponding to the sample audio, and the length of the sample audio is greater than the target audio length; According to the target audio length, clipping the sample audio in the training sample set to obtain an audio segment set corresponding to the sample audio, wherein the length of the audio segments in the audio segment set is the target audio length; The audio segment in the audio segment set corresponding to the sample audio is used as input data, and the sample label corresponding to the sample audio is used as expected output data to train and generate an audio recognition model.

2. The method according to claim 1, characterized in that The step of clipping the sample audio in the training sample set according to the target audio length to obtain a set of audio segments corresponding to the sample audio includes: Determine the length of the sample audio in the training sample set to obtain the sample audio length; Determining the number of clippings of the sample audio in the training sample set based on the target audio length and the sample audio length, wherein the number of clippings represents the number of audio segments obtained by clipping the sample audio; According to the target audio length, the sample audio in the training sample set is cropped into the cropped number of audio segments to obtain an audio segment set corresponding to the sample audio.

3. The method according to claim 2, characterized in that The step of determining the number of sample audios to be cropped in the training sample set based on the target audio length and the sample audio length includes: Determining a quotient of the sample audio length and the target audio length; Based on the quotient value, the number of clippings of the sample audio in the training sample set is determined.

4. The method according to claim 3, characterized in that The step of determining the number of sample audio samples to be cut in the training sample set based on the quotient value comprises: When the quotient value is an integer, the quotient value is determined as the number of clippings of the sample audio in the training sample set; In the case that the quotient is a non-integer, a result of rounding up the quotient is determined as the number of sample audio clips in the training sample set.

5. The method according to claim 4, characterized in that When the quotient value is a non-integer, the step of cutting the sample audio in the training sample set into the cut number of audio segments according to the target audio length to obtain the audio segment set corresponding to the sample audio includes: Determine a result of rounding the quotient upwards; Determine, based on the result, the sample audio length, and the target audio length, cropping position information of the sample audio in the training sample set, wherein the number of cropping positions indicated by the cropping position information is one less than the cropping number; According to the target audio length, at the cropping position indicated by the cropping position information, the sample audio in the training sample set is cropped into the cropping number of audio segments to obtain an audio segment set corresponding to the sample audio.

6. The method according to any one of claims 1 to 5, characterized in that: The method uses an audio segment in an audio segment set corresponding to the sample audio as input data, uses a sample label corresponding to the sample audio as expected output data, and trains and generates an audio recognition model, including: Obtain a first model set, wherein the number of first models in the first model set is the number of audio segments obtained by cropping the sample audio; For each sample audio in the training sample set, each audio segment in the audio segment set corresponding to the sample audio is respectively determined as input data of each first model in the first model set, and actual output data of each first model is obtained, and the actual output data of each first model is used as input data of the second model, and the sample label corresponding to the sample audio is used as the expected output data of the second model, and each first model is trained respectively, wherein the first model is used to train to obtain an audio recognition model, and the actual output data of the first model represents the recognition result of the audio segment input to the first model; The first model that has been trained is determined as the audio recognition model.

7. The method according to claim 6, characterized in that The step of respectively determining each audio segment in the audio segment set corresponding to the sample audio as input data of each first model in the first model set includes: For each audio segment in the audio segment set corresponding to the sample audio, perform the following steps: Randomly determine a target first model from the first model set, wherein the target first model is a first model of the input data corresponding to the sample audio that has not been determined; The audio segment is determined as input data of the currently randomly determined target first model to determine input data corresponding to the sample audio of the currently randomly determined target first model.

8. The method according to claim 6, characterized in that The model structures of the first models in the first model set are consistent.

9. The method according to any one of claims 1 to 5, characterized in that: There are overlapping parts among the audio segments in the audio segment set corresponding to the cropped sample audio.

10. An audio device, characterized in that: The audio device is used to implement the method described in any one of claims 1 to 9 above; or, The audio device is used to perform audio recognition using an audio recognition model trained and generated by the method described in any one of claims 1 to 9.

Citation Information

Cited By

  • Audio track data labeling method and device, equipment and storage medium

    CN119694334A