Deep learning-based sound recognition method and device, cloud device, and computer device

Abstract

The application relates to the field of sound recognition, in particular to a sound recognition method and device based on deep learning, a cloud device and a computer device, which greatly improve the accuracy of sound recognition classification. The scheme comprises the following steps: a classification head based on sigmoid is used to expand new labels for pictures in a training set, and the expansion label rule is that when a first sound source category and a second sound source category are classified, a value obtained by using a sigmoid activation function is used as a value of a label bit of the expansion label, the number of the label bits corresponds to the number of the sound source categories to be classified; a light-weight neural network model is used to identify and judge the first sound source category and the second sound source category, if the first sound source category is misjudged as the second sound source category, a loss degree of the judgment error is calculated according to the difference of the label bits, and the correlation between the first sound source category and the second sound source category is established according to the loss degree; and different sound sources are identified through the trained light-weight neural network model. The application is suitable for sound recognition classification.

Description

Technical Field

[0001] This invention relates to the field of voice recognition, and more specifically to a voice recognition method, apparatus, cloud device, and computer device based on deep learning. Background Technology

[0002] Sound recognition systems are primarily used to classify sound signals captured by devices, determining the source of the captured sound. Examples include dog barks, bird calls, baby cries, and car horns. They can be used for early warning purposes, such as alerting users to baby cries, nighttime dog barks, and car horns in no-honking zones. They can also be used for automated data processing and analysis, such as monitoring the time and frequency of bird calls and other protected animal calls in nature reserves for accurate tracking.

[0003] While existing sound recognition technologies can extract features from audio signals and classify common sound categories using deep learning models trained on classification deep networks, misidentification often occurs when these models are used extensively over extended periods. For example, a dog's bark might sometimes be misidentified as a baby crying or a bird chirping, resulting in inaccurate classification results. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a sound recognition method, device, cloud device and computer device based on deep learning, which greatly improves the accuracy of sound recognition and classification.

[0005] The present invention achieves the above objectives by adopting the following technical solution: Firstly, the present invention provides a deep learning-based sound recognition method, comprising:

[0006] Creating a lightweight neural network model: In the first layer of the neural network model, the preprocessed Mel spectrum is fed into BatchNorm for batch normalization; grouped depthwise separable convolutions are used consecutively to extract features from the Mel spectrum; a fully connected layer is connected as the feature layer of the model, and the feature maps extracted by the convolutions are compressed into feature vectors and connected to the softmax cross-entropy loss function; the feature layer is connected to a fully connected layer with multiple categories, and after this fully connected layer, each neuron is connected to a sigmoid binary cross-entropy loss function;

[0007] Training a lightweight neural network model: New labels are amplified on the images in the training set using a sigmoid-based classification head. The rule for amplifying the labels is: when classifying at least the first sound source category and the second sound source category, the value obtained by the corresponding sigmoid activation function is used as the label bit value of the amplified label for the first sound source category and the second sound source category. The number of label bits corresponds to the number of sound source categories that need to be classified.

[0008] A lightweight neural network model is used to identify and judge the first sound source category and the second sound source category. If the first sound source category is misclassified as the second sound source category, or the second sound source category is misclassified as the first sound source category, the degree of loss of judgment error is calculated based on the difference of label bits. The difference of label bits is proportional to the degree of loss of judgment error. The correlation between the first sound source category and the second sound source category is established based on the degree of loss.

[0009] Different sound sources are identified using a trained lightweight neural network model.

[0010] Furthermore, training lightweight neural network models specifically includes:

[0011] The correlation between the first sound source category and the second sound source category is judged. If the correlation between the first sound source category and the second sound source category is lower than the set threshold, the amplification labels of the first sound source category and the second sound source category are changed to reduce the degree of misjudgment loss between the first sound source category and the second sound source category.

[0012] Furthermore, training lightweight neural network models specifically includes:

[0013] When classifying the first, second, and third sound source categories, the correlation between these categories is assessed. If the correlation between the first and second sound source categories is greater than the correlation between the second and third sound source categories, or between the first and third sound source categories, then when the amplified or predicted label includes the first sound source category, the loss levels of the second and third sound source categories are normalized. Normalization specifically involves dividing the calculated loss level by the number of label differences, performing inverse differentiation on the resulting value, and updating the weights. The inverse calculation method is as follows:

[0014]

[0015]

[0016] m represents the degree of difference between the label and the prediction, and the degree of difference is normalized. Lsigmoidn represents the nth binary cross-entropy loss. f1 represents the category other than the first sound source category. f2 represents the category other than the first sound source category in the predicted label or the amplified label. h represents the result of the model output after passing through the activation function before entering the loss function. L represents the loss obtained by measuring the classification difference.

[0017] The cross-entropy loss for binary classification is calculated as follows:

[0018] y i For amplified true labels, For predicted labels.

[0019] Furthermore, during backpropagation, if the value obtained by the first sound source category through the sigmoid activation function is greater than a set threshold, while the values ​​obtained by other sound sources through the sigmoid activation function are less than the set threshold, then the backpropagation value is calculated using f1.

[0020] In a second aspect, the present invention provides a deep learning-based voice recognition device, the device comprising:

[0021] The model creation module is used to create lightweight neural network models. Specifically, in the first layer of the neural network model, the preprocessed Mel spectrum is fed into BatchNorm for batch normalization; grouped depthwise separable convolutions are used consecutively to extract features from the Mel spectrum; a fully connected layer is connected as the model's feature layer, the feature maps extracted by the convolutions are compressed into feature vectors, and then the softmax cross-entropy loss function is applied; the feature layer is connected to a fully connected layer with multiple categories, and after this fully connected layer, each neuron is connected to a sigmoid binary cross-entropy loss function.

[0022] The model training module is used to train a lightweight neural network model. Specifically, it includes amplifying new labels for images in the training set using a sigmoid-based classification head. The rule for amplifying labels is: when classifying at least the first sound source category and the second sound source category, the value obtained by the corresponding sigmoid activation function is used as the label bit value of the amplified label for the first sound source category and the second sound source category. The number of label bits corresponds to the number of sound source categories that need to be classified.

[0023] A lightweight neural network model is used to identify and judge the first sound source category and the second sound source category. If the first sound source category is misclassified as the second sound source category, or the second sound source category is misclassified as the first sound source category, the degree of loss of judgment error is calculated based on the difference of label bits. The difference of label bits is proportional to the degree of loss of judgment error. The correlation between the first sound source category and the second sound source category is established based on the degree of loss.

[0024] The sound source recognition module is used to identify different sound sources using a trained lightweight neural network model.

[0025] Furthermore, the model training module is specifically used to determine the correlation between the first sound source category and the second sound source category. If the correlation between the first sound source category and the second sound source category is lower than the set threshold, the amplified labels of the first sound source category and the second sound source category are changed to reduce the degree of misjudgment loss between the first sound source category and the second sound source category.

[0026] Furthermore, the model training module is also specifically used to determine the correlation between the first, second, and third sound source categories when classifying them. If the correlation between the first and second sound source categories is greater than the correlation between the second and third sound source categories or between the first and third sound source categories, and when the amplified or predicted label contains the first sound source category, the loss of the second and third sound source categories is normalized. Normalization specifically involves dividing the calculated loss by the number of label differences, taking the inverse derivative of the obtained value, and updating the weights. The inverse calculation method is as follows:

[0027]

[0028]

[0029] m represents the degree of difference between the label and the prediction, and the degree of difference is normalized. Lsigmoidn represents the nth binary cross-entropy loss. f1 represents the category other than the first sound source category. f2 represents the category other than the first sound source category in the predicted label or the amplified label.

[0030] The cross-entropy loss for binary classification is calculated as follows:

[0031] y i For amplified true labels, For predicted labels.

[0032] Furthermore, the model training module is also used to calculate the backpropagation value by using f1 if the value obtained by the first sound source category through the sigmoid activation function is greater than a set threshold, and the values ​​obtained by other sound sources through the sigmoid activation function are less than the set threshold.

[0033] Thirdly, the present invention provides a cloud device, the cloud device including a server, wherein the deep learning-based voice recognition device described above is installed on the server.

[0034] Fourthly, the present invention provides a computer device including a memory storing program instructions, wherein when the program instructions are executed, the deep learning-based voice recognition method described above is performed.

[0035] The beneficial effects of this invention are as follows:

[0036] This invention employs a lightweight neural network model that is more compatible with the Mel spectrum. During model training, it establishes corresponding correlations for audio samples with high relevance, i.e., those that are easily confused. Ultimately, it can effectively identify and classify multiple audio categories on the same model, greatly improving the accuracy of sound recognition. Attached Figure Description

[0037] Figure 1 This is a flowchart of a deep learning-based sound recognition method provided in an embodiment of the present invention;

[0038] Figure 2 This is a structural block diagram of a deep learning-based voice recognition device provided in an embodiment of the present invention. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0040] like Figure 1 As shown, this invention provides a deep learning-based sound recognition method, specifically including:

[0041] Creating a lightweight neural network model: In the first layer of the neural network model, the preprocessed Mel spectrum is fed into BatchNorm for batch normalization; grouped depthwise separable convolutions are used consecutively to extract features from the Mel spectrum; a fully connected layer is connected as the feature layer of the model, and the feature maps extracted by the convolutions are compressed into feature vectors and connected to the softmax cross-entropy loss function; the feature layer is connected to a fully connected layer with multiple categories, and after this fully connected layer, each neuron is connected to a sigmoid binary cross-entropy loss function;

[0042] Training a lightweight neural network model: New labels are amplified on the images in the training set using a sigmoid-based classification head. The rule for amplifying the labels is: when classifying at least the first sound source category and the second sound source category, the value obtained by the corresponding sigmoid activation function is used as the label bit value of the amplified label for the first sound source category and the second sound source category. The number of label bits corresponds to the number of sound source categories that need to be classified.

[0043] A lightweight neural network model is used to identify and judge the first sound source category and the second sound source category. If the first sound source category is misclassified as the second sound source category, or the second sound source category is misclassified as the first sound source category, the degree of loss of judgment error is calculated based on the difference of label bits. The difference of label bits is proportional to the degree of loss of judgment error. The correlation between the first sound source category and the second sound source category is established based on the degree of loss.

[0044] Different sound sources are identified using a trained lightweight neural network model.

[0045] For example, in this embodiment, during model training, the first sound source category is puppy barking, and the second sound source category is baby crying. When classifying puppy barking and baby crying, the value obtained from the sigmoid activation function is used as the label value for the amplified label. The sigmoid activation function can obtain a value between 0 and 1. In this embodiment, two categories are classified, so the label for puppy barking can be set to 00, and the label for baby crying can be set to 11. When using a lightweight neural network model to identify and classify puppy barking and baby crying, if the model misclassifies puppy barking as baby crying (i.e., misclassifies the label 00 as 11), both label bits are different, so the model suffers a misclassification loss of 2. This establishes a correlation between puppy barking and baby crying. During training, this correlation helps the model to more accurately distinguish between puppy barking and baby crying.

[0046] In one embodiment of the present invention, training the lightweight neural network model further includes:

[0047] The correlation between the first sound source category and the second sound source category is judged. If the correlation between the first sound source category and the second sound source category is lower than the set threshold, the amplification labels of the first sound source category and the second sound source category are changed to reduce the degree of misjudgment loss between the first sound source category and the second sound source category.

[0048] In this embodiment, if the first sound source category is an infant crying and the second sound source category is a car horn, the correlation between the two is low, meaning that an infant crying and a car horn are not easily confused. Therefore, the degree of misjudgment loss between infant crying and car horn can be reduced by changing the labels. For example, if the original label for infant crying is set to 11 and the label for car horn is 00, the degree of misjudgment loss is 2. Then, the label for car horn can be changed to 10 or 11, reducing the degree of misjudgment loss to 1 or 0.

[0049] In one embodiment of the present invention, training the lightweight neural network model further includes:

[0050] When classifying the first, second, and third sound source categories, the correlation between these categories is assessed. If the correlation between the first and second sound source categories is greater than the correlation between the second and third sound source categories, or between the first and third sound source categories, then when the amplified or predicted label includes the first sound source category, the loss levels of the second and third sound source categories are normalized. Normalization specifically involves dividing the calculated loss level by the number of label differences, performing inverse differentiation on the resulting value, and updating the weights. The inverse calculation method is as follows:

[0051]

[0052]

[0053] m represents the degree of difference between the label and the prediction, and the degree of difference is normalized. Lsigmoidn represents the nth binary cross-entropy loss. f1 represents the category other than the first sound source category. f2 represents the category other than the first sound source category in the predicted label or the amplified label.

[0054] The cross-entropy loss for binary classification is calculated as follows:

[0055] y i For amplified true labels, For predicted labels.

[0056] In this embodiment, if the first sound source category is a dog barking, the second sound source category is a baby crying, and the third sound source category is a car horn, then when classifying dog barking, baby crying, and car horn, a relevance judgment is needed. Dog barking and baby crying are easily confused into the same category, thus having high attention or relevance. Dog barking and car horn, as well as baby crying and car horn, are not easily confused, thus having low attention or relevance. Therefore, the judgment result is that the relevance between dog barking and baby crying is higher than that between dog barking and car horn, and between baby crying and car horn. In this case, the loss of the sound source label with low attention (i.e., low relevance) is reduced. If the amplified true label or the model-predicted label contains dog barking, in addition to calculating the loss according to the set label, the loss of baby crying and car horn is normalized.

[0057] like Figure 2 As shown, based on the above method, the present invention also provides a deep learning-based voice recognition device, the device comprising:

[0058] The model creation module is used to create lightweight neural network models. Specifically, in the first layer of the neural network model, the preprocessed Mel spectrum is fed into BatchNorm for batch normalization; grouped depthwise separable convolutions are used consecutively to extract features from the Mel spectrum; a fully connected layer is connected as the model's feature layer, the feature maps extracted by the convolutions are compressed into feature vectors, and then the softmax cross-entropy loss function is applied; the feature layer is connected to a fully connected layer with multiple categories, and after this fully connected layer, each neuron is connected to a sigmoid binary cross-entropy loss function.

[0059] The model training module is used to train a lightweight neural network model. Specifically, it includes amplifying new labels for images in the training set using a sigmoid-based classification head. The rule for amplifying labels is: at least the first sound source category and the second sound source category are classified. The value obtained by the sigmoid activation function is used as the value of the label bit of the amplified label. The number of label bits corresponds to the number of sound source categories that need to be classified.

[0060] A lightweight neural network model is used to identify and judge the first sound source category and the second sound source category. If the first sound source category is misclassified as the second sound source category, or the second sound source category is misclassified as the first sound source category, the degree of loss of judgment error is calculated based on the difference of label bits. The difference of label bits is proportional to the degree of loss of judgment error. The correlation between the first sound source category and the second sound source category is established based on the degree of loss.

[0061] The sound source recognition module is used to identify different sound sources using a trained lightweight neural network model.

[0062] The present invention also provides a cloud device, which includes a server, and the deep learning-based voice recognition device described in the present invention is installed on the server.

[0063] The present invention also provides a computer device, including a memory storing program instructions, which, when executed, perform the deep learning-based voice recognition method as described in the present invention.

[0064] The following detailed description is based on a specific embodiment provided by the present invention.

[0065] The specific steps are as follows:

[0066] Creating lightweight neural network models:

[0067] (1) In the first layer of the neural network, the preprocessed Mel spectrum is fed into BatchNorm for batch normalization. This operation normalizes the Mel spectrum to a distribution with a mean of 0 and a variance of 1, which helps to avoid excessive changes in the value range between network layers and alleviates the gradient explosion and vanishing problems during the optimization process. In addition, the Mel spectrum may be affected by factors such as volume differences and noise between different audio sources. Using Batch Normalization can make the network more robust to these differences.

[0068] The data preprocessing includes: reading audio data and normalizing the corresponding data; performing STFT (Short-time Fourier Transform) on the audio to obtain the STFT matrix; obtaining the weight matrix of the Mel filter bank according to the sampling rate and the set fast Fourier audio data window size; and then calculating the Mel spectrogram.

[0069] (2) Continuous use of grouped depthwise separable convolutions for feature extraction of the Mel spectrum. In this invention, depthwise separable convolutions are decomposed into depthwise convolutions and pointwise convolutions. The kernel size of the depthwise convolution part is set to 5*5 to ensure that the receptive field of the convolution kernel can cover the features of the Mel spectrum in local continuous transformations, adapting to the characteristics of the Mel spectrum. At the same time, grouped convolutions are added to the depthwise convolutions, and the number of groups is set to 1 / 4 of the input channels. This reduces the number of convolution parameters while ensuring that there are enough channels for information exchange.

[0070] (3) A fully connected layer is used as the feature layer of the model to compress the feature maps extracted by convolution into feature vectors. The softmax cross-entropy loss function is then applied.

[0071] (4) In the second stage of training the lightweight neural network model, while maintaining the above network structure, it is necessary to create another classification head for the neural network to perform refined category constraint training. Therefore, in addition to the above model structure, the feature layer needs to be connected to a fully connected layer with n categories, where n is the number of categories and is an integer greater than or equal to 1. After this fully connected layer, each neuron is connected to a sigmoid binary cross-entropy loss function.

[0072] Training a lightweight neural network model:

[0073] The model training is divided into two stages. The first stage is similar to the conventional classification of images by neural networks, that is, the model weights are updated through backpropagation using the softmax cross-entropy loss function, thereby achieving the optimization effect.

[0074] Phase Two: Based on the model constructed above, a sigmoid-based classification head is added to the original model structure. New labels are then added to the images in the training set (after the audio data was transformed into Mel-spectrum data, it has become image data). The rules for adding labels are as follows:

[0075] Suppose we need to classify four sound source categories: dog barking, baby crying, car horns, and bird calls. After the first stage of training, we find that dog barking and baby crying are easily confused into one category, while car horns and bird calls are easily confused into another (one category is misclassified as another). Because we can obtain a value between 0 and 1 after the sigmoid activation function (n sigmoids result in n such values), we set the labels as dog barking 1000, car horns 1100, bird calls 1110, and baby crying 1111.

[0076] The formula for the sigmoid activation function is: x represents the feature value extracted by the model.

[0077] The principle behind this design is that when using the softmax cross-entropy loss function for model training, there's no way to establish the loss correlation between labels. That is, if the model identifies a dog barking as a baby crying or a car horn, the loss is the same. However, because in some cases the features of a dog barking are more similar to those of a baby crying, the softmax cross-entropy loss function assigns the same loss value to both results. While a dog barking can be well distinguished from a car horn, it cannot be well distinguished from a baby crying. Therefore, at this stage, this invention, based on the characteristics of the sigmoid cross-entropy loss function, differentiates the losses for similar categories. For example, if the model incorrectly identifies a dog barking as a baby crying—that is, the label 1000 is assigned as 1111—the latter differs from the former in all three label positions, resulting in an error loss of 3. Following this method, if the model identifies a dog barking as a bird call, the loss is 2; if the model identifies a dog barking as a car horn, the loss is 1. At this point, this method establishes the correlation between dog barks and sounds like baby crying, bird calls, and car horns. Since the model has completed the first stage of training, it can theoretically distinguish between categories with very different features, such as dog barks and car horns. Therefore, the newly added loss head can be used to further constrain categories that are prone to misclassification.

[0078] However, the loss calculation in this invention also has some inappropriate aspects. For example, it also measures the differences between categories that are not considered, and this measurement itself may be inappropriate. For instance, when the labels are set as described above, the loss for misjudging a baby crying and a bird call is 1, while the loss for a baby crying and a car horn is 2. Such a setting may not meet people's cognitive or attentional needs (because baby crying and bird calls, as well as baby crying and car horns, are not easily confused and are not the focus of differentiation). Therefore, it is necessary to reduce or uniformly set the loss for label relationships that are not of interest or have low relevance to 1. That is, if the real label or the predicted label contains a dog bark, the loss is calculated according to the labels described above. In other cases, the loss is normalized at this time, that is, the calculated loss is divided by the number of label differences, and this value is used for backpropagation and weight update. The formula below shows the gradient calculation method during backpropagation. The formula in the first row represents the categories other than dog barking, and m represents the degree of difference between the label and the prediction (the degree of difference is normalized); the second row represents the categories in the prediction or real label that contain dog barking, and the degree of difference is considered in the loss:

[0079]

[0080]

[0081] m represents the degree of difference between the label and the prediction, which is normalized. Lsigmoidn represents the nth binary cross-entropy loss. f1 represents the category other than the first sound source category. f2 represents the category other than the first sound source category in the predicted label or the amplified label. h represents the result of the model output after passing through the activation function before entering the loss function. L is the L2 loss.

[0082] The cross-entropy loss for binary classification is calculated as follows:

[0083] y i For amplified true labels, For predicted labels.

[0084] because Since f1 is a probability value, when choosing between f1 and f2 formulas as described above, a threshold, such as 0.5, needs to be defined. If the value obtained by the sigmoid activation function for a dog bark is greater than 0.5, while the values ​​obtained by the sigmoid activation function for other sound sources are less than 0.5, then the backpropagation value is calculated using f1.

[0085] Therefore, in the second stage, the total loss will be expressed as L 总 =L1+L2, where L1 is the loss obtained from the softmax classification head, and L2 is the loss obtained by measuring the classification difference.

[0086] L2=Lsigmoid1+Lsigmoid2+Lsigmoid3+...Lsigmoidn

[0087] Where Lsigmoidn is the sigmoid cross-entropy loss established between each predicted value and each amplified tag.

[0088] After the two-stage training process described above, most of the easily confused samples will be further distinguished, thereby improving the accuracy of the model in sound recognition.

[0089] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A deep learning-based voice recognition method, characterized in that, include: Creating a lightweight neural network model: In the first layer of the neural network model, the preprocessed Mel spectrum is fed into BatchNorm for batch normalization; grouped depthwise separable convolutions are used consecutively to extract features from the Mel spectrum; a fully connected layer is connected as the feature layer of the model, and the feature maps extracted by the convolutions are compressed into feature vectors and connected to the softmax cross-entropy loss function; the feature layer is connected to a fully connected layer with multiple categories, and after this fully connected layer, each neuron is connected to a sigmoid binary cross-entropy loss function; Training the lightweight neural network model: Training is divided into two stages. The first stage updates the weights of the lightweight neural network model through backpropagation using the softmax cross-entropy loss function. The second stage introduces a sigmoid classifier to augment the labels based on the lightweight neural network model, as detailed below: New labels are amplified on images in the training set using a sigmoid-based classification head. The rule for amplifying the labels is as follows: when classifying at least the first sound source category and the second sound source category, the value obtained by the corresponding sigmoid activation function is used as the label bit value of the amplified label for the first sound source category and the second sound source category. The number of label bits is equal to the number of sound source categories to be classified. A lightweight neural network model is used to identify and judge the first sound source category and the second sound source category. If the first sound source category is misclassified as the second sound source category, or the second sound source category is misclassified as the first sound source category, the degree of loss of judgment error is calculated based on the difference of label bits. The difference of label bits is proportional to the degree of loss of judgment error. The correlation between the first sound source category and the second sound source category is established based on the degree of loss. Different sound source categories are identified using a trained lightweight neural network model.

2. The deep learning-based voice recognition method according to claim 1, characterized in that, Training lightweight neural network models also includes: The correlation between the first sound source category and the second sound source category is judged. If the correlation between the first sound source category and the second sound source category is lower than the set threshold, the amplification labels of the first sound source category and the second sound source category are changed to reduce the degree of misjudgment loss between the first sound source category and the second sound source category.

3. The deep learning-based voice recognition method according to claim 1, characterized in that, Training lightweight neural network models also includes: When classifying the first, second, and third sound source categories, the correlation between these categories is assessed. If the correlation between the first and second sound source categories is greater than the correlation between the second and third sound source categories, or between the first and third sound source categories, then when the amplified or predicted label includes the first sound source category, the loss levels of the second and third sound source categories are normalized. Normalization specifically involves dividing the calculated loss level by the number of label differences, performing inverse differentiation on the resulting value, and updating the weights. The inverse calculation method is as follows: m represents the degree of difference between the label and the prediction, and the degree of difference is normalized. Lsigmoidn represents the nth binary cross-entropy loss. f1 represents the category other than the first sound source category. f2 represents the category other than the first sound source category in the predicted label or the amplified label. h represents the result of the model output after passing through the activation function before entering the loss function. L represents the loss obtained by measuring the classification difference. The cross-entropy loss for binary classification is calculated as follows: , For amplified true labels, For predicted labels.

4. The deep learning-based voice recognition method according to claim 3, characterized in that, During backpropagation, if the value obtained by the sigmoid activation function for the first sound source category is greater than the set threshold, and the values ​​obtained by the sigmoid activation function for other sound sources are less than the set threshold, then the backpropagation value is calculated using f1.

5. A deep learning-based voice recognition device, used to implement the deep learning-based voice recognition method as described in any one of claims 1-4, characterized in that, The device includes: The model creation module is used to create lightweight neural network models. Specifically, in the first layer of the neural network model, the preprocessed Mel spectrum is fed into BatchNorm for batch normalization; grouped depthwise separable convolutions are used consecutively to extract features from the Mel spectrum; a fully connected layer is connected as the model's feature layer, the feature maps extracted by the convolutions are compressed into feature vectors, and then the softmax cross-entropy loss function is applied; the feature layer is connected to a fully connected layer with multiple categories, and after this fully connected layer, each neuron is connected to a sigmoid binary cross-entropy loss function. The model training module is used to train the lightweight neural network model. The training is divided into two stages. In the first stage, the weights of the lightweight neural network model are updated through backpropagation using the softmax cross-entropy loss function. In the second stage, a sigmoid classifier is introduced to augment the labels based on the lightweight neural network model, as detailed below: New labels are amplified on images in the training set using a sigmoid-based classification head. The rule for amplifying the labels is as follows: when classifying at least the first sound source category and the second sound source category, the value obtained by the corresponding sigmoid activation function is used as the label bit value of the amplified label for the first sound source category and the second sound source category. The number of label bits is equal to the number of sound source categories to be classified. A lightweight neural network model is used to identify and judge the first sound source category and the second sound source category. If the first sound source category is misclassified as the second sound source category, or the second sound source category is misclassified as the first sound source category, the degree of loss of judgment error is calculated based on the difference of label bits. The difference of label bits is proportional to the degree of loss of judgment error. The correlation between the first sound source category and the second sound source category is established based on the degree of loss. The sound source recognition module is used to identify different sound sources using a trained lightweight neural network model.

6. The deep learning-based voice recognition device according to claim 5, characterized in that, The model training module is specifically used to determine the correlation between the first sound source category and the second sound source category. If the correlation between the first sound source category and the second sound source category is lower than the set threshold, the amplified labels of the first sound source category and the second sound source category are changed to reduce the degree of misjudgment loss between the first sound source category and the second sound source category.

7. The deep learning-based voice recognition device according to claim 5, characterized in that, The model training module is also specifically used to determine the correlation between the first, second, and third sound source categories when classifying them. If the correlation between the first and second sound source categories is greater than the correlation between the second and third sound source categories, and the amplified or predicted label contains the first sound source category, the loss of the second and third sound source categories is normalized. Normalization specifically involves dividing the calculated loss by the number of label differences, taking the inverse derivative of the obtained value, and updating the weights. The inverse calculation method is as follows: m represents the degree of difference between the label and the prediction, and the degree of difference is normalized. Lsigmoidn represents the nth binary cross-entropy loss. f1 represents the category other than the first sound source category. f2 represents the category other than the first sound source category in the predicted label or the amplified label. The cross-entropy loss for binary classification is calculated as follows: , For amplified true labels, For predicted labels.

8. The deep learning-based voice recognition device according to claim 7, characterized in that, The model training module is also used to calculate the backpropagation value by using f1 if the value obtained by the first sound source category through the sigmoid activation function is greater than the set threshold, and the values ​​obtained by other sound sources through the sigmoid activation function are less than the set threshold.

9. A cloud device, the cloud device comprising a server, characterized in that, The deep learning-based voice recognition device as described in any one of claims 5-8 is installed on the server.

10. A computer device comprising a memory storing program instructions, characterized in that, When the program instructions are executed, the deep learning-based sound recognition method as described in any one of claims 1-4 is performed.

Images