Noise identification method, device, electronic equipment and medium

Abstract

The present application discloses a method, device, electronic device and medium for noise recognition. In the present application, after obtaining the speech data to be recognized, the speech data to be recognized is input into a target soft threshold residual network having a soft threshold function with a first slope and a soft threshold function with a second slope to obtain a noise recognition result, and based on the noise recognition result, the noise data in the speech data to be recognized is determined. By applying the technical solution of the present application, the purpose of more comprehensive filtering of noise signals in speech can be achieved by adopting two soft threshold residual networks with different slopes. This avoids the problem of unclear denoising effect caused by the deep residual shrinkage network with only a single slope in the related art.

Description

Technical Field

[0001] The present application relates to data processing technology, and in particular to a method, device, electronic device and medium for noise recognition. Background Art

[0002] With the development of science and technology, the proportion of high-quality products in the production of the whole country is increasing. Whether it is a production product or a daily life product, as the product is used more and more, it will inevitably wear out. Therefore, an accurate fault diagnosis system directly determines the quality of the product.

[0003] Furthermore, in the related art, judging the operating sound of many products is an important method for diagnosing their faults. Currently, there are methods in the related art that often use soft thresholding for denoising. Among them, soft thresholding (Soft Threshlding) is a classic method, especially very practical in the field of signal noise reduction. The natural nonlinear properties of soft thresholding are very suitable for use in the calculation and transmission process of deep neural networks. Therefore, the deep residual shrinkage network (Residual Shrinkage Network) has also proven its practicality in the field of signal noise reduction.

[0004] However, the deep residual shrinkage network in the related art only has a soft threshold parameter of a slope, which also leads to the phenomenon that its denoising effect is not obvious. Summary of the Invention

[0005] The embodiments of the present application provide a method, device, electronic device and medium for noise recognition. The embodiments of the present application are used to solve the problem in the related art that the deep residual shrinkage network has only a soft threshold parameter with a slope, resulting in an unclear denoising effect.

[0006] According to one aspect of an embodiment of the present application, a noise recognition method is provided, comprising:

[0007] Obtaining voice data to be recognized;

[0008] Inputting the to-be-recognized speech data into a target soft threshold residual network to obtain a noise recognition result, wherein the target soft threshold residual network has a soft threshold function with a first slope and a soft threshold function with a second slope, and the first slope is different from the second slope;

[0009] According to the noise recognition result, the noise data in the speech data to be recognized is determined

[0010] Optionally, in another embodiment of the method of the present application, before obtaining the voice data to be recognized, the method further includes:

[0011] Obtain a first input feature, perform at least two convolution operations on the first input feature, perform a batch normalization operation, and perform a linear rectification function operation on the first input feature to obtain a first output result;

[0012] Performing an absolute value algorithm and a global average pooling operation on the first output result to obtain a second output result;

[0013] The first slope and the second slope are obtained based on the second output result.

[0014] Optionally, in another embodiment of the method of the present application, obtaining the first slope and the second slope based on the second output result includes:

[0015] Performing a convolution operation on the second output result, a batch normalization operation, a full connection operation, and a linear rectification function operation on the second output result to obtain a third output result;

[0016] Perform sigmoid function conversion on the third output result to obtain a target soft threshold function.

[0017] Optionally, in another embodiment of the above method of the present application, after obtaining the first output result, the method further includes:

[0018] Performing a first absolute value algorithm on the first output result and a global average pooling operation on the first output result to obtain a fourth output result;

[0019] A full connection operation is performed on the fourth output result, a batch normalization operation is performed, a linear rectification function operation is performed, and a full connection operation is performed to obtain the first slope value.

[0020] Optionally, in another embodiment of the above method of the present application, after obtaining the first slope value, the method further includes:

[0021] Performing a second absolute value algorithm on the first output result and a global average pooling operation on the first output result to obtain a fifth output result;

[0022] A full connection operation is performed on the fifth output result, a batch normalization operation is performed, a linear rectification function operation is performed, and a full connection operation is performed to obtain the second slope value.

[0023] Optionally, in another embodiment of the above method of the present application, after obtaining the second slope value, the method further includes:

[0024] generating a soft threshold function having the first slope and a soft threshold function having the second slope according to the first slope value, the second slope value, and the target soft threshold function;

[0025] The target soft threshold residual network is constructed according to the soft threshold function with the first slope and the soft threshold function with the second slope.

[0026] Optionally, in another embodiment of the above method of the present application, inputting the speech data to be recognized into a target soft threshold residual network to obtain a noise recognition result includes:

[0027] Acquire the speech data to be recognized, perform a convolution operation on the speech data to be recognized, extract semantic information from the speech data to be recognized, and obtain an output result of a first dimension;

[0028] Using the soft threshold function of the first slope and the soft threshold function of the second slope, removing noise redundancy of the output result of the first dimension;

[0029] Performing a batch normalization operation on the output result of the first dimension to remove noise redundancy, a global average pooling operation, and a linear rectification function operation to obtain an output result of the second dimension;

[0030] The noise recognition result is obtained based on the output result of the second dimension.

[0031] Optionally, in another embodiment of the above method of the present application, obtaining the noise recognition result based on the output result of the second dimension includes:

[0032] Performing a conversion operation to convert the output result of the second dimension into an N-dimensional vector;

[0033] A full connection operation and a softmax function operation are performed on the N-dimensional vector to obtain the noise recognition result.

[0034] According to another aspect of an embodiment of the present application, a noise recognition device is provided, comprising:

[0035] An acquisition module is configured to acquire speech data to be recognized;

[0036] a generating module configured to input the speech data to be recognized into a target soft threshold residual network to obtain a noise recognition result, wherein the target soft threshold residual network has a soft threshold function with a first slope and a soft threshold function with a second slope, and the first slope is different from the second slope;

[0037] The determination module is configured to determine the noise data in the speech data to be recognized according to the noise recognition result.

[0038] According to another aspect of the embodiments of the present application, an electronic device is provided, including:

[0039] a memory for storing executable instructions; and

[0040] A display, configured to display with the memory to execute the executable instructions to complete the operation of any of the above-mentioned noise recognition methods.

[0041] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided for storing computer-readable instructions, wherein the instructions, when executed, perform the operations of any of the above-mentioned noise recognition methods.

[0042] In this application, after obtaining the speech data to be recognized, the speech data to be recognized is input into a target soft threshold residual network having a soft threshold function with a first slope and a soft threshold function with a second slope to obtain a noise recognition result. Based on the noise recognition result, the noise data in the speech data to be recognized is determined. By applying the technical solution of this application, the purpose of more comprehensive filtering of noise signals in speech can be achieved by using two soft threshold residual networks with different slopes. This avoids the problem of poor denoising effect caused by deep residual shrinkage networks with only a single slope in the related art.

[0043] The technical solution of the present application is further described in detail below through the accompanying drawings and examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and, together with the description, serve to explain the principles of the present application.

[0045] The present application can be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0046] Figure 1 A schematic diagram of a noise identification method proposed in this application;

[0047] Figure 2-Figure 8 Schematic diagram of the soft threshold residual network based on the same slope generation proposed in this application;

[0048] Figures 9-11 Schematic diagram of the soft threshold residual network based on different slope generation proposed in this application;

[0049] Figure 12 A schematic diagram of the structure of the noise identification device of this application;

[0050] Figure 13 Schematic diagram of the electronic device structure for noise recognition in this application. DETAILED DESCRIPTION

[0051] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangements of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present application.

[0052] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0053] The following description of at least one exemplary embodiment is merely illustrative in nature and is not intended to limit the present disclosure, its application, or uses.

[0054] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0055] It should be noted that like reference numerals and letters refer to like items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0056] In addition, the technical solutions between the various embodiments of the present application can be combined with each other, but it must be based on the fact that ordinary technicians in this field can implement it. When the combination of technical solutions is mutually contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0057] It should be noted that all directional indications in the embodiments of the present application (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0058] In one embodiment, the present application also proposes a method, device, target terminal and medium for noise identification.

[0059] Figure 1 The following schematically shows a flow chart of a noise identification method according to an embodiment of the present application. Figure 1 As shown, the method includes:

[0060] S101: Acquire speech data to be recognized.

[0061] Furthermore, in the related art, in the deep residual network (ResNet), the rectified linear unit (ReLU) is a common, universal and effective nonlinear activation function, and its mathematical expression is

[0062]

[0063] Among them, x and y are real numbers, representing input and output respectively; its derivative is

[0064]

[0065] Similarly, the above x and y are also real numbers, representing input and output respectively. Further, the function shape of Relu and its derivative shape are as follows Figure 2 As shown. The corresponding deep residual shrinkage network (Residual Shrinkage Network), its activation function is the soft thresholding function, mathematically expressed as

[0066]

[0067] In the above expression, τ is the threshold, x and y are real numbers, representing input and output respectively. The shape of the function is as follows: Figure 3 Its derivative is

[0068]

[0069] Among them, τ is the threshold, and the shape of the derivative is as follows Figure 4 Furthermore, since the soft thresholding function is a function whose value is 0 within the threshold range and maintains a slope of 1 outside the threshold, similar to ReLU, it can suppress noise interference within the threshold range and eliminate easy features.

[0070] Furthermore, soft thresholding, that is, a soft thresholding function, is a function that shrinks the input data toward zero.

[0071] Furthermore, for the design of the attention depth residual shrinkage network with the same slope, the design of the entire network is a continuous stacking of soft thresholding modules with the same slope based on the attention mechanism (Attention Same Slope Unit), abbreviated as ASSU, such as Figure 5-Figure 6 As shown. It is a deep residual contraction network with the same slope, and its generated architecture is as follows:

[0072] In step 1, the application can first obtain the input features as CxWx1. It should be noted that in this feature, C corresponds to the output channel, W refers to the width of the feature, and 1 refers to the height of the feature, because the feature height in a noisy signal is 1. Because different deep learning development frameworks may not arrange the channels in the same position, some frameworks may be Wx1xC. In one approach, the application can place the channel in the first dimension and save this input separately for the final residual addition.

[0073] In step 2, the present application may perform a CBN operation on the features of CxWx1, where CBN refers to the abbreviation of the convolution (Con) + batch normalization operation BN (Batch Normalization) + Relu operation.

[0074] Step 3: Perform another CBN operation on the above output to extract more advanced semantic information.

[0075] In step 4, an attention mechanism is used to automatically derive the threshold of the output of step 2. This part first uses absolute value + global average pooling (GAP) to obtain the output.

[0076] In step 5, the application can perform full connection (FC) + batch normalization operation BN (Batch Normalization) + Relu + full connection (FC) operations on the output obtained in step 3, and then perform sigmoid function to obtain the output of the threshold τ.

[0077] In step 6, an attention mechanism with the same slope is automatically derived for the output of step 3. It should be noted that this part first uses absolute value + global average pooling (GAP) to obtain the output.

[0078] In step seven, the present application can further perform full connection (FC) + batch normalization operation BN (Batch Normalization) + Relu + full connection (FC) operations on the output of step six, thereby directly obtaining the output of the slope value α.

[0079] In step 8, the present application can construct a soft thresholding function with the same slope based on the threshold τ and slope value α obtained in steps 5 and 7, respectively. The output of the soft thresholding function with the same slope is then added to the original input, i.e., the saved input in step 1, and the result of the addition is used as the output. This completes the design description of a complete soft thresholding module with the same slope (ASSU).

[0080] In addition, the present application can bring the slope α automatically derived by the network into the soft thresholding function to replace the original soft thresholding function with a slope of 1. The expression of this new function is

[0081]

[0082] Among them, τ is the threshold, x and y are real numbers, representing input and output respectively, α is the slope, and the shape of the function is as follows Figure 7 The corresponding mathematical expression of the derivative is

[0083]

[0084] Here τ is the threshold, x and y are real numbers, representing input and output respectively, and α is the slope. The shape of the function is as follows: Figure 8 shown.

[0085] Furthermore, after obtaining the soft threshold function with a fixed slope, the present application can also generate the corresponding attention depth residual shrinkage network based on it. The specific steps are as follows:

[0086] In step 1, the input is the original noisy signal with a dimension of CxWx1. First, a convolution operation is performed to extract the semantic information, and the dimension is still CxWx1.

[0087] In step 2, the image is passed through several soft thresholding modules (ASSU) with the same slope mentioned above to remove noise redundancy.

[0088] Step 3: After a batch normalization (BN) + Relu + global average pooling (GAP) operation, the output is converted to an Nx1x1 dimension, where N is a hyperparameter and the channel dimension.

[0089] Step 4: Then, through the reshape operation, the Nx1x1 feature is transformed into a vector of length N. The N-dimensional vector is fully connected (FC) and then softmax is performed to directly output the noise recognition result.

[0090] Furthermore, for a deep residual shrinkage network with the same slope, after training, the network can accurately output fault diagnosis in noisy environments. Because the network design can automatically derive the same slope based on different noise signals, it also has higher accuracy.

[0091] However, since the slope of the deep residual shrinkage network combined with the residual network (ResNet) is fixed to 1, it cannot guarantee that the network can effectively respond to changes in external noise. In order to improve the noise interference resistance and detection accuracy of the entire network.

[0092] To address the above problem, the present application can adopt a network structure with different slopes to allow the network to automatically derive multiple slope values, thereby ensuring that the network can dynamically and in real time calculate the slope value under the current noise, thereby effectively suppressing the interference of noise.

[0093] S102: Input the speech data to be recognized into a target soft threshold residual network to obtain a noise recognition result, wherein the target soft threshold residual network has a soft threshold function with a first slope and a soft threshold function with a second slope, and the first slope is different from the second slope.

[0094] Furthermore, with the development of science and technology, the proportion of mechanization and productization in the country's production is increasing, whether it is for industrial products or daily life products. High-quality products must rely on high-quality production processes, which will inevitably drive the continuous improvement of mechanical production processes. Whether it is production tools or daily life products, as the use of products increases, the products will inevitably wear out. Therefore, accurate fault diagnosis systems directly determine the quality of products.

[0095] For example, rotating bearings account for a significant proportion of the components used in many mechanical manufacturing applications and in everyday products. Bearing wear is a common phenomenon in both manufacturing and everyday household appliances. Interpreting the sound of rotating bearings is a key method for fault diagnosis. However, bearing rotation, whether in production or everyday life, is often accompanied by a significant amount of noise and redundant signals. Therefore, simply judging the extent of damage based on sound alone can lead to significant errors.

[0096] Currently, traditional methods use statistical learning to analyze sound signal noise. However, these methods often require many parameters to be set manually. Setting these parameters is inherently complex and often requires extensive statistical experimentation. However, different environments can affect these parameters, such as the internal and external structure of the machine, the material of the bearings, and even the ambient temperature and humidity. These hyperparameters can affect the accuracy of these parameters, making them difficult to adapt uniformly. Soft thresholding, a classic method, is particularly useful in signal noise reduction. However, as previously mentioned, the threshold in the soft thresholding function is a hyperparameter, and setting a reasonable value is a challenging task.

[0097] The recent proliferation of the Internet of Things, big data, and mobile devices, particularly the explosive development of deep learning, has made intelligent detection and recognition technologies based on deep learning possible. Unlike traditional methods, deep learning approaches can automatically learn the parameter characteristics of perturbed signals and derive the correct and appropriate parameters, making them extremely practical and valuable. Furthermore, the inherently nonlinear properties of soft thresholding are well-suited for use in the computation and transmission processes of deep neural networks. The Deep Residual Network (ResNet), a classic deep learning network, has been successfully applied in many fields. A network combining a deep residual network with a nonlinear soft thresholding function, namely the Deep Residual Shrinkage Network (RSN), has also demonstrated its practical application in signal noise reduction. The RSN utilizes an attention mechanism (similar to the Squeeze-and-Excitation Network) to automatically set thresholds, eliminating the need for manual thresholding.

[0098] Therefore, since the fixed slope mentioned above cannot be adjusted according to the strength of the noise signal. Therefore, in the embodiment of the present application, the speech data to be recognized can be recognized based on the pre-trained target soft threshold residual network with a soft threshold function of a first slope and a soft threshold function of a second slope, thereby obtaining the corresponding noise recognition result. It can be understood that the target soft threshold residual network using soft threshold functions of two slopes can dynamically calculate the gradient of the slope by the network, which can greatly improve the response of the entire network model to the external noise signal, thereby achieving the purpose of improving the accuracy and precision of fault diagnosis.

[0099] S103: Determine noise data in the speech data to be recognized according to the noise recognition result.

[0100] In this application, after obtaining the speech data to be recognized, the speech data to be recognized is input into a target soft threshold residual network having a soft threshold function with a first slope and a soft threshold function with a second slope to obtain a noise recognition result. Based on the noise recognition result, the noise data in the speech data to be recognized is determined. By applying the technical solution of this application, the purpose of more comprehensive filtering of noise signals in speech can be achieved by using two soft threshold residual networks with different slopes. This avoids the problem of poor denoising effect caused by deep residual shrinkage networks with only a single slope in the related art.

[0101] Optionally, in a possible implementation manner of the present application, before S101 (obtaining the voice data to be recognized), the following steps are further included:

[0102] Obtain a first input feature, perform at least two convolution operations on the first input feature, perform a batch normalization operation, and perform a linear rectification function operation to obtain a first output result;

[0103] Performing an absolute value algorithm on the first output result and performing a global average pooling operation to obtain a second output result;

[0104] Based on the second output result, a first slope and a second slope are obtained.

[0105] The first input feature may be CxWx1, where C refers to the output channel, W refers to the width of the feature, and 1 corresponds to the height of the feature, because the feature height in the noise signal is 1.

[0106] Furthermore, the present application can also perform a CBN operation on the features of CxWx1, where the CBN refers to the linear rectification function operation of the convolution operation (Con) + batch normalization operation BN (Batch Normalization) + Relu, thereby obtaining the corresponding first output result.

[0107] Optionally, the present application may also re-perform the above-mentioned convolution operation (Con) + batch normalization operation BN (Batch Normalization) + Relu linear rectification function operation on the first output result, thereby achieving the purpose of extracting higher-level semantic information.

[0108] Next, after obtaining the first output, an attention mechanism with automatic threshold derivation can be applied to the output. Specifically, the absolute value algorithm and global average pooling operations can be performed on the output to obtain the corresponding second output.

[0109] Optionally, in a possible implementation of the present application, obtaining the first slope and the second slope based on the second output result includes:

[0110] Performing a convolution operation on the second output result, a batch normalization operation, a full connection operation, and a linear rectification function operation on the second output result to obtain a third output result;

[0111] The third output result is subjected to sigmoid function to obtain the target soft threshold function.

[0112] Furthermore, after obtaining the corresponding second output result, the present application can further perform a full connection (FC) + batch normalization operation BN (Batch Normalization) + Relu linear rectification function operation + full connection (FC) operation on the output result to obtain a corresponding third output result. The third output result can then be further subjected to a sigmoid function operation to obtain the output of the target soft threshold function τ.

[0113] Optionally, in a possible implementation manner of the present application, after obtaining the first output result, the method further includes:

[0114] Performing a first absolute value algorithm on the first output result and a global average pooling operation on the first output result to obtain a fourth output result;

[0115] A full connection operation is performed on the fourth output result, a batch normalization operation is performed, a linear rectification function operation is performed, and a full connection operation is performed to obtain a first slope value.

[0116] Furthermore, after obtaining the target soft threshold function τ, the present application can also perform an attention mechanism for automatically deriving the same slope on the first output result. Specifically, this part can obtain the fourth output result by performing an absolute value algorithm + a global average pooling (GAP) operation. And the fourth output result is subjected to a full connection (FC) operation + a batch normalization operation BN (Batch Normalization) + a Relu linear rectification function operation + a full connection (FC) operation to directly obtain the output result of the slope value α (i.e., the first slope value).

[0117] Optionally, in a possible implementation manner of the present application, after obtaining the first slope value, the method further includes:

[0118] Performing a second absolute value algorithm on the first output result and performing a global average pooling operation on the first output result to obtain a fifth output result;

[0119] A full connection operation is performed on the fifth output result, a batch normalization operation is performed, a linear rectification function operation is performed, and a full connection operation is performed to obtain a second slope value.

[0120] Similarly, the present application can also perform the same slope automatic derivation attention mechanism on the first output result again. Specifically, this part can obtain the third output result by performing the absolute value algorithm + global average pooling (GAP) operation. And perform the full connection (FC) operation + batch normalization operation BN (BatchNormalization) + Relu execution linear rectification function operation + full connection (FC) operation on the third output result to directly obtain the output result of the slope value b (i.e., the second slope value).

[0121] Optionally, in a possible implementation manner of the present application, after obtaining the second slope value, the method further includes:

[0122] generating a soft threshold function having a first slope and a soft threshold function having a second slope according to the first slope value, the second slope value, and the target soft threshold function;

[0123] A target soft threshold residual network is constructed according to the soft threshold function with the first slope and the soft threshold function with the second slope.

[0124] Furthermore, after obtaining the first slope value α, the second slope value b and the target soft threshold function τ, the present application can construct the soft threshold function of the first slope and the soft threshold function of the second slope. Subsequently, the output results obtained by the two soft threshold functions with different slopes are added together, and the result of the addition is used as the output result of the target soft threshold residual network (i.e., the noise recognition result). For example, Figure 9 As shown in Figure 1, the design description of a complete soft thresholding module (ADSU) with different slopes is completed. The slopes α and b automatically derived by the network are respectively introduced into the soft thresholding function to replace the original soft thresholding function with a slope of 1. The expression of this new function is

[0125]

[0126] Similarly, τ is the threshold, x and y are real numbers, representing input and output respectively, α and b are slopes, and the shape of the function is as follows Figure 10 The corresponding mathematical expression of the derivative is

[0127]

[0128] Similarly, τ is the threshold, x and y are real numbers, representing input and output respectively, α and b are slopes, and the shape of the function is as follows Figure 11 shown.

[0129] Optionally, in a possible implementation of the present application, inputting the speech data to be recognized into a target soft threshold residual network to obtain a noise recognition result includes:

[0130] Acquire the speech data to be recognized, perform a convolution operation on the speech data to be recognized, extract semantic information from the speech data to be recognized, and obtain an output result of the first dimension;

[0131] Using a soft threshold function with a first slope and a soft threshold function with a second slope, removing noise redundancy from an output result of the first dimension;

[0132] Performing a batch normalization operation on the output result of the first dimension to remove noise redundancy, a global average pooling operation, and a linear rectification function operation to obtain an output result of the second dimension;

[0133] Based on the output knot of the second dimension, the noise recognition result is obtained.

[0134] Optionally, in a possible implementation of the present application, obtaining a noise recognition result based on the output result of the second dimension includes:

[0135] Perform a conversion operation to convert the output result of the second dimension into an N-dimensional vector;

[0136] Perform a full connection operation on the N-dimensional vector and a softmax function operation to obtain the noise recognition result.

[0137] Furthermore, the present application can obtain the speech data to be recognized, and perform a convolution operation on the speech data to be recognized, extract the semantic information in the speech data to be recognized, and obtain an output result of the first dimension. For example, the input is the original noisy signal with a dimension of CxWx1. First, a convolution operation is performed to extract the semantic information, and the dimension is still CxWx1. Then, several soft thresholding modules (ADSU) with different slopes mentioned above are used to remove noise redundancy.

[0138] Furthermore, the present application can transform the output of the first dimension of the noise-reduced image into an Nx1x1 dimension, where N is a hyperparameter representing the channel dimension, by performing batch normalization (BN) + ReLU + global average pooling (GAP) operations on the first dimension to remove noise and redundancy. The output of the second dimension is obtained.

[0139] Furthermore, the present application can also reshape the output result of the second dimension to convert the Nx1x1 feature into a vector of length N, perform a full connection (FC) operation on the N-dimensional vector, and then perform a softmax operation to directly output the final noise recognition result.

[0140] In this application, after obtaining the speech data to be recognized, the speech data to be recognized is input into a target soft threshold residual network having a soft threshold function with a first slope and a soft threshold function with a second slope to obtain a noise recognition result. Based on the noise recognition result, the noise data in the speech data to be recognized is determined. By applying the technical solution of this application, the purpose of more comprehensive filtering of noise signals in speech can be achieved by using two soft threshold residual networks with different slopes. This avoids the problem of poor denoising effect caused by deep residual shrinkage networks with only a single slope in the related art.

[0141] In one embodiment, the noise recognition method in the present application can be applied to household appliances. For example, it can be deployed on a smart speaker. When the smart speaker detects that there is currently user-generated command voice data, the command voice data can be input into the soft threshold residual network based on two different slopes deployed on itself, thereby obtaining the noise recognition result corresponding to the command voice data. It can be understood that the noise recognition result may include background noise, ambient noise, white noise, etc.

[0142] Furthermore, during the process of recognizing command voice data, the present application can perform a convolution operation on the voice data to be recognized and extract semantic information from the voice data to be recognized, thereby obtaining an output result of the first dimension. Furthermore, a soft threshold function with a first slope and a soft threshold function with a second slope can be used to remove noise redundancy from the output result of the first dimension, thereby obtaining preliminary processed data.

[0143] Furthermore, the smart speaker may also perform a batch normalization operation, a global average pooling operation, and a linear rectification function operation on the output result of the first dimension of the preliminary processed data to obtain an output result of the second dimension, so as to subsequently obtain the noise recognition result based on the output result of the second dimension.

[0144] Furthermore, after obtaining the noise recognition result corresponding to the voice data, the smart speaker can determine the noise data contained in the command voice data based on the noise recognition result and remove it, thereby achieving a more accurate user-generated command voice.

[0145] In another embodiment of the present application, Figure 6 As shown, the present application also provides a device for noise recognition. The device includes an acquisition module 201, a generation module 202, and a determination module 203, wherein:

[0146] An acquisition module 201 is configured to acquire speech data to be recognized;

[0147] a generating module 202 configured to input the speech data to be recognized into a target soft threshold residual network to obtain a noise recognition result, wherein the target soft threshold residual network has a soft threshold function with a first slope and a soft threshold function with a second slope, and the first slope is different from the second slope;

[0148] The determination module 203 is configured to determine the noise data in the speech data to be recognized according to the noise recognition result.

[0149] In this application, after obtaining the speech data to be recognized, the speech data to be recognized is input into a target soft threshold residual network having a soft threshold function with a first slope and a soft threshold function with a second slope to obtain a noise recognition result. Based on the noise recognition result, the noise data in the speech data to be recognized is determined. By applying the technical solution of this application, the purpose of more comprehensive filtering of noise signals in speech can be achieved by using two soft threshold residual networks with different slopes. This avoids the problem of poor denoising effect caused by deep residual shrinkage networks with only a single slope in the related art.

[0150] In another embodiment of the present application, the acquisition module 201 further includes:

[0151] An acquisition module 201 is configured to acquire a first input feature, perform at least two convolution operations on the first input feature, perform a batch normalization operation, and perform a linear rectification function operation to obtain a first output result;

[0152] an acquisition module 201 configured to perform an absolute value algorithm and a global average pooling operation on the first output result to obtain a second output result;

[0153] The acquisition module 201 is configured to obtain the first slope and the second slope based on the second output result.

[0154] In another embodiment of the present application, the acquisition module 201 further includes:

[0155] An acquisition module 201 is configured to perform a convolution operation, a batch normalization operation, a full connection operation, and a linear rectification function operation on the second output result to obtain a third output result;

[0156] The acquisition module 201 is configured to perform sigmoid function on the third output result to obtain a target soft threshold function.

[0157] In another embodiment of the present application, the acquisition module 201 further includes:

[0158] an acquisition module 201 configured to perform a first absolute value algorithm and a global average pooling operation on the first output result to obtain a third output result;

[0159] The acquisition module 201 is configured to perform a full connection operation on the third output result, a batch normalization operation, a linear rectification function operation, and a full connection operation to obtain the first slope value.

[0160] In another embodiment of the present application, the acquisition module 201 further includes:

[0161] an acquisition module 201 configured to perform a second absolute value algorithm and a global average pooling operation on the first output result to obtain a third output result;

[0162] The acquisition module 201 is configured to perform a full connection operation on the third output result, a batch normalization operation, a linear rectification function operation, and a full connection operation to obtain the second slope value.

[0163] In another embodiment of the present application, the acquisition module 201 further includes:

[0164] an acquisition module 201 configured to generate a soft threshold function having the first slope and a soft threshold function having the second slope according to the first slope value, the second slope value, and the target soft threshold function;

[0165] The acquisition module 201 is configured to construct the target soft threshold residual network according to the soft threshold function with the first slope and the soft threshold function with the second slope.

[0166] In another embodiment of the present application, the acquisition module 201 further includes:

[0167] The acquisition module 201 is configured to acquire the speech data to be recognized, perform a convolution operation on the speech data to be recognized, extract semantic information from the speech data to be recognized, and obtain an output result of a first dimension;

[0168] An acquisition module 201 is configured to remove noise redundancy from an output result of the first dimension by using a soft threshold function of the first slope and a soft threshold function of the second slope;

[0169] An acquisition module 201 is configured to perform a batch normalization operation, a global average pooling operation, and a linear rectification function operation on the output result of the first dimension after removing noise and redundancy, to obtain an output result of the second dimension;

[0170] The acquisition module 201 is configured to obtain the noise recognition result based on the output result of the second dimension.

[0171] In another embodiment of the present application, the acquisition module 201 further includes:

[0172] An acquisition module 201 is configured to perform a conversion operation to convert the output result of the second dimension into an N-dimensional vector;

[0173] The acquisition module 201 is configured to perform a full connection operation and a softmax function operation on the N-dimensional vector to obtain the noise recognition result.

[0174] Figure 13 3 is a block diagram of a logical structure of an electronic device according to an exemplary embodiment. For example, the electronic device 300 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0175] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory including instructions, and the above instructions can be executed by an electronic device processor to complete the above-mentioned network monitoring method, the method comprising: obtaining voice data to be recognized; inputting the voice data to be recognized into a target soft threshold residual network to obtain a noise recognition result, wherein the target soft threshold residual network has a soft threshold function with a first slope and a soft threshold function with a second slope, and the first slope is different from the second slope; according to the noise recognition result, determining the noise data in the voice data to be recognized. Optionally, the above instructions can also be executed by the processor of the electronic device to complete the other steps involved in the above-mentioned exemplary embodiment. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a tape, a floppy disk, and an optical data storage device.

[0176] In an exemplary embodiment, an application / computer program product is also provided, comprising one or more instructions that can be executed by a processor of an electronic device to perform the aforementioned network monitoring method, comprising: obtaining speech data to be recognized; inputting the speech data to be recognized into a target soft threshold residual network to obtain a noise recognition result, wherein the target soft threshold residual network has a soft threshold function with a first slope and a soft threshold function with a second slope, the first slope being different from the second slope; and determining noise data in the speech data to be recognized based on the noise recognition result. Optionally, the aforementioned instructions can also be executed by a processor of an electronic device to perform the other steps involved in the aforementioned exemplary embodiment.

[0177] Figure 13 This is an example diagram of a computer device 30. Those skilled in the art will appreciate that Figure 13 This is merely an example of the computer device 30 and does not constitute a limitation of the computer device 30 . The computer device 30 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the computer device 30 may also include input and output devices, network access devices, buses, etc.

[0178] The processor 302 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor 302 may be any conventional processor. The processor 302 is the control center of the computer device 30 and connects various components of the entire computer device 30 using various interfaces and lines.

[0179] The memory 301 can be used to store computer-readable instructions 303. The processor 302 implements various functions of the computer device 30 by running or executing the computer-readable instructions or modules stored in the memory 301 and accessing the data stored in the memory 301. The memory 301 may mainly include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function (such as a sound playback function or an image playback function); the data storage area may store data generated based on the use of the computer device 30. In addition, the memory 301 may include a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, a read-only memory (ROM), a random access memory (RAM), or other non-volatile / volatile storage devices.

[0180] If the modules integrated into computer device 30 are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the present invention can also implement all or part of the processes in the above-mentioned method embodiments by instructing the relevant hardware through computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium. When executed by a processor, the computer-readable instructions can implement the steps of each of the above-mentioned method embodiments.

[0181] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0182] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A noise identification method, characterized in that: include: Obtaining voice data to be recognized; Inputting the to-be-recognized speech data into a target soft threshold residual network to obtain a noise recognition result, wherein the target soft threshold residual network has a soft threshold function with a first slope and a soft threshold function with a second slope, and the first slope is different from the second slope; Determining noise data in the speech data to be recognized according to the noise recognition result; The soft threshold function is expressed as follows: Wherein, τ is a threshold, x and y are real numbers representing input and output respectively, α is the first slope, and b is the second slope.

2. The method according to claim 1, wherein Before acquiring the voice data to be recognized, the method further includes: Obtain a first input feature, perform at least two convolution operations on the first input feature, perform a batch normalization operation, and perform a linear rectification function operation on the first input feature to obtain a first output result; Performing an absolute value algorithm and a global average pooling operation on the first output result to obtain a second output result; The first slope and the second slope are obtained based on the second output result.

3. The method according to claim 2, wherein The obtaining the first slope and the second slope based on the second output result includes: Performing a convolution operation on the second output result, a batch normalization operation, a full connection operation, and a linear rectification function operation on the second output result to obtain a third output result; Perform sigmoid function conversion on the third output result to obtain a target soft threshold function.

4. The method according to claim 3, wherein After obtaining the first output result, the method further includes: Performing a first absolute value algorithm on the first output result and a global average pooling operation on the first output result to obtain a fourth output result; A full connection operation is performed on the fourth output result, a batch normalization operation is performed, a linear rectification function operation is performed, and a full connection operation is performed to obtain the first slope value.

5. The method according to claim 4, wherein After obtaining the first slope value, the method further includes: Performing a second absolute value algorithm on the first output result and a global average pooling operation on the first output result to obtain a fifth output result; A full connection operation is performed on the fifth output result, a batch normalization operation is performed, a linear rectification function operation is performed, and a full connection operation is performed to obtain the second slope value.

6. The method according to claim 5, wherein After obtaining the second slope value, the method further includes: generating a soft threshold function having the first slope and a soft threshold function having the second slope according to the first slope value, the second slope value, and the target soft threshold function; The target soft threshold residual network is constructed according to the soft threshold function with the first slope and the soft threshold function with the second slope.

7. The method according to claim 1, wherein The step of inputting the speech data to be recognized into a target soft threshold residual network to obtain a noise recognition result includes: Acquire the speech data to be recognized, perform a convolution operation on the speech data to be recognized, extract semantic information from the speech data to be recognized, and obtain an output result of a first dimension; Using the soft threshold function of the first slope and the soft threshold function of the second slope, removing noise redundancy of the output result of the first dimension; Performing a batch normalization operation on the output result of the first dimension to remove noise redundancy, a global average pooling operation, and a linear rectification function operation to obtain an output result of the second dimension; The noise recognition result is obtained based on the output result of the second dimension.

8. The method according to claim 7, wherein The obtaining of the noise recognition result based on the output result of the second dimension includes: Performing a conversion operation to convert the output result of the second dimension into an N-dimensional vector; A full connection operation and a softmax function operation are performed on the N-dimensional vector to obtain the noise recognition result.

9. A noise recognition device, characterized in that: include: An acquisition module is configured to acquire speech data to be recognized; a generating module configured to input the speech data to be recognized into a target soft threshold residual network to obtain a noise recognition result, wherein the target soft threshold residual network has a soft threshold function with a first slope and a soft threshold function with a second slope, and the first slope is different from the second slope; a determination module, configured to determine noise data in the speech data to be recognized based on the noise recognition result; The soft threshold function in the generation module is expressed as follows: Wherein, τ is a threshold, x and y are real numbers representing input and output respectively, α is the first slope, and b is the second slope.

10. A household appliance, characterized in that: include: a memory for storing executable instructions; as well as, A processor is configured to communicate with the memory to execute the executable instructions to thereby complete the operation of the noise recognition method according to any one of claims 1 to 8.

11. A computer-readable storage medium for storing computer-readable instructions, characterized in that: When the instructions are executed, the operation of the noise recognition method according to any one of claims 1 to 8 is performed.

Images