Voice processing method, device, computer equipment, and storage medium

Abstract

The present application relates to a speech processing method, apparatus, computer equipment, storage medium and computer program product. The method comprises obtaining audio data and image information of the location where the audio data is collected; determining scene information corresponding to the audio data according to the image information; determining candidate noise data corresponding to the scene information from a noise library, comparing the candidate noise data with the audio data, and determining target noise data in the audio data; removing the target noise data from the audio data, and outputting the denoised audio data, thereby reducing the influence of environmental noise on the audio data, and subsequently performing speech recognition on the denoised audio data, which can improve the accuracy of speech recognition.

Description

Technical Field

[0001] The present application relates to the field of computer applications, and in particular to a speech processing method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] With the development of computer technology, speech recognition technology has emerged. Speech recognition technology is a human-computer interaction technology that is widely used in various life interaction scenarios. In recent years, the application of deep neural networks in the field of speech recognition has become more and more popular, making speech recognition technology more usable. Despite this, factors such as environmental noise still have a great impact on the performance of speech recognition systems, reducing the accuracy of speech recognition. Summary of the invention

[0003] Based on this, it is necessary to provide a speech processing method, apparatus, computer device, computer-readable storage medium and computer program product to address the above technical issues.

[0004] In a first aspect, the present application provides a speech processing method. The method comprises:

[0005] Acquire audio data and image information of a location where the audio data is collected;

[0006] Determining scene information corresponding to the audio data according to the image information;

[0007] Determine candidate noise data corresponding to the scene information from a noise library, compare the candidate noise data with the audio data, and determine target noise data in the audio data;

[0008] The target noise data is removed from the audio data, and the noise-reduced audio data is output.

[0009] In one embodiment, comparing the candidate noise data with the audio data to determine the target noise data in the audio data includes:

[0010] Separating the audio data to obtain voice audio data and environmental audio data;

[0011] The candidate noise data is compared with the ambient audio data to determine target noise data in the ambient audio data.

[0012] In one embodiment, comparing the candidate noise data with the ambient audio data to determine the target noise data in the ambient audio data includes:

[0013] Calculating the similarity between the candidate noise data and the ambient audio data;

[0014] The candidate noise data whose similarity is greater than or equal to the similarity threshold is determined as the target noise data in the environmental audio data.

[0015] In one embodiment, removing the target noise data from the audio data and outputting the noise-reduced audio data includes:

[0016] Eliminating the target noise data from the ambient audio data;

[0017] The ambient audio data from which the target noise data is removed and the speech audio data are merged to output the noise-reduced audio data.

[0018] In one embodiment, the method further comprises:

[0019] Identify the environmental audio data from which the target noise data is removed, and obtain a pseudo label of the environmental audio data from which the target noise data is removed; the pseudo label is used to characterize the noise type of the environmental audio data;

[0020] Mixing the labeled training data for training the noise classification model and the pseudo-labeled environmental audio data to obtain mixed training data;

[0021] The noise classification model is trained according to the mixed training data to obtain an updated noise classification model.

[0022] In one embodiment, the identifying the ambient audio data from which the target noise data is removed to obtain a pseudo label of the ambient audio data from which the target noise data is removed includes:

[0023] The environmental audio data from which the target noise data is removed is input into the noise classification model to obtain a pseudo label for the environmental audio data from which the target noise data is removed.

[0024] In a second aspect, the present application also provides a speech processing device. The device comprises:

[0025] An information acquisition module, used to acquire audio data and image information of a location where the audio data is collected;

[0026] a location determination module, configured to determine scene information corresponding to the audio data according to the image information;

[0027] A noise determination module, configured to determine candidate noise data corresponding to the scene information from a noise library, and compare the candidate noise data with the audio to determine target noise data in the audio information;

[0028] The output module is used to remove the target noise data from the audio data and output the denoised audio data.

[0029] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0030] Acquire audio data and image information of a location where the audio data is collected;

[0031] Determining scene information corresponding to the audio data according to the image information;

[0032] Determine candidate noise data corresponding to the scene information from a noise library, compare the candidate noise data with the audio data, and determine target noise data in the audio data;

[0033] The target noise data is removed from the audio data, and the noise-reduced audio data is output.

[0034] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0035] Acquire audio data and image information of a location where the audio data is collected;

[0036] Determining scene information corresponding to the audio data according to the image information;

[0037] Determine candidate noise data corresponding to the scene information from a noise library, compare the candidate noise data with the audio data, and determine target noise data in the audio data;

[0038] The target noise data is removed from the audio data, and the noise-reduced audio data is output.

[0039] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0040] Acquire audio data and image information of a location where the audio data is collected;

[0041] Determining scene information corresponding to the audio data according to the image information;

[0042] Determine candidate noise data corresponding to the scene information from a noise library, compare the candidate noise data with the audio data, and determine target noise data in the audio data;

[0043] The target noise data is removed from the audio data, and the noise-reduced audio data is output.

[0044] The above-mentioned speech processing method, device, computer equipment, storage medium and computer program product obtain audio data and image information of the location where the audio data is collected; determine the scene information corresponding to the audio data according to the image information; determine the candidate noise data corresponding to the scene information from the noise library, compare the candidate noise data with the audio data, and determine the target noise data in the audio data; remove the target noise data from the audio data and output the audio data after noise reduction. First, the audio data is obtained and the image information of the location where the audio data is collected is obtained to determine the scene information corresponding to the audio data, then the candidate noise data corresponding to the scene information is queried from the noise library, the target noise data in the audio data is determined through comparison, and then the target noise data is removed and the audio data after noise reduction is output, thereby reducing the influence of environmental noise on the audio data. Speech recognition is subsequently performed on the denoised audio data, which can improve the accuracy of speech recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A diagram of an application environment of a speech processing method in an embodiment;

[0046] Figure 2 is a flow chart of a speech processing method in one embodiment;

[0047] Figure 3 A schematic diagram of a noise library system of a speech processing method in one embodiment;

[0048] Figure 4 A schematic diagram of obtaining pseudo labels in a speech processing method in one embodiment;

[0049] Figure 5 A training schematic diagram of building a noise classification model in a speech processing method in one embodiment;

[0050] Figure 6 is a structural block diagram of a speech processing device in one embodiment;

[0051] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0053] The speech processing method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The server 104 obtains the audio data in the terminal 102 and the image information of the location where the audio data is collected; determines the scene information corresponding to the audio data according to the image information; determines the candidate noise data corresponding to the scene information from the noise library, compares the candidate noise data with the audio data, and determines the target noise data in the audio data; removes the target noise data from the audio data, and outputs the denoised audio data. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0054] It can be understood that the voice processing method provided in the embodiment of the present application can be applied not only to a system including a terminal and a server, but also to a terminal or a server alone.

[0055] In one embodiment, Figure 2 As shown, a speech processing method is provided, which is applied to Figure 1 The server 104 or the terminal 102 in the example is used as an example to illustrate, and the following steps are included:

[0056] Step 202: Acquire audio data and image information of the location where the audio data is collected.

[0057] The audio data may be a sound signal collected in real time or a sound signal stored in a storage space after collection. The user may collect the audio data of the user's location through an audio input device such as a microphone of the terminal. The audio data may include voice data and environmental sound data.

[0058] Image information refers to the image information of the location where the audio data is collected. The user can use the terminal's camera to take a picture of the location where the audio data is collected.

[0059] In a possible implementation, a processor of the terminal obtains audio data collected by a microphone and a picture taken by a camera of the location where the audio data was collected.

[0060] In another possible implementation, the terminal uploads to the server the audio data collected by the microphone and a picture of the location where the audio data was collected, taken by the camera.

[0061] Step 204: determine scene information corresponding to the audio data according to the image information.

[0062] Among them, scene information refers to the scene where the audio information is collected, such as an indoor scene, a subway scene, a school scene, a train station scene, and a restaurant scene, but not limited to these.

[0063] In a possible implementation, a processor of the terminal determines scene information corresponding to the audio data according to the acquired image information.

[0064] In another possible implementation, the terminal determines the scene information corresponding to the audio data according to the acquired image information, and uploads the audio information and the scene information corresponding to the audio information to the server together.

[0065] Step 206: determine candidate noise data corresponding to the scene information from the noise library, compare the candidate noise data with the audio data, and determine target noise data in the audio data.

[0066] Among them, the collected noise data set is classified in advance according to the noise data corresponding to each scene, and then stored in the noise library according to the correspondence between the scene and the noise data. Candidate noise data refers to a variety of noise data corresponding to a scene in the noise library. The target noise data can be determined according to the similarity between the audio data and the candidate noise data. Specifically, the processor queries the noise data corresponding to the scene information in the noise library as candidate noise data based on the scene information, and then compares the candidate noise data with the audio data for similarity, and determines the target noise data in the audio data according to the similarity comparison result. In one possible implementation, the candidate noise data with a similarity greater than a similarity threshold is selected as the target noise data; in another possible implementation, the candidate noise data with the greatest similarity is selected as the target noise data.

[0067] Step 208: remove the target noise data from the audio data and output the denoised audio data.

[0068] Among them, noise reduction refers to reducing the impact of target noise on audio data.

[0069] Specifically, the processor removes the target noise data from the audio data and outputs the noise-reduced audio data.

[0070] In the above-mentioned speech processing method, audio data and image information of the location where the audio data is collected are obtained; scene information corresponding to the audio data is determined according to the image information; candidate noise data corresponding to the scene information is determined from a noise library, the candidate noise data is compared with the audio data, and target noise data in the audio data is determined; the target noise data is removed from the audio data, and the denoised audio data is output. First, the audio data is obtained and the image information of the location where the audio data is collected is obtained to determine the scene information corresponding to the audio data, and then the candidate noise data corresponding to the scene information is queried from the noise library, and the target noise data in the audio data is determined after comparison, and then the target noise data is removed, and the denoised audio data is output, thereby improving the accuracy of speech recognition.

[0071] In one embodiment, comparing the candidate noise data with the audio data to determine the target noise data in the audio data includes: separating the audio data to obtain speech audio data and ambient audio data; comparing the candidate noise data with the ambient audio data to determine the target noise data in the ambient audio data.

[0072] Among them, the voice audio data is the voice audio information sent by the user; the environmental audio data refers to the environmental audio information when the audio data is collected.

[0073] Specifically, the processor separates the audio data into speech audio information and environmental audio information according to the near track and the far track, and then compares the similarity between the candidate noise data and the environmental audio data to determine the target noise data in the environmental audio data.

[0074] In this embodiment, by separating the audio data into speech audio data and environmental audio data, and determining the target noise data from the environmental audio data, the accuracy of determining the target noise data can be improved.

[0075] In one embodiment, comparing candidate noise data with ambient audio data to determine target noise data in the ambient audio data includes: calculating the similarity between the candidate noise data and the ambient audio data; and determining candidate noise data having a similarity greater than or equal to a similarity threshold as the target noise data in the ambient audio data.

[0076] Among them, the similarity is used to indicate the similarity between the candidate noise data and the ambient audio data. The higher the similarity, the closer the candidate noise data is to the ambient audio data. The similarity between the candidate noise data and the ambient audio data is determined based on the waveform.

[0077] Specifically, the processor calculates the similarity between the candidate noise data and the ambient audio data according to the waveform of the candidate noise data and the waveform of the ambient audio data, and compares the similarity threshold to determine the candidate noise data as the target noise data in the ambient audio data if the similarity between the candidate noise data and the ambient audio data is greater than or equal to the similarity threshold. If the similarity between the candidate noise data and the ambient audio data is less than the similarity threshold, the candidate noise data is not used as the target noise data in the ambient audio data.

[0078] In this embodiment, by calculating the similarity, the accuracy of determining the target noise data can be improved.

[0079] In one embodiment, removing target noise data from audio data and outputting noise-reduced audio data includes: removing target noise data from ambient audio data; merging ambient audio data from which target noise data is removed and speech audio data, and outputting noise-reduced audio data.

[0080] Specifically, the processor removes the target noise data from the ambient audio data according to the waveform of the target noise data, and then merges the ambient audio data from which the target noise data is removed with the voice audio data to output the noise-reduced audio data.

[0081] In this embodiment, the noise reduction effect can be achieved by eliminating the target noise data and then merging the ambient audio data from which the target noise data is eliminated and the speech audio book.

[0082] In one embodiment, the above-mentioned speech processing method includes: identifying the environmental audio data from which the target noise data is eliminated, and obtaining a pseudo-label for the environmental audio data from which the target noise data is eliminated; the pseudo-label is used to characterize the noise type of the environmental audio data; the labeled training data used to train the noise classification model and the pseudo-labeled environmental audio data are mixed to obtain mixed training data; the noise classification model is trained according to the mixed training data to obtain an updated noise classification model.

[0083] Among them, pseudo labels refer to the approximate labels of unlabeled noise data samples, which are used to simulate the noise types of manually labeled environmental audio data. Noise classification models refer to models used to classify noise data according to scenes. Training data refers to data used to train noise classification models during noise data mining. Large and diverse noise sample data can build high-quality noise classification models.

[0084] Specifically, the processor identifies the ambient audio data after the target noise data is removed, determines the pseudo-label of the ambient audio data after the target noise data is removed based on the ambient audio data after the target noise data is removed, mixes the labeled training data and the pseudo-labeled ambient audio data for training the noise classification model to obtain mixed training data, trains the noise classification model based on the mixed training data, and obtains an updated noise classification model.

[0085] In this embodiment, a new noise classification model is obtained by training the ambient audio data after removing the target noise data and the labeled training data for training the noise classification model, so as to achieve the effect of updating the noise database.

[0086] In one embodiment, identifying the ambient audio data from which target noise data is removed and obtaining a pseudo label for the ambient audio data from which the target noise data is removed includes: inputting the ambient audio data from which the target noise data is removed into a noise classification model to obtain a pseudo label for the ambient audio data from which the target noise data is removed.

[0087] Specifically, the processor extracts features from the ambient audio data after removing the target noise, transforms the features into vectors through convolution kernel, linear rectification function and maximum pooling, and then inputs them into the noise classification model, calculates through S-type function and pooling function, and obtains the pseudo labels of the ambient audio data after removing the target noise data. The features include the batch number, channel, time and frequency of the ambient audio data after removing the target noise data.

[0088] In this embodiment, by acquiring labels, it is possible to easily construct a noise classification model.

[0089] In order to better understand the speech processing method provided in the embodiment of the present application, the specific steps of the speech processing method are described below in conjunction with a detailed embodiment:

[0090] (1) The processor obtains audio data and image information of the location where the audio data is collected;

[0091] (2) The processor separates the audio data into voice audio data and ambient audio data;

[0092] (3) the processor determines the scene at the location where the audio information is collected based on the image information at the location where the audio information is collected;

[0093] (4) The processor queries the noise data corresponding to the scene in the noise library as candidate noise data according to the scene information, wherein the noise library system schematic diagram is as follows: Figure 3 As shown, the noise library uses scenes to distinguish the noise data corresponding to each scene. Different locations correspond to different scenes, and different scenes correspond to different noise data.

[0094] (5) the processor calculates the similarity between the candidate noise data and the ambient audio data;

[0095] (6) The processor compares the similarity between the candidate noise data and the ambient audio data, and determines the candidate noise data whose similarity is greater than or equal to the similarity threshold as the target noise data in the ambient audio data.

[0096] (7) The processor removes the target noise data from the ambient audio data;

[0097] (8) The processor then fuses the ambient audio data with the speech audio data after the target noise data is removed, and outputs the audio data after noise reduction.

[0098] (9) The processor extracts features from the ambient audio data after the target noise is removed, wherein the features include batch number, channel, time, and frequency; the features are then vectorized through convolution kernel, linear rectification function (ReLU) and maximum pooling, input into the long short-term memory neural network for classification through the fully connected layer, and calculated through S-type function and pooling function to obtain the pseudo label of the ambient audio data after the target noise data is removed, such as Figure 4 shown.

[0099] (10) The processor mixes the labeled training data and the pseudo-labeled ambient audio data used to train the noise classification model to obtain mixed training data, and trains the noise classification model according to the mixed training data to obtain an updated noise classification model, such as Figure 5 shown.

[0100] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0101] Based on the same inventive concept, the embodiment of the present application also provides a speech processing device for implementing the speech processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more speech processing device embodiments provided below can refer to the limitations on the speech processing method above, and will not be repeated here.

[0102] In one embodiment, Figure 6 As shown, a speech processing device is provided, including: an information acquisition module 610, a scene determination module 620, a noise determination module 630 and an output module 640, wherein:

[0103] An information acquisition module 610 is used to acquire audio data and image information of a location where the audio data is collected;

[0104] A scene determination module 620, configured to determine scene information corresponding to the audio data according to the image information;

[0105] A noise determination module 630 is used to determine candidate noise data corresponding to the scene information from a noise library, and compare the candidate noise data with the audio to determine target noise data in the audio information;

[0106] The output module 640 is used to remove the target noise data from the audio data and output the denoised audio data.

[0107] In one embodiment, the speech processing device includes: a separation module for separating the audio data to obtain speech audio data and environmental audio data; and a noise determination module 630 for comparing the candidate noise data with the environmental audio data to determine target noise data in the environmental audio data.

[0108] In one embodiment, the speech processing device includes: a calculation module for calculating the similarity between the candidate noise data and the ambient audio data; and a noise determination module 630 for determining the candidate noise data with a similarity greater than or equal to a similarity threshold as the target noise data in the ambient audio data.

[0109] In one embodiment, the speech processing device comprises: a removal module for removing the target noise data from the ambient audio data; and an output module 640 for merging the ambient audio data from which the target noise data is removed and the speech audio data, and outputting the denoised audio data.

[0110] In one embodiment, the speech processing device includes: a label acquisition module, a mixing module and a model determination module. The label acquisition module is used to identify the environmental audio data from which the target noise data is removed, and obtain a pseudo label of the environmental audio data from which the target noise data is removed; the pseudo label is used to characterize the noise type of the environmental audio data; the mixing module is used to mix the labeled training data for training the noise classification model and the pseudo-labeled environmental audio data to obtain mixed training data; the model determination module is used to train the noise classification model according to the mixed training data to obtain an updated noise classification model.

[0111] In one embodiment, the label acquisition module is used to input the environmental audio data from which the target noise data is removed into the noise classification model to obtain a pseudo label for the environmental audio data from which the target noise data is removed.

[0112] Each module in the above-mentioned speech processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0113] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store voice processing data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a voice processing method is implemented.

[0114] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0115] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0116] Acquire audio data and image information of a location where the audio data is collected;

[0117] Determining scene information corresponding to the audio data according to the image information;

[0118] Determine candidate noise data corresponding to the scene information from the noise library, compare the candidate noise data with the audio data, and determine target noise data in the audio data;

[0119] Remove target noise data from the audio data and output the denoised audio data.

[0120] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0121] Separate the audio data to obtain voice audio data and environmental audio data;

[0122] The candidate noise data is compared with the ambient audio data to determine the target noise data in the ambient audio data.

[0123] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0124] Calculate the similarity between candidate noise data and environmental audio data;

[0125] The candidate noise data with a similarity greater than or equal to a similarity threshold is determined as the target noise data in the environmental audio data.

[0126] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0127] Eliminate target noise data from environmental audio data;

[0128] The ambient audio data and the speech audio data after the target noise data are removed are merged to output the denoised audio data.

[0129] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0130] Identify the ambient audio data from which target noise data is removed, and obtain a pseudo label of the ambient audio data from which the target noise data is removed; the pseudo label is used to characterize the noise type of the ambient audio data;

[0131] Mixing the labeled training data used to train the noise classification model and the pseudo-labeled environmental audio data to obtain mixed training data;

[0132] The noise classification model is trained according to the mixed training data to obtain an updated noise classification model.

[0133] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0134] The ambient audio data from which the target noise data is removed is input into the noise classification model to obtain a pseudo label for the ambient audio data from which the target noise data is removed.

[0135] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0136] Acquire audio data and image information of a location where the audio data is collected;

[0137] Determining scene information corresponding to the audio data according to the image information;

[0138] Determine candidate noise data corresponding to the scene information from the noise library, compare the candidate noise data with the audio data, and determine target noise data in the audio data;

[0139] Remove target noise data from the audio data and output the denoised audio data.

[0140] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0141] Separate the audio data to obtain voice audio data and environmental audio data;

[0142] The candidate noise data is compared with the ambient audio data to determine the target noise data in the ambient audio data.

[0143] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0144] Calculate the similarity between candidate noise data and environmental audio data;

[0145] The candidate noise data with a similarity greater than or equal to a similarity threshold is determined as the target noise data in the environmental audio data.

[0146] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0147] Eliminate target noise data from environmental audio data;

[0148] The ambient audio data and the speech audio data after the target noise data are removed are merged to output the denoised audio data.

[0149] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0150] Identify the ambient audio data from which target noise data is removed, and obtain a pseudo label of the ambient audio data from which the target noise data is removed; the pseudo label is used to characterize the noise type of the ambient audio data;

[0151] Mixing the labeled training data used to train the noise classification model and the pseudo-labeled environmental audio data to obtain mixed training data;

[0152] The noise classification model is trained according to the mixed training data to obtain an updated noise classification model.

[0153] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0154] The ambient audio data from which the target noise data is removed is input into the noise classification model to obtain a pseudo label for the ambient audio data from which the target noise data is removed.

[0155] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0156] Acquire audio data and image information of a location where the audio data is collected;

[0157] Determining scene information corresponding to the audio data according to the image information;

[0158] Determine candidate noise data corresponding to the scene information from the noise library, compare the candidate noise data with the audio data, and determine target noise data in the audio data;

[0159] Remove target noise data from the audio data and output the denoised audio data.

[0160] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0161] Separate the audio data to obtain voice audio data and environmental audio data;

[0162] The candidate noise data is compared with the ambient audio data to determine the target noise data in the ambient audio data.

[0163] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0164] Calculate the similarity between candidate noise data and environmental audio data;

[0165] The candidate noise data with a similarity greater than or equal to a similarity threshold is determined as the target noise data in the environmental audio data.

[0166] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0167] Eliminate target noise data from environmental audio data;

[0168] The ambient audio data and the speech audio data after the target noise data are removed are merged to output the denoised audio data.

[0169] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0170] Identify the ambient audio data from which target noise data is removed, and obtain a pseudo label of the ambient audio data from which the target noise data is removed; the pseudo label is used to characterize the noise type of the ambient audio data;

[0171] Mixing the labeled training data used to train the noise classification model and the pseudo-labeled environmental audio data to obtain mixed training data;

[0172] The noise classification model is trained according to the mixed training data to obtain an updated noise classification model.

[0173] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0174] The ambient audio data from which the target noise data is removed is input into the noise classification model to obtain a pseudo label for the ambient audio data from which the target noise data is removed.

[0175] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0176] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0177] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0178] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A speech processing method, characterized in that: The method comprises: Acquire audio data and image information of a location where the audio data is collected; Determining scene information corresponding to the audio data according to the image information; Determine candidate noise data corresponding to the scene information from a noise library, compare the candidate noise data with the audio data, and determine target noise data in the audio data; the noise library stores the correspondence between the scene and the noise data; The target noise data is removed from the audio data, and the noise-reduced audio data is output; the noise-reduced audio data is used for speech recognition.

2. The method according to claim 1, characterized in that The comparing the candidate noise data with the audio data to determine the target noise data in the audio data includes: Separating the audio data to obtain voice audio data and environmental audio data; The candidate noise data is compared with the ambient audio data to determine target noise data in the ambient audio data.

3. The method according to claim 2, characterized in that The comparing the candidate noise data with the ambient audio data to determine the target noise data in the ambient audio data includes: Calculating the similarity between the candidate noise data and the ambient audio data; The candidate noise data whose similarity is greater than or equal to the similarity threshold is determined as the target noise data in the environmental audio data.

4. The method according to claim 2, characterized in that: The removing the target noise data from the audio data and outputting the noise-reduced audio data comprises: Eliminating the target noise data from the ambient audio data; The ambient audio data from which the target noise data is removed and the speech audio data are merged to output the noise-reduced audio data.

5. The method according to claim 4, characterized in that The method further comprises: Identify the environmental audio data from which the target noise data is removed, and obtain a pseudo label of the environmental audio data from which the target noise data is removed; the pseudo label is used to characterize the noise type of the environmental audio data; Mixing the labeled training data used to train the noise classification model and the pseudo-labeled environmental audio data to obtain mixed training data; The noise classification model is trained according to the mixed training data to obtain an updated noise classification model.

6. The method according to claim 5, characterized in that The step of identifying the ambient audio data from which the target noise data is removed, and obtaining a pseudo label of the ambient audio data from which the target noise data is removed, comprises: The environmental audio data from which the target noise data is removed is input into the noise classification model to obtain a pseudo label for the environmental audio data from which the target noise data is removed.

7. A speech processing device, characterized in that: The device comprises: An information acquisition module, used to acquire audio data and image information of a location where the audio data is collected; A scene determination module, used to determine scene information corresponding to the audio data according to the image information; A noise determination module, configured to determine candidate noise data corresponding to the scene information from a noise library, compare the candidate noise data with the audio data, and determine target noise data in the audio data; the noise library stores the correspondence between scenes and noise data; The output module is used to remove the target noise data from the audio data and output the denoised audio data; the denoised audio data is used for speech recognition.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images