Voice signal processing method and device and electronic equipment

Abstract

The invention discloses a voice signal processing method and device and electronic equipment. In the application, signal enhancement is performed on a first voice signal, and weight prediction processing is performed in combination with the first voice signal and a second voice signal to obtain a first mixed weight corresponding to the first voice signal; and then, mixing the first voice signal with the second voice signal according to the first mixing weight, performing voice recognition on the mixed voice signal, and taking a recognition result as a recognition result of the first voice signal. In the scheme, the enhanced voice signal is superposed in the original voice, so that an effective signal can be perceived more easily during voice recognition; besides, the original signal and the second voice signal are mixed according to a certain proportion, so that the influence of nonlinear noise caused by voice enhancement in the second voice signal is reduced, and the accuracy of a voice recognition result is improved.

Description

Technical Field

[0001] The present application relates to the field of signal processing technology, and in particular to a method, device and electronic device for processing speech signals. Background Art

[0002] Speech recognition technology, also known as automatic speech recognition (ASR), aims to convert the lexical content of human speech into computer-readable input. Speech recognition is a foundational technology for intelligent interaction. However, current speech recognition technology suffers from low accuracy and is in urgent need of improvement. Summary of the Invention

[0003] The embodiments of the present application provide a speech signal processing method, apparatus, and electronic device, which can improve the accuracy of speech recognition results.

[0004] A speech signal processing method, comprising:

[0005] Performing signal enhancement on the first speech signal to obtain a second speech signal;

[0006] Performing weight prediction processing based on the first speech signal and the second speech signal to obtain a first mixing weight corresponding to the first speech signal;

[0007] mixing the first speech signal with the second speech signal according to the first mixing weight to obtain a mixed speech signal;

[0008] Perform speech recognition on the mixed speech signal to obtain a recognition result.

[0009] Accordingly, an embodiment of the present application further provides a speech signal processing device, comprising:

[0010] a speech enhancement unit, configured to perform signal enhancement on the first speech signal to obtain a second speech signal;

[0011] a processing unit, configured to perform weight prediction processing based on the first speech signal and the second speech signal to obtain a first mixing weight corresponding to the first speech signal;

[0012] a mixing unit, configured to mix the first speech signal with the second speech signal according to the first mixing weight to obtain a mixed speech signal;

[0013] The speech recognition unit is used to perform speech recognition on the mixed speech signal to obtain a recognition result.

[0014] In addition, an embodiment of the present application further provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the processor is used to run the computer program in the memory to implement the steps in the speech signal processing method provided in the embodiment of the present application.

[0015] In addition, an embodiment of the present application further provides a computer-readable storage medium, which stores a plurality of instructions suitable for loading by a processor to execute the steps in the speech signal processing method provided in the embodiment of the present application.

[0016] In addition, an embodiment of the present application also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the steps in the speech signal processing method provided in the embodiment of the present application.

[0017] In an embodiment of the present application, the first voice signal is enhanced and weight prediction processing is performed on the first voice signal and the second voice signal. A first mixing weight corresponding to the first voice signal is obtained; then, the first voice signal is mixed with the second voice signal according to the first mixing weight, and voice recognition is performed on the mixed voice signal, and the recognition result is used as the recognition result of the first voice signal. In this solution, the most suitable mixing weight when the first voice signal is mixed is determined by introducing weight prediction, and the first voice signal and the second voice signal are mixed for voice recognition, that is, the enhanced voice signal is superimposed on the original voice, so that the effective signal can be more easily perceived during voice recognition; in addition, the original first voice signal and the enhanced second voice signal are mixed in a certain proportion, rather than directly performing voice recognition on the second voice signal, which reduces the influence of nonlinear noise in the second voice signal caused by voice enhancement, thereby improving the accuracy of the voice recognition result. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 Schematic diagram of a scenario of a speech signal processing method provided in an embodiment of the present application;

[0020] Figure 2 Schematic diagram of the flow of the speech signal processing method provided in the embodiment of the present application;

[0021] Figure 31 is a flow chart of a method for speech enhancement adapted for speech recognition provided in an embodiment of the present application;

[0022] Figure 4 Schematic diagram of the network structure of the weight prediction model provided in the embodiment of the present application;

[0023] Figure 5 This is a schematic diagram of another application scenario of the speech signal processing method provided in an embodiment of the present application;

[0024] Figure 6 Schematic diagram of the structure of the speech signal processing device provided in an embodiment of the present application;

[0025] Figure 7 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0027] In practical applications, speech enhancement techniques are often used to pre-process noisy speech before speech recognition in noisy environments. However, speech enhancement introduces nonlinear noise into the original speech signal, which the speech recognition model has not been trained to handle. Directly combining a speech recognition model with a speech enhancement model can easily affect speech recognition performance.

[0028] Based on this, the embodiments of the present application introduce a weight prediction method, which mixes the first voice signal and the second voice signal for voice recognition. The weight prediction method is used to predict the optimal mixing weight when the first voice signal and the second voice signal are mixed. This ensures that voice recognition can receive more comprehensive information, while reducing the impact of nonlinear noise in the second voice signal and improving the accuracy of the voice recognition results. In addition, this solution is not limited by the models used for speech enhancement and speech recognition. When changing the speech enhancement model or speech recognition model, there is no need to retrain the model, and the speech enhancement model or speech recognition model can be changed at will.

[0029] The embodiments of the present application provide a method, apparatus, and electronic device for processing voice signals, which can improve the intelligence of voice signal processing. The voice signal processing apparatus can be integrated into an electronic device, which can be a server, a terminal, or other device.

[0030] Among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services, and big data and artificial intelligence platforms. The terminal can be a mobile phone, a computer, an intelligent voice interaction device, a smart home appliance, a car terminal, an aircraft, etc., but is not limited to these. The terminal and the server can be directly or indirectly connected through wired or wireless communication, and this application does not limit this.

[0031] In order to better understand the speech signal processing method, device, and electronic device provided in the embodiments of the present application, the application environment applicable to the embodiments of the present application is described below.

[0032] See also Figure 1 , Figure 1 FIG1 shows a schematic diagram of an application environment of the voice signal processing method provided by an embodiment of the present application. As an implementation method, each step in the voice signal processing method provided by the embodiment of the present application can be applied to the same electronic device. The electronic device can be Figure 1 The server 10 shown in FIG. 1 can be connected to the terminal device 20 via a network. The network is used as a medium for providing a communication link between the server 10 and the terminal device 20. The network can include various connection types, such as wired communication links, wireless communication links, etc., which are not limited in the embodiments of the present application. Alternatively, in other embodiments, the electronic device can also be a smartphone, a laptop computer, etc.

[0033] It should be understood that Figure 1 The server 10, network, and terminal device 20 are merely illustrative. Any number of servers, networks, and terminal devices may be provided as needed. For example, the server 10 may be a physical server or a server cluster consisting of multiple servers, and the terminal device 20 may be a mobile phone, tablet, desktop computer, laptop computer, or the like. It will be appreciated that embodiments of the present application may also allow multiple terminal devices 20 to access the server 10 simultaneously.

[0034] In some embodiments, the terminal device 20 can record the user's voice to obtain the user's voice data. Further, the terminal device 20 sends the user's voice data to the server 10 via the network. After the server 10 receives the user's voice data, it can process the received voice data using the voice signal processing method of the embodiment of the present application.

[0035] As another embodiment, each step in the speech signal processing method provided in the embodiment of the present application can also be applied to different electronic devices. For example, the speech signal enhancement step is applied to electronic device A, the weight prediction step is applied to electronic device B, the speech signal mixing and recognition step is applied to electronic device C, and so on. In addition, any two of the speech signal enhancement step, the weight prediction step, and the speech signal mixing and recognition step provided in the embodiment of the present application can be applied to the same electronic device. For example, the speech signal enhancement step, the speech signal mixing and recognition step are applied to electronic device A, and the weight prediction step is applied to electronic device B. The embodiment of the present application does not limit the electronic device to which the above method is applied.

[0036] It should be noted that Figure 1 The scenario diagram of the speech signal processing application shown is only an example. The speech signal processing application and scenario described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided by the embodiment of the present application. Ordinary technicians in this field can know that with the evolution of speech signal processing applications and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.

[0037] It can be understood that in the specific implementation of this application, related data such as attribute data, attribute sets and attribute subsets are involved. When the following embodiments of this application are applied to specific products or technologies, permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0038] It should be noted that the order of description of the following embodiments is not intended to limit the preferred order of the embodiments.

[0039] This embodiment will be described from the perspective of a voice signal processing device. The voice signal processing device can be specifically integrated into an electronic device, which can be a server or a terminal. Among them, the terminal can be a mobile phone, a computer, an intelligent voice interaction device, a smart home appliance, a vehicle terminal, an aircraft, etc., but is not limited to these.

[0040] An embodiment of the present application provides a speech signal processing method, including: performing signal enhancement on a first speech signal to obtain a second speech signal; performing weight prediction processing based on the first speech signal and the second speech signal to obtain a first mixing weight corresponding to the first speech signal; mixing the first speech signal and the second speech signal according to the first mixing weight to obtain a mixed speech signal; and performing speech recognition on the mixed speech signal to obtain a recognition result.

[0041] like Figure 2 As shown, the specific process of the voice signal processing method is as follows:

[0042] 101. Perform signal enhancement on the first speech signal to obtain a second speech signal.

[0043] Signal enhancement, also known as speech enhancement, refers to the technology used to extract useful speech signals from a noisy background, suppressing and reducing the noise interference when the speech signal is interfered with or even drowned out by various noises. The goal is to extract the purest possible original speech from the noisy speech.

[0044] Specifically, a microphone or other recording device may be used to capture the user's voice to obtain a first voice signal, wherein the first voice signal may include an original voice signal (i.e., a pure original user voice) and a noise signal (e.g., background noise, echo, electrical noise, and other non-voice components).

[0045] In a specific implementation, the acquired first voice signal can be pre-processed, such as filtering, noise removal, and gain control. To enable the electronic device to understand and store the data, the first voice signal can be converted from an analog signal to a digital signal using an analog-to-digital converter. This process is called sampling and is accompanied by operations such as quantization and encoding. The resulting digital signal is then stored in a corresponding storage area, such as a hard disk, solid-state drive, or memory, and transferred to the electronic device for processing when needed.

[0046] Furthermore, the data integrity and quality of the voice signal can be loaded and checked on the electronic device, and time and frequency domain analysis can be performed to understand its characteristics. Next, key voice features such as Mel Frequency Cepstral Coefficients (MFCCs), pitch, and rhythm are extracted, while ensuring that the voice signal is synchronized with other relevant data. In addition, long voice signals can be segmented into shorter frames or segments to suit specific processing algorithms. In addition, the voice signal can be normalized to adjust the signal's energy level or dynamic range for subsequent processing.

[0047] In this embodiment, the first speech signal can be enhanced based on a trained speech enhancement model to obtain a second speech signal. There are many types of speech enhancement models, such as a model based on a dual-path recurrent neural network (Deep Parallel Recurrent Neural Network, DPRNN), a deep neural network (Deep Neural Networks, DNN), a convolutional neural network (Convolutional Neural Networks, CNN), a long short-term memory network (Long Short-Term Memory, LSTM), a generative adversarial network (Generative Adversarial Networks, GAN), and the like.

[0048] 102. Perform weight prediction processing based on the first speech signal and the second speech signal to obtain a first mixing weight corresponding to the first speech signal.

[0049] In an embodiment of the present application, a variety of different weight prediction methods may be included. For example, in one embodiment, weight prediction processing may be performed by signal processing. Specifically, a mapping relationship between different signal-to-noise ratios and mixing weights may be pre-set, and then the signal-to-noise ratio of the first speech signal may be calculated by signal processing. Finally, based on the calculated signal-to-noise ratio and the pre-set mapping relationship, a first mixing weight corresponding to the first speech signal may be determined.

[0050] For another example, in one embodiment, weight prediction processing can also be performed by model prediction. Specifically, the first speech signal and the second speech signal can be input into a weight prediction model, and the weight prediction model outputs a first mixing weight corresponding to the first speech signal.

[0051] It should be noted that the weight prediction model needs to be pre-built and trained in the embodiment of the present application. That is, in one embodiment, before the first speech signal is enhanced, the following process may also be included:

[0052] Constructing a training set, where the training set includes training samples and sample labels, where the training samples include a first speech sample and a second speech sample, where the second speech sample is obtained by performing signal enhancement on the first speech sample, and the sample label represents a sample signal-to-noise ratio of the first speech sample;

[0053] Obtaining a basic prediction model, which includes: a basic prediction network and a preset weight mapping network;

[0054] Taking the first speech sample and the second speech sample as inputs of a basic prediction network, and outputting a predicted signal-to-noise ratio of the first speech sample;

[0055] According to the difference between the predicted signal-to-noise ratio and the sample signal-to-noise ratio, the basic prediction network is trained to obtain a trained signal-to-noise ratio prediction network;

[0056] Based on the trained signal-to-noise ratio prediction network and the preset weight mapping network, a weight prediction model is constructed.

[0057] Specifically, in the process of building a weight prediction model, it is first necessary to build a comprehensive training set. This training set consists of multiple training samples, each of which includes a noisy speech signal (i.e., the first speech sample) and the corresponding enhanced speech signal (i.e., the second speech sample). The sample labels corresponding to these speech samples, i.e., the signal-to-noise ratio of the first speech sample, are an important indicator for evaluating the signal enhancement effect. In practical applications, clean speech signals and noise signals can be obtained, and the signal-to-noise ratio of the mixed noise is randomly selected within a certain signal-to-noise ratio range (i.e., a value interval, such as 0-30db) for mixing, to obtain the first speech sample and its corresponding sample signal-to-noise ratio.

[0058] The signal-to-noise ratio (SNR) is an important parameter for measuring signal quality. It represents the ratio of the power of the useful signal to the power of the background noise. SNR has a wide range of applications in communications, audio processing, video processing, and other electronic signal processing fields.

[0059] To establish a basic prediction model, a basic prediction network must be designed. This network accepts a noisy speech signal and an enhanced speech signal as input and outputs a predicted signal-to-noise ratio (SNR) for the corresponding speech signal. Furthermore, a pre-set weight mapping network is required to map the predicted SNR to a fixed decimal within a certain range, which serves as the mixing weight for the noisy speech signal when mixing speech signals.

[0060] During the training phase, the first and second speech samples are fed into the basic prediction network, which then outputs a predicted signal-to-noise ratio (SNR). This predicted SNR is then compared with the actual sample SNR label, and the difference between the two is calculated. This difference is a key measure of the model's prediction accuracy, and is used to train the basic prediction network through backpropagation, adjusting network parameters to minimize prediction error.

[0061] After a sufficient number of training iterations, a trained SNR prediction network is obtained, which can more accurately predict the SNR of the speech signal. Finally, this trained prediction network is combined with the preset weight mapping network to construct a complete weight prediction model.

[0062] In one embodiment, when the basic prediction network is trained based on the difference between the predicted signal-noise ratio and the sample signal-noise ratio to obtain the trained signal-noise ratio prediction network, a model loss can be constructed based on the difference between the predicted signal-noise ratio and the sample signal-noise ratio, and the network parameters of the basic prediction network are iteratively updated based on the model loss until the model loss converges, or the number of iterations is greater than or equal to a preset number, and finally the trained signal-noise ratio prediction network is obtained.

[0063] In this embodiment, the basic prediction network may include a convolutional layer and a linear layer. In specific implementation, the minimum mean square error (MSE) or mean absolute error (MAE) may be used as a loss function for gradient update.

[0064] Therefore, when processing the first and second speech signals based on the trained weight prediction model, the signal-to-noise ratio of the first speech signal can be predicted using the trained signal-to-noise ratio prediction network. Based on the signal-to-noise ratio, the corresponding mixing weight is determined using a preset weight mapping network. That is, in one embodiment, when performing weight prediction processing based on the first and second speech signals to obtain the first mixing weight corresponding to the first speech signal, the following process can be included:

[0065] Obtain a weight prediction model, which includes a signal-to-noise ratio prediction network and a weight mapping network;

[0066] Processing the first speech signal and the second speech signal based on a signal-to-noise ratio prediction network to obtain a signal-to-noise ratio of the first speech signal;

[0067] The signal-to-noise ratio is processed according to a weight mapping network to obtain a first mixing weight.

[0068] During specific implementation, the first speech signal and the second speech signal can be spliced ​​in the channel dimension as the input of the weight prediction model. For example, assuming that the dimensions of both signals are N*1*T, N represents the batch size, and T represents the duration of the speech signal, then after splicing the two in the channel dimension, the input dimension of the network can be obtained as N*2*T. Then, based on the first speech signal and the second speech signal, the signal-to-noise ratio of the first speech signal is predicted by the signal-to-noise ratio prediction network in the weight prediction model. Finally, the signal-to-noise ratio of the first speech signal is processed according to the weight mapping network in the weight prediction model to obtain the first mixing weight corresponding to the first speech signal.

[0069] In one embodiment, when the signal-to-noise ratio is processed according to the weight mapping network to obtain the first mixing weight, the following process may be included:

[0070] Determine the value interval corresponding to the sample signal-to-noise ratio;

[0071] Obtain the ratio between the signal-to-noise ratio and the size of the value interval through the weight mapping network;

[0072] The floating weight is weighted according to the ratio, and a preset operation is performed on the basic weight and the weighted floating weight to obtain a first mixed weight.

[0073] Specifically, the ratio between the signal-to-noise ratio of the first speech signal and the size of the value interval can be obtained through the weight mapping network in the weight prediction model, and based on the determined ratio, the floating weight is weighted through the weight mapping network, and then the basic weight and the weighted floating weight are subjected to preset operations (for example, addition and summation) to obtain the final weight calculation result, and the weight calculation result is used as the first mixed weight of the first speech signal.

[0074] It should be noted that in this embodiment, the base weight and the floating weight constitute the total weight, and the ratio of the base weight to the floating weight can be set according to actual circumstances. For example, if the total weight is 1 and the base weight is 0.5, the floating weight is 0.5; for another example, if the total weight is 1 and the base weight is 0.6, the floating weight is 0.4.

[0075] In practical applications, the weight prediction network can be constructed based on a linear transformation function, the formula of which is as follows:

[0076]

[0077] Among them, W represents the first mixing weight, x represents the signal-to-noise ratio of the first speech signal; n represents the size of the value interval corresponding to the sample signal-to-noise ratio, that is, the difference between the upper limit and the lower limit of the value interval. If the value interval is [a, b], then the value of n is ab. For example, if the value interval is [0, 30], then the value of n is 30; m represents the basic weight, which can be any real number in the interval (0, 1).

[0078] In one embodiment, if the lower limit value (i.e., a) of the value interval [a, b] is not 0, then when calculating the first mixed weight, it is necessary to calculate the difference between the signal-to-noise ratio of the first speech signal and the lower limit value, and obtain the ratio between the difference and the size of the value interval (i.e., n), and then perform weighted processing on the floating weight according to the obtained ratio, and perform preset operations on the basic weight and the weighted floating weight to obtain the first mixed weight.

[0079] It should be noted that if the signal-to-noise ratio of the first speech signal is greater than the upper limit of the above-mentioned value range, the signal-to-noise ratio of the first speech signal is updated to the upper limit, that is, the upper limit of the value range is assigned to the signal-to-noise ratio of the first speech signal, and the assigned signal-to-noise ratio is input into the weight mapping network for processing.

[0080] 103. Mix the first speech signal and the second speech signal according to a first mixing weight to obtain a mixed speech signal.

[0081] Among them, speech mixing aims to balance the intelligibility and naturalness of speech, while minimizing the noise distortion that may be generated during the enhancement process, and ultimately generating a clearer speech output. In specific implementation, the mixing weight corresponding to the second speech signal can be calculated based on the known weight information, so that the first speech signal and the second speech signal can achieve a better mixing effect. That is, in a real-time manner, when the first speech signal and the second speech signal are mixed according to the first mixing weight, the following process can be included:

[0082] Determining a second mixing weight corresponding to the second speech signal according to the total weight and the first mixing weight;

[0083] Performing weighted processing on the first speech signal based on the first mixing weight, and performing weighted processing on the second speech signal based on the second mixing weight;

[0084] The weighted first speech signal and the weighted second speech signal are superimposed to obtain a mixed speech signal.

[0085] In a specific implementation, if the mixed voice signal only includes the first voice signal and the second voice signal, the weight difference between the total weight and the first mixed weight can be obtained as the mixed weight corresponding to the second voice signal. Subsequently, the voice signals are weighted based on their corresponding mixed weights, and the weighted voice signals are superimposed to obtain a mixed voice signal. The mixed voice can reduce the noise distortion after enhancement. The higher the signal-to-noise ratio, the lower the noise component in the first voice signal. Therefore, the mixed weight of the second voice signal should be lower. Conversely, the higher the mixed weight of the second voice signal is required, so that the noise reduction effect is more obvious.

[0086] 104. Perform speech recognition on the mixed speech signal to obtain a recognition result.

[0087] In the embodiment of the present application, a deep model-based speech recognition method can be used to perform speech recognition on the mixed speech signal. Among them, the deep model-based speech recognition method is a method that uses deep learning technology to process and analyze speech signals and then realize speech-to-text conversion.

[0088] For example, during speech recognition of mixed speech, speech features, such as Mel-Spectrum Energy Features (MSEF), are first extracted from the mixed speech signal. These features are then matched to corresponding phonemes or words using a trained acoustic model. Furthermore, to improve the accuracy of speech recognition, a language model is used to predict the probability of word sequences—that is, the likelihood of a word sequence occurring in a specific context—to enhance recognition accuracy and language fluency. Finally, the outputs of the acoustic and language models are combined to generate the final text sequence.

[0089] In practical applications, the recognition results can also be post-processed, such as grammar checking, word correction, etc., to improve the quality of the recognized text. It is understood that the speech recognition results can be used for display, storage or further processing.

[0090] As can be seen from the above, the speech signal processing method provided in the embodiment of the present application performs signal enhancement on the first speech signal and performs weight prediction processing on the first speech signal and the second speech signal to obtain a first mixing weight corresponding to the first speech signal; then, the first speech signal is mixed with the second speech signal according to the first mixing weight, and speech recognition is performed on the mixed speech signal, and the recognition result is used as the recognition result of the first speech signal. In this solution, the most suitable mixing weight when the first speech signal is mixed is determined by introducing weight prediction, and the first speech signal and the second speech signal are mixed for speech recognition, that is, the enhanced speech signal is superimposed on the original speech, so that the effective signal can be more easily perceived during speech recognition; in addition, the original signal and the second speech signal are mixed in a certain proportion, rather than directly performing speech recognition on the second speech signal, which reduces the influence of nonlinear noise in the second speech signal due to speech enhancement, thereby improving the accuracy of the speech recognition result.

[0091] In practical applications, to further improve the accuracy of signal-to-noise ratio prediction, multiple speech enhancement models with different model structures can be used to enhance the first speech signal. That is, in one embodiment, when enhancing the first speech signal, the first speech signal can be enhanced based on multiple speech enhancement models with different model structures to obtain multiple second speech signals.

[0092] Since speech enhancement models with different model structures have their specific advantages and application scenarios, in the embodiments of the present application, multiple speech enhancement models with different model structures will be introduced to process the first speech signal to adapt to speech signals in different scenarios or with different signal characteristics. For example, a model based on a dual-path recurrent neural network, which combines a convolutional neural network and a long short-term memory network, is suitable for processing long-sequence speech data; a deep neural network is suitable for speech enhancement tasks, but may face the problem of too many parameters when processing long-sequence data; a convolutional neural network can accurately obtain the local features of the input speech signal and is suitable for restoring the high-frequency components of the speech signal; a long short-term memory network is suitable for processing sequence data with time dependence, etc.

[0093] That is, in one embodiment, there are multiple second speech signals; each second speech signal is obtained by enhancing the first speech signal through a speech enhancement model, and the model structure of each speech enhancement model is different; then, when determining the second mixing weight corresponding to the second speech signal based on the total weight and the first mixing weight, the following process may be included:

[0094] determining a weight difference between the total weight and the first blending weight;

[0095] determining a degree of match between the first speech signal and each speech enhancement model;

[0096] A weight is assigned to each second speech signal according to the weight difference and the matching degree to obtain a second mixing weight corresponding to each second speech signal.

[0097] Specifically, the weight difference between the total weight and the first mixed weight is obtained and assigned to the multiple second speech signals to obtain a more optimized mixed speech signal. In specific implementation, the weights can be assigned based on the degree of match between the first speech signal and each speech enhancement model, with a higher degree of match being assigned a higher weight, and a lower degree of match being assigned a lower weight.

[0098] Ultimately, the enhanced speech signal and the first speech signal are blended using the first and second blending weights to generate an output signal that comprehensively considers the strengths of each model. This signal preserves the content and characteristics of the original speech while also reducing noise and distortion through enhancement processing, improving speech clarity and auditory comfort. This approach leverages the strengths of different models to achieve superior speech enhancement results.

[0099] In one embodiment, when determining the matching degree between the first speech signal and each speech enhancement model, the following process may be included:

[0100] Performing signal analysis on the first speech signal to obtain a signal analysis result;

[0101] Get the description information corresponding to each speech enhancement model;

[0102] The signal analysis result is matched with the description information corresponding to each speech enhancement model to obtain the matching degree between the first speech signal and each speech enhancement model.

[0103] Specifically, in-depth signal analysis of the speech signal is a crucial first step. This process involves using various signal processing techniques to extract key features of the speech signal, such as Mel-Frequency Cepstral Coefficients (MFCCs), pitch, rhythm, intonation, and other acoustic characteristics. These features reveal the intrinsic properties of the speech signal and provide essential information for subsequent enhancement processing.

[0104] During specific implementation, descriptive information of each speech enhancement model can be obtained. The descriptive information may include the type of model, the optimization target (such as a specific type of background noise), and the model's performance on different speech features. For example, some models may be particularly good at processing low-frequency noise, while other models may perform better in maintaining the naturalness of speech. Afterwards, the signal analysis results are matched with the descriptive information of each speech enhancement model to calculate the degree of matching. The degree of matching reflects the consistency between the signal features and the model characteristics. For example, if the speech signal to be processed has obvious low-frequency noise characteristics, then the degree of matching with the enhancement model specially designed to suppress this type of noise will be higher.

[0105] Ultimately, this matching information is used to guide the selection and fusion strategy of the second speech signal. In this way, the system can intelligently select the most appropriate one or more second speech signals to mix with the first speech signal, which not only improves the quality of the mixed speech signal but also increases the flexibility and adaptability of the speech enhancement system.

[0106] In practical applications, in order to further improve the application effect of the weight prediction model in actual scenarios, the model parameters can be fine-tuned based on user feedback to improve the adaptability of the model to the scenario. That is, in one embodiment, after performing speech recognition on the mixed speech signal and obtaining the recognition result, the following process can also be included:

[0107] In response to receiving feedback information regarding the speech recognition result, performing content analysis on the feedback information to obtain a content analysis result;

[0108] When the analysis results meet the preset conditions, the model parameters of the weight prediction model are adjusted based on the content analysis results.

[0109] Feedback information may include corrections, evaluations, or other annotations to the recognition results. For example, after the speech signal processing system provides a recognition result, the user may provide corresponding feedback on the recognition result. If the recognition result matches the user input, the user may confirm the recognition result through the interactive system or rate the matching degree of the recognition result through the evaluation system, such as if the user inputs a matching degree of 100%.

[0110] It is understandable that due to the lack of corpus or the weak generalization ability of the weight prediction model, inaccurate recognition results may occur. In this case, the user can provide corresponding feedback on the recognition results, such as correcting or instructing the speech signal processing system to re-perform speech recognition (that is, when the analysis results meet the preset conditions). At this time, the model parameters of the weight prediction model can be adjusted based on the content analysis results. For example, the user's feedback indicates that the speech recognition system has errors in certain words or phrases. The system will adjust the parameters of the language model or acoustic model based on these feedbacks. For example, if "parking lot" is often misrecognized, the probability of phonemes or words related to "parking lot" can be increased to adjust the model parameters of the weight prediction model. After the adjustment is completed, speech recognition is performed based on the adjusted weight prediction model.

[0111] According to the method described in the above embodiment, the following examples will be given to further illustrate the details. In another embodiment of the present application, a speech enhancement method adapted for speech recognition is provided. Figure 3 and Figure 4 , the speech enhancement method adapted for speech recognition provided in this embodiment is described in detail.

[0112] This embodiment proposes a weight prediction model. The introduction of this model achieves the effect of not needing to retrain the speech enhancement model or the speech recognition model, and also not needing to jointly train the two models. Figure 3 The weighted prediction model takes the noisy speech signal (i.e., the first speech signal) and the enhanced speech signal (i.e., the second speech signal) as input, predicts the signal-to-noise ratio of the noisy speech, and outputs a mixing weight based on the signal-to-noise ratio. The enhanced and noisy speech are then weighted according to the mixing weight and superimposed to produce a mixed speech. This mixed speech is then input into the speech recognition model to obtain the final recognition result. The specific model training method is as follows:

[0113] (1) Data Preparation: Assuming the speech signal sampling rate (SampleRate) is 16kHz, obtain clean speech and noise, and randomly select the signal-to-noise ratio (SNR) of the mixed noise in the range of 0-30dB to mix the noise to obtain noisy speech. Using the noisy speech and clean speech, calculate the frame-level signal-to-noise ratio (SNR_frame) with a frame length of 1s and a frame shift of 0.5s. Input the noisy speech into the trained speech enhancement model to obtain the enhanced speech.

[0114] (2) Model training: The model uses noisy speech and enhanced speech as training inputs. Assume that the dimensions of both signals are N*1*T, where N represents the batch size (batch_size) and T represents the speech duration. The two are concatenated in the channel dimension to obtain the network input with a dimension of N*2*T. If the convolution downsampling rate is DownSample=16, the dimension of the network output Predict is N*(T / DownSample) and the dimension of the output label is N*(T / SampleRate / h). The frame-level dimension of the label is repeated to obtain the dimension of the output label of N*(T / DownSample). Then, the minimum mean square error is used as the loss function for gradient update. The loss function is as follows:

[0115]

[0116] Among them, N represents the number of training samples (i.e. batch size), i represents the sample number, and y i represents the label of training sample i (i.e., the true signal-to-noise ratio), represents the predicted signal-to-noise ratio of training sample i.

[0117] refer to Figure 4 The model fuses two input data (i.e., the sample noisy speech signal and the sample enhanced speech signal) through the convolution layer, then obtains the hidden layer features through the linear layer, and finally uses the mapping layer to map the signal-to-noise ratio (SNR) to a fixed decimal in the range of 0.5-1.0 as the mixing weight W. The mapping formula is as follows:

[0118] W=0.5+0.5*SNR / 30

[0119] Specifically, the model predicts the signal-to-noise ratio (SNR) and maps the predicted SNR to generate hybrid weights. A lower SNR corresponds to a higher weight for the enhanced speech signal; a higher SNR corresponds to a higher weight for the noisy speech signal. This ensures that the noise component of the second speech signal is reduced while also minimizing the distortion noise introduced by the second speech signal, thus achieving a variable noise reduction strategy for ASR in different noise scenarios.

[0120] In specific implementation, the noisy speech signal (i.e., the first speech signal) is processed by a speech enhancement model to obtain an enhanced speech signal (i.e., the second speech signal); then, the noisy speech signal and the enhanced speech signal are input into a fusion ratio prediction model to obtain a mixing weight; then, the mixed speech is obtained based on the mixing weight; finally, the mixed speech is input into a speech recognition model to obtain the final recognition result.

[0121] It can be seen that this embodiment uses the fusion ratio prediction model to predict the fusion weight, which solves the adaptation problem of the speech enhancement model and the speech recognition model and avoids the complex scheme of joint training of the two. In addition, through the learnable hybrid weight, the enhanced speech and the noisy speech are fused according to different signal-to-noise ratios, which can achieve a more accurate recognition effect.

[0122] refer to Figure 5 The speech signal processing method provided in the embodiments of the present application can be applied to any scenario that requires speech recognition, and the method can be applied to products in these scenarios, such as vehicle-mounted systems, intelligent voice systems, industrial control systems, smart home systems, etc.

[0123] For example, taking the in-vehicle system as an example, the driver needs to set up navigation, play music or answer calls while driving, but the noise inside the vehicle (such as engine noise, wind noise, and passenger conversations) may affect the accuracy of voice recognition. Using the voice signal processing method provided in the embodiment of the present application, the user voice signal can be collected by a microphone, and then the user voice signal is enhanced, and the user voice signal and the second voice signal are input into a weight prediction model together, and the weight prediction model outputs the mixing ratio of the user voice signal, and the user voice signal is mixed with the second voice signal according to the mixing ratio (i.e. W, 1-W). Finally, the mixed voice signal is input into the voice recognition model, and the corresponding voice control instructions (such as instruction 1, instruction 2, instruction 3) are triggered based on the voice recognition results. By applying the voice signal processing method provided in the present embodiment in the in-vehicle system, the driver's voice instructions can be clearly captured even in a noisy in-car environment, and these instructions are converted into corresponding vehicle control operations.

[0124] In order to better implement the above method, an embodiment of the present application further provides a voice signal processing device, which can be integrated into an electronic device.

[0125] For example, Figure 6 As shown, the speech signal processing device may include: a speech enhancement unit 302, a processing unit 301, a mixing unit 302 and a speech recognition unit 303, as follows:

[0126] The speech enhancement unit 301 is configured to perform signal enhancement on the first speech signal to obtain a second speech signal;

[0127] The processing unit 302 is configured to perform weight prediction processing based on the first speech signal and the second speech signal to obtain a first mixing weight corresponding to the first speech signal;

[0128] a mixing unit 303, configured to mix the first speech signal and the second speech signal according to a first mixing weight to obtain a mixed speech signal;

[0129] The speech recognition unit 304 is configured to perform speech recognition on the mixed speech signal to obtain a recognition result.

[0130] Optionally, in some implementations, when performing weight prediction processing based on the first speech signal and the second speech signal to obtain a first mixing weight corresponding to the first speech signal, the processing unit 302 is further configured to:

[0131] A weight prediction model is obtained, which includes a signal-to-noise ratio prediction network and a weight mapping network; based on the signal-to-noise ratio prediction network, a first speech signal and a second speech signal are processed to obtain a signal-to-noise ratio of the first speech signal; and the signal-to-noise ratio is processed according to the weight mapping network to obtain a first mixing weight.

[0132] Optionally, in some embodiments, the device further comprises:

[0133] A first construction unit is configured to construct a training set, the training set including training samples and sample labels, the training samples including a first speech sample and a second speech sample, the second speech sample being obtained by performing signal enhancement on the first speech sample, and the sample label representing a sample signal-to-noise ratio of the first speech sample;

[0134] An acquisition unit, configured to acquire a basic prediction model, the basic prediction model comprising: a basic prediction network and a preset weight mapping network;

[0135] a prediction unit, configured to take the first speech sample and the second speech sample as inputs of the basic prediction network and output a predicted signal-to-noise ratio of the first speech sample;

[0136] a training unit, configured to train the basic prediction network according to a difference between the predicted signal-to-noise ratio and the sample signal-to-noise ratio, to obtain a trained signal-to-noise ratio prediction network;

[0137] The second construction unit is used to construct a weight prediction model based on the trained signal-to-noise ratio prediction network and the preset weight mapping network.

[0138] Optionally, in some embodiments, when processing the signal-to-noise ratio according to the weight mapping network to obtain the first mixing weight, the processing unit 302 is further configured to:

[0139] Determine the value interval corresponding to the sample signal-to-noise ratio; obtain the ratio between the signal-to-noise ratio and the size of the value interval through the weight mapping network; weight the floating weight according to the ratio, and perform a preset operation on the basic weight and the weighted floating weight to obtain a first mixed weight.

[0140] Optionally, in some embodiments, the device may further include:

[0141] a determination unit, configured to determine whether a value of the signal-to-noise ratio of the first speech signal is greater than an upper limit of the value interval;

[0142] The updating unit is configured to update the value of the signal-to-noise ratio to the upper limit value when the judging unit determines that the signal-to-noise ratio is yes.

[0143] Optionally, in some embodiments, the device may further include:

[0144] an analyzing unit, configured to, in response to receiving feedback information regarding the speech recognition result, perform content analysis on the feedback information to obtain a content analysis result;

[0145] An adjustment unit is used to adjust the model parameters of the weight prediction model based on the content analysis result when the analysis result meets the preset conditions.

[0146] Optionally, in some embodiments, the mixing unit 303 is further configured to:

[0147] Based on the total weight and the first mixing weight, a second mixing weight corresponding to the second voice signal is determined; the first voice signal is weighted based on the first mixing weight, and the second voice signal is weighted based on the second mixing weight; the weighted first voice signal and the weighted second voice signal are superimposed to obtain a mixed voice signal.

[0148] Optionally, in some embodiments, there are multiple second speech signals; each second speech signal is obtained by enhancing the first speech signal through a speech enhancement model, and the model structure of each speech enhancement model is different; when determining the second mixing weight corresponding to the second speech signal based on the total weight and the first mixing weight, the mixing unit 303 is further used to:

[0149] Determine the weight difference between the total weight and the first mixed weight; determine the matching degree between the first speech signal and each speech enhancement model; assign a weight to each second speech signal based on the weight difference and the matching degree to obtain a second mixed weight corresponding to each second speech signal.

[0150] Optionally, in some implementations, when determining the matching degree between the first speech signal and each speech enhancement model, the mixing unit 303 is further configured to:

[0151] Perform signal analysis on the first speech signal to obtain a signal analysis result; obtain descriptive information corresponding to each speech enhancement model; match the signal analysis result with the descriptive information corresponding to each speech enhancement model to obtain a matching degree between the first speech signal and each speech enhancement model.

[0152] As can be seen from the above, the speech signal processing device provided in the embodiment of the present application performs signal enhancement on the first speech signal and performs weight prediction processing on the first speech signal and the second speech signal to obtain a first mixing weight corresponding to the first speech signal; the first speech signal and the second speech signal are mixed according to the first mixing weight, and speech recognition is performed on the mixed speech signal, and the recognition result is used as the recognition result of the first speech signal. In this solution, the most suitable mixing weight when the first speech signal is mixed is determined by introducing weight prediction, and the first speech signal and the second speech signal are mixed for speech recognition, that is, the enhanced speech signal is superimposed on the original speech, so that the effective signal can be more easily perceived during speech recognition; in addition, the original signal and the second speech signal are mixed in a certain proportion, rather than directly performing speech recognition on the second speech signal, which reduces the influence of nonlinear noise in the second speech signal caused by speech enhancement, thereby improving the accuracy of the speech recognition result.

[0153] The present application also provides an electronic device, such as Figure 7 , which shows a schematic diagram of the structure of the electronic device involved in the embodiment of the present application, specifically:

[0154] The electronic device may include one or more processing core processors 401, one or more computer-readable storage media memories 402, a power supply 403, an input unit 404 and other components. Those skilled in the art will understand that Figure 7 The electronic device structure shown in the figure does not constitute a limitation of the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently.

[0155] Processor 401 is the control center of the electronic device. It connects all parts of the electronic device using various interfaces and circuits. It performs various functions of the electronic device and processes data by running or executing software programs and / or modules stored in memory 402 and accessing data stored in memory 402. Optionally, processor 401 may include one or more processing cores. Preferably, processor 401 may integrate an application processor and a modem processor. The application processor primarily processes the operating system, user interface, and computer programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 401.

[0156] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and scheduling by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, a computer program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0157] The electronic device also includes a power supply 403 for supplying power to various components. Preferably, the power supply 403 can be logically connected to the processor 401 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply 403 can also include one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components.

[0158] The electronic device may further include an input unit 404, which may be configured to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0159] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail here. It should be noted that the electronic device in this embodiment may specifically be the first server, that is, the server where the SSO authentication center is located. Specifically in this embodiment, the processor 401 in the electronic device will load the executable files corresponding to the processes of one or more computer programs into the memory 402 according to the following instructions, and the processor 401 will run the computer program stored in the memory 402 to implement various functions, as follows:

[0160] The first speech signal is enhanced to obtain a second speech signal; weight prediction processing is performed based on the first speech signal and the second speech signal to obtain a first mixing weight corresponding to the first speech signal; the first speech signal and the second speech signal are mixed according to the first mixing weight to obtain a mixed speech signal; speech recognition is performed on the mixed speech signal to obtain a recognition result.

[0161] In one embodiment, when performing weight prediction processing based on the first speech signal and the second speech signal to obtain a first mixing weight corresponding to the first speech signal, the processor 401 may be specifically configured to:

[0162] Obtaining a weight prediction model, the weight prediction model including a signal-to-noise ratio prediction network and a weight mapping network;

[0163] Based on the signal-to-noise ratio prediction network, the first speech signal and the second speech signal are processed to obtain a signal-to-noise ratio of the first speech signal;

[0164] The signal-to-noise ratio is processed according to the weight mapping network to obtain a first mixing weight.

[0165] In one embodiment, before acquiring the first voice signal, the processor 401 may further be configured to:

[0166] A training set is constructed, where the training set includes training samples and sample labels, where the training samples include a first speech sample and a second speech sample, where the second speech sample is obtained by performing signal enhancement on the first speech sample, and the sample label represents a sample signal-to-noise ratio of the first speech sample; a basic prediction model is obtained, where the basic prediction model includes: a basic prediction network and a preset weight mapping network; the first speech sample and the second speech sample are used as inputs of the basic prediction network, and a predicted signal-to-noise ratio of the first speech sample is output; the basic prediction network is trained based on a difference between the predicted signal-to-noise ratio and the sample signal-to-noise ratio to obtain a trained signal-to-noise ratio prediction network; and a weight prediction model is constructed based on the trained signal-to-noise ratio prediction network and the preset weight mapping network.

[0167] In one embodiment, when processing the signal-to-noise ratio according to the weight mapping network to obtain the first mixing weight, the processor 401 may be specifically configured to:

[0168] Determine the value interval corresponding to the sample signal-to-noise ratio; obtain the ratio between the signal-to-noise ratio and the size of the value interval through the weight mapping network; weight the floating weight according to the ratio, and perform a preset operation on the basic weight and the weighted floating weight to obtain a first mixed weight.

[0169] In one embodiment, before processing the signal-to-noise ratio according to the weight mapping network to obtain the first mixing weight, the processor 401 may further be configured to:

[0170] Determine whether the signal-to-noise ratio of the first speech signal is greater than an upper limit of the value interval; if so, update the signal-to-noise ratio to the upper limit.

[0171] In one embodiment, after performing speech recognition on the mixed speech signal and obtaining a recognition result, the processor 401 may further be configured to:

[0172] In response to receiving feedback information regarding the speech recognition result, content analysis is performed on the feedback information to obtain a content analysis result; when the analysis result meets a preset condition, model parameters of the weight prediction model are adjusted based on the content analysis result.

[0173] In one embodiment, when mixing the first speech signal and the second speech signal according to the first mixing weight to obtain a mixed speech signal, the processor 401 may be specifically configured to:

[0174] Based on the total weight and the first mixing weight, a second mixing weight corresponding to the second voice signal is determined; the first voice signal is weighted based on the first mixing weight, and the second voice signal is weighted based on the second mixing weight; the weighted first voice signal and the weighted second voice signal are superimposed to obtain a mixed voice signal.

[0175] In one embodiment, there are multiple second speech signals; each second speech signal is obtained by enhancing the first speech signal through a speech enhancement model, and the model structure of each speech enhancement model is different; when performing signal enhancement on the first speech signal to obtain the second speech signal, the processor 401 can specifically be used to:

[0176] Determine the weight difference between the total weight and the first mixed weight; determine the matching degree between the first speech signal and each speech enhancement model; assign a weight to each second speech signal based on the weight difference and the matching degree to obtain a second mixed weight corresponding to each second speech signal.

[0177] In one embodiment, when determining the matching degree between the first speech signal and each speech enhancement model, the processor 401 may be specifically configured to:

[0178] Perform signal analysis on the first speech signal to obtain a signal analysis result; obtain descriptive information corresponding to each speech enhancement model; match the signal analysis result with the descriptive information corresponding to each speech enhancement model to obtain a matching degree between the first speech signal and each speech enhancement model.

[0179] The specific implementation of the above operations can be found in the previous embodiments and will not be described in detail here.

[0180] From the above, it can be seen that in the embodiment of the present application, the electronic device performs signal enhancement on the first voice signal and performs weight prediction processing on the first voice signal and the second voice signal to obtain a first mixing weight corresponding to the first voice signal; then, the first voice signal is mixed with the second voice signal according to the first mixing weight, and voice recognition is performed on the mixed voice signal, and the recognition result is used as the recognition result of the first voice signal. In this solution, the most suitable mixing weight when the first voice signal is mixed is determined by introducing weight prediction, and the first voice signal and the second voice signal are mixed for voice recognition, that is, the enhanced voice signal is superimposed on the original voice, so that the effective signal can be more easily perceived during voice recognition; in addition, the original signal and the second voice signal are mixed in a certain proportion, rather than directly performing voice recognition on the second voice signal, which reduces the influence of nonlinear noise in the second voice signal due to voice enhancement, thereby improving the accuracy of the voice recognition result.

[0181] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0182] To this end, an embodiment of the present application provides a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute the steps of any of the speech signal processing methods provided in the embodiments of the present application. For example, the instructions can execute the following steps:

[0183] The first speech signal is enhanced to obtain a second speech signal; weight prediction processing is performed based on the first speech signal and the second speech signal to obtain a first mixing weight corresponding to the first speech signal; the first speech signal and the second speech signal are mixed according to the first mixing weight to obtain a mixed speech signal; speech recognition is performed on the mixed speech signal to obtain a recognition result.

[0184] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0185] The computer-readable storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0186] Since the instructions stored in the computer-readable storage medium can execute the steps in any of the speech signal processing methods provided in the embodiments of the present application, the beneficial effects that can be achieved by any of the speech signal processing methods provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0187] According to one aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in various optional implementations of the scheduling aspect described above.

[0188] The above is a detailed introduction to a speech signal processing method, device and electronic device provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core ideas. At the same time, for technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A speech signal processing method, characterized in that: include: Performing signal enhancement on the first speech signal to obtain a second speech signal; Performing weight prediction processing based on the first speech signal and the second speech signal to obtain a first mixing weight corresponding to the first speech signal; mixing the first speech signal and the second speech signal according to the first mixing weight to obtain a mixed speech signal; Perform speech recognition on the mixed speech signal to obtain a recognition result.

2. The speech signal processing method according to claim 1, wherein: The performing weight prediction processing based on the first speech signal and the second speech signal to obtain a first mixing weight corresponding to the first speech signal includes: Processing the first speech signal and the second speech signal based on a signal-to-noise ratio prediction network in a weighted prediction model to obtain a signal-to-noise ratio of the first speech signal; The signal-to-noise ratio is processed according to a weight mapping network in the weight prediction model to obtain a first mixing weight.

3. The speech signal processing method according to claim 2, wherein: The method further comprises: Constructing a training set, the training set including training samples and sample labels, the training samples including a first speech sample and a second speech sample, the second speech sample being obtained by performing signal enhancement on the first speech sample, and the sample label representing a sample signal-to-noise ratio of the first speech sample; Taking the first speech sample and the second speech sample as inputs of a basic prediction network, and outputting a predicted signal-to-noise ratio of the first speech sample; Training the basic prediction network according to the difference between the predicted signal-to-noise ratio and the sample signal-to-noise ratio to obtain a signal-to-noise ratio prediction network; Based on the trained signal-to-noise ratio prediction network and the preset weight mapping network, a weight prediction model is constructed.

4. The speech signal processing method according to claim 3, wherein: The processing of the signal-to-noise ratio according to the weight mapping network to obtain a first mixing weight includes: Determining a value interval corresponding to the sample signal-to-noise ratio; Obtaining, through the weight mapping network, a ratio between the signal-to-noise ratio and the size of the value interval; The floating weight is weighted according to the ratio, and a preset operation is performed on the basic weight and the weighted floating weight to obtain a first mixed weight.

5. The speech signal processing method according to claim 4, characterized in that: Before processing the signal-to-noise ratio according to the weight mapping network to obtain the first mixing weight, the method further includes: Determining whether a value of the signal-to-noise ratio of the first speech signal is greater than an upper limit of the value range; If so, the value of the signal-to-noise ratio is updated to the upper limit value.

6. The speech signal processing method according to any one of claims 1 to 5, characterized in that: The step of mixing the first speech signal with the second speech signal according to the first mixing weight to obtain a mixed speech signal includes: determining a second mixing weight corresponding to the second speech signal according to the total weight and the first mixing weight; Performing weighted processing on the first speech signal based on the first mixing weight, and performing weighted processing on the second speech signal based on the second mixing weight; The first speech signal after weighted processing and the second speech signal after weighted processing are superimposed to obtain a mixed speech signal.

7. The speech signal processing method according to claim 6, characterized in that: There are multiple second speech signals; each second speech signal is obtained by enhancing the first speech signal through a speech enhancement model, and the model structure of each speech enhancement model is different; The determining, according to the total weight and the first mixing weight, a second mixing weight corresponding to the second speech signal includes: Obtaining a weight difference between the total weight and the first mixed weight; Determining a degree of match between the first speech signal and each speech enhancement model; A weight is assigned to each second speech signal according to the weight difference and the matching degree to obtain a second mixing weight corresponding to each second speech signal.

8. The speech signal processing method according to claim 7, wherein: Determining a matching degree between the first speech signal and each speech enhancement model includes: performing signal analysis on the first speech signal to obtain a signal analysis result; Get the description information corresponding to each speech enhancement model; The signal analysis result is matched with the description information corresponding to each speech enhancement model to obtain a matching degree between the first speech signal and each speech enhancement model.

9. A speech signal processing device, characterized in that: include: a speech enhancement unit, configured to perform signal enhancement on the first speech signal to obtain a second speech signal; a processing unit, configured to perform weight prediction processing based on the first speech signal and the second speech signal to obtain a first mixing weight corresponding to the first speech signal; a mixing unit, configured to mix the first speech signal with the second speech signal according to the first mixing weight to obtain a mixed speech signal; The speech recognition unit is used to perform speech recognition on the mixed speech signal to obtain a recognition result.

10. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the steps in the speech signal processing method according to any one of claims 1 to 8.

Images