Intelligent audio noise reduction method and system for voice communication

Abstract

The invention discloses an intelligent audio noise reduction method and system for voice communication. The method comprises the following steps: firstly, extracting a voice signal to be processed and receiving an environment noise signal collected by a first sound sensor; performing signal fragment waveform feature extraction on the to-be-processed voice signal and the environmental noise signal to obtain a sequence of voice signal fragment waveform semantic feature vectors and a sequence of environmental noise signal fragment waveform semantic feature vectors; a semantic noise reduction module is used for processing the sequence of the waveform semantic feature vectors of the voice signal segments and the sequence of the waveform semantic feature vectors of the environmental noise signal segments so as to obtain the sequence of the waveform feature vectors of the voice signal segments after environmental noise is filtered out; and performing inter-signal fragment correlation analysis and signal restoration on the sequence of the waveform feature vectors of the speech signal fragments after the environmental noise is filtered out to obtain a speech signal after noise reduction.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent audio noise reduction, and in particular to an intelligent audio noise reduction method and system for voice calls. Background Art

[0002] With the popularity of mobile communications and remote work, voice calls have become an indispensable part of people's daily life and work. However, in actual calls, environmental noise often affects the call quality, leading to communication difficulties and unclear information transmission.

[0003] Traditional noise reduction methods often use fixed filters to filter and reduce noise on voice signals during calls. This noise reduction method has limited effect and cannot adapt to complex and changing environments. Therefore, an optimized intelligent audio noise reduction method and system for voice calls is expected. Summary of the invention

[0004] The present disclosure is made in view of the above problems. One object of the present disclosure is to provide an intelligent audio noise reduction method and system for voice calls.

[0005] An embodiment of the present disclosure provides an intelligent audio noise reduction method for voice calls, which includes:

[0006] Extracting a speech signal to be processed;

[0007] Receiving an environmental noise signal collected by a first sound sensor;

[0008] Performing signal segment waveform feature extraction on the speech signal to be processed and the ambient noise signal to obtain a sequence of speech signal segment waveform semantic feature vectors and a sequence of ambient noise signal segment waveform semantic feature vectors;

[0009] Using a semantic noise reduction module, the sequence of semantic feature vectors of the speech signal segment waveform and the sequence of semantic feature vectors of the environmental noise signal segment waveform are processed to obtain a sequence of feature vectors of the speech signal segment waveform after filtering out the environmental noise; and

[0010] The sequence of waveform feature vectors of the speech signal segments after the environmental noise is filtered out is subjected to signal segment correlation analysis and signal restoration to obtain the denoised speech signal.

[0011] For example, according to an intelligent audio noise reduction method for voice calls according to an embodiment of the present disclosure, signal segment waveform feature extraction is performed on the voice signal to be processed and the ambient noise signal to obtain a sequence of speech signal segment waveform semantic feature vectors and a sequence of ambient noise signal segment waveform semantic feature vectors, including:

[0012] Performing signal preprocessing on the speech signal to be processed and the ambient noise signal to obtain a sequence of speech signal segments and a sequence of ambient noise signal segments; and

[0013] A deep learning network model is used to extract waveform features of the sequence of speech signal segments and the sequence of environmental noise signal segments respectively to obtain a sequence of waveform semantic feature vectors of the speech signal segments and a sequence of waveform semantic feature vectors of the environmental noise signal segments.

[0014] For example, according to an intelligent audio noise reduction method for voice calls according to an embodiment of the present disclosure, signal preprocessing is performed on the voice signal to be processed and the ambient noise signal to obtain a sequence of voice signal segments and a sequence of ambient noise signal segments, including:

[0015] Performing time alignment on the speech signal to be processed and the ambient noise signal based on the timestamp to obtain an aligned speech signal to be processed and an aligned ambient noise signal; and

[0016] Signal segmentation is performed on the aligned speech signal to be processed and the aligned ambient noise signal based on a predetermined time scale to obtain a sequence of the speech signal segments and a sequence of the ambient noise signal segments.

[0017] For example, according to an intelligent audio noise reduction method for voice calls according to an embodiment of the present disclosure, the deep learning network model is used to extract waveform features of the sequence of voice signal segments and the sequence of environmental noise signal segments to obtain a sequence of waveform semantic feature vectors of the voice signal segments and a sequence of waveform semantic feature vectors of the environmental noise signal segments, including:

[0018] The sequence of speech signal segments and the sequence of environmental noise signal segments are respectively passed through a signal fluctuation feature extractor based on a convolutional neural network model to obtain a sequence of waveform semantic feature vectors of the speech signal segments and a sequence of waveform semantic feature vectors of the environmental noise signal segments.

[0019] For example, according to an intelligent audio noise reduction method for voice calls according to an embodiment of the present disclosure, a semantic noise reduction module is used to process a sequence of semantic feature vectors of the speech signal segment waveform and a sequence of semantic feature vectors of the ambient noise signal segment waveform to obtain a sequence of speech signal segment waveform feature vectors after filtering out ambient noise, including:

[0020] The sequence of the waveform semantic feature vectors of the speech signal segments and the sequence of the waveform semantic feature vectors of the environmental noise signal segments are processed by the following semantic noise reduction formula to obtain the sequence of the waveform feature vectors of the speech signal segments after filtering out the environmental noise;

[0021] The semantic denoising formula is:

[0022]

[0023] Among them, ω i is the i-th waveform feature vector of the speech signal segment after the environmental noise is filtered out in the sequence of waveform feature vectors of the speech signal segment after the environmental noise is filtered out, is the i-th speech signal segment waveform semantic feature vector in the sequence of speech signal segment waveform semantic feature vectors, is the i-th environmental noise signal segment waveform semantic feature vector in the sequence of environmental noise signal segment waveform semantic feature vectors, represents the square of the L2 norm of the eigenvector, and - represents the element-wise subtraction process.

[0024] For example, according to an intelligent audio noise reduction method for voice calls according to an embodiment of the present disclosure, the sequence of waveform feature vectors of the voice signal segments after filtering out the ambient noise is subjected to signal segment correlation analysis and signal restoration to obtain the noise-reduced voice signal, including:

[0025] Passing the sequence of waveform feature vectors of the speech signal segments after the environmental noise is filtered out through a semantic association encoder between semantic signal segments based on a Transformer layer to obtain a sequence of waveform feature vectors of the speech signal segments after the environmental noise is filtered out in the context; and

[0026] The sequence of waveform feature vectors of the speech signal segment after the context filtering out the environmental noise is decoded and restored to obtain the denoised speech signal.

[0027] For example, according to an intelligent audio noise reduction method for voice calls according to an embodiment of the present disclosure, the sequence of waveform feature vectors of the speech signal segment after filtering out the ambient noise is passed through a semantic association encoder between semantic signal segments based on a Transformer layer to obtain a sequence of waveform feature vectors of the speech signal segment after the ambient noise is filtered out from the context, including:

[0028] Arranging the sequence of waveform feature vectors of the speech signal segments after filtering out the environmental noise in one dimension to obtain a global waveform feature vector of the speech signal after filtering out the environmental noise;

[0029] Calculating the product between the global waveform feature vector of the speech signal after filtering out the environmental noise and the transposed vector of each waveform feature vector of the speech signal segment after filtering out the environmental noise in the sequence of the waveform feature vectors of the speech signal segment after filtering out the environmental noise to obtain multiple self-attention association matrices;

[0030] Normalizing each of the plurality of self-attention association matrices to obtain a plurality of standardized self-attention association matrices;

[0031] Passing each of the plurality of standardized self-attention association matrices through a Softmax classification function to obtain a plurality of probability values; and

[0032] Each probability value in the multiple probability values ​​is used as a weight to weight each waveform feature vector of the speech signal segment after the environmental noise is filtered out in the sequence of waveform feature vectors of the speech signal segment after the environmental noise is filtered out to obtain the sequence of waveform feature vectors of the speech signal segment after the environmental noise is filtered out in the context.

[0033] For example, according to an intelligent audio noise reduction method for voice calls according to an embodiment of the present disclosure, decoding and restoring a sequence of waveform feature vectors of speech signal segments after the context filtering out environmental noise to obtain the noise-reduced speech signal includes:

[0034] The sequence of waveform feature vectors of the speech signal segments after the context filtering and the environmental noise is removed is passed through a decoder-based signal restorer to obtain the denoised speech signal.

[0035] The embodiment of the present disclosure also provides an intelligent audio noise reduction system for voice calls, which includes:

[0036] A speech signal extraction module to be processed, used to extract the speech signal to be processed;

[0037] An environmental noise signal receiving module, used to receive the environmental noise signal collected by the first sound sensor;

[0038] A waveform feature extraction module, used for performing signal segment waveform feature extraction on the speech signal to be processed and the ambient noise signal to obtain a sequence of speech signal segment waveform semantic feature vectors and a sequence of ambient noise signal segment waveform semantic feature vectors;

[0039] A filtering module, configured to process the sequence of the speech signal segment waveform semantic feature vectors and the sequence of the environmental noise signal segment waveform semantic feature vectors using a semantic noise reduction module to obtain a sequence of the speech signal segment waveform feature vectors after filtering out the environmental noise; and

[0040] The analysis and restoration module is used to perform signal segment correlation analysis and signal restoration on the sequence of waveform feature vectors of the speech signal segments after the environmental noise is filtered out to obtain the denoised speech signal.

[0041] According to the intelligent audio noise reduction method and system for voice calls according to the embodiments of the present disclosure, a sound sensor is used to collect existing environmental noise signals, and an intelligent algorithm is used to extract the waveform implicit semantic features of the environmental noise signals to capture the noise characteristics of the environmental noise signals, thereby filtering out noise from the processed voice signals based on the noise characteristics of the environmental noise signals, and restoring and reconstructing the signals after noise filtering to achieve intelligent audio noise reduction during voice calls. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings of the embodiments of the present disclosure. Obviously, the drawings described below only relate to some embodiments of the present disclosure, and are not intended to limit the present disclosure.

[0043] Figure 1 A schematic diagram of the application architecture of an intelligent audio noise reduction method for voice calls in an embodiment of the present disclosure is shown;

[0044] Figure 2 A flow chart of an intelligent audio noise reduction method for voice calls in an embodiment of the present disclosure is shown;

[0045] Figure 3 A flowchart of sub-step S530 of the intelligent audio noise reduction method for voice calls in an embodiment of the present disclosure is shown;

[0046] Figure 4 A flowchart of sub-step S550 of the intelligent audio noise reduction method for voice calls in an embodiment of the present disclosure is shown;

[0047] Figure 5 A schematic diagram of the structure of an intelligent audio noise reduction system for voice calls in an embodiment of the present disclosure is shown;

[0048] Figure 6 An application scenario diagram of an intelligent audio noise reduction method for voice calls in an embodiment of the present disclosure is shown; and

[0049] Figure 7 A schematic diagram of a storage medium according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present disclosure will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present disclosure, rather than all embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work also fall within the scope of protection of the present disclosure.

[0051] The terms used in this specification are those general terms currently widely used in the art in consideration of the functions of the present disclosure, but these terms may vary according to the intention of a person of ordinary skill in the art, precedents, or new technologies in the art. In addition, specific terms may be selected by the applicant, and in this case, their detailed meanings will be described in the detailed description of the present disclosure. Therefore, the terms used in the specification should not be understood as simple names, but rather as an overall description based on the meaning of the terms and the present disclosure.

[0052] Although the present disclosure makes various references to certain modules in the system according to embodiments of the present disclosure, any number of different modules can be used and run on a user terminal and / or server. The modules are illustrative only, and different aspects of the systems and methods can use different modules.

[0053] Flowcharts are used in this disclosure to illustrate the operations performed by the system according to the embodiments of the present disclosure. It should be understood that the preceding or following operations are not necessarily performed precisely in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.

[0054] Figure 1 A schematic diagram of the application architecture of an intelligent audio noise reduction method for voice calls in an embodiment of the present disclosure is shown, including a server 100 and a terminal device 200.

[0055] The terminal device 200 and the server 100 can be connected via the Internet to achieve mutual communication. Optionally, the above-mentioned Internet uses standard communication technology and / or protocol. The Internet is usually the Internet, but it can also be any network, including but not limited to any combination of a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network or a virtual private network. In some embodiments, the data exchanged through the network is represented by technologies and / or formats including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above-mentioned data communication technologies.

[0056] The server 100 can provide various network services for the terminal device 200, wherein the server 100 can be a single server, a server cluster consisting of several servers, or a cloud computing center. Specifically, the server 100 may include a processor 110 (Center Processing Unit, CPU), a memory 120, an input device 130, and an output device 140, etc. The input device 130 may include a keyboard, a mouse, a touch screen, etc., and the output device 140 may include a display device, such as a liquid crystal display (Liquid Crystal Display, LCD), a cathode ray tube (Cathode Ray Tube, CRT), etc.

[0057] The memory 120 may include a read-only memory (ROM) and a random access memory (RAM), and provides the processor 110 with program instructions and data stored in the memory 120. In the embodiment of the present disclosure, the memory 120 may be used to store the program of the intelligent audio noise reduction method for voice calls in the embodiment of the present disclosure.

[0058] The processor 110 calls the program instructions stored in the memory 120, and the processor 110 is used to execute the steps of any one of the intelligent audio noise reduction methods for voice calls in the embodiments of the present disclosure according to the obtained program instructions.

[0059] In addition, the application architecture diagram in the embodiment of the present disclosure is intended to more clearly illustrate the technical solution in the embodiment of the present disclosure, and does not constitute a limitation on the technical solution provided in the embodiment of the present disclosure. Of course, for other application architectures and business applications, the technical solution provided in the embodiment of the present disclosure is also applicable to similar problems.

[0060] The following is a non-restrictive description of the intelligent audio noise reduction method for voice calls provided according to at least one embodiment of the present disclosure through several examples or embodiments. As described below, different features in these specific examples or embodiments can be combined with each other without conflicting with each other to obtain new examples or embodiments, and these new examples or embodiments also fall within the scope of protection of the present disclosure.

[0061] In response to this technical problem, the technical concept of the present application is: using sound sensors to collect existing environmental noise signals, combining intelligent algorithms to extract the waveform implicit semantic features of the environmental noise signals to capture the noise characteristics of the environmental noise signals, and then filtering out the noise of the processed voice signals based on the noise characteristics of the environmental noise signals, and restoring and reconstructing the signals after noise filtering to achieve intelligent audio noise reduction during voice calls.

[0062] Based on this, Figure 2 The flowchart of the intelligent audio noise reduction method for voice calls in the embodiment of the present disclosure is shown. For example, the intelligent audio noise reduction method for voice calls can be executed by a server, which can be Figure 1 The server 100 shown in FIG. Figure 2 As shown, the intelligent audio noise reduction method for voice calls according to the embodiment of the present disclosure includes the steps of: S510, extracting a voice signal to be processed; S520, receiving an ambient noise signal collected by a first sound sensor; S530, performing signal segment waveform feature extraction on the voice signal to be processed and the ambient noise signal to obtain a sequence of voice signal segment waveform semantic feature vectors and a sequence of ambient noise signal segment waveform semantic feature vectors; S540, using a semantic noise reduction module to process the sequence of voice signal segment waveform semantic feature vectors and the sequence of ambient noise signal segment waveform semantic feature vectors to obtain a sequence of voice signal segment waveform feature vectors after filtering out ambient noise; and, S550, performing signal segment correlation analysis and signal restoration on the sequence of voice signal segment waveform feature vectors after filtering out ambient noise to obtain a denoised voice signal.

[0063] Specifically, in the technical solution of the present application, first, extract the voice signal to be processed; and receive the environmental noise signal collected by the first sound sensor. Among them, the sound sensor is a device for detecting and measuring sound signals in the surrounding environment, which can convert the sound signal into an electrical signal for subsequent processing and analysis. Environmental noise signals refer to various noise signals generated in the environment, including sound signals generated by various noise sources such as traffic, machinery and equipment, and background human voices. These noise signals will interfere with normal voice call signals and affect call quality and clarity. In the technical solution of the present application, the purpose of receiving the environmental noise signal collected by the first sound sensor is to obtain a reference signal of environmental noise in the process of intelligent audio noise reduction. In this way, by collecting and analyzing the environmental noise signal, the noise characteristics in the current environment can be better understood, so as to carry out noise reduction processing on the voice signal to be processed in a targeted manner. It is worth mentioning that the deployment position of the first sound sensor should take into account the ability to effectively capture the noise signal of the surrounding environment, and usually choose a location near the call device or susceptible to environmental noise interference. In the actual application scenario of the present application, the method of deploying the first sound sensor should depend on the specific application scenario and system design requirements.

[0064] Considering that the voice signal to be processed and the ambient noise signal may be collected by different sensors, for example, in a specific example of the present application, the voice signal to be processed is recorded and collected by a microphone, and the ambient noise signal is recorded and collected by the first sound sensor, and the performance differences and acquisition methods between the sensor devices will lead to signal misalignment. It should be understood that if the voice signal to be processed and the ambient noise signal are subjected to noise reduction processing without alignment, it will affect the effect of noise reduction and even introduce more interference. In order to solve or alleviate this problem, in the technical solution of the present application, it is expected to time-align the voice signal to be processed and the ambient noise signal based on the timestamp to obtain the aligned voice signal to be processed and the aligned ambient noise signal. Here, the method of time alignment based on the timestamp can make the voice signal to be processed and the ambient noise signal consistent in time, and ensure that the two can be processed correspondingly at the same time point in the subsequent signal analysis and signal noise reduction process, without errors and offsets caused by the time mismatch between the signals, ensuring the time synchronization and consistency of the two.

[0065] Next, the aligned speech signal to be processed and the aligned ambient noise signal are segmented based on a predetermined time scale to obtain a sequence of speech signal segments and a sequence of ambient noise signal segments. That is, the aligned speech signal to be processed and the aligned ambient noise signal with a longer time span are divided into small segments by signal segmentation, so that the waveform characteristics of each segment can be more easily captured. Subsequently, the sequence of speech signal segments and the sequence of ambient noise signal segments are respectively passed through a signal fluctuation feature extractor based on a convolutional neural network model to obtain a sequence of speech signal segment waveform semantic feature vectors and a sequence of ambient noise signal segment waveform semantic feature vectors. Among them, a convolutional neural network (CNN) is a deep learning model, which is generally used to process data with a grid structure. Specifically, CNN generally includes multiple convolutional layers and pooling layers, which can effectively extract features from data and perform well in tasks such as classification and recognition. It should be understood that the sequence of ambient noise signal segments contains rich waveform information about noise characteristics, and the sequence of speech signal segments contains more complex speech waveform information. In the technical solution of the present application, the convolutional neural network model is used to construct the signal fluctuation feature extractor, so as to capture the local neighborhood implicit waveform correlation characteristics contained in the sequence of each of the speech signal segments and each of the environmental noise signal segments respectively by making use of the excellent performance of the convolutional neural network model in extracting abstract feature representations, thereby characterizing the waveform characteristics of the environmental noise and the waveform characteristics of the speech containing complex sound elements.

[0066] Accordingly, in step S530, if Figure 3 As shown, signal segment waveform feature extraction is performed on the speech signal to be processed and the ambient noise signal to obtain a sequence of speech signal segment waveform semantic feature vectors and a sequence of ambient noise signal segment waveform semantic feature vectors, including: S531, signal preprocessing is performed on the speech signal to be processed and the ambient noise signal to obtain a sequence of speech signal segments and a sequence of ambient noise signal segments; and, S532, waveform feature extraction is performed on the sequence of speech signal segments and the sequence of ambient noise signal segments respectively using a deep learning network model to obtain a sequence of speech signal segment waveform semantic feature vectors and a sequence of ambient noise signal segment waveform semantic feature vectors.

[0067] Among them, in step S531, the speech signal to be processed and the ambient noise signal are subjected to signal preprocessing to obtain a sequence of speech signal segments and a sequence of ambient noise signal segments, including: time-aligning the speech signal to be processed and the ambient noise signal based on a timestamp to obtain an aligned speech signal to be processed and an aligned ambient noise signal; and signal segmenting the aligned speech signal to be processed and the aligned ambient noise signal based on a predetermined time scale to obtain a sequence of speech signal segments and a sequence of ambient noise signal segments.

[0068] Wherein, in step S532, the deep learning network model is used to perform waveform feature extraction on the sequence of speech signal segments and the sequence of environmental noise signal segments respectively to obtain a sequence of waveform semantic feature vectors of the speech signal segments and a sequence of waveform semantic feature vectors of the environmental noise signal segments, including: passing the sequence of speech signal segments and the sequence of environmental noise signal segments through a signal fluctuation feature extractor based on a convolutional neural network model respectively to obtain a sequence of waveform semantic feature vectors of the speech signal segments and a sequence of waveform semantic feature vectors of the environmental noise signal segments.

[0069] Since both the speech signal to be processed and the environmental noise signal contain complex waveform features, for example, the speech waveform features in the speech signal to be processed are affected by factors such as the speaker's voice, speech speed and emotion, while the signal waveform features in the environmental noise signal are affected by various environmental factors (such as noise type, intensity, etc.), the relationship between these factors may be nonlinear. This results in the speech signal to be processed and the environmental noise signal itself having complex nonlinear features. At the same time, in actual application scenarios, there may be interweaving between the speech signal to be processed and the environmental noise signal, which further increases the complexity between the two. Therefore, if the corresponding differential processing is directly performed on the two, it may not be possible to effectively filter out the noise, and may even lead to the loss of a lot of effective information. In the technical solution of the present application, a semantic denoising module is further used to process the sequence of semantic feature vectors of the speech signal segment waveform and the sequence of semantic feature vectors of the environmental noise signal segment waveform to obtain a sequence of speech signal segment waveform feature vectors after filtering out the environmental noise. The semantic denoising module suppresses and filters noise by performing nonlinear differential interaction on the sequence of semantic feature vectors of the waveform of the speech signal segment and the sequence of semantic feature vectors of the waveform of the environmental noise signal segment, while retaining the key waveform feature information of the speech signal. In this way, compared with the traditional linear denoising method, it can better retain the details and features of the speech signal and avoid excessive processing of the signal to cause information loss.

[0070] Correspondingly, in step S540, the sequence of the semantic feature vectors of the speech signal segment waveform and the sequence of the semantic feature vectors of the ambient noise signal segment waveform are processed using a semantic denoising module to obtain a sequence of the speech signal segment waveform feature vectors after filtering out the ambient noise, including: processing the sequence of the semantic feature vectors of the speech signal segment waveform and the sequence of the semantic feature vectors of the ambient noise signal segment waveform using the following semantic denoising formula to obtain a sequence of the speech signal segment waveform feature vectors after filtering out the ambient noise; wherein the semantic denoising formula is:

[0071]

[0072] Among them, ω i is the i-th waveform feature vector of the speech signal segment after the environmental noise is filtered out in the sequence of waveform feature vectors of the speech signal segment after the environmental noise is filtered out, is the i-th speech signal segment waveform semantic feature vector in the sequence of speech signal segment waveform semantic feature vectors, is the i-th environmental noise signal segment waveform semantic feature vector in the sequence of environmental noise signal segment waveform semantic feature vectors, represents the square of the L2 norm of the eigenvector, and - represents the element-wise subtraction process.

[0073] Then, the sequence of waveform feature vectors of the speech signal segments after the environmental noise is filtered out is passed through a semantic association encoder between semantic signal segments based on the Transformer layer to obtain a sequence of waveform feature vectors of the speech signal segments after the environmental noise is filtered out in the context. That is, the Transformer model is used to capture the characteristic of global dependencies when processing sequence data to capture and characterize the speech content and semantic meaning context association information expressed by the sequence of waveform feature vectors of the speech signal segments after the environmental noise is filtered out, so as to comprehensively integrate the global semantic information in the speech signal to be processed. Finally, the sequence of waveform feature vectors of the speech signal segments after the environmental noise is filtered out in the context is passed through a signal restorer based on a decoder to obtain a denoised speech signal. Here, the decoder can convert the sequence of waveform feature vectors of the speech signal segments after the environmental noise is filtered out in the context back into the form of a speech signal, thereby achieving denoising and restoration reconstruction of the speech signal to be processed.

[0074] Accordingly, in step S550, if Figure 4As shown, the sequence of waveform feature vectors of the speech signal segments after the environmental noise is filtered out is subjected to signal segment association analysis and signal restoration to obtain the denoised speech signal, including: S551, passing the sequence of waveform feature vectors of the speech signal segments after the environmental noise is filtered out through a semantic signal segment semantic association encoder based on a Transformer layer to obtain a sequence of waveform feature vectors of the speech signal segments after the environmental noise is filtered out in the context; and, S552, decoding and restoring the sequence of waveform feature vectors of the speech signal segments after the environmental noise is filtered out in the context to obtain the denoised speech signal.

[0075] Among them, in step S551, the sequence of waveform feature vectors of the speech signal segments after filtering out the environmental noise is passed through a semantic association encoder between semantic signal segments based on a Transformer layer to obtain a sequence of waveform feature vectors of the speech signal segments after filtering out the environmental noise in context, including: arranging the sequence of waveform feature vectors of the speech signal segments after filtering out the environmental noise in one dimension to obtain a global waveform feature vector of the speech signal after filtering out the environmental noise; calculating the transpose vector of the global waveform feature vector of the speech signal after filtering out the environmental noise and each waveform feature vector of the speech signal segments after filtering out the environmental noise in the sequence of the waveform feature vectors of the speech signal segments after filtering out the environmental noise. to obtain multiple self-attention association matrices; respectively standardize each of the multiple self-attention association matrices to obtain multiple standardized self-attention association matrices; pass each of the multiple standardized self-attention association matrices through a Softmax classification function to obtain multiple probability values; and respectively use each of the multiple probability values ​​as a weight to weight each of the waveform feature vectors of the speech signal segment after filtering out the environmental noise in the sequence of the waveform feature vectors of the speech signal segment after filtering out the environmental noise to obtain the sequence of the waveform feature vectors of the speech signal segment after filtering out the environmental noise in the context.

[0076] Among them, in step S552, the sequence of waveform feature vectors of the speech signal fragments after the context filtering out the environmental noise is decoded and restored to obtain the denoised speech signal, including: passing the sequence of waveform feature vectors of the speech signal fragments after the context filtering out the environmental noise through a decoder-based signal restorer to obtain the denoised speech signal.

[0077] In the above technical solution, the waveform feature vectors of each speech signal segment after filtering out the environmental noise in the sequence of waveform feature vectors of the speech signal segment after filtering out the environmental noise respectively represent the semantic features of the signal waveform of each speech signal segment after filtering out the environmental noise. When the sequence of waveform feature vectors of the speech signal segment after filtering out the environmental noise passes through the semantic signal segment semantic association encoder based on the Transformer layer, the semantic signal segment semantic association encoder based on the Transformer layer captures the contextual semantic waveform association features of the local signal domain-global signal domain of the waveform feature vectors of each speech signal segment after filtering out the environmental noise based on the multi-head attention mechanism, which improves the richness of the feature expression of each speech signal segment after filtering out the environmental noise. However, this will also cause the sequence of waveform feature vectors of the speech signal segment after filtering out the environmental noise in the context to be offset from the sequence of waveform feature vectors of the speech signal segment after filtering out the environmental noise in the semantic space, affecting the accuracy of the decoding generation of the sequence of waveform feature vectors of the speech signal segment after filtering out the environmental noise by the decoder-based signal restorer.

[0078] Based on this, in a preferred embodiment, the sequence of waveform feature vectors of the speech signal segment after the context filtering out the environmental noise is passed through a decoder-based signal restorer to obtain the denoised speech signal, including:

[0079] Cascading the sequences of waveform feature vectors of the speech signal segments after the context filtering out the environmental noise to obtain a cascade feature vector of the waveform of the speech signal after the environmental noise is filtered out;

[0080] Calculate the sum of the absolute values ​​of the eigenvalues ​​of the cascaded feature vectors of the speech signal waveform after filtering out the environmental noise to obtain a first cascaded and modulated value of the speech signal waveform after filtering out the environmental noise, and calculate the square root of the sum of the squares of the eigenvalues ​​of the cascaded feature vectors of the speech signal waveform after filtering out the environmental noise to obtain a second cascaded and modulated value of the speech signal waveform after filtering out the environmental noise;

[0081] After performing dot-wise subtraction on the eigenvector of the speech signal waveform cascade after filtering out environmental noise and the second cascade of the speech signal waveform cascade after filtering out environmental noise and the modulation value, perform dot-wise multiplication with the number of eigenvalues ​​of the eigenvector of the speech signal waveform cascade after filtering out environmental noise and the inverse of the first cascade of the speech signal waveform cascade after filtering out environmental noise and the modulation value, and take the inverse of each eigenvalue to obtain the first cascade of the speech signal waveform cascade after filtering out environmental noise;

[0082] After performing dot-wise subtraction on the cascaded feature vector of the speech signal waveform after filtering out environmental noise and the first cascaded speech signal waveform after filtering out environmental noise and modulation value, perform dot-wise multiplication with the square root of the number of eigenvalues ​​of the cascaded feature vector of the speech signal waveform after filtering out environmental noise and the reciprocal of the second cascaded speech signal waveform after filtering out environmental noise and modulation value, and take the reciprocal of each eigenvalue to obtain the second cascaded speech signal waveform after filtering out environmental noise phase conversion vector;

[0083] Subtract the dot product vector of the second speech signal waveform cascade phase transition vector after filtering out the environmental noise from the first speech signal waveform cascade phase transition vector after filtering out the environmental noise and the weighted hyperparameter to obtain an optimized speech signal waveform cascade feature vector after filtering out the environmental noise;

[0084] The optimized speech signal waveform cascade feature vector after filtering out environmental noise is passed through the decoder-based signal restorer to obtain the noise-reduced speech signal.

[0085] Here, the optimized expression of the cascade feature vector of the speech signal waveform after filtering out the environmental noise, denoted as V, is:

[0086]

[0087]

[0088]

[0089]

[0090]

[0091] v i ∈V∈R n

[0092] Wherein, V is the cascade feature vector of the speech signal waveform after filtering out the environmental noise, R represents a real number set, and v i represents the eigenvalue of the ith position of the cascaded feature vector of the speech signal waveform after filtering out the ambient noise, n represents the number of eigenvalues ​​of the cascaded feature vector of the speech signal waveform after filtering out the ambient noise, α represents the first cascaded speech signal waveform and modulation value after filtering out the ambient noise, β represents the second cascaded speech signal waveform and modulation value after filtering out the ambient noise, ⊙ represents the point multiplication by position, Indicates positional subtraction, (·) ⊙-1Represents the inverse of each eigenvalue of the calculated eigenvector, V1 represents the cascade phase transition vector of the speech signal waveform after the first environmental noise is filtered out, V2 represents the cascade phase transition vector of the speech signal waveform after the second environmental noise is filtered out, ω represents the weighted hyperparameter, and V′ represents the optimized cascade feature vector of the speech signal waveform after the environmental noise is filtered out.

[0093] In the preferred example, the difference in the difference in modulation representation of the eigenvalue of the cascade feature vector of the speech signal waveform after filtering out the environmental noise relative to the feature set of the vector as a whole of the cascade feature vector of the speech signal waveform after filtering out the environmental noise is used as semantic change intensity information, and a phase-like transformation corresponding to the position-based intensity modulation is performed through different and modulation representations, so that the aggregation enhancement of semantic change phase perception can enhance the axial aggregation receptive field along the feature aggregation direction by performing a spatial translation operation based on alternating stacking under the scale equalization of the vector set of the cascade feature vector of the speech signal waveform after filtering out the environmental noise, thereby enhancing the perception effect of the aggregated semantics of the cascade feature vector of the speech signal waveform after filtering out the environmental noise for the detailed semantic changes, so as to improve the expression effect of the cascade feature vector of the speech signal waveform after filtering out the environmental noise, and improve the accuracy of its signal decoding generation through a decoder-based signal restorer.

[0094] Based on the above embodiments, see Figure 5 The figure shows a schematic diagram of the structure of an intelligent audio noise reduction system 800 for voice calls in an embodiment of the present disclosure. The intelligent audio noise reduction system 800 for voice calls includes: a to-be-processed voice signal extraction module 810, for extracting a to-be-processed voice signal; an environmental noise signal receiving module 820, for receiving an environmental noise signal collected by a first sound sensor; a waveform feature extraction module 830, for performing signal segment waveform feature extraction on the to-be-processed voice signal and the environmental noise signal to obtain a sequence of voice signal segment waveform semantic feature vectors and a sequence of environmental noise signal segment waveform semantic feature vectors; a filtering module 840, for processing the sequence of voice signal segment waveform semantic feature vectors and the sequence of environmental noise signal segment waveform semantic feature vectors using a semantic noise reduction module to obtain a sequence of voice signal segment waveform feature vectors after filtering out environmental noise; and an analysis and restoration module 850, for performing signal segment correlation analysis and signal restoration on the sequence of voice signal segment waveform feature vectors after filtering out environmental noise to obtain a denoised voice signal.

[0095] Here, those skilled in the art will appreciate that the specific functions and operations of the various modules in the intelligent audio noise reduction system 800 for voice calls have been described in detail above. Figures 2 to 5The invention has been described in detail in the description of the intelligent audio noise reduction method for voice calls, and therefore, its repeated description will be omitted.

[0096] Figure 6 FIG. 1 is an application scenario diagram of the intelligent audio noise reduction method for voice calls according to an embodiment of the present disclosure. Figure 6 As shown, in this application scenario, first, the speech signal to be processed is obtained (for example, Figure 6 D1 as shown in FIG. 1 ) and an ambient noise signal collected by a first sound sensor (eg, Figure 6 Then, the speech signal to be processed and the ambient noise signal are input to a server (for example, Figure 6 In S) shown in , the server can use the intelligent audio noise reduction algorithm for voice calls to process the voice signal to be processed and the ambient noise signal to obtain a noise-reduced voice signal.

[0097] Based on the above embodiments, another exemplary embodiment of an electronic device is also provided in the embodiments of the present disclosure. In some possible implementations, the electronic device in the embodiments of the present disclosure may include a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor can implement the steps of the intelligent audio noise reduction method for voice calls in the above embodiments when executing the program.

[0098] For example, the electronic device is disclosed in the present invention Figure 1 Taking the server 100 in the example as an example, the processor in the electronic device is the processor 110 in the server 100, and the memory in the electronic device is the memory 120 in the server 100.

[0099] An embodiment of the present disclosure also provides a computer-readable storage medium. Figure 7 FIG. 1 is a schematic diagram of a computer-readable storage medium 1000 according to an embodiment of the present disclosure. Figure 7 As shown, the computer-readable storage medium 1000 stores computer-executable instructions 1001. When the computer-executable instructions 1001 are executed by the processor, the intelligent audio noise reduction method for voice calls according to the embodiments of the present disclosure described with reference to the above figures can be executed. The computer-readable storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. The volatile memory may, for example, include a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may, for example, include a read-only memory (ROM), a hard disk, a flash memory, etc.

[0100] The embodiments of the present disclosure also provide a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the intelligent audio noise reduction method for voice calls according to the embodiments of the present disclosure.

[0101] Those skilled in the art will appreciate that the contents disclosed in this disclosure may be subject to various modifications and improvements. For example, the various devices or components described above may be implemented by hardware, or by software, firmware, or a combination of some or all of the three.

[0102] In addition, although the present disclosure makes various references to certain units in the system according to embodiments of the present disclosure, any number of different units may be used and run on the client and / or server. The units are illustrative only, and different aspects of the systems and methods may use different units.

[0103] Those skilled in the art will appreciate that all or part of the steps in the above method can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk. Optionally, all or part of the steps in the above embodiment can also be implemented using one or more integrated circuits. Accordingly, each module / unit in the above embodiment can be implemented in the form of hardware or in the form of a software functional module. The present disclosure is not limited to any particular form of combination of hardware and software.

[0104] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present disclosure belongs. It should also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology and should not be interpreted in an idealized or highly formal sense, unless explicitly defined as such herein.

[0105] The above is an explanation of the present disclosure and should not be considered as a limitation thereof. Although several exemplary embodiments of the present disclosure have been described, it will be readily understood by those skilled in the art that many modifications may be made to the exemplary embodiments without departing from the novel teachings and advantages of the present disclosure. Therefore, all such modifications are intended to be included within the scope of the present disclosure as defined in the claims. It should be understood that the above is an explanation of the present disclosure and should not be considered as being limited to the specific embodiments disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the appended claims. The present disclosure is defined by the claims and their equivalents.

Claims

1. An intelligent audio noise reduction method for voice calls, characterized in that: include: Extracting a speech signal to be processed; Receiving an environmental noise signal collected by a first sound sensor; Extracting signal segment waveform features from the speech signal to be processed and the ambient noise signal to obtain a sequence of speech signal segment waveform semantic feature vectors and a sequence of ambient noise signal segment waveform semantic feature vectors; Using a semantic noise reduction module, the sequence of the speech signal segment waveform semantic feature vectors and the sequence of the environmental noise signal segment waveform semantic feature vectors are processed to obtain a sequence of the speech signal segment waveform feature vectors after filtering out the environmental noise; as well as The sequence of waveform feature vectors of the speech signal segments after the environmental noise is filtered out is subjected to signal segment correlation analysis and signal restoration to obtain the denoised speech signal.

2. The intelligent audio noise reduction method for voice calls according to claim 1, characterized in that: Extracting signal segment waveform features from the speech signal to be processed and the ambient noise signal to obtain a sequence of speech signal segment waveform semantic feature vectors and a sequence of ambient noise signal segment waveform semantic feature vectors, including: Performing signal preprocessing on the speech signal to be processed and the ambient noise signal to obtain a sequence of speech signal segments and a sequence of ambient noise signal segments; and A deep learning network model is used to extract waveform features of the sequence of speech signal segments and the sequence of environmental noise signal segments respectively to obtain a sequence of waveform semantic feature vectors of the speech signal segments and a sequence of waveform semantic feature vectors of the environmental noise signal segments.

3. The intelligent audio noise reduction method for voice calls according to claim 2, characterized in that: The method of performing signal preprocessing on the speech signal to be processed and the ambient noise signal to obtain a sequence of speech signal segments and a sequence of ambient noise signal segments comprises: Performing time alignment on the speech signal to be processed and the ambient noise signal based on the timestamp to obtain an aligned speech signal to be processed and an aligned ambient noise signal; and Signal segmentation is performed on the aligned speech signal to be processed and the aligned ambient noise signal based on a predetermined time scale to obtain a sequence of the speech signal segments and a sequence of the ambient noise signal segments.

4. The intelligent audio noise reduction method for voice calls according to claim 3, characterized in that: The deep learning network model is used to extract waveform features of the sequence of speech signal segments and the sequence of environmental noise signal segments to obtain a sequence of waveform semantic feature vectors of the speech signal segments and a sequence of waveform semantic feature vectors of the environmental noise signal segments, including: The sequence of speech signal segments and the sequence of environmental noise signal segments are respectively passed through a signal fluctuation feature extractor based on a convolutional neural network model to obtain a sequence of waveform semantic feature vectors of the speech signal segments and a sequence of waveform semantic feature vectors of the environmental noise signal segments.

5. The intelligent audio noise reduction method for voice calls according to claim 4, characterized in that: The method further comprises: using a semantic noise reduction module to process the sequence of the speech signal segment waveform semantic feature vectors and the sequence of the environmental noise signal segment waveform semantic feature vectors to obtain a sequence of the speech signal segment waveform feature vectors after filtering out the environmental noise, including: The sequence of the speech signal segment waveform semantic feature vectors and the sequence of the environmental noise signal segment waveform semantic feature vectors are processed using the following semantic noise reduction formula to obtain the sequence of the speech signal segment waveform feature vectors after filtering out the environmental noise; The semantic denoising formula is: Among them, ω i is the i-th waveform feature vector of the speech signal segment after the environmental noise is filtered out in the sequence of waveform feature vectors of the speech signal segment after the environmental noise is filtered out, is the i-th speech signal segment waveform semantic feature vector in the sequence of speech signal segment waveform semantic feature vectors, is the i-th environmental noise signal segment waveform semantic feature vector in the sequence of environmental noise signal segment waveform semantic feature vectors, represents the square of the L2 norm of the eigenvector, and - represents the element-wise subtraction process.

6. The intelligent audio noise reduction method for voice calls according to claim 5, characterized in that: The sequence of waveform feature vectors of the speech signal segments after filtering out the environmental noise is subjected to signal segment correlation analysis and signal restoration to obtain a denoised speech signal, including: Passing the sequence of waveform feature vectors of the speech signal segments after the environmental noise is filtered out through a semantic association encoder between semantic signal segments based on a Transformer layer to obtain a sequence of waveform feature vectors of the speech signal segments after the environmental noise is filtered out in the context; and The sequence of waveform feature vectors of the speech signal segment after the context filtering out the environmental noise is decoded and restored to obtain the denoised speech signal.

7. The intelligent audio noise reduction method for voice calls according to claim 6, characterized in that: The sequence of waveform feature vectors of the speech signal segments after the environmental noise is filtered out is passed through a semantic association encoder between semantic signal segments based on a Transformer layer to obtain a sequence of waveform feature vectors of the speech signal segments after the environmental noise is filtered out from the context, including: Arranging the sequence of waveform feature vectors of the speech signal segments after filtering out the environmental noise in one dimension to obtain a global waveform feature vector of the speech signal after filtering out the environmental noise; Calculating the product between the global waveform feature vector of the speech signal after filtering out the environmental noise and the transposed vector of each waveform feature vector of the speech signal segment after filtering out the environmental noise in the sequence of the waveform feature vectors of the speech signal segment after filtering out the environmental noise to obtain multiple self-attention association matrices; Normalizing each of the plurality of self-attention association matrices to obtain a plurality of standardized self-attention association matrices; Passing each of the plurality of standardized self-attention association matrices through a Softmax classification function to obtain a plurality of probability values; and Each probability value in the multiple probability values ​​is used as a weight to weight each waveform feature vector of the speech signal segment after the environmental noise is filtered out in the sequence of waveform feature vectors of the speech signal segment after the environmental noise is filtered out to obtain the sequence of waveform feature vectors of the speech signal segment after the environmental noise is filtered out in the context.

8. The intelligent audio noise reduction method for voice calls according to claim 7, characterized in that: Decoding and restoring the sequence of waveform feature vectors of the speech signal fragments after the context filtering out the environmental noise to obtain the denoised speech signal, including: The sequence of waveform feature vectors of the speech signal segments after the context filtering and the environmental noise is removed is passed through a decoder-based signal restorer to obtain the denoised speech signal.

9. An intelligent audio noise reduction system for voice calls, characterized in that: include: A speech signal extraction module to be processed, used to extract the speech signal to be processed; An environmental noise signal receiving module, used to receive the environmental noise signal collected by the first sound sensor; A waveform feature extraction module, used for performing signal segment waveform feature extraction on the speech signal to be processed and the ambient noise signal to obtain a sequence of speech signal segment waveform semantic feature vectors and a sequence of ambient noise signal segment waveform semantic feature vectors; A filtering module, used for processing the sequence of semantic feature vectors of the speech signal segment waveform and the sequence of semantic feature vectors of the environmental noise signal segment waveform using a semantic noise reduction module to obtain a sequence of feature vectors of the speech signal segment waveform after filtering out the environmental noise; as well as The analysis and restoration module is used to perform signal segment correlation analysis and signal restoration on the sequence of waveform feature vectors of the speech signal segments after the environmental noise is filtered out to obtain the denoised speech signal.

Images