An in-vehicle intercommunication method, device, storage medium and electronic terminal

Abstract

This invention discloses an in-vehicle communication method, device, storage medium, and electronic terminal, comprising the steps of: acquiring a first speech signal from a sound source seat, the first speech signal including a user speech signal and an ambient speech signal; creating a temporal convolutional network model, the temporal convolutional network model including an encoder, a residual network, and a decoder, the temporal convolutional network model being used to extract the user speech signal from the first speech signal to obtain a second speech signal; performing feedback suppression and speech enhancement on the second speech signal using an adaptive filtering algorithm to obtain a third speech signal, and sending the third speech signal to a speaker in a non-sound source seat. This invention optimizes feature extraction and processing, improves speech waveform accuracy and phase estimation effect, and adapts to the complex acoustic environment inside a vehicle; it greatly improves the quality of in-vehicle communication, facilitating communication between the driver and rear passengers without distraction, reducing driving safety hazards.

Description

Technical Field

[0001] This invention relates to the field of in-vehicle voice processing technology, specifically to an in-vehicle communication method, device, storage medium, and electronic terminal. Background Technology

[0002] With the development of the automotive industry, in-vehicle communication systems are becoming increasingly important for improving the in-vehicle communication experience and ensuring driving safety. Their main purpose is to create a high-quality communication environment for passengers, allowing drivers to easily communicate with rear passengers and avoiding safety hazards caused by turning their heads or shifting their gaze. However, current in-vehicle communication systems have many problems. Some systems do not support simultaneous two-way communication, or when two people are speaking at the same time, issues such as incomplete echo cancellation and howling may occur, resulting in poor call quality and even threatening driving safety.

[0003] In-vehicle communication systems need to integrate microphones, speakers, voice processing, and in-vehicle audio signal processing. In voice front-end processing, echo cancellation is crucial to call quality. Existing neural network-based echo cancellation algorithms mostly employ time-frequency masking strategies, which have significant shortcomings: they require short-time Fourier transforms, which negatively impact echo cancellation; and when reconstructing the phase of the speech signal during echo cancellation, neural networks based on time-frequency training strategies have defects in phase estimation, resulting in insufficient accuracy of the reconstructed waveform.

[0004] Therefore, how to accurately extract effective user voice from a mixed signal containing user voice and environmental voice, and further optimize call quality, has become an urgent technical challenge. Summary of the Invention

[0005] The purpose of this invention is to address the above-mentioned problems by providing an in-vehicle communication method, device, storage medium, and electronic terminal.

[0006] The technical solution of this invention is: an in-vehicle communication method, comprising the following steps: acquiring a first speech signal from a sound source seat, the first speech signal including a user speech signal and an environmental speech signal; creating a temporal convolutional network model, the temporal convolutional network model including an encoder, a residual network, and a decoder, the temporal convolutional network model being used to extract the user speech signal from the first speech signal to obtain a second speech signal; the residual network including normalization and N convolutional blocks. , ... The N convolutional blocks are connected via residual paths, and the convolutional blocks are activated by an activation function. The input is a convolutional block The residual path output, where i and N are both positive integers. The encoder is used to convert the first speech signal to obtain a feature space representation of the first speech signal; the residual network is used to extract features from the feature space representation and output the masking value of the user speech signal; the decoder is used to convert the masking value to obtain a second speech signal; the second speech signal is subjected to howling suppression and speech enhancement through an adaptive filtering algorithm to obtain a third speech signal, and the third speech signal is sent to the speaker of the non-sound source seat.

[0007] As an improvement to an embodiment of the present invention, the convolutional block The convolution is a depthwise separable convolution; the expression for the depthwise separable convolution is: , ,in, For the convolution block Input, and The sizes are respectively A convolution kernel of 1, For convolution operations, This is for concatenation operations.

[0008] As an improvement to this embodiment of the invention, an increasing expansion factor is introduced into the residual network. The convolutional block expansion factor .

[0009] As an improvement to this embodiment of the invention, the step of "converting the first speech signal to obtain a feature space representation of the first speech signal" specifically includes: converting the discrete waveform of the first speech signal using the encoder to obtain a feature space representation of the first speech signal; the discrete waveform expression of the first speech signal is: Where M is the number of source signals of the first speech signal. Let j be the source signal of the first speech signal, where j and M are both positive integers. .

[0010] As an improvement to this embodiment of the invention, the step of "extracting features from the feature space representation and outputting the masking value of the user's speech signal" specifically includes: extracting features from the feature space representation using the temporal convolutional network model and outputting M source signals. The masking value; , ,in, As constraints, H() is an optional activation function. is the basis function of the encoder.

[0011] As an improvement to this embodiment of the invention, the step of "converting the masking value to obtain a second speech signal" specifically includes: reconstructing the masking value using a decoder to obtain a second speech signal. ,in, are the basis functions of the decoder.

[0012] As an improvement to this embodiment of the invention, the adaptive filtering algorithm is normalized least squares adaptive filtering.

[0013] To achieve one of the aforementioned objectives, one embodiment of the present invention provides an in-vehicle communication device, comprising the following modules: a signal acquisition module for acquiring a first speech signal from a sound source seat, the first speech signal including a user speech signal and an environmental speech signal; a model creation module for creating a temporal convolutional network model, the temporal convolutional network model including an encoder, a residual network, and a decoder, the temporal convolutional network model being used to extract the user speech signal from the first speech signal to obtain a second speech signal; the residual network including normalization and N convolutional blocks. , ... The N convolutional blocks are connected via residual paths, and the convolutional blocks are activated by an activation function. The input is a convolutional block The residual path output; the encoder is used to convert the first speech signal to obtain a feature space representation of the first speech signal; the residual network is used to extract features from the feature space representation and output the masking value of the user speech signal; the decoder is used to convert the masking value to obtain a second speech signal; the signal optimization module is used to perform howling suppression and speech enhancement on the second speech signal through an adaptive filtering algorithm to obtain a third speech signal, and send the third speech signal to the speaker of the non-sound source seat.

[0014] To achieve one of the above-mentioned objectives, one embodiment of the present invention provides a storage medium storing program instructions, which, when executed, implement the in-vehicle communication method as described in any of the preceding claims.

[0015] To achieve one of the above-mentioned objectives, one embodiment of the present invention provides an electronic terminal, including a processor and a memory, wherein the memory stores program instructions, and the processor executes the program instructions to implement the in-vehicle communication method as described in any of the preceding claims.

[0016] The in-vehicle communication method, device, storage medium, and electronic terminal provided in this invention have the following advantages: This invention uses a temporal convolutional network model to extract user speech from in-vehicle speech, thereby eliminating echo; it uses an adaptive filtering algorithm to suppress howling and enhance speech, solving the problem of in-vehicle call quality; it facilitates communication between the driver and rear passengers without distraction, reducing driving safety hazards; compared with traditional algorithms, it does not require short-time Fourier transform, optimizes feature extraction and processing, improves speech waveform accuracy and phase estimation effect, and adapts to the complex acoustic environment inside the vehicle. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the in-vehicle communication method described in this invention;

[0018] Figure 2 This is a schematic diagram of the temporal convolutional network model described in this invention;

[0019] Figure 3 This is a schematic diagram illustrating an application scenario of the in-vehicle communication method described in this invention;

[0020] Figure 4 This is a schematic diagram of the structure of the in-vehicle communication device described in this invention;

[0021] Figure 5 This is a schematic diagram of the structure of the electronic terminal described in this invention. Detailed Implementation

[0022] The present invention will now be described in detail with reference to the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional modifications made by those skilled in the art based on these embodiments are included within the scope of protection of the present invention.

[0023] If the present invention involves orientation (e.g., up, down, left, right, front, back, outside, inside, etc.) in its description, then the orientations involved need to be defined.

[0024] The scope of the embodiments described herein includes the entire scope of the claims and all available equivalents thereof. Throughout this document, the terms “first,” “second,” etc., are used only to distinguish one element from another without requiring or implying any actual relationship or order between the elements. Indeed, a first element can also be referred to as a second element, and vice versa. Furthermore, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a structure, apparatus, or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a structure, apparatus, or device. Without further limitations, an element defined by the phrase “comprising one…” does not exclude the presence of other identical elements in the structure, apparatus, or device that includes said element. The various embodiments described herein are presented in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably.

[0025] The terms "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer" used in this document to indicate orientation or positional relationships are based on the orientation or positional relationships shown in the accompanying drawings and are used only for the convenience of describing this document and simplifying the description. They do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as limiting the invention. In the description herein, unless otherwise specified and limited, the terms "installed," "connected," and "linked" should be interpreted broadly. For example, they can refer to mechanical or electrical connections, or internal connections between two elements, or direct connections or indirect connections through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.

[0026] Embodiment 1 of the present invention provides a method for in-vehicle communication, such as... Figure 1 As shown, it includes the following steps:

[0027] Step 101: Acquire the first speech signal from the sound source location, the first speech signal including the user's speech signal and the environmental speech signal;

[0028] In the car interior environment, such as Figure 2As shown, in-vehicle voice signals are collected through a first microphone 201, a second microphone 202, a third microphone 203, and a fourth microphone 204 installed inside the vehicle; the received voice signals are played back through a first speaker 301, a second speaker 302, a third speaker 303, and a fourth speaker 304 installed inside the vehicle. A Fourier transform is performed on the in-vehicle voice signal collected by each microphone to obtain the frequency domain expression of the voice signal collected by each microphone. ,in, These correspond to microphones 201-204 respectively. The time-domain speech signals collected from each microphone are then used to extract the speech frequency domain amplitude signal. The speech signal in each frequency domain is segmented according to a preset frequency domain sub-band, which can be: 80-250Hz, 250-500Hz, 500-1000Hz, 1000-2000Hz, or 2000-4000Hz. For each sub-band, according to the formula... Calculate the signal energy, where, For the first The microphone collects signals at the... The energy carried by a person's height For the microphone signal at the 1st The frequency domain amplitude of the sub-band. Then, the weighted energy of the corresponding signal from each microphone is calculated. ,in, For the first Subband weighting coefficients. Compare the weighted energy of the four microphones. The microphone with the highest energy value collects the in-vehicle voice signal as the first voice signal, and the seat corresponding to the microphone with the highest energy value is the sound source seat. The seats corresponding to the other microphones are non-sound source seats. The first voice sound includes user voice and environmental voice signals. The environmental voice signals include speaker echo signals from non-sound source seats, in-vehicle equipment noise, external noise, howling signals, and other environmental interference sounds.

[0029] Step 102: Create a temporal convolutional network model, which includes an encoder, a residual network, and a decoder. The temporal convolutional network model is used to extract the user's speech signal from the first speech signal to obtain a second speech signal. Here, the temporal convolutional network model is a deep learning architecture designed specifically for processing time-series signals. It effectively captures long-short-term dependencies in the signal through methods such as causal convolution, dilated convolution, and residual connections, and is particularly suitable for speech separation tasks.

[0030] The residual network includes normalization and N convolutional blocks. , ... And activation function, preferably, an increasing expansion factor is introduced into the residual network. The convolutional block expansion factor The N convolutional blocks are connected through residual paths. The input is a convolutional block The residual path output, where i and N are both positive integers. The residual network described above can exponentially expand the receptive field without increasing the number of parameters or computational complexity, enabling it to simultaneously capture both microscopic details and macroscopic structures in speech signals. The sparse connectivity of dilated convolutions reduces redundant computations between parameters, improving model training efficiency and mitigating the risk of overfitting, allowing the model to maintain good generalization ability even with limited training data. Through the connections of the residual paths, the gradient vanishing problem caused by dilated convolutions is effectively alleviated, ensuring smooth information transfer in deep networks and further optimizing the model's ability to represent speech signals.

[0031] In this invention, the convolution block The convolution is a depthwise separable convolution; the expression for the depthwise separable convolution is: , ,in, For the convolution block Input, and The sizes are respectively A convolution kernel of 1, For convolution operations, For concatenation operations. Here, depthwise separable convolution can significantly reduce the number of parameters and computational load, reducing the computational overhead of the in-vehicle communication system when processing voice signals. At the same time, depthwise separable convolution can finely extract features within a channel and flexibly fuse features across channels, adapting to the diverse features of voice signals in the complex acoustic environment of an in-vehicle environment, improving the model's ability to distinguish between user voice and environmental voice, and extracting user voice more efficiently.

[0032] In specific implementation, a model architecture consisting of an encoder, a residual network, and a decoder is constructed. After receiving the first speech signal, the encoder extracts features from the temporal signal through a one-dimensional convolutional layer, converting it into a feature space representation. The residual network includes a normalization operation, N convolutional blocks, and an activation function. The N convolutional blocks are connected through residual paths, and the input of each convolutional block is the output of the residual path of the previous convolutional block. The residual network performs deep feature mining on the feature space representation and outputs a masking value used to identify user speech features. Here, preferably, the activation function used in this invention is PReLU. Compared with the ReLU activation function, PReLU introduces learnable parameters, which can adaptively adjust the slope of the negative half-axis. This allows the network to better retain negative feature information when processing complex speech signals containing user speech and environmental noise, avoiding information loss due to feature truncation and effectively improving the model's ability to capture features of different speech signals.

[0033] The decoder transforms the masking values ​​back to the time domain through deconvolution, generating a second speech signal containing only the user's voice. The time-domain convolutional network model effectively separates user speech from environmental noise, improving in-vehicle call quality. To optimize model performance, mixed speech data from different in-vehicle environments can be collected for training. The network parameters are adjusted using a mean squared error loss function and backpropagation algorithm to adapt to complex acoustic environments. The trained model can be deployed on an in-vehicle processing chip or in the cloud to achieve real-time processing of the first speech signal, providing a foundation for subsequent speech enhancement and howling suppression.

[0034] Step 103: The second speech signal is subjected to howling suppression and speech enhancement through an adaptive filtering algorithm to obtain a third speech signal, and the third speech signal is sent to the speaker at the non-sound source seat. Preferably, the adaptive filtering algorithm is normalized least squares adaptive filtering.

[0035] In practice, a Fourier transform is performed on the current frame of the second speech signal to obtain the frequency domain signal. The ratio of the average power of the frame's spectrum to the power of each frequency point and the average power is calculated. Frequency points with ratios greater than a preset threshold are labeled as howling frequencies. Based on the number of howling frequencies, a corresponding number of notch filter transfer functions are preset and cascaded to suppress howling in the frequency domain speech signal. Parameters such as control bandwidth α and attenuation depth β in the transfer functions can be assigned values ​​according to actual working conditions or prior information. Then, the noise signal is estimated through the silence segment, and the initial coefficients of the filter are given. Using the current frame speech signal as the target signal, combined with the estimated noise signal, a normalized least squares adaptive filtering algorithm is used to further remove residual noise, resulting in the third speech signal. Finally, the third speech signal is sent to the speaker at the non-sound source seat.

[0036] In this embodiment, the step of "converting the first speech signal to obtain a feature space representation of the first speech signal" specifically includes: converting the discrete waveform of the first speech signal using the encoder to obtain a feature space representation of the first speech signal; the discrete waveform expression of the first speech signal is: Where M is the number of source signals of the first speech signal. Let j be the source signal of the first speech signal, where j and M are both positive integers. .

[0037] In practice, the encoder can perform a sliding convolution operation on the discrete waveform using one-dimensional convolution to generate a feature map. Then, the feature map is normalized, and an activation function is used to introduce a nonlinear transformation to obtain a feature space representation. This feature space representation preserves the time-frequency and spatial features of the first speech signal, providing a structured input for the residual network and effectively supporting the extraction and separation of user speech signals.

[0038] In this embodiment, "extracting features from the feature space representation and outputting the masking value of the user's speech signal" specifically includes: extracting features from the feature space representation using the temporal convolutional network model and outputting M source signals. The masking value; , ,in, As constraints, H() is an optional activation function. is the basis function of the encoder.

[0039] In practice, through the residual network Each convolutional block is used to extract features layer by layer, generating intermediate feature representations. Then it can be done through mapping functions. Calculate each source signal masking value The mapping process satisfies the constraints. To ensure energy conservation. The basis functions here. As the basis of feature transformation in the encoder, it is used to map the masking values ​​back to the original signal space, and the final output masking value is... Characterizes the feature space belonging to the first By multiplying the probability distribution of each source signal element-wise with the feature space representation, the feature components corresponding to the user's speech signal can be separated, thus realizing the extraction of the user's speech.

[0040] In this embodiment, "converting the masking value to obtain the second speech signal" specifically includes: reconstructing the masking value using a decoder to obtain the second speech signal. ,in, are the basis functions of the decoder.

[0041] In practice, the decoder first receives the M source signal masking values ​​output by the residual network. Through basis functions Mapping it back to the time-domain signal space can be achieved through linear combination operations. To achieve, among which, The basis functions of the decoder define the mapping from the feature space to the time domain. To ensure the accuracy of the reconstructed signal, the basis functions... With encoder basis functions Duality condition must be met This is to achieve lossless reconstruction of the speech signal. Understandably, this mapping process converts the masking values ​​representing the user's speech features into a time-domain waveform, generating a second speech signal containing only the user's speech, thus completing the task of extracting the user's speech from the first speech signal.

[0042] Embodiment 2 of the present invention provides an in-vehicle communication device, such as... Figure 4 As shown, it includes the following modules:

[0043] Signal acquisition module 401 is used to acquire a first speech signal from the sound source location, the first speech signal including user speech signal and environmental speech signal;

[0044] Model creation module 402 is used to create a temporal convolutional network model, which includes an encoder, a residual network, and a decoder. The temporal convolutional network model is used to extract the user's speech signal from the first speech signal to obtain a second speech signal. The residual network includes a normalization function and N convolutional blocks. , ... The N convolutional blocks are connected via residual paths, and the convolutional blocks are activated by an activation function. The input is a convolutional block The encoder is used to convert the first speech signal to obtain a feature space representation of the first speech signal; the residual network is used to extract features from the feature space representation and output the masking value of the user speech signal; the decoder is used to convert the masking value to obtain a second speech signal.

[0045] The signal optimization module 403 is used to perform howling suppression and speech enhancement on the second speech signal through an adaptive filtering algorithm to obtain a third speech signal, and to send the third speech signal to the speaker of the non-sound source seat.

[0046] Embodiment 3 of the present invention provides a storage medium storing program instructions, which, when executed, implement the in-vehicle communication method as described in any of the preceding embodiments.

[0047] Embodiment 4 of the present invention provides an electronic terminal, such as Figure 5 As shown, it includes a processor and a memory, the memory storing program instructions, and the processor executing the program instructions to implement the in-vehicle communication method as described in any of the preceding claims.

[0048] This invention can be an apparatus, method, and / or computer program product. A computer program product may include a readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.

[0049] Storage media can be tangible devices that hold and store instructions for use by instruction execution devices. Storage media can include, but are not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof.

[0050] It should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0051] The detailed descriptions listed above are merely specific descriptions of feasible embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. All equivalent embodiments or modifications made without departing from the spirit of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for in-vehicle communication, characterized in that, Includes the following steps: Acquire a first speech signal from the sound source location, the first speech signal including user speech signal and environmental speech signal; A temporal convolutional network model is created, which includes an encoder, a residual network, and a decoder. The temporal convolutional network model is used to extract the user's speech signal from the first speech signal to obtain a second speech signal. The residual network includes normalization and N convolutional blocks. , ... The N convolutional blocks are connected via residual paths, and the convolutional blocks are activated by an activation function. The input is a convolutional block The residual path output, where i and N are both positive integers. ; The encoder is used to convert the first speech signal to obtain a feature space representation of the first speech signal, specifically including: converting the discrete waveform of the first speech signal to obtain a feature space representation of the first speech signal using the encoder; the discrete waveform expression of the first speech signal is: Where M is the number of source signals of the first speech signal. Let j be the source signal of the first speech signal, where j and M are both positive integers. ; The residual network is used to extract features from the feature space representation and output the masking value of the user's speech signal. Specifically, it includes: extracting features from the feature space representation through the temporal convolutional network model and outputting M source signals. The masking value; , ,in, As constraints, H() is an optional activation function. The encoder uses the base function; the decoder converts the masking value to obtain the second speech signal. The second speech signal is subjected to feedback suppression and speech enhancement using an adaptive filtering algorithm to obtain a third speech signal, which is then sent to a speaker at a non-sound source seat.

2. The in-vehicle communication method according to claim 1, characterized in that, The convolutional block The convolution is a depthwise separable convolution; the expression for the depthwise separable convolution is: , ,in, For the convolution block Input, and The sizes are respectively A convolution kernel of 1, For convolution operations, This is for concatenation operations.

3. The in-vehicle communication method according to claim 1, characterized in that, An increasing expansion factor is introduced into the residual network. The convolutional block expansion factor .

4. The in-vehicle communication method according to claim 1, characterized in that, The step of "converting the masking value to obtain the second speech signal" specifically includes: reconstructing the masking value using a decoder to obtain the second speech signal. ,in, are the basis functions of the decoder.

5. The in-vehicle communication method according to claim 1, characterized in that, The adaptive filtering algorithm is normalized least squares adaptive filtering.

6. An in-vehicle communication device, characterized in that, Includes the following modules: The signal acquisition module is used to acquire the first speech signal of the sound source seat, the first speech signal including the user speech signal and the environmental speech signal; The model creation module is used to create a temporal convolutional network model, which includes an encoder, a residual network, and a decoder. This model is used to extract the user's speech signal from the first speech signal to obtain a second speech signal. The residual network includes a normalization function and N convolutional blocks. , ... The N convolutional blocks are connected via residual paths, and the convolutional blocks are activated by an activation function. The input is a convolutional block The residual path output; the encoder is used to convert the first speech signal to obtain a feature space representation of the first speech signal, specifically including: converting the discrete waveform of the first speech signal to obtain a feature space representation of the first speech signal through the encoder; the discrete waveform expression of the first speech signal is: Where M is the number of source signals of the first speech signal. Let j be the source signal of the first speech signal, where j and M are both positive integers. The residual network is used to extract features from the feature space representation and output the masking value of the user's speech signal. Specifically, it includes: extracting features from the feature space representation through the temporal convolutional network model and outputting M source signals. The masking value; , ,in, As constraints, H() is an optional activation function. The encoder uses the base function; the decoder converts the masking value to obtain the second speech signal. The signal optimization module is used to suppress howling and enhance speech in the second speech signal using an adaptive filtering algorithm to obtain a third speech signal, and then send the third speech signal to the speaker in the non-sound source seat.

7. A storage medium storing program instructions, characterized in that, When the program instructions are executed, the in-vehicle communication method as described in any one of claims 1 to 5 is implemented.

8. An electronic terminal, characterized in that, It includes a processor and a memory, the memory storing program instructions, and the processor executing the program instructions to implement the in-vehicle communication method as described in any one of claims 1 to 5.

Images