A vehicle-mounted multi-audio-zone interaction method and device, electronic equipment and storage medium

Abstract

The application relates to a vehicle-mounted multi-sound-zone interaction method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a microphone receiving signal and a reference signal; converting the microphone receiving signal and the reference signal to a short-time time-frequency domain; performing echo cancellation on the signal converted to the short-time time-frequency domain; determining to which sound zone the signal after the echo cancellation belongs; separating the signal after the echo cancellation to obtain a separated signal of each sound zone; converting the separated signal of each sound zone to obtain a final time-domain output of each sound zone; determining an actual wake-up position of a sound zone signal according to the final time-domain output of each sound zone; and inputting the signal at the actual wake-up position into a recognition engine to complete voice interaction. The application separates the signals of each sound zone, inputs the separated signals into corresponding wake-up engines to determine the actual wake-up position, inputs the signal at the actual wake-up position into the recognition engine, thereby completing vehicle-mounted multi-sound-zone interaction, and realizing separate interaction of each sound zone.

Description

Technical Field

[0001] This invention relates to the field of in-vehicle multi-zone interaction technology, specifically to an in-vehicle multi-zone interaction method, device, electronic device, and storage medium. Background Technology

[0002] Currently, there are two methods for in-vehicle voice separation: one is to use beamforming technology or blind source separation technology to achieve multi-range voice separation. Beamforming technology is not sufficient to suppress interference from other voice ranges, which can easily lead to misidentification of other voice ranges; while blind source separation technology separates signals of each voice range by solving a separation matrix, but its output channel order is uncertain. The above two voice separation methods can easily lead to inaccurate in-vehicle interaction, and in existing technologies, the voice ranges of in-vehicle interaction are mixed together, resulting in a poor user experience. Summary of the Invention

[0003] This invention provides a vehicle-mounted multi-zone interactive method, device, electronic device, and storage medium that can solve the above-mentioned technical problems.

[0004] In a first aspect, embodiments of the present invention provide a vehicle-mounted multi-zone interaction method, comprising:

[0005] Acquire the microphone received signal and the reference signal;

[0006] Convert the microphone received signal and the reference signal to the short-time time-frequency domain;

[0007] Echo cancellation is performed on the signal converted to the short-time frequency domain;

[0008] Determine which frequency range the echo-cancelled signal belongs to;

[0009] The separated signals for each pitch range are obtained by separating the echo-cancelled signals.

[0010] The separated signals of each pitch range are converted to obtain the final time-domain output of each pitch range;

[0011] The actual wake-up position of the tone zone signal is determined based on the final time-domain output of each tone zone;

[0012] The signal from the actual wake-up location is input into the recognition engine to recognize the command and perform the operation.

[0013] After completing the corresponding command, the voice interaction is completed through TTS feedback.

[0014] Furthermore, in the above-mentioned in-vehicle multi-zone interaction method, converting the microphone received signal and the reference signal to the short-time time-frequency domain includes:

[0015] The microphone received signal and the reference signal are converted to the short-time time-frequency domain by performing a short-time Fourier transform.

[0016] Furthermore, in the above-mentioned vehicle-mounted multi-zone interaction method, echo cancellation is performed on the signal converted to the short-time time-frequency domain, including:

[0017] Linear echo cancellation is performed on the signal converted to the short-time frequency domain;

[0018] Residual echo suppression is applied to the signal after linear echo cancellation.

[0019] Echo cancellation is achieved through an adaptive filtering method, while residual echo suppression is achieved through deep learning to suppress residual echoes and non-human voice noise.

[0020] Furthermore, in the above-mentioned in-vehicle multi-zone interaction method, determining which zone the echo-cancelled signal belongs to includes:

[0021] Calculate the energy of the echo-cancelled signal based on the echo-cancelled signal.

[0022] The time delay of the echo-cancelled signal between different microphone pairs is calculated using the generalized cross-correlation method based on the distance between each microphone.

[0023] The acoustic register of the echo-cancelled signal is determined by the energy of the echo-cancelled signal and the time delay of different microphone pairs.

[0024] Furthermore, in the above-mentioned vehicle-mounted multi-zone interaction method, the separation of the echo-cancelled signal to obtain the separation signal of each zone includes: zone-level separation and zone-level separation;

[0025] First-level separation of audio regions includes: if the current echo-cancelled signal belongs to the first audio region, then the corresponding adaptive filter of the first audio region is not updated; if the current echo-cancelled signal does not belong to the first audio region, then the corresponding adaptive filter of the first audio region is updated to suppress signals outside the first audio region.

[0026] The two-stage separation of pitch ranges includes: taking the signal corresponding to the first pitch range as the near-end signal and the signals corresponding to other pitch ranges as the far-end signals, and feeding them into an adaptive filter to further suppress the signals of other pitch ranges, thereby obtaining the final separated signals of each pitch range.

[0027] Furthermore, in the above-mentioned in-vehicle multi-zone interaction method, the output signal of each zone is converted to obtain the final time-domain output of each zone, including:

[0028] The final time-domain output of each tone range is obtained by performing a short-time inverse Fourier transform on the output signal of each tone range.

[0029] Furthermore, in the above-mentioned in-vehicle multi-zone interaction method, determining the wake-up position of the zone signal based on the final time-domain output of each zone includes:

[0030] The final time-domain output of each tone zone is sent to the wake-up engine, and the time-domain output data is buffered.

[0031] Determine if there is only one location to wake up. If so, there is no need to calculate the wake word energy, and the corresponding location is taken as the actual wake-up location.

[0032] If there are multiple wake-up locations, calculate the signal energy corresponding to the wake-up time for each wake-up location, and determine the location with the highest signal energy as the actual wake-up location.

[0033] Secondly, embodiments of the present invention also provide an in-vehicle multi-zone interactive device, comprising:

[0034] Acquisition module: used to acquire the microphone received signal and reference signal;

[0035] First conversion module: used to convert the microphone received signal and the reference signal to the short-time frequency domain;

[0036] Echo cancellation module: used to cancel the echo of signals converted to the short-time frequency domain;

[0037] Judgment module: Used to determine which frequency range the echo-cancelled signal belongs to;

[0038] Separation module: Used to separate the echo-cancelled signal to obtain the separated signal for each tone range;

[0039] The second conversion module is used to convert the separated signals of each pitch range to obtain the final time-domain output of each pitch range;

[0040] Determining module: used to determine the actual wake-up position of the tone zone signal based on the final time-domain output of each tone zone;

[0041] Input module: Used to input the signal from the actual wake-up location into the recognition engine, recognize the commands, and perform the operation;

[0042] Voice interaction module: Used to complete voice interaction by providing TTS feedback after the corresponding command operation is completed.

[0043] Thirdly, embodiments of the present invention also provide an electronic device, including: a processor and a memory;

[0044] The processor executes any of the above-mentioned in-vehicle multi-zone interaction methods by calling programs or instructions stored in memory.

[0045] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a program or instructions that cause a computer to execute any of the above-mentioned in-vehicle multi-zone interaction methods.

[0046] The beneficial effects of this invention are as follows: This invention acquires microphone received signals and reference signals; converts the microphone received signals and reference signals to the short-time time-frequency domain; performs echo cancellation on the converted short-time time-frequency domain signals; determines which sound zone the echo-cancelled signal belongs to; separates the echo-cancelled signals to obtain the separated signals for each sound zone; converts the separated signals for each sound zone to obtain the final time-domain output for each sound zone; determines the actual wake-up position of the sound zone signal based on the final time-domain output of each sound zone; and inputs the signal at the actual wake-up position into the recognition engine to complete voice interaction. This invention separates the signals for each sound zone, sends the separated signals to the corresponding wake-up engine to determine the actual wake-up position, and inputs the signal at the actual wake-up position into the recognition engine, thereby completing in-vehicle multi-sound-zone interaction and realizing individual interaction for each sound zone. Attached Figure Description

[0047] Figure 1 A vehicle-mounted multi-zone interactive method provided in this embodiment of the invention. Figure 1 ;

[0048] Figure 2 A vehicle-mounted multi-zone interactive method provided in this embodiment of the invention. Figure 2 ;

[0049] Figure 3 A vehicle-mounted multi-zone interactive method provided in this embodiment of the invention. Figure 3 ;

[0050] Figure 4 This is a schematic diagram of a microphone array and a four-zone in-vehicle sound system provided in an embodiment of the present invention;

[0051] Figure 5 This is a schematic diagram of a microphone array and a six-zone in-vehicle audio system provided in an embodiment of the present invention;

[0052] Figure 6 A diagram of an in-vehicle multi-zone interactive device provided in an embodiment of the present invention;

[0053] Figure 7 This is a schematic block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0054] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0055] Figure 1A vehicle-mounted multi-zone interactive method provided in this embodiment of the invention. Figure 1 .

[0056] In a first aspect, embodiments of the present invention provide a vehicle-mounted multi-zone interaction method, combined with Figure 1 It includes nine steps, S101 to S109:

[0057] S101: Acquire microphone received signal and reference signal.

[0058] Specifically, in this embodiment of the invention, the microphone received signal is the signal received by the microphone, and the reference signal is the echo reference signal.

[0059] The microphones in this invention are arranged as follows: the front microphones are placed near the sun visor, with no specific requirements, as long as the microphone pickup hole is not obstructed from the mouth of the person at the corresponding position; the rear microphones can be placed near the armrest. It is not necessary to know the precise coordinates of the microphones inside the vehicle, only the distance between each microphone needs to be measured.

[0060] S102: Convert the microphone received signal and the reference signal to the short-time frequency domain.

[0061] Specifically, in this embodiment of the invention, the microphone received signal and the reference signal are converted to the short-time time-frequency domain by a short-time Fourier transform.

[0062] S103: Perform echo cancellation on the signal converted to the short-time frequency domain.

[0063] Specifically, in this embodiment of the invention, echo cancellation of the signal converted to the short-time frequency domain is achieved through linear echo cancellation and residual echo suppression.

[0064] S104: Determine which frequency range the echo-cancelled signal belongs to.

[0065] Specifically, in this embodiment of the invention, determining which audio region the echo-cancelled signal belongs to is accomplished by calculating the signal energy and the time delay relationship between each microphone pair.

[0066] S105: Separate the echo-cancelled signal to obtain the separated signal for each tone range.

[0067] Specifically, in this embodiment of the invention, the separation signal of each pitch region obtained after echo cancellation is achieved by pitch region first-level separation and pitch region second-level separation, both of which are completed by adaptive filtering.

[0068] S106: The separated signals of each pitch range are converted to obtain the final time-domain output of each pitch range.

[0069] Specifically, in this embodiment of the invention, the separated signals of each pitch range are subjected to short-time inverse Fourier transform to obtain the final time-domain output of each pitch range.

[0070] S107: Determine the actual wake-up position of the tone zone signal based on the final time-domain output of each tone zone.

[0071] Specifically, in this embodiment of the invention, the final time domain output of each sound zone is sent to the corresponding wake-up engine. If the wake-up is successful, the corresponding position signal is sent to the recognition engine. If multiple positions are woken up at the same time, the wake-up word energy is calculated based on the wake-up word time information returned by the wake-up engine. The position with the higher wake-up word energy is the actual wake-up position.

[0072] S108: Input the signal from the actual wake-up location into the recognition engine, recognize the command, and operate.

[0073] S109: After completing the corresponding command operation, the voice interaction is completed through TTS feedback.

[0074] Specifically, in this embodiment of the invention, the signal from the actual wake-up location is sent to the recognition engine, while other signals are considered false wake-ups and are not sent to the recognition engine, thereby completing the in-vehicle multi-zone interaction.

[0075] Furthermore, in the above-mentioned in-vehicle multi-zone interaction method, converting the microphone received signal and the reference signal to the short-time time-frequency domain includes:

[0076] The microphone received signal and the reference signal are converted to the short-time time-frequency domain by performing a short-time Fourier transform.

[0077] Figure 2 A vehicle-mounted multi-zone interactive method provided in this embodiment of the invention. Figure 2 .

[0078] Furthermore, in the aforementioned in-vehicle multi-zone interaction method, the signal converted to the short-time frequency domain undergoes echo cancellation, combined with... Figure 2 It includes two steps, S201 to S202:

[0079] S201: Perform linear echo cancellation on the signal converted to the short-time frequency domain;

[0080] S202: Perform residual echo suppression on the signal after linear echo cancellation;

[0081] Echo cancellation is achieved through an adaptive filtering method, while residual echo suppression is achieved through deep learning to suppress residual echoes and non-human voice noise.

[0082] Specifically, in this embodiment of the invention, linear echo cancellation is performed on the signal converted to the short-time frequency domain, and residual echo suppression is performed on the signal after linear echo cancellation. This filters out the sound played from the in-vehicle speaker and non-human noise received by the microphone, such as wind noise, tire noise, and engine noise. The signal after echo cancellation in the short-time frequency domain is represented as S. i , where i represents the i-th microphone.

[0083] Figure 3 A vehicle-mounted multi-zone interactive method provided in this embodiment of the invention. Figure 3 .

[0084] Furthermore, in the aforementioned in-vehicle multi-zone interaction method, determining which zone the echo-cancelled signal belongs to, combined with... Figure 3 It includes three steps, S301 to S303:

[0085] S301: Calculate the signal energy after echo cancellation based on the signal after echo cancellation.

[0086] Specifically, in this embodiment of the invention, based on the echo-cancelled signal S i Calculate the signal energy P after echo cancellation. i , where i represents the i-th microphone.

[0087] S302: Calculate the time delay of the echo-cancelled signal between different microphone pairs using the generalized cross-correlation method based on the distance between each microphone.

[0088] Specifically, in this embodiment of the invention, the time delay of the echo-cancelled signal between different microphone pairs is calculated using the generalized cross-correlation method based on the distance between each microphone, and denoted as D. ij (j≠i), where i represents the i-th microphone.

[0089] S303: Determine the audio range of the echo-cancelled signal based on the signal energy after echo cancellation and the time delay of different microphone pairs after echo cancellation.

[0090] Figure 4 This is a schematic diagram of a microphone array and a four-zone in-vehicle sound system provided in an embodiment of the present invention;

[0091] Figure 5 This is a schematic diagram of a microphone array and a six-zone in-vehicle audio system provided in an embodiment of the present invention;

[0092] Specifically, in the embodiments of the present invention, combined with Figure 4 and Figure 5 If P1>P k (k≠1) and D 1kIf (k≠1)<0, then the signal after echo cancellation belongs to the driver's zone. The driver's signal is closest to mic1, so the signal energy should be the largest, and the signal arrives at mic1 first. The time delay of mic1 and other mics is estimated to be negative. The judgment of other zones is similar to that of the driver's zone.

[0093] In some embodiments, the present invention can also be extended to an octave or a decibel range by simply adding microphones at the corresponding positions and providing the distance between each microphone.

[0094] Furthermore, in the above-mentioned vehicle-mounted multi-zone interaction method, the separation of the echo-cancelled signal to obtain the separation signal of each zone includes: zone-level separation and zone-level separation;

[0095] First-level separation of audio regions includes: if the current echo-cancelled signal belongs to the first audio region, then the corresponding adaptive filter of the first audio region is not updated; if the current echo-cancelled signal does not belong to the first audio region, then the corresponding adaptive filter of the first audio region is updated to suppress signals outside the first audio region.

[0096] Specifically, in this embodiment of the invention, the first sound zone can be any sound zone within the vehicle. Based on the sound zone determination of the current echo-cancelled signal, the corresponding adaptive filter is controlled to update and complete sound zone separation. If the current echo-cancelled signal belongs to the driver's sound zone, the adaptive filter corresponding to the driver's sound zone does not update its parameters to protect the driver's sound zone signal from damage. Otherwise, the adaptive filter updates its parameters to filter out signals from other sound zones. The near-end signal of each sound zone's adaptive filter is the corresponding microphone signal, and the far-end signal is the microphone signal from other locations. The first-level sound zone separation output signal is denoted as... i represents the i-th microphone.

[0097] The two-stage separation of pitch ranges includes: taking the signal corresponding to the first pitch range as the near-end signal and the signals corresponding to other pitch ranges as the far-end signals, and feeding them into an adaptive filter to further suppress the signals of other pitch ranges, thereby obtaining the final separated signals of each pitch range.

[0098] Specifically, in this embodiment of the invention, the signals corresponding to each tone zone are... As a near-end signal, the corresponding signals for other registers As the far-end signal, it is fed again into the adaptive filter to further suppress signals from other registers, resulting in the final separated signals for each register, denoted as... i represents the i-th microphone.

[0099] Furthermore, in the above-mentioned in-vehicle multi-zone interaction method, the output signal of each zone is converted to obtain the final time-domain output of each zone, including:

[0100] The final time-domain output of each tone zone is obtained by performing a short-time inverse Fourier transform on the output signal of each tone zone.

[0101] Furthermore, in the above-mentioned in-vehicle multi-zone interaction method, determining the wake-up position of the zone signal based on the final time-domain output of each zone includes:

[0102] The final time-domain output of each tone zone is sent to the wake-up engine, and the time-domain output data is buffered.

[0103] Determine if there is only one location to wake up. If so, there is no need to calculate the wake word energy, and the corresponding location is taken as the actual wake-up location.

[0104] If there are multiple wake-up locations, calculate the signal energy corresponding to the wake-up time for each wake-up location, and determine the location with the highest signal energy as the actual wake-up location.

[0105] Specifically, in this embodiment of the invention, the final time-domain output of each audio region is sent to the wake-up engine, and the final time-domain output is buffered. If a location is woken up, there is no need to calculate the wake-up word energy, and it is directly used as the actual wake-up location. If multiple locations are woken up, the signal energy of the wake-up time corresponding to the wake-up location is calculated, and the location with the largest energy is selected as the actual wake-up location.

[0106] Figure 6 This is a diagram of a vehicle-mounted multi-zone interactive device provided in an embodiment of the present invention.

[0107] Secondly, embodiments of the present invention also provide an in-vehicle multi-zone interactive device, combined with Figure 6 ,include:

[0108] Acquisition module 601: used to acquire microphone received signals and reference signals.

[0109] Specifically, in this embodiment of the invention, the acquisition module 601 acquires the microphone received signal and the reference signal. The microphone received signal is the signal received by the microphone, and the reference signal is the echo reference signal.

[0110] The microphones in this invention are arranged as follows: the front microphones are placed near the sun visor, with no specific requirements, as long as the microphone pickup hole is not obstructed from the mouth of the person at the corresponding position; the rear microphones can be placed near the armrest. It is not necessary to know the precise coordinates of the microphones inside the vehicle, only the distance between each microphone needs to be measured.

[0111] First conversion module 602: used to convert the microphone received signal and the reference signal to the short-time frequency domain.

[0112] Specifically, in this embodiment of the invention, the first conversion module 602 performs a short-time Fourier transform on the microphone received signal and the reference signal to convert them to the short-time time-frequency domain.

[0113] Echo cancellation module 603: Used to cancel the echo of signals converted to the short-time frequency domain.

[0114] Specifically, in this embodiment of the invention, the echo cancellation module 603 performs echo cancellation on the signal converted to the short-time frequency domain through linear echo cancellation and residual echo suppression.

[0115] Judgment module 604: Used to determine which tone range the echo-cancelled signal belongs to.

[0116] Specifically, in this embodiment of the invention, the determination module 604 determines which audio region the echo-cancelled signal belongs to by calculating the signal energy and the time delay relationship between each microphone pair.

[0117] Separation module 605: Used to separate the echo-cancelled signal to obtain the separated signal for each tone range.

[0118] Specifically, in this embodiment of the invention, the separation module 605 separates the echo-cancelled signal to obtain the separation signal for each pitch range by first-level pitch range separation and second-level pitch range separation. Both first-level pitch range separation and second-level pitch range separation are completed through adaptive filtering.

[0119] The second conversion module 606 is used to convert the separated signals of each pitch range to obtain the final time-domain output of each pitch range.

[0120] Specifically, in this embodiment of the invention, the second conversion module 606 performs a short-time inverse Fourier transform on the separated signals of each pitch range to obtain the final time-domain output of each pitch range.

[0121] Determining module 607: used to determine the actual wake-up position of the tone zone signal based on the final time-domain output of each tone zone.

[0122] Specifically, in this embodiment of the invention, the determining module 607 sends the final time domain output of each sound zone to the corresponding wake-up engine. If the wake-up is successful, the corresponding position signal is sent to the recognition engine. If multiple positions are woken up at the same time, the wake-up word energy is calculated based on the wake-up word time information returned by the wake-up engine, and the position with the higher wake-up word energy is determined as the actual wake-up position.

[0123] Input module 608: Used to input the signal from the actual wake-up location into the recognition engine, recognize the command, and operate.

[0124] Voice interaction module 609: Used to complete voice interaction through TTS feedback after completing the corresponding command operation.

[0125] Specifically, in this embodiment of the invention, the input module 608 and the voice interaction module 609 input the signal of the actual wake-up location into the recognition engine, while other signals are pseudo-wake-ups and are not sent to the recognition engine, thereby completing the in-vehicle multi-zone interaction.

[0126] Thirdly, embodiments of the present invention also provide an electronic device, including: a processor and a memory;

[0127] The processor executes any of the above-mentioned in-vehicle multi-zone interaction methods by calling programs or instructions stored in memory.

[0128] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a program or instructions that cause a computer to execute any of the above-mentioned in-vehicle multi-zone interaction methods.

[0129] Figure 7 This is a schematic block diagram of an electronic device provided in an embodiment of this disclosure.

[0130] like Figure 7 As shown, the electronic device includes at least one processor 701, at least one memory 702, and at least one communication interface 703. The various components of the electronic device are coupled together via a bus system 704. The communication interface 703 is used for information transmission with external devices. It is understood that the bus system 704 is used to implement communication between these components. In addition to a data bus, the bus system 704 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 7 The general designated all buses as Bus System 704.

[0131] It is understood that the memory 702 in this embodiment may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory.

[0132] In some implementations, memory 702 stores elements such as executable units or data structures, or subsets thereof, or extended sets thereof: operating systems and applications.

[0133] The operating system includes various system programs, such as the framework layer, core library layer, and driver layer, used to implement various basic business functions and handle hardware-based tasks. The application programs include various applications, such as media players and browsers, used to implement various application functions. The program implementing any method in the in-vehicle multi-zone interaction method provided in this embodiment of the invention can be included in the application programs.

[0134] In this embodiment of the invention, the processor 701 executes the steps of various embodiments of the in-vehicle multi-zone interaction method provided by the present invention by calling the program or instructions stored in the memory 702, specifically, the program or instructions stored in the application program.

[0135] Collect the microphone received signal and the reference signal;

[0136] Convert the microphone received signal and the reference signal to the short-time time-frequency domain;

[0137] Echo cancellation is performed on the signal converted to the short-time frequency domain;

[0138] Determine which frequency range the echo-cancelled signal belongs to;

[0139] The separated signals for each pitch range are obtained by separating the echo-cancelled signals.

[0140] The separated signals of each pitch range are converted to obtain the final time-domain output of each pitch range;

[0141] The actual wake-up position of the tone zone signal is determined based on the final time-domain output of each tone zone;

[0142] The signal from the actual wake-up location is input into the recognition engine to recognize the command and perform the operation.

[0143] After completing the corresponding command, the voice interaction is completed through TTS feedback.

[0144] Any method in the in-vehicle multi-zone interaction method provided in this embodiment of the invention can be applied to, or implemented by, the processor 701. The processor 701 can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 701 or by instructions in software form. The processor 701 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor.

[0145] The steps of any method in the in-vehicle multi-zone interaction method provided in this embodiment of the invention can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software units in the decoding processor. The software units can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 702, and processor 701 reads the information in memory 702 and combines it with hardware to complete the steps of the method.

[0146] Those skilled in the art will understand that although some embodiments described herein include certain features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of the invention and form different embodiments.

[0147] Those skilled in the art will understand that the descriptions of the various embodiments have different focuses, and for parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0148] Although embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention. All such modifications and variations fall within the scope defined by the appended claims. The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0149] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A vehicle-mounted multi-zone interactive method, characterized in that, include: Acquire the microphone received signal and the reference signal; The microphone received signal and the reference signal are converted to the short-time frequency domain; Echo cancellation is performed on the signal converted to the short-time frequency domain; Determine which frequency range the echo-cancelled signal belongs to; The echo-cancelled signal is separated to obtain the separated signal for each tone region; The separated signals of each pitch range are converted to obtain the final time-domain output of each pitch range; The actual wake-up position of the tone zone signal is determined based on the final time-domain output of each tone zone; The signal from the actual wake-up location is input into the recognition engine to recognize the command and perform the operation. After completing the corresponding command operation, the voice interaction is completed through TTS feedback; The process of separating the echo-cancelled signal to obtain the separation signal for each pitch range includes: primary pitch range separation and secondary pitch range separation; The first-level separation of the sound register includes: if the current echo-cancelled signal belongs to the first sound register, then the corresponding adaptive filter of the first sound register is not updated; if the current echo-cancelled signal does not belong to the first sound register, then the corresponding adaptive filter of the first sound register is updated to suppress signals outside the first sound register. The two-stage separation of the sound registers includes: taking the signal corresponding to the first sound register as the near-end signal and the signals corresponding to other sound registers as the far-end signals, and sending them into an adaptive filter to further suppress the signals of other sound registers, thereby obtaining the final separated signals of each sound register.

2. The in-vehicle multi-zone interaction method according to claim 1, characterized in that, The step of converting the microphone received signal and the reference signal to the short-time time-frequency domain includes: The microphone received signal and the reference signal are converted to the short-time time-frequency domain by performing a short-time Fourier transform.

3. The in-vehicle multi-zone interaction method according to claim 1, characterized in that, The echo cancellation of the signal converted to the short-time frequency domain includes: Linear echo cancellation is performed on the signal converted to the short-time frequency domain; Residual echo suppression is applied to the signal after linear echo cancellation. Echo cancellation is achieved through an adaptive filtering method, while residual echo suppression is achieved through deep learning to suppress residual echoes and non-human voice noise.

4. The in-vehicle multi-zone interaction method according to claim 1, characterized in that, The determination of which frequency range the echo-cancelled signal belongs to includes: Calculate the energy of the echo-cancelled signal based on the echo-cancelled signal. The time delay of the echo-cancelled signal between different microphone pairs is calculated using the generalized cross-correlation method based on the distance between each microphone. The echo-cancelled signal is determined to belong to a specific frequency range based on its energy and the time delay of different microphone pairs.

5. The in-vehicle multi-zone interaction method according to claim 1, characterized in that, The conversion of the output signal for each pitch range to obtain the final time-domain output for each pitch range includes: The output signal of each tone zone is subjected to a short-time inverse Fourier transform to obtain the final time-domain output of each tone zone.

6. The in-vehicle multi-zone interaction method according to claim 1, characterized in that, The step of determining the wake-up position of the tone zone signal based on the final time-domain output of each tone zone includes: The final time-domain output of each tone zone is sent to the wake-up engine, and the time-domain output data is buffered. Determine if there is only one location to wake up. If so, there is no need to calculate the wake word energy, and the corresponding location is taken as the actual wake-up location. If there are multiple wake-up locations, calculate the signal energy corresponding to the wake-up time for each wake-up location, and determine the location with the highest signal energy as the actual wake-up location.

7. A vehicle-mounted multi-zone interactive device, employing the vehicle-mounted multi-zone interactive method according to any one of claims 1 to 6, characterized in that, include: Acquisition module: used to acquire the microphone received signal and reference signal; First conversion module: used to convert the microphone received signal and the reference signal to the short-time frequency domain; Echo cancellation module: used to cancel the echo of signals converted to the short-time frequency domain; Judgment module: Used to determine which frequency range the echo-cancelled signal belongs to; Separation module: used to separate the echo-cancelled signal to obtain the separation signal for each tone range; The second conversion module is used to convert the separated signals of each tone range to obtain the final time-domain output of each tone range; Determining module: used to determine the actual wake-up position of the tone zone signal based on the final time-domain output of each tone zone; Input module: Used to input the signal from the actual wake-up location into the recognition engine, recognize the commands, and perform the operation; Voice interaction module: Used to complete voice interaction by providing TTS feedback after the corresponding command operation is completed.

8. An electronic device, characterized in that, include: Processor and memory; The processor executes a vehicle multi-zone interaction method as described in any one of claims 1 to 6 by calling programs or instructions stored in the memory.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program or instructions that cause a computer to perform a vehicle-mounted multi-zone interaction method as described in any one of claims 1 to 6.

Images