Mobile equipment voice interception method and system based on vehicle-mounted multi-voice-area scene

Abstract

The invention relates to the technical field of vehicle-mounted voice access, in particular to a mobile device voice interception method and system based on a vehicle-mounted multi-voice-area scene, and the method comprises the steps: collecting audio signals of different areas in a vehicle-mounted system, and carrying out the preprocessing of the audio signals; dividing the preprocessed audio signal into a plurality of sound areas according to different spatial sources; sequentially judging whether voice exists in the divided audio signals or not, and if the voice does not exist in the audio signals, directly ending the process; if the voice exists in the audio signal, determining a main voice area of the audio signal through a sound source localization algorithm; inputting the determined audio signal of the main sound area into a preset human mouth and equipment sound production detection model, and if the model detects that a sound source is mobile equipment, judging that the sound source is invalid input; and if the model detects that the sound source is a human mouth, starting voice recognition, and analyzing and executing a voice instruction of the user. According to the invention, the external playing human voice and the real user voice can be effectively distinguished in the vehicle-mounted multi-voice-area scene.

Description

Technical Field

[0001] This application relates to the field of in-vehicle voice technology, and in particular to a method and system for intercepting voice from mobile devices in a multi-voice zone in-vehicle scenarios. Background Technology

[0002] With the development of in-vehicle voice interaction systems, the accuracy of speech recognition and voice wake-up in multi-zone environments has become crucial for improving user experience. Voice Activity Detection (VAD), as a pre-processing module for speech processing, is primarily used to determine which time periods in the audio stream contain speech signals and which are silence or non-speech noise, thus providing reliable input for subsequent functions such as speech recognition and speech enhancement. In complex environments, VAD systems typically combine spatial audio processing technologies such as Sound Source Localization (SSL) and beamforming, using multi-microphone arrays to estimate the direction of sound sources or enhance sound from specific directions to achieve speech detection of the target speaker. These technologies are widely used in communication, in-vehicle voice systems, and other scenarios to improve the accuracy and robustness of speech detection.

[0003] However, existing technologies still have significant limitations in multi-zone in-vehicle scenarios. Specifically, when external devices, such as mobile phones, tablets, or in-vehicle screens, are playing human voices, traditional VAD systems combining SSL or beamforming can easily misinterpret the externally played voice as the real user's voice, thus triggering incorrect commands. This is because the spectral characteristics of externally played human voices are highly similar to those of natural human voices, and the sound source direction may be close to the target user, making it difficult to distinguish them using traditional acoustic features such as energy, zero-crossing rate, or MFCC. This problem is prevalent in multi-device in-vehicle scenarios, seriously affecting the reliability of voice interaction systems and user experience. Therefore, effectively distinguishing between externally played human voices and real user voices in multi-zone in-vehicle scenarios remains a challenge. Summary of the Invention

[0004] This application provides a method and system for intercepting voice signals from mobile devices in a multi-zone in-vehicle environment, which can effectively distinguish between externally played human voices and real user voices in such environments. The technical solution provided in this application is as follows: Firstly, this application provides a method for intercepting voice messages on mobile devices in a multi-zone vehicle environment, the method comprising: The audio signals from different regions are collected in the vehicle system and preprocessed. The preprocessed audio signals are then divided into multiple sound zones according to their different spatial sources. The process involves sequentially determining whether speech exists in the segmented audio signals. If no speech is found in the audio signals, the process ends directly. If the audio signal contains speech, the main sound region of the audio signal is determined by a sound source localization algorithm; The audio signal of the main sound range is input into the preset human mouth and device sound detection model. If the model detects that the sound source is a mobile device, it is judged as an invalid input. If the model detects that the sound source is a human mouth, it will enable speech recognition, parse and execute the user's voice commands.

[0005] In one specific implementation scheme, the step of acquiring audio signals from different regions in the vehicle system and preprocessing the audio signals, and then dividing the preprocessed audio signals into multiple sound zones according to different spatial sources, includes: The in-vehicle system uses a multi-microphone array to collect audio signals from different areas inside the vehicle. Blind source separation and echo cancellation are performed on the acquired audio signals; The processed audio signal is divided into multiple sound zones according to its spatial origin, with each sound zone corresponding to a specific area inside the vehicle.

[0006] In a specific feasible implementation, the step of sequentially determining whether speech exists in the segmented audio signal includes: The audio signals of each divided sound region are sequentially input into the speech activity detection module, which is based on the speech activity detection algorithm and is used to determine whether the audio of the corresponding sound region contains speech. The speech activity detection module analyzes the audio signal of each vocal range, extracts its acoustic features for discrimination, and determines whether there is a speech signal in the current time period of that vocal range.

[0007] In one specific feasible implementation, the process of constructing the human mouth and device sound emission detection model includes: A dataset is constructed, which includes real human mouth speech collected in various scenarios as well as device speech collected by playing from mobile phones, tablets and speakers of different brands; The samples in the dataset cover different distances, angles, reverberation, and noise conditions; By comparing various network structures and input features, a network model suitable for voice recognition was determined and optimized. The selected network model was then trained in a supervised manner using a large-scale dataset to obtain a voice detection model for human mouths and devices.

[0008] In one specific implementation scheme, if the model detects that the sound source is a human mouth, then starting speech recognition, parsing and executing the user's voice commands includes: If the detection result of the human mouth and device sound detection model is that the human mouth is making a sound, activate the voice command recognition function and start the voice recognition process; The system uses a recognition model to receive and parse user-input voice commands, and maps the parsing results to corresponding control operations to fulfill the user's interaction needs.

[0009] In one specific implementation scheme, if the audio signal contains speech, determining the main vocal range of the audio signal using a sound source localization algorithm includes: After receiving an audio signal from a region marked as containing speech, the input audio signal is processed using a sound source localization method based on a multi-microphone array. The direction parameters of the sound source are calculated and mapped to a preset spatial region inside the vehicle. Based on the spatial mapping relationship, the main region corresponding to the audio signal is identified.

[0010] Secondly, this application provides a mobile device voice interception system based on a multi-zone vehicle environment, employing the following technical solution: A mobile device voice interception system based on a multi-zone vehicle environment includes: The audio acquisition module is used to acquire audio signals from different areas in the vehicle system and preprocess the audio signals, dividing the preprocessed audio signals into multiple sound zones according to different spatial sources; The speech detection module is used to sequentially determine whether there is speech in the segmented audio signal. If there is no speech in the audio signal, the process ends directly. A sound source localization module is used to determine the main sound region of the audio signal if speech is present in the audio signal using a sound source localization algorithm. The sound detection module is used to input the audio signal of the determined main sound range into the preset human mouth and device sound detection model. If the model detects that the sound source is a mobile device, it is judged as invalid input. The speech recognition module is used to enable speech recognition if the model detects that the sound source is a human mouth, and then parses and executes the user's voice commands.

[0011] Thirdly, this application provides an electronic device, the device including a processor and a memory; the memory stores a program, the program being loaded and executed by the processor to implement a mobile device voice interception method based on a multi-zone vehicle scenario as described in the first aspect.

[0012] Fourthly, this application provides a computer-readable storage medium storing a program that, when executed by a processor, is used to implement a mobile device voice interception method based on a multi-zone vehicle scenario as described in the first aspect.

[0013] In summary, the beneficial effects of this application include at least the following: (1) By sequentially executing a multi-level detection process in a multi-zone vehicle environment, including audio signal preprocessing, speech activity detection, sound source localization, and recognition of human mouth and device speech, the system can effectively distinguish between real human speech and speech played by mobile devices, thereby achieving the function of intercepting mobile device speech. This technology ensures that the vehicle voice control system only responds to the speech of real users, avoiding erroneous control of the vehicle state caused by external devices simulating speech or playing voice commands, and significantly improving the reliability and safety of the system interaction process.

[0014] (2) The detection method has good reproducibility and adaptability, and can maintain stable recognition capabilities in various in-vehicle scenarios. By constructing a large-scale training dataset covering various environmental conditions and using a deep learning model to analyze the sound source's vocal mechanism characteristics, it can accurately determine the sound source type even in noisy environments, reverberant scenarios, and multi-angle sound sources, thus ensuring the system's robustness in complex in-vehicle environments. This effect not only avoids interference from mobile device voice on vehicle operation, but also provides users with a safe, convenient, and continuously optimizeable voice interaction experience.

[0015] Audio signals from different areas within the vehicle are collected using a multi-microphone array and pre-processed, then divided into multiple sound zones based on their spatial origin. Speech activity detection is performed on each sound zone sequentially to filter out signals containing speech. The audio containing speech is further localized to determine its primary sound zone. The audio from the primary sound zone is then input into a human mouth and device voice detection model. A deep learning model analyzes time-frequency characteristics, speech initiation characteristics, and differences in vocal tract response to determine whether the speech is naturally produced by a human mouth or played from an external device. Finally, the speech recognition module is activated for audio determined to be produced by a human mouth, parsing and executing the user's voice commands. This solution effectively distinguishes between externally played human voices and genuine user voices in complex multi-sound zone environments within a vehicle, achieving precise interception of mobile device voice and preventing accidental system operation triggered by external device voice. This solves the current technical challenge of reliably determining the source of speech in in-vehicle voice interaction systems under multi-source interference, improving the safety and accuracy of voice interaction.

[0016] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the mobile device voice interception method based on a multi-zone vehicle environment in this application embodiment.

[0018] Figure 2This is a schematic diagram of the overall process of the mobile device voice interception method based on the multi-voice zone scenario in the vehicle, as described in this application.

[0019] Figure 3 This is a structural block diagram of a mobile device voice interception system based on a multi-zone vehicle scenario in this application embodiment.

[0020] Figure 4 This is a block diagram of an electronic device for voice interception of mobile devices based on a multi-zone vehicle environment, as described in this application embodiment. Detailed Implementation

[0021] The specific embodiments of this application will be described in further detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate this application, but are not intended to limit the scope of this application.

[0022] Optionally, this application uses the mobile device voice interception method based on the vehicle multi-zone scenario provided in various embodiments as an example for application in an electronic device. The electronic device is a terminal or a server. The terminal can be a computer, tablet computer, etc. This embodiment does not limit the type of electronic device.

[0023] Reference Figure 1 This is a flowchart illustrating a method for intercepting voice signals from a mobile device in a multi-zone vehicle environment, according to an embodiment of this application. The method includes at least the following steps: Step S101: Collect audio signals from different areas in the vehicle system and preprocess the audio signals. Divide the preprocessed audio signals into multiple sound zones according to different spatial sources.

[0024] In step S101, audio signals from different areas inside the vehicle are acquired using a multi-microphone array in the in-vehicle system, and preprocessing is performed on the acquired signals. Specifically, blind source separation and echo cancellation are performed on the acquired audio signals. The processed audio signals are divided into multiple sound zones based on their spatial source, such as sound zone 1, sound zone 2, up to sound zone n. Each sound zone corresponds to a specific area inside the vehicle. By performing this step, echo and aliasing signals can be suppressed before the signal enters the subsequent processing link, and sound zone division based on spatial source can be formed, thereby providing clearer and more independent audio input for subsequent speech detection and sound source analysis.

[0025] Step S102: Determine whether there is speech in the divided audio signal. If there is no speech in the audio signal, the process ends directly.

[0026] In step S102, the audio signals of each divided sound region are sequentially input to the speech activity detection module. The speech activity detection module, based on the speech activity detection (VAD) algorithm, determines whether the audio in the corresponding sound region contains speech. Specifically, the speech activity detection module analyzes the audio signal of each sound region, extracts its acoustic features for discrimination, and determines whether a speech signal exists in that sound region during the current time period. If the detection result indicates that the audio signal does not contain a speech signal, the processing flow for that sound region ends directly and does not proceed to subsequent processing steps.

[0027] Step S103: If there is speech in the audio signal, determine the main sound region of the audio signal through the sound source localization algorithm.

[0028] In step S103, for the audio signal detected as containing speech, the main source location of the speech signal is further determined using a sound source localization algorithm, thereby determining the main sound region where the sound source is located. Specifically, after receiving the audio signal marked as containing speech, a sound source localization method based on a multi-microphone array, such as a time delay estimation algorithm, generalized cross-correlation phase transform, or beam pointing search, is used to process the input audio signal, calculate the direction parameters of the sound source, and map these direction parameters to a preset spatial area inside the vehicle. Combining the spatial mapping relationship, the main sound region corresponding to the audio signal can be further confirmed.

[0029] Step S104: Input the audio signal of the determined main sound range into the preset human mouth and device sound detection model. If the model detects that the sound source is a mobile device, it is judged as invalid input.

[0030] In step S104, the audio signal of the determined main sound range is input to the pre-built human mouth and device sound detection model to determine whether the audio signal is actually produced by a human mouth.

[0031] Specifically, this step treats human mouth sounds and device sounds as two different sound source types, constructing a deep learning model capable of identifying the sound production mechanism. This model performs multi-dimensional feature analysis on the input audio signal, including time-frequency features, sound production initiation features, and differences in vocal tract response, to determine whether the audio originates from natural human mouth sounds or is played from an external device. If the detection model indicates that the sound is from a device, it is considered invalid input, and the process ends.

[0032] In implementation, a large-scale dataset was first constructed, containing real human mouth speech collected in various scenarios, such as dialogues, commands, and wake words, as well as device speech collected from different brands of mobile phones, tablets, and speakers. The dataset samples were required to cover different distances, angles, reverberation, and noise conditions to ensure the model's generalization ability. After comparing various network structures and input features, a deep model suitable for speech pattern recognition was finally determined and optimized. Supervised training was performed on the selected deep network using the constructed large-scale dataset. Preprocessed audio samples with extracted features and their mouth / device labels were input into the network. Cross-entropy loss was used, and optimizers such as Adam were employed for small-batch iterative updates of the network parameters until training converged. The resulting model constitutes the human mouth and device speech detection model of this application. The detection method exhibits good reproducibility and adaptability, maintaining stable recognition capabilities in various in-vehicle scenarios. By constructing a large-scale training dataset covering various environmental conditions and employing a deep learning model to analyze the sound source's emission mechanism characteristics, the system can accurately determine the sound source type even in noisy environments, reverberant scenarios, and multi-angle sound sources, thus ensuring the system's robustness in complex in-vehicle environments. This not only avoids interference from mobile device voice commands on vehicle operation but also provides users with a safe, convenient, and continuously optimizeable voice interaction experience.

[0033] Step S105: If the model detects that the sound source is a human mouth, start speech recognition, parse and execute the user's voice command.

[0034] In step S105, if the detection result of the human mouth and device sound detection model is that the human mouth is producing sound, then the voice command recognition function is activated, and the voice recognition process is initiated. The existing recognition model is used to receive and parse the user's input voice commands, and the parsing results are mapped to corresponding control operations, such as executing system functions like opening a window or opening the trunk, to fulfill the user's interaction needs.

[0035] In summary, combining Figure 2This system collects audio signals from different areas within the vehicle using a multi-microphone array and preprocesses them, dividing them into multiple sound zones based on their spatial origin. Then, it sequentially performs speech activity detection on the audio in each zone to filter out signals containing speech. The audio containing speech is further localized to determine its main sound zone. The audio from the main sound zone is then input into a human mouth and device voice detection model. A deep learning model analyzes time-frequency characteristics, speech initiation characteristics, and differences in vocal tract response to determine whether the speech is naturally produced by a human mouth or played from an external device. Finally, the audio determined to be produced by a human mouth activates the speech recognition module, parsing and executing the user's voice commands. This solution effectively distinguishes between externally played human voices and genuine user voices in complex multi-sound zone environments within a vehicle, achieving precise interception of mobile device voice and preventing accidental system operation triggered by external device voice. This solves the current technical challenge of in-vehicle voice interaction systems being unable to reliably determine the source of speech under multi-source interference, thus improving the safety and accuracy of voice interaction.

[0036] Figure 3 This is a structural block diagram of a mobile device voice interception system based on a multi-zone vehicle environment, according to an embodiment of this application. The system includes at least the following modules: The audio acquisition module is used to acquire audio signals from different areas in the vehicle system and preprocess the audio signals, dividing the preprocessed audio signals into multiple sound zones according to different spatial sources; The speech detection module is used to determine whether there is speech in the segmented audio signal. If there is no speech in the audio signal, the process ends directly. The sound source localization module is used to determine the main sound region of the audio signal if speech is present in the audio signal, using a sound source localization algorithm. The sound detection module is used to input the audio signal of the determined main sound range into the preset human mouth and device sound detection model. If the model detects that the sound source is a mobile device, it is judged as invalid input. The speech recognition module is used to enable speech recognition if the model detects that the sound source is a human mouth, and then parses and executes the user's voice commands.

[0037] For relevant details, please refer to the above method implementation examples.

[0038] Figure 4 This is a block diagram of an electronic device provided in one embodiment of this application. The device includes at least a processor 401 and a memory 402.

[0039] Processor 401 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 401 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 401 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 401 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 401 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0040] The memory 402 may include one or more computer-readable storage media, which may be non-transitory. The memory 402 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 402 is used to store at least one instruction, which is executed by the processor 401 to implement the mobile device voice interception method based on a multi-zone vehicle environment provided in the method embodiments of this application.

[0041] In some embodiments, the electronic device may also optionally include: a peripheral device interface and at least one peripheral device. The processor 401, memory 402, and peripheral device interface can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface via a bus, signal line, or circuit board. Indicatively, peripheral devices include, but are not limited to: radio frequency circuits, touch displays, audio circuits, and power supplies.

[0042] Of course, electronic devices may also include fewer or more components, and this embodiment does not limit this.

[0043] Optionally, this application also provides a computer-readable storage medium storing a program that is loaded and executed by a processor to implement the mobile device voice interception method based on a multi-zone vehicle scenario described in the above method embodiments.

[0044] Optionally, this application also provides a computer product including a computer-readable storage medium storing a program, which is loaded and executed by a processor to implement the mobile device voice interception method based on the in-vehicle multi-zone scenario described in the above method embodiments.

[0045] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0046] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for intercepting voice messages on mobile devices in a multi-zone vehicle environment, characterized in that, The method includes: The audio signals from different regions are collected in the vehicle system and preprocessed. The preprocessed audio signals are then divided into multiple sound zones according to their different spatial sources. The process involves sequentially determining whether speech exists in the segmented audio signals. If no speech is found in the audio signals, the process ends directly. If the audio signal contains speech, the main sound region of the audio signal is determined by a sound source localization algorithm; The audio signal of the main sound range is input into the preset human mouth and device sound detection model. If the model detects that the sound source is a mobile device, it is judged as an invalid input. If the model detects that the sound source is a human mouth, it will enable speech recognition, parse and execute the user's voice commands.

2. The method for intercepting voice messages on mobile devices based on in-vehicle multi-zone scenarios according to claim 1, characterized in that, The process of acquiring audio signals from different regions in the vehicle system and preprocessing the audio signals, then dividing the preprocessed audio signals into multiple sound zones according to different spatial sources, includes: The in-vehicle system uses a multi-microphone array to collect audio signals from different areas inside the vehicle. Blind source separation and echo cancellation are performed on the acquired audio signals; The processed audio signal is divided into multiple sound zones according to its spatial source, with each sound zone corresponding to a specific area inside the vehicle.

3. The method for intercepting voice messages on mobile devices based on in-vehicle multi-zone scenarios according to claim 1, characterized in that, The step of sequentially determining whether speech exists in the segmented audio signal includes: The audio signals of each divided sound region are sequentially input into the speech activity detection module, which is based on the speech activity detection algorithm and is used to determine whether the audio of the corresponding sound region contains speech. The speech activity detection module analyzes the audio signal of each vocal range, extracts its acoustic features for discrimination, and determines whether there is a speech signal in the current time period of that vocal range.

4. The method for intercepting voice messages on mobile devices based on in-vehicle multi-zone scenarios according to claim 1, characterized in that, The process of constructing the human mouth and device sound emission detection model includes: A dataset is constructed, which includes real human mouth speech collected in various scenarios as well as device speech collected by playing from mobile phones, tablets and speakers of different brands. The samples in the dataset cover different distances, angles, reverberation, and noise conditions; By comparing various network structures and input features, a network model suitable for voice generation mode recognition was determined and optimized. The selected network model was then trained in a supervised manner using a large-scale dataset to obtain a voice generation detection model for human mouth and device.

5. The method for intercepting voice messages on mobile devices based on in-vehicle multi-zone scenarios according to claim 1, characterized in that, If the model detects a human mouth as the sound source, then enabling speech recognition, parsing and executing the user's voice commands includes: If the detection result of the human mouth and device sound detection model is that the human mouth is making a sound, activate the voice command recognition function and start the voice recognition process; The system uses a recognition model to receive and parse user-input voice commands, and maps the parsing results to corresponding control operations to fulfill the user's interaction needs.

6. The method for intercepting voice messages on mobile devices based on in-vehicle multi-zone scenarios according to claim 1, characterized in that, If the audio signal contains speech, determining the main vocal range of the audio signal using a sound source localization algorithm includes: After receiving an audio signal from a region marked as containing speech, the input audio signal is processed using a sound source localization method based on a multi-microphone array. The direction parameters of the sound source are calculated and mapped to a preset spatial region inside the vehicle. Based on the spatial mapping relationship, the main region corresponding to the audio signal is identified.

7. A voice interception system for mobile devices based on in-vehicle multi-zone scenarios, characterized in that, include: The audio acquisition module is used to acquire audio signals from different areas in the vehicle system and preprocess the audio signals, dividing the preprocessed audio signals into multiple sound zones according to different spatial sources; The speech detection module is used to sequentially determine whether there is speech in the segmented audio signal. If there is no speech in the audio signal, the process ends directly. A sound source localization module is used to determine the main sound region of the audio signal if speech is present in the audio signal using a sound source localization algorithm. The sound detection module is used to input the audio signal of the determined main sound range into the preset human mouth and device sound detection model. If the model detects that the sound source is a mobile device, it is judged as invalid input. The speech recognition module is used to enable speech recognition if the model detects that the sound source is a human mouth, and then parses and executes the user's voice commands.

8. An electronic device, characterized in that, The device includes a processor and a memory; the memory stores a program, which is loaded and executed by the processor to implement a mobile device voice interception method based on a multi-zone vehicle environment as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The storage medium stores a program, which, when executed by a processor, is used to implement a mobile device voice interception method based on a multi-zone vehicle environment as described in any one of claims 1 to 6.

Images