Audio processing method and system based on vibration signal outside vehicle

Abstract

The invention provides an audio processing method and system based on an external vibration signal of a vehicle, which can acquire the external vibration signal of the vehicle through a single or a plurality of vibration sensors and perform audio processing, and comprises the following steps of: firstly deploying the vibration sensors, and acquiring vehicle vibration signal data by picking up remote sound outside the vehicle; vehicle vibration signal data are reconstructed into an external audio signal through the DSP module, an internal audio signal is acquired through the digital microphone, then algorithm processing is carried out on two different audio data through the DSP module, external output voice is acquired, and a traditional external microphone is easily affected by the environment and is low in signal-to-noise ratio during high-speed driving; the vehicle passive perception capability is insufficient, and sound events in a specific direction are difficult to capture, so that the accuracy of out-of-vehicle voice interaction recognition and the perception capability of an environment sound source can be improved, the disability condition of a traditional microphone in a complex environment is avoided, and the reliability is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of external vehicle audio pickup and processing, and more specifically, particularly relates to an audio processing method and system based on external vehicle vibration signals. Background Art

[0002] Under the wave of the development of vehicle intelligence and networking, the vehicle's demand for perceiving external environmental information has become increasingly urgent. The external vehicle microphone, as a key sensor for the vehicle to obtain external sound information, plays an important role in scenarios such as automatic emergency braking warning, pedestrian detection, road noise monitoring, and vehicle-road-person interaction. For example, in an intelligent driving system, the external vehicle microphone can assist the vehicle in making obstacle avoidance decisions by capturing the footsteps of pedestrians and the honking of vehicles; in the scenario of external vehicle voice interaction, clear and accurate audio collection is the basis for realizing natural dialogue between humans and vehicles.

[0003] Traditional external vehicle microphones are easily affected by environmental factors such as wind noise, rain, and snow, have a low signal-to-noise ratio when driving at high speeds, and are prone to failure in complex environments, with low reliability. In addition, traditional external vehicle microphones have insufficient passive sensing capabilities for the vehicle and are difficult to capture sound events in specific directions, such as honking and emergency vehicle alarms. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention provides an audio processing method and system based on external vehicle vibration signals to solve the technical problems of insufficient passive sensing capabilities and low reliability of traditional external vehicle microphones in the prior art.

[0005] The purpose and efficacy of an audio processing method and system based on external vehicle vibration signals of the present invention are achieved by the following specific technical means:

[0006] An audio processing method based on external vehicle vibration signals includes the following steps:

[0007] Deploy vibration sensors, where the vibration sensors are one or more;

[0008] When the vehicle is running or stationary, the vibration sensors installed on the vehicle body collect the acoustic wave vibrations caused by the arrival of acoustic waves outside the vehicle on the vehicle surface to obtain vehicle vibration signal data, and collect in-vehicle audio based on a digital microphone to obtain in-vehicle audio signals;

[0009] Perform acoustic signal reconstruction on the vehicle vibration signal data to obtain an external vehicle audio signal;

[0010] Input the in-vehicle audio signal and the external vehicle audio signal into the DSP module. The DSP module contains a specific self-developed speech algorithm library, and the specific self-developed speech algorithm library includes a speech interaction algorithm library and an alarm sound detection algorithm library for FSD;

[0011] Based on the DSP module, perform acoustic interference cancellation on the in-vehicle audio signal, and perform noise reduction on the in-vehicle audio signal and the out-of-vehicle audio signal. Perform secondary noise reduction on the in-vehicle audio signal and the out-of-vehicle audio signal based on a traditional algorithm noise reduction model or DNN;

[0012] Based on the DSP module, perform frequency adjustment and AGC control on the in-vehicle audio signal and the out-of-vehicle audio signal, and perform enhancement processing on the out-of-vehicle audio signal, retaining only the data of the single enhanced out-of-vehicle sound signal.

[0013] As a further solution of the present invention, the performing acoustic interference cancellation on the in-vehicle audio signal based on the DSP module includes:

[0014] Eliminate the in-vehicle call audio data and the infotainment audio data through a multi-band residual adaptive filter, and retain only the external sound audio data;

[0015] Or, eliminate the in-vehicle call audio data and the infotainment audio data through a multi-band residual adaptive filter and an echo suppressor, and retain only the external sound audio data.

[0016] As a further solution of the present invention, the method further includes:

[0017] The in-vehicle audio signal includes the in-vehicle call audio data and the infotainment audio data.

[0018] As a further solution of the present invention, the performing noise reduction on the in-vehicle audio signal and the out-of-vehicle audio signal includes:

[0019] Detect the stationary and transient noises in the received and transmitted in-vehicle audio signal and out-of-vehicle audio signal.

[0020] As a further solution of the present invention, the performing frequency adjustment and AGC control on the in-vehicle audio signal and the out-of-vehicle audio signal based on the DSP module includes:

[0021] Perform transient signal equalization on the in-vehicle audio signal and the out-of-vehicle audio signal;

[0022] Perform frequency adjustment and signal automatic gain control on the in-vehicle audio signal and the out-of-vehicle audio signal.

[0023] As a further solution of the present invention, the deploying the vibration sensor and the digital microphone includes:

[0024] The vibration sensor is deployed to a rigid contact part inside the vehicle body, and the digital microphone is deployed inside the vehicle.

[0025] As a further solution of the present invention, the single enhanced external sound signal data is represented as audio signal data that only retains the external audio.

[0026] As a further solution of the present invention, the method further includes:

[0027] Deploy a digital microphone.

[0028] An audio processing system based on an external vehicle vibration signal, comprising:

[0029] A vibration sensor, the vibration sensor is installed at a rigid contact part inside the vehicle body, and the vibration sensor is used to collect the acoustic wave vibration caused by the external acoustic wave reaching the vehicle surface, and obtain vehicle vibration signal data;

[0030] A digital microphone, the digital microphone is used to collect the in-vehicle audio, obtain the in-vehicle audio signal, and the digital microphone is installed inside the vehicle;

[0031] A DSP module, the DSP module is used to reconstruct the acoustic signal of the vehicle vibration signal data to obtain an external vehicle audio signal, and is used to perform audio processing on the external vehicle audio signal and the in-vehicle audio signal, and retain a single enhanced external sound signal data.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] The present invention provides a method and system for collecting external vehicle vibration signals by a single or multiple vibration sensors and performing acoustic architecture reconstruction and processing, which is used to enhance the vehicle environment perception ability. First, the vibration sensors are deployed. When the vehicle is running or stationary, the vibration sensors collect the acoustic wave vibration caused by the external acoustic wave reaching the vehicle surface - by picking up the long-distance sound outside the vehicle, obtain vehicle vibration signal data, collect the in-vehicle audio based on a digital microphone to obtain the in-vehicle audio signal, and then reconstruct the acoustic signal of the vehicle vibration signal data through the DSP module to obtain the external vehicle audio signal, and can process the in-vehicle audio signal and the external vehicle audio signal to output the external output voice. The traditional external vehicle microphone is easily affected by environmental factors such as wind noise, rain and snow, and has a low signal-to-noise ratio when driving at high speed, and the vehicle's passive perception ability is insufficient, making it difficult to capture sound events in specific directions (such as honking, emergency vehicle alarms). This can improve the accuracy of external vehicle voice interaction recognition and the perception ability of environmental sound sources, and can effectively prevent related electromagnetic interference. At the same time, the traditional MEMS microphone is prone to failure in complex environments, while the vibration sensor of the present invention is installed inside the vehicle body, and the installation position can ensure effective vibration, so it can adapt to complex environments, such as extreme environments and vibration environments, and has high reliability. Description of the Drawings

[0034] Figure 1 It is the flowchart of the steps of an audio processing method based on the vehicle exterior vibration signal of the present invention. Specific implementation mode

[0035] The following further describes in detail the implementation mode of the present invention in conjunction with the drawings and embodiments. The following embodiments are used to illustrate the technical solutions of the present invention, but cannot be used to limit the protection scope of the present invention.

[0036] Embodiment 1:

[0037] As shown in the Figure 1 accompanying drawings:

[0038] The present invention provides an audio processing method based on the vehicle exterior vibration signal, which is applicable to vehicle exterior audio pickup and processing, and includes the following steps:

[0039] Step S1, deploy vibration sensors.

[0040] Specifically, in addition to deploying vibration sensors, digital microphones can also be deployed. The vibration sensors are deployed to the rigid contact parts inside the vehicle body, and the digital microphones are deployed inside the vehicle.

[0041] It can be understood that traditional MEMS microphones are prone to failure in complex environments, while the vehicle exterior microphone made based on vibration sensors can be installed at the rigid contact parts inside the vehicle body, and the installation position can ensure effective vibration, so as to adapt to complex environments, such as extreme environments and vibration environments, and has high reliability.

[0042] Step S2, when the vehicle is running or stationary, the vibration sensors installed on the vehicle body collect the acoustic wave vibrations caused by the arrival of the acoustic waves outside the vehicle on the vehicle surface, obtain vehicle vibration signal data, and collect the in-vehicle audio based on the digital microphones to obtain in-vehicle audio signals.

[0043] Step S3, perform acoustic signal reconstruction on the vehicle vibration signal data to obtain vehicle exterior audio signals.

[0044] Step S4, input the in-vehicle audio signal and the vehicle exterior audio signal into the DSP module. The DSP module contains a specific self-developed voice algorithm library, and the specific self-developed voice algorithm library includes a voice interaction algorithm library and an alarm sound detection algorithm library for FSD.

[0045] Step S5, based on the DSP module, perform acoustic interference cancellation on the in-vehicle audio signal, and perform noise reduction on the in-vehicle audio signal and the vehicle exterior audio signal. Perform secondary noise reduction on the in-vehicle audio signal and the vehicle exterior audio signal based on the traditional algorithm noise reduction model or DNN.

[0046] In this embodiment, the in-vehicle call audio data and the infotainment audio data are eliminated by a multi-band residual adaptive filter, and only the external sound audio data is retained.

[0047] It can be understood that the multi-band residual adaptive filter is a refined noise reduction technology for complex noise scenarios, which eliminates residual band-specific noise. Its core is to decompose the signal into multiple sub-bands, and in each sub-band, an adaptive algorithm such as LMS is used to dynamically track and suppress the residual noise components, and finally merged and reconstructed into a high signal-to-noise ratio signal. By means of sub-band refined processing, the deficiencies of global noise reduction are made up, which is especially suitable for scenarios that need to balance real-time performance and multi-band noise residue.

[0048] Furthermore, the in-vehicle audio signal includes the in-vehicle call audio data and the infotainment audio data. The multi-band residual adaptive filter and the echo suppressor eliminate the in-vehicle call and infotainment sounds, and only the external sound is retained.

[0049] Alternatively, the in-vehicle call audio data and the infotainment audio data are eliminated by a multi-band residual adaptive filter and an echo suppressor, and only the external sound audio data is retained.

[0050] It can be understood that the echo suppressor is a key audio processing device for eliminating echo interference during a call. It dynamically suppresses the echo component by real-time monitoring and analyzing the sound signal, accurately identifying and separating the delayed echo generated by acoustic feedback or line reflection, and using algorithms such as adaptive filtering and signal cancellation.

[0051] In this embodiment, the stationary and transient noises in the received and transmitted in-vehicle audio signal and the out-of-vehicle audio signal are detected and attenuated, such as the stationary and transient noises generated by traffic, bars, passing vehicles, etc.

[0052] In this embodiment, transient signal equalization is performed on the in-vehicle audio signal and the out-of-vehicle audio signal, and frequency adjustment and signal automatic gain control are performed on the in-vehicle audio signal and the out-of-vehicle audio signal.

[0053] Step S6, based on the DSP module, frequency adjustment and AGC control are performed on the in-vehicle audio signal and the out-of-vehicle audio signal, and enhancement processing is performed on the out-of-vehicle audio signal, and only the single enhanced out-of-vehicle sound signal data is retained.

[0054] Specifically, the single enhanced out-of-vehicle sound signal data is represented as the audio signal data that only retains the out-of-vehicle audio.

[0055] It is understandable that the external microphone made by using a vibration sensor and adopting the method provided by the present invention acquires an external vehicle audio signal through a vibration sensor module, and can process the external vehicle audio signal and the internal vehicle audio signal through relevant algorithms. Subsequently, only the audio outside the vehicle body is output, which can improve the accuracy of audio recognition and effectively prevent relevant electromagnetic interference. At the same time, traditional MEMS microphones are prone to failure in complex environments, while this external vehicle microphone is installed inside the vehicle body, and the installation position can ensure effective vibration, so as to adapt to complex environments (such as extreme environments and vibration environments) and has high reliability.

[0056] In this embodiment, when the vehicle is running or stationary, vibration signals caused by sound waves hitting the vehicle surface are collected based on the vibration sensor (external sounds causing vehicle body vibration), and the vibration signals collected by the vibration sensor are converted into audio signals to collect external vehicle audio. At the same time, when the vibration sensor is collecting data, a digital microphone inside the vehicle collects internal vehicle audio (internal vehicle sounds). Based on the DSP board, audio processing operations are performed on the external vehicle audio and the internal vehicle audio, so as to perform audio noise reduction, audio gain control, and eliminate the differences between internal and external audio. After the processing is completed, only the external vehicle audio is retained, and the DSP board contains relevant algorithms for performing the above operations; secondly, the microphone based on the vibration sensor is arranged inside the vehicle body and can be installed at any rigid contact part inside the vehicle body, and can ensure a position with effective vibration, preventing failure caused by the influence of the external environment on traditional MEMS microphones; finally, the audio data processed by the DSP can output digital audio or analog audio. The output digital audio can be transmitted to the subsequent device based on the A2B bus; the output analog audio can be directly transmitted to the subsequent device. Using the A2B bus for transmission can reduce the connection harness of the whole vehicle, reduce audio interference, reduce the vehicle weight, and meet the lightweight requirement; at the same time, the A2B bus can ensure the normal operation of multiple nodes inside the vehicle.

[0057] An audio processing system based on external vehicle vibration signals, comprising:

[0058] A vibration sensor, which is installed at a rigid contact part inside the vehicle body, and is used for collecting acoustic wave vibrations caused by external sound waves reaching the vehicle surface to obtain vehicle vibration signal data;

[0059] A digital microphone, which is used for collecting internal vehicle audio to obtain internal vehicle audio signals, and is installed inside the vehicle;

[0060] A DSP module, which is used for reconstructing acoustic signals from vehicle vibration signal data to obtain external vehicle audio signals, and is used for performing audio processing on the external vehicle audio signals and the internal vehicle audio signals to retain a single enhanced external vehicle sound signal data.

[0061] Specific usage and functions of the first embodiment:

[0062] First, deploy the vibration sensor and the digital microphone. When the vehicle is running or stationary, the vibration sensor collects the acoustic wave vibrations caused by the arrival of acoustic waves outside the vehicle on the vehicle surface. By picking up the distant sounds outside the vehicle, vehicle vibration signal data is obtained. Based on the digital microphone, in-vehicle audio is collected to obtain in-vehicle audio signals. Then, the DSP module reconstructs the acoustic signals from the vehicle vibration signal data to obtain out-of-vehicle audio signals, and can process the in-vehicle audio signals and the out-of-vehicle audio signals to output out-of-vehicle output speech. Traditional out-of-vehicle microphones are easily affected by environmental factors such as wind noise, rain, and snow, and have a low signal-to-noise ratio when driving at high speeds, with insufficient passive perception ability of the vehicle and difficulty in capturing sound events in specific directions (such as honking, emergency vehicle alarms). Using this can improve the accuracy of out-of-vehicle voice interaction recognition and the perception ability of environmental sound sources, and can effectively prevent related electromagnetic interference. At the same time, traditional MEMS microphones are prone to failure in complex environments, while the vibration sensor of the present invention is installed inside the vehicle body, and the installation position can ensure effective vibration, so it can adapt to complex environments, such as extreme environments and vibration environments, with high reliability.

[0063] It should be understood that the term "and / or" in this article is merely a description of the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. In addition, the character " / " in this article generally represents an "or" relationship between the associated objects before and after, but it may also represent an "and / or" relationship, which can be specifically understood by referring to the context before and after.

[0064] It should be understood that in the embodiments of the present invention, the magnitudes of the serial numbers of the above processes do not mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0065] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. An audio processing method based on an external vehicle vibration signal, characterized in that, It includes the following steps: Deploy one or more vibration sensors; When the vehicle is running or stationary, the vibration sensors installed on the vehicle body collect the acoustic vibrations caused by external sound waves reaching the vehicle surface to obtain vehicle vibration signal data, and collect in-vehicle audio based on a digital microphone to obtain in-vehicle audio signals; Reconstruct the acoustic signal from the vehicle vibration signal data to obtain an out-of-vehicle audio signal; Input the in-vehicle audio signal and the out-of-vehicle audio signal into a DSP module, which contains a specific self-developed speech algorithm library. The specific self-developed speech algorithm library includes a speech interaction algorithm library and an alarm sound detection algorithm library for FSD; Based on the DSP module, perform acoustic interference cancellation on the in-vehicle audio signal, and perform noise reduction on the in-vehicle audio signal and the out-of-vehicle audio signal. Perform secondary noise reduction on the in-vehicle audio signal and the out-of-vehicle audio signal based on a traditional algorithm noise reduction model or DNN; Based on the DSP module, perform frequency adjustment and AGC control on the in-vehicle audio signal and the out-of-vehicle audio signal, and perform enhancement processing on the out-of-vehicle audio signal to retain a single enhanced out-of-vehicle sound signal data.

2. The audio processing method based on an external vehicle vibration signal according to claim 1, wherein The acoustic interference cancellation of the in-vehicle audio signal based on the DSP module includes: Eliminate in-vehicle call audio data and infotainment audio data through a multi-band residual adaptive filter, and only retain external sound audio data; Or, eliminate in-vehicle call audio data and infotainment audio data through a multi-band residual adaptive filter and an echo suppressor, and only retain external sound audio data.

3. The audio processing method based on an external vehicle vibration signal according to claim 2, wherein, The method further includes: The in-vehicle audio signal includes the in-vehicle call audio data and the infotainment audio data.

4. The audio processing method based on an external vehicle vibration signal according to claim 1, characterized in that The noise reduction of the in-vehicle audio signal and the out-of-vehicle audio signal includes: Detect the stationary and transient noise in the reception and transmission of the in-vehicle audio signal and the out-of-vehicle audio signal.

5. The audio processing method based on an external vehicle vibration signal according to claim 1, wherein The frequency adjustment and AGC control of the in-vehicle audio signal and the out-of-vehicle audio signal based on the DSP module includes: Perform transient signal equalization on the in-vehicle audio signal and the out-of-vehicle audio signal; Perform frequency adjustment and signal automatic gain control on the in-vehicle audio signal and the out-of-vehicle audio signal.

6. The audio processing method based on the vehicle exterior vibration signal according to claim 1, wherein, The deployment of the vibration sensors and the digital microphone includes: Deploy the vibration sensors to the rigid contact parts inside the vehicle body, and deploy the digital microphone inside the vehicle.

7. The audio processing method based on an external vehicle vibration signal according to claim 1, wherein The single enhanced out-of-vehicle sound signal data is represented as audio signal data that only retains the out-of-vehicle audio.

8. The audio processing method based on an external vehicle vibration signal according to claim 1, characterized in that The method further includes: Deploy the digital microphone.

9. An audio processing system based on an external vehicle vibration signal, characterized in that, It includes: Vibration sensors, which are installed at the rigid contact parts inside the vehicle body and are used to collect the acoustic vibrations caused by external sound waves reaching the vehicle surface to obtain vehicle vibration signal data; Digital microphones, which are used to collect in-vehicle audio to obtain in-vehicle audio signals and are installed inside the vehicle; A DSP module, which is used to reconstruct acoustic signals from vehicle vibration signal data to obtain an external vehicle audio signal, and is also used to perform audio processing on the external vehicle audio signal and the internal vehicle audio signal to retain a single enhanced external vehicle sound signal data.

Images