Vehicle external sound intelligent screening and sound scene feedback system

Abstract

The invention discloses a vehicle external sound intelligent screening and sound scene feedback system, and relates to the technical field of vehicles. The system comprises a data sensing module used for collecting external sound data and scene state data of a vehicle and transmitting the data to a data processing module; the data processing module is used for carrying out classification judgment on the external sound data of the vehicle to obtain a classification result, processing the external sound data of the vehicle according to a sound processing strategy corresponding to the classification result and the scene state data of the vehicle to obtain first sound data, and sending the first sound data to the server; the classification result and the first sound data are transmitted to an interaction output module; and the interactive output module is used for carrying out safety prompt according to the classification result and outputting the first sound data. According to the invention, the driving safety and the environment perception capability of people in the vehicle can be improved.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and in particular to a vehicle external sound intelligent filtering and soundscape feedback system. Background Technology

[0002] Physical sound insulation technologies are typically used in vehicles, such as multi-layered laminated soundproof glass, reinforced sealing strips at door seams, and sound-absorbing felt and damping sheets added to the body panel structure to physically block external noise from entering the vehicle. However, this method reduces the ability of occupants to perceive the external environment through sound, making it difficult for them to distinguish between harmful noise and beneficial sound information (such as safety alarms and natural sounds), thus affecting driving safety. Summary of the Invention

[0003] The main objective of this application is to provide an intelligent external sound filtering and soundscape feedback system for vehicles, which aims to improve driving safety and the environmental perception capabilities of occupants.

[0004] To achieve the above objectives, one aspect of this application proposes a vehicle external sound intelligent filtering and soundscape feedback system, the system comprising: The data perception module is used to collect external sound data and scene status data of the vehicle and transmit them to the data processing module; The data processing module is used to classify and judge the external sound data to obtain a classification result, and then process the external sound data according to the sound processing strategy corresponding to the classification result and the scene state data to obtain the first sound data. Subsequently, the classification result and the first sound data are transmitted to the interactive output module. The interactive output module is used to provide safety prompts based on the classification results and output the first sound data.

[0005] In some embodiments, the data sensing module includes: A multi-channel microphone array unit is used to collect the external sound data and send it to the data processing module; The scene parameter acquisition unit is used to acquire the scene state data and send it to the data processing module.

[0006] In some embodiments, the external sound data includes at least one of safety alarm sounds, natural sounds, and harmful noises, and the scene state data includes at least the current speed of the vehicle.

[0007] In some embodiments, the data processing module includes: The sound classification unit is used to analyze the external sound data through a pre-trained sound classification model, obtain the classification results, and transmit them to the sound enhancement unit and the interactive output module respectively. The sound enhancement unit is configured to: when the classification result indicates that the external sound data contains the safety alarm sound, perform directional gain processing on the safety alarm sound contained in the external sound data; and / or when the classification result indicates that the external sound data contains the natural sound, perform feature frequency band extraction and detail enhancement processing on the natural sound contained in the external sound data; and / or when the classification result indicates that the external sound data contains the harmful noise, perform harmful noise filtering processing on the harmful noise contained in the external sound data; and then transmit the processed external sound data as initial sound data to the sound adjustment unit. The sound adjustment unit is used to acquire the vehicle's historical speed, and when it is determined that the vehicle's current speed is greater than the historical speed, and the classification result indicates that the external sound data contains the safety alarm sound and the natural sound, the unit reduces the output volume of the natural sound contained in the initial sound data, and then uses the adjusted initial sound data as the first sound data and transmits it to the interactive output module.

[0008] In some embodiments, the sound classification model is built based on a CNN-LSTM neural network model.

[0009] In some embodiments, the data processing module further includes: The sound preprocessing unit is used to preprocess the external sound data before transmitting it to the sound classification unit; wherein the preprocessing method includes at least one of frame windowing, filtering and noise reduction, and spatial enhancement.

[0010] In some embodiments, the interactive output module includes: The central control interaction unit is used to obtain the natural sound output configuration information input by the user through the interface and transmit it to the audio output unit when it is determined that the classification result represents that the external sound data contains the safety alarm sound and the natural sound. An audio output unit is used to control an audio playback device to output the first sound data according to the natural sound output configuration information; A safety alert unit is used to control an information display device to provide a visual alert about the type and source direction of the safety alarm sound carried by the classification result when it is determined that the classification result indicates that the external sound data contains the safety alarm sound.

[0011] In some embodiments, the audio playback device is a headrest speaker.

[0012] In some embodiments, the information display device includes an AR-HUD device and a dashboard. The AR-HUD device is used to display an icon corresponding to the type of safety alarm sound carried by the classification result and an arrow indicating the direction of the source. The dashboard is used to display text prompts representing the classification result.

[0013] In some embodiments, the safety alert unit is also used to control the ambient lights located in the driver's area to flash as a warning.

[0014] The embodiments of this application include at least the following beneficial effects: by using a data perception module to collect external sound data and scene status data of the vehicle, using a data processing module to classify and judge the external sound data of the vehicle to obtain a classification result, and then processing the external sound data of the vehicle according to the sound processing strategy corresponding to the classification result and the scene status data of the vehicle to obtain first sound data, and using an interactive output module to provide safety prompts and output the first sound data according to the classification result, driving safety and the environmental perception ability of the occupants of the vehicle can be improved.

[0015] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0016] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a schematic diagram of the composition of a vehicle external sound intelligent filtering and soundscape feedback system provided in an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the reference to "embodiment" herein means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments. The implementation methods described in the following exemplary embodiments do not represent all implementation methods consistent with the embodiments of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0018] It is understood that the terms "first," "second," etc., used in this application may be used to describe various concepts herein, but unless specifically stated otherwise, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words "if" or "when" as used herein may be interpreted as "when," "in response to a determination," or "at least one," "multiple," "each," "any," etc., as used in this application, at least one includes one, two, or more than two, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0020] Intelligent interaction technology based on external vehicle sounds is a key development direction in the field of smart cockpits. The evolution of this technology has consistently revolved around the dynamic balance between noise control and environmental information perception. Currently, this technology has gone through three development stages; however, from the perspective of actual user experience, each stage has certain implementation shortcomings, as detailed below.

[0021] The first stage of development involved using physical sound insulation technologies in vehicles, such as multi-layered laminated soundproof glass, reinforced sealing strips at door seams, and sound-absorbing felt and damping sheets added to the vehicle's sheet metal structure to physically block external noise from entering the vehicle. However, this physical sound insulation technology reduces the ability of occupants to perceive the external environment through sound; that is, occupants may find it difficult to distinguish between harmful noise and beneficial sound information (such as safety alarms and natural sounds), thus affecting driving safety.

[0022] The second stage of development involves using active noise cancellation technology in vehicles. This involves using error microphones placed in the cabin to collect residual noise in real time. A controller then analyzes the waveform characteristics of the residual noise and emits a canceling sound wave with the opposite phase and the same amplitude as the original noise through a speaker. Ultimately, noise cancellation is achieved using the principle of sound wave interference. However, this active noise cancellation technology cannot recognize the semantic information of sound and will indiscriminately cancel all collected external sounds. If external safety alarms are weakened, it increases the risk of vehicle accidents; if natural external sounds are weakened, it degrades the user's driving experience.

[0023] The third stage of development involves using preset soundscape technology in vehicles. This involves playing pre-recorded, locally stored high-quality natural sounds or white noise through the car's audio system to mask unpleasant noises inside the vehicle. However, this preset soundscape technology is completely disconnected from the actual external environment. For example, it might play the sound of heavy rain inside the car when it's sunny outside, or play birdsong when an emergency vehicle is approaching. This not only fails to enhance the user's perception of the external environment but also further interferes with the user's auditory judgment, affecting driving safety.

[0024] In view of this, this application proposes a vehicle external sound intelligent filtering and soundscape feedback system. This solution uses a data perception module to collect vehicle external sound data and scene status data, uses a data processing module to classify and judge the vehicle's external sound data to obtain classification results, and then processes the vehicle's external sound data according to the sound processing strategy corresponding to the classification results and the vehicle's scene status data to obtain first sound data. The interactive output module uses an interactive output module to provide safety prompts based on the classification results and output the first sound data, which can improve driving safety and the environmental perception ability of the occupants of the vehicle.

[0025] Please refer to Figure 1 , Figure 1 This is a schematic diagram of the composition of a vehicle external sound intelligent filtering and soundscape feedback system provided in an embodiment of this application. The system includes a data sensing module, a data processing module, and an interactive output module. The data sensing module is connected to the data processing module, and the data processing module is connected to the interactive output module.

[0026] In practical applications, the data perception module collects external sound data and scene status data of the vehicle and transmits them to the data processing module. The data processing module first classifies and judges the external sound data of the vehicle to obtain a classification result. Then, based on the sound processing strategy corresponding to the classification result and the scene status data of the vehicle, the external sound data of the vehicle is processed to obtain the first sound data. Subsequently, the classification result and the first sound data are transmitted to the interactive output module. The interactive output module provides a safety prompt based on the classification result and outputs the first sound data.

[0027] In some embodiments, the data sensing module includes a multi-channel microphone array unit and a scene parameter acquisition unit, both of which are connected to the data processing module. In practical applications, the multi-channel microphone array unit acquires external sound data of the vehicle and sends it to the data processing module, while the scene parameter acquisition unit acquires scene state data of the vehicle and sends it to the data processing module.

[0028] The vehicle's external sound data includes at least one of the following: safety alarm sounds (such as horns of nearby vehicles, ambulance sirens, fire truck sirens, etc.), natural sounds (such as raindrops, birdsong, etc.), and harmful noises (such as road noise, construction noise, etc.); the vehicle's scene status data includes the vehicle's current speed and window status information, as well as rainfall data outside the vehicle.

[0029] Specifically, the multi-channel microphone array unit can employ a ring-shaped microphone array, achieving omnidirectional sound acquisition from outside the vehicle through uniform arrangement. It can pinpoint the location of sound sources in different directions and supports wide-band sound capture to ensure the acquisition of safety alarm sounds, natural sounds, and harmful noise. The scene parameter acquisition unit includes a rain sensor, a window status sensor, and a vehicle speed sensor. The rain sensor can be installed on the windshield to collect rainfall data outside the vehicle. The window status sensor is used to obtain information about the vehicle's window status (e.g., whether all windows are fully closed or fully open, or only some windows are closed). The vehicle speed sensor is used to collect the vehicle's current speed.

[0030] In some embodiments, the data processing module includes a sound classification unit, a sound enhancement unit, and a sound adjustment unit. The sound classification unit is connected to the sound enhancement unit, the multi-channel microphone array unit, and the interactive output module, respectively. The sound enhancement unit is connected to the sound adjustment unit, which is connected to the scene parameter acquisition unit and the interactive output module, respectively. In practical applications, the sound classification unit analyzes the vehicle's external sound data using a pre-trained sound classification model, obtains classification results, and transmits them to the sound enhancement unit and the interactive output module. When the sound enhancement unit determines that the classification result indicates the vehicle's external sound data contains a safety alarm sound, it performs directional gain processing on the safety alarm sound to suppress background interference, enabling the driver to clearly perceive it. And / or when the classification result indicates the vehicle's external sound data contains natural sounds, it performs feature frequency band extraction and detail enhancement processing on the natural sounds to preserve the natural texture of the sound. And / or when the classification result indicates the vehicle's external sound data contains... When harmful noise is present, the harmful noise contained in the vehicle's external sound data is filtered out. This can be achieved using active noise cancellation technology to reduce the impact of harmful noise on the driving experience. The processed external sound data is then used as the initial sound data and transmitted to the sound adjustment unit. The sound adjustment unit obtains the vehicle's historical speed, which can be understood as the vehicle's speed at the previous moment. If it is determined that the vehicle's current speed is greater than the historical speed (i.e., the vehicle's speed has increased), and the classification result indicates that the vehicle's external sound data contains both safety alarm sounds and natural sounds, the output volume of the natural sounds contained in the initial sound data is reduced to improve the recognizability of the safety alarm sounds. The adjusted initial sound data is then used as the first sound data and transmitted to the interactive output module.

[0031] The degree to which the output volume of the natural sound is reduced can be determined based on the increase in vehicle speed. For example, when the increase in vehicle speed falls within the range of [0, 5 km / h], the reduction in the output volume of the natural sound can be any value within the range of [0, 5%]. Furthermore, the high-frequency bands of the natural sound contained in the initial sound data can be further compressed.

[0032] The sound classification model is built on a CNN-LSTM neural network model, which includes a CNN (Convolutional Neural Network) feature extraction layer, an LSTM (Long Short-Term Memory) temporal analysis layer, and a classification output layer connected in sequence. In the CNN feature extraction layer, the spectral features of the sound data are extracted through convolutional operations to identify the frequency distribution pattern of the sound data. In the LSTM temporal analysis layer, the temporal features of the sound data are processed to capture the dynamic characteristics such as the periodicity of sirens and the suddenness of horn sounds. In the classification output layer, the recognition results of various sounds are output through a fully connected network.

[0033] The sound classification model is obtained by training the CNN-LSTM neural network model. The training methods may include, but are not limited to: collecting diverse sound data samples to form an initial sample set, covering different types of safety alarm sounds, different types of natural sounds, different types of harmful noise, and different types of mixed sounds, where the mixed sounds are obtained by mixing at least two of the safety alarm sounds, natural sounds, and harmful noise; expanding the initial sample set using data augmentation techniques to obtain a final sample set, where the expansion processing includes adding vehicle background noise, adjusting the speed and / or pitch of the sound, and simulating at least one of different spatial reverberation effects; dividing the final sample set using a stratified sampling method to obtain a training set, a validation set, and a test set to ensure a balanced distribution of sound data samples across categories; training the CNN-LSTM neural network model based on the training set, using a Bayesian optimization algorithm to automatically optimize the network hyperparameters during training; and evaluating the performance of the trained CNN-LSTM neural network model using a cross-validation method based on the validation set. It should be noted that the weighted cross-entropy loss function can be used as the loss function required during model training. Furthermore, since safety alarm sounds need to be given more attention during driving, the classification loss part for safety alarm sounds is given a relatively high weight in the weighted cross-entropy loss function.

[0034] As an improved implementation, the sound classification model can be compressed before deployment to achieve lightweight deployment while maintaining recognition accuracy, meeting the requirements of in-vehicle embedded environments. The sound classification model can be iteratively optimized online. In the cloud, based on periodically collected user feedback data (such as false positives and false negatives) and sound data samples from new scenarios, the model parameters are updated using incremental training, the feature extraction layer is frozen, and only the classification decision layer is optimized. Then, OTA (Over-The-Air) updates are pushed to the model, enabling seamless model upgrades on the vehicle and improving adaptability to different regions and scenarios. Since the classification results output by the sound classification model generally include sound type, source direction, and confidence level, if the confidence level is determined to be lower than a preset confidence threshold, the sound classification model can be re-analyzed on the vehicle's external sound data to improve classification reliability.

[0035] In some embodiments, the data processing module may further include a sound preprocessing unit, which is connected to both the sound classification unit and the multi-channel microphone array unit. The sound preprocessing unit preprocesses the vehicle's external sound data before transmitting it to the sound classification unit, thereby providing reliable data input for the sound classification model. The preprocessing methods include at least one of frame-segmentation and windowing, filtering and denoising, and spatial enhancement. Frame-segmentation and windowing involves dividing continuous vehicle external sound data into short data blocks and smoothing them to avoid distortion caused by data truncation. Filtering and denoising involves removing low-frequency mechanical noise from the vehicle's external sound data using high-pass filtering to retain effective audio frequencies. Spatial enhancement involves using beamforming technology to dynamically adjust the weights of each channel's sound data according to the sound source direction to enhance the target sound energy.

[0036] In some embodiments, the interactive output module includes a central control interaction unit, an audio output unit, and a safety prompt unit. The central control interaction unit is connected to the sound classification unit and the audio output unit, respectively. The audio output unit is connected to the sound adjustment unit, and the safety prompt unit is connected to the sound classification unit. In practical applications, when the central control interaction unit determines that the classification result represents the vehicle's external sound data containing both safety alarm sounds and natural sounds, it acquires the natural sound output configuration information input by the user through the interface and transmits it to the audio output unit. Based on the natural sound output configuration information, the audio output unit controls an audio playback device to output the first sound data; this audio playback device can be a seat headrest speaker. When the safety prompt unit determines that the classification result represents the vehicle's external sound data containing safety alarm sounds, it controls an information display device to provide a visual prompt regarding the type and source direction of the safety alarm sound carried by the classification result.

[0037] The interface refers to the intuitive soundscape control interface provided by the central control interaction unit. This interface includes a natural sound type selection area and a parameter adjustment area. The natural sound type selection area primarily displays the type of natural sound (such as raindrops, birdsong, etc.) carried by the classification result. When the classification result carries only one type of natural sound, only that natural sound is displayed in the selection area and is selected by default. When the classification result carries at least two types of natural sound, at least two natural sounds are displayed in the selection area, and the user can select at least one preferred natural sound. This parameter adjustment area primarily displays the mixing ratios of different types of natural sounds. When the classification result carries only one type of natural sound, only that natural sound is displayed in this parameter adjustment area with a default mixing ratio of 100%, or it can be displayed directly without setting a mixing ratio. When the classification result carries at least two types of natural sounds, at least two types of natural sounds are displayed in this parameter adjustment area, and users can enter the mixing ratio for each type of natural sound. For example, when the classification result carries the types of natural sounds such as raindrops and birdsong, the user can set the mixing ratio for raindrops to 60% and the mixing ratio for birdsong to 40%. It should be noted that when the user completes the natural sound selection operation in this natural sound type selection area, the parameter adjustment area can be adjusted accordingly, meaning that only the previously selected natural sounds are displayed in this parameter adjustment area. Understandably, the natural sound output configuration information includes the type of natural sound selected by the user in the natural sound type selection area and the mixing ratio for the selected natural sound entered in the parameter adjustment area. The settings for the natural sound type and mixing ratio can also be determined by analyzing the user's input voice information. Furthermore, the soundscape control interface also includes a status display area, which primarily displays the types of all sounds carried by the classification result.

[0038] The information display device includes an AR-HUD device (i.e., an augmented reality head-up display) and an instrument panel. The AR-HUD device displays icons corresponding to the types of safety alarm sounds carried by the classification result and arrows indicating the source direction of the safety alarm sounds carried by the classification result. The instrument panel displays text prompts representing the classification result to help the driver quickly locate the safety alarm sounds and make appropriate responses.

[0039] In addition, this safety alert unit also controls the ambient lighting in the driver's area to flash as a warning, increasing the driver's attention to the safety alarm sounds within their line of sight. Under normal circumstances, the AR-HUD device can flash icons corresponding to the type of safety alarm sound, with the flashing frequency matching that of the ambient lighting.

[0040] The intelligent vehicle external sound filtering and soundscape feedback system proposed in this application has at least the following beneficial effects: In terms of enhancing driving safety, multi-dimensional sound recognition and directional prompts ensure that drivers can promptly perceive relevant safety sounds around the vehicle (such as ambulance sirens and horns from vehicles approaching from the side). This is especially beneficial in low-visibility environments such as rain and fog, supplementing visual perception and reducing the risk of accidents. Furthermore, safety prompts within the driver's line of sight are provided through AR-HUD devices, the instrument panel, and ambient lighting in the driver's area, shortening hazard response time. Regarding optimizing the driving experience, users can customize natural sound combinations. Dynamic adaptation algorithms simulate realistic environmental sound effects and specifically suppress harmful noises such as road noise and wind noise while retaining valuable natural environmental sounds, thus improving driving comfort.

[0041] The following is a specific application example of urban commuting in rainy weather. This urban commuting scenario covers vehicles driving on city roads during the morning rush hour in moderate rain, with all vehicle windows closed. The solution provided in this application embodiment will be further explained in conjunction with this specific application example.

[0042] In the data perception module, external sound data of the vehicle is collected through a microphone array, including the sound of raindrops hitting the body, the sound of a car horn coming from the left, construction noise in the distance, and sporadic bird calls. The frequency range of the raindrop impact sound is 20-200Hz, the frequency range of the car horn coming from the left is 800-1200Hz, the frequency range of the construction noise in the distance is 200-800Hz, and the frequency range of the sporadic bird calls is 1000-3000Hz. The rain density outside the vehicle is detected by a rain sensor. When the rain density outside the vehicle is determined to be greater than or equal to 50 drops / minute, the weather outside the vehicle is determined to be moderate rain. The window status sensor detects that all windows of the vehicle are fully closed, which can determine that the vehicle is in a closed environment. The vehicle speed sensor detects that the vehicle speed is 45km / h, which can determine that the vehicle is in a medium-speed commuting state.

[0043] In the data processing module, the external sound data of the vehicle collected by the microphone array is high-pass filtered to remove vehicle chassis resonance noise with a frequency less than 100Hz. Then, beamforming technology is used to enhance the sound data to further amplify the sound source signal from the direction of oncoming traffic on the left. The pre-processed external sound data of the vehicle is input into a pre-trained sound classification model for analysis to obtain classification results. The classification results include safety alarm sounds, natural sounds, and harmful noises. The safety alarm sounds include the horn sound of oncoming traffic on the left with its sound source direction 30° to the left front. The natural sounds include raindrop sounds and birdsong. The harmful noises include construction noise. The horn sound of oncoming traffic on the left is directionally amplified and converted into an analog warning tone with a frequency band of 500-800Hz to suppress interference from other sound source directions. The characteristic frequency bands of the raindrop sounds and birdsong are extracted, and the mixed construction noise is filtered out by a noise reduction algorithm. Since the classification results include the safety alarm sounds and the natural sounds, the output volume of the natural sounds is set to a medium volume to avoid masking the warning of the safety alarm sounds. In addition, the sensitivity of the microphone array can be increased by 20% to compensate for the sound attenuation caused by the vehicle windows being closed.

[0044] In the interactive output module, a 3D left-front 30-degree arrow is displayed via an AR-HUD device to indicate the direction of the horn sound source, while a horn icon is superimposed as a prompt, flashing at a frequency of 2 times per second. A pop-up window appears on the instrument panel displaying the text "Left-side approach warning." The ambient light in the driver's area can also be controlled to flash yellow at a frequency of 2 times per second, synchronized with the horn icon. Raindrop and bird song icons are automatically highlighted on the soundscape control interface. Users can set the mixing ratio of raindrop sound to 60% and bird song sound to 40% on this interface, and then output the sound through the seat headrest speakers in a surround sound effect.

[0045] It should be noted that if the horn sound from the vehicle on the left stops afterward, the AR-HUD device will fade out its currently displayed 3D left front 30-degree arrow and horn icon within 1 second, the instrument panel will close the pop-up window, and the ambient lights set in the driver's area will be restored to a constantly lit or normally closed state.

[0046] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0047] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, an implementation process comprising a series of steps is not necessarily limited to those explicitly listed, but may include other steps not explicitly listed or inherent to these implementation processes.

[0048] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0049] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A vehicle external sound intelligent filtering and soundscape feedback system, characterized in that, The system includes: The data perception module is used to collect external sound data and scene status data of the vehicle and transmit them to the data processing module; The data processing module is used to classify and judge the external sound data to obtain a classification result, and then process the external sound data according to the sound processing strategy corresponding to the classification result and the scene state data to obtain the first sound data. Subsequently, the classification result and the first sound data are transmitted to the interactive output module. The interactive output module is used to provide safety prompts based on the classification results and output the first sound data.

2. The intelligent vehicle external sound filtering and soundscape feedback system according to claim 1, characterized in that, The data sensing module includes: A multi-channel microphone array unit is used to collect the external sound data and send it to the data processing module; The scene parameter acquisition unit is used to acquire the scene state data and send it to the data processing module.

3. The intelligent vehicle external sound filtering and soundscape feedback system according to claim 1, characterized in that, The external sound data includes at least one of safety alarm sounds, natural sounds, and harmful noises, and the scene status data includes at least the current vehicle speed.

4. The intelligent vehicle external sound filtering and soundscape feedback system according to claim 3, characterized in that, The data processing module includes: The sound classification unit is used to analyze the external sound data through a pre-trained sound classification model, obtain the classification results, and transmit them to the sound enhancement unit and the interactive output module respectively. The sound enhancement unit is configured to: when the classification result indicates that the external sound data contains the safety alarm sound, perform directional gain processing on the safety alarm sound contained in the external sound data; and / or when the classification result indicates that the external sound data contains the natural sound, perform feature frequency band extraction and detail enhancement processing on the natural sound contained in the external sound data; and / or when the classification result indicates that the external sound data contains the harmful noise, perform harmful noise filtering processing on the harmful noise contained in the external sound data; and then transmit the processed external sound data as initial sound data to the sound adjustment unit. The sound adjustment unit is used to acquire the vehicle's historical speed, and when it is determined that the vehicle's current speed is greater than the historical speed, and the classification result indicates that the external sound data contains the safety alarm sound and the natural sound, the unit reduces the output volume of the natural sound contained in the initial sound data, and then uses the adjusted initial sound data as the first sound data and transmits it to the interactive output module.

5. The intelligent vehicle external sound filtering and soundscape feedback system according to claim 4, characterized in that, The sound classification model is built based on the CNN-LSTM neural network model.

6. The intelligent vehicle external sound filtering and soundscape feedback system according to claim 4, characterized in that, The data processing module further includes: The sound preprocessing unit is used to preprocess the external sound data before transmitting it to the sound classification unit; wherein the preprocessing method includes at least one of frame windowing, filtering and noise reduction, and spatial enhancement.

7. The intelligent vehicle external sound filtering and soundscape feedback system according to claim 3, characterized in that, The interactive output module includes: The central control interaction unit is used to obtain the natural sound output configuration information input by the user through the interface and transmit it to the audio output unit when it is determined that the classification result represents that the external sound data contains the safety alarm sound and the natural sound. An audio output unit is used to control an audio playback device to output the first sound data according to the natural sound output configuration information; A safety alert unit is used to control an information display device to provide a visual alert about the type and source direction of the safety alarm sound carried by the classification result when it is determined that the classification result indicates that the external sound data contains the safety alarm sound.

8. The intelligent vehicle external sound filtering and soundscape feedback system according to claim 7, characterized in that, The audio playback device is a headrest speaker.

9. The intelligent vehicle external sound filtering and soundscape feedback system according to claim 7, characterized in that, The information display device includes an AR-HUD device and a dashboard. The AR-HUD device is used to display icons corresponding to the type of safety alarm sound carried by the classification result and arrows indicating the direction of the source. The dashboard is used to display text prompts representing the classification result.

10. The intelligent vehicle external sound filtering and soundscape feedback system according to claim 7, characterized in that, The safety alert unit is also used to control the ambient lighting in the driver's area to flash as a warning.

Images