Vehicle early warning method, device and equipment based on environmental sound signal perception and medium

Abstract

The invention provides a vehicle early warning method and device based on environmental sound signal perception, equipment and a medium, and the method comprises the steps: collecting vehicle driving information and a surrounding initial environmental sound signal, and carrying out the noise reduction of the initial environmental sound signal, so as to generate a standard environmental sound signal; specific sound features are extracted from standard environment sound signals, the features are identified by using a pre-trained sound identification model, so that the type of an actual sound event is determined, and a proper early warning level is evaluated and determined according to the combination of vehicle driving information and the identified sound event. According to the early warning level, a vehicle alarm system is activated to execute corresponding early warning measures, and timely warning is provided for a driver; according to the method, corresponding early warning is carried out on the driver through recognition of the environment sound, safety accidents caused by the fact that the driver cannot hear the environment sound are avoided, and the driving safety is effectively improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a vehicle early warning method, device, equipment and medium based on environmental sound signal perception. Background Art

[0002] As consumers' requirements for driving comfort continue to increase, passenger cars have made significant progress in cab sound insulation technology. Traditionally, vehicles use physical noise reduction methods, such as applying sound insulation materials or replacing quieter parts to reduce external noise entering the car. In recent years, with the rapid development of the smart car industry, active noise reduction technology has gradually been introduced into car design. This technology uses a microphone installed in the car to capture the noise source signal, and generates a sound waveform with an opposite phase through calculation, which is released through the speaker to achieve the effect of offsetting the original noise, creating a quieter and more comfortable riding environment for passengers.

[0003] However, while enjoying better sound insulation, this also brings some new challenges. Especially when the vehicle is in certain specific situations, such as shouts in emergency situations, warning sirens from other vehicles, and alarms from special vehicles such as police cars and ambulances may not be transmitted to the driver's ears in a timely and effective manner. These important and indicative sound signals (such as emergency shouts, alarms or warning sirens) may be difficult to detect because the vehicle's sound insulation is too good. This situation increases the difficulty for the driver to respond quickly and accurately to the external environment, and in extreme cases may also lead to serious traffic accidents. Summary of the invention

[0004] In view of the above-mentioned shortcomings of the prior art, the present application provides a vehicle warning method, device, equipment and medium based on environmental sound signal perception to solve the above-mentioned technical problems.

[0005] The present application provides a vehicle warning method based on environmental sound signal perception, the method comprising: obtaining vehicle driving information and initial environmental sound signals around the vehicle, and performing noise reduction processing on the initial environmental sound signals to obtain a standard environmental sound signal after noise reduction, wherein the vehicle driving information is parameter information used to describe the current motion state of the vehicle; extracting sound features of the standard environmental sound signal, and recognizing the sound features based on a trained sound recognition model to obtain sound events represented by the standard environmental sound signal, wherein the sound events are used to represent human voice information, environmental sound information, and alarm sound information of the vehicle's surrounding environment; determining a current warning level of the vehicle's surrounding environment based on the vehicle driving information and the sound events, and triggering a vehicle alarm system to execute a corresponding warning plan according to the current warning level to warn the vehicle driver.

[0006] In one embodiment of the present application, the initial ambient sound signal is subjected to noise reduction processing, including: dividing the initial ambient sound signal into multiple single-frame audios, and performing frequency domain transformation on each single-frame audio to obtain an amplitude spectrum and a phase spectrum of each single-frame audio; selecting a leading silent segment data from the initial ambient sound signal, and calculating the average energy of the frequency domain spectrum lines of the leading silent segment data as a noise estimation value; adjusting the amplitude spectrum of each single-frame audio based on the noise estimation value, and performing an inverse frequency domain transformation on the adjusted amplitude spectrum and the original phase spectrum to obtain a standard ambient sound signal after noise reduction.

[0007] In one embodiment of the present application, extracting the sound features of the standard ambient sound signal includes: identifying the audio data of the standard ambient sound signal and dividing the audio data into multiple data categories, the data categories including at least human voice, ambient sound, and alarm sound; extracting a first sound characteristic from the human voice, extracting a second sound characteristic from the background noise, extracting a third sound characteristic from the alarm sound, and determining the first sound feature, the second sound feature, and the third sound feature as the sound features of the standard ambient sound signal.

[0008] In one embodiment of the present application, before identifying the sound features based on the trained sound recognition model, it also includes: labeling the historical environmental sound signals based on the sound features, and dividing the labeled information to obtain at least a training data set and a verification data set; training the initial neural network model based on the training data set to learn the initial mapping relationship between the input environmental sound signals and the sound events; verifying the initial mapping relationship based on the verification data set, and optimizing the initial mapping relationship based on the verification result to obtain an optimized neural network model, and determining the optimized neural network model as the sound recognition model.

[0009] In one embodiment of the present application, the current warning level of the vehicle's surrounding environment is determined based on the vehicle driving information and the sound event, including: identifying the sound event to obtain a recognition result; the recognition result includes keyword information obtained based on the keywords of the human voice information, the road type and climate conditions of the vehicle's surrounding environment determined based on the ambient sound information, the alarm category, alarm volume, and alarm duration determined based on the alarm sound information, and the current speed of the vehicle obtained based on the driving information; based on preset scoring rules, each of the recognition results is scored to obtain a comprehensive score, and the current warning level of the vehicle's surrounding environment is determined based on the comprehensive score.

[0010] In one embodiment of the present application, after the vehicle driver is warned, the method further includes: monitoring the driver's reaction after the warning, at least including monitoring the driver's braking action, steering operation and acceleration behavior to determine whether the driver has executed countermeasures; if the driver is not detected to execute countermeasures within a preset time, the current warning level is upgraded, and the warning is re-issued based on the upgraded warning level until the driver executes countermeasures, and the countermeasures are determined based on a warning measure mapping table, and the warning measure mapping table is used to characterize the mapping relationship between the countermeasures of each warning level.

[0011] In one embodiment of the present application, before obtaining the initial ambient sound signal around the vehicle, it also includes: detecting the switch status of all windows of the vehicle, the switch status of the windows including fully closed, fully open and partially closed; when it is detected that all windows are in a fully closed state, activating a sound receiving device in the vehicle; monitoring the sound volume of the current environment, if the monitored ambient sound volume exceeds a preset threshold, triggering the sound receiving device to start collecting the initial ambient sound signal around the vehicle.

[0012] The present application provides a vehicle warning device based on environmental sound signal perception, the device comprising: a sound signal acquisition and preprocessing module, used to obtain vehicle driving information and initial environmental sound signals around the vehicle, and perform noise reduction processing on the initial environmental sound signals to obtain standard environmental sound signals after noise reduction; a sound event recognition module, used to extract sound features of the standard environmental sound signals, and recognize the sound features based on a trained sound recognition model to obtain sound events represented by the standard environmental sound signals; a warning execution module, used to determine a current warning level of the vehicle's surrounding environment based on the vehicle driving information and the sound events, and trigger a vehicle alarm system to execute a corresponding warning plan according to the current warning level to warn the vehicle driver.

[0013] The present application provides an electronic device, including a processor, a memory and a communication bus; the communication bus is used to connect the processor and the memory; the processor is used to execute a computer program stored in the memory to implement the vehicle warning method based on environmental sound signal perception as described above.

[0014] The present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is used to enable a computer to execute the vehicle warning method based on environmental sound signal perception as described above.

[0015] Beneficial effects of the present application: The vehicle warning method based on environmental sound signal perception in the present application first performs noise reduction processing on the initial environmental sound signal to generate a clear standard environmental sound signal, then extracts specific sound features from it, and uses a pre-trained sound recognition model to identify these features, so as to accurately determine the type of sound event that has occurred; combined with the current vehicle driving information, evaluates potential risks and determines the appropriate warning level, and then activates the vehicle alarm system to execute corresponding warning measures to provide timely warnings for the driver. This method not only overcomes the problem that key external sounds (such as emergency shouts, alarms or warning sirens) are difficult to detect due to the good sound insulation effect in the car, but also improves the response speed and accuracy to the surrounding dynamic changes through intelligent environmental perception technology. Therefore, the present invention effectively solves the problem of information transmission barriers caused by excessive sound insulation by innovatively integrating sound signal processing and intelligent warning mechanisms, and effectively improves the safety in the field of intelligent transportation.

[0016] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0018] Figure 1 is a schematic diagram of an implementation environment of a vehicle warning method based on environmental sound signal perception shown in an exemplary embodiment of the present application;

[0019] Figure 2 is a flowchart of a vehicle warning method based on environmental sound signal perception, shown in an exemplary embodiment of the present application;

[0020] Figure 3 is a schematic diagram of the installation position of a sound collection device shown in an exemplary embodiment of the present application;

[0021] Figure 4 is a block diagram of a vehicle warning device based on environmental sound signal perception shown in an exemplary embodiment of the present application;

[0022] Figure 5 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0023] The following will describe the implementation methods of the present application with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for illustrating the present application, not for limiting the scope of protection of the present application.

[0024] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application, and thus the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.

[0025] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application. However, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present application difficult to understand.

[0026] Figure 1 It is a schematic diagram of the implementation environment of a vehicle warning method based on environmental sound signal perception shown in an exemplary embodiment of the present application.

[0027] like Figure 1 As shown, the implementation environment of the vehicle early warning method based on environmental sound signal perception includes a data acquisition device 101 and a computer device 102. Among them, the data acquisition device 101 is responsible for collecting raw data during the operation of the vehicle, which includes but is not limited to driving environment information, driver information and vehicle status information. Driving environment information is further subdivided into environment types (such as urban roads, highways, rural roads, etc.) and sound information (such as background noise levels, special sound events such as whistles, alarms, etc.). Its data acquisition equipment can be various sensors installed on the vehicle (such as microphone arrays, environmental sound sensors, etc.), or it can be data collected by external devices after being connected to the vehicle wirelessly or wired. This application does not impose any restrictions on the specific implementation form of the data acquisition device.

[0028] The computer device 102 is used to process the raw data collected by the data acquisition device 101, specifically including: constructing an initial sound recognition model, and using the historical data in the raw data to train it to obtain an optimized sound recognition model. The real-time sound signal in the raw data is then input into the model to extract and identify sound features to determine the sound events occurring in the current environment. According to the current driving information of the vehicle and the identified sound events, the computer device 102 can evaluate the potential risk level and trigger corresponding early warning measures accordingly. The computer device 102 can be a high-performance processor, a dedicated vehicle-mounted computing platform, or an embedded system integrated in the vehicle. This application also does not impose any restrictions on the specific type of computer equipment. In this way, the system can provide drivers with timely and effective safety warnings in complex driving environments and improve driving safety.

[0029] Figure 2 It is a flowchart of the steps of a vehicle warning method based on environmental sound signal perception shown in an exemplary embodiment of the present application.

[0030] like Figure 2 As shown, in an exemplary embodiment, the vehicle warning method based on environmental sound signal perception includes at least steps S210 to S230, which are described in detail as follows:

[0031] Step S210, obtain vehicle driving information and the initial ambient sound signal around the vehicle, and perform noise reduction processing on the initial ambient sound signal to obtain a standard ambient sound signal after noise reduction, wherein the vehicle driving information is parameter information used to describe the current motion state of the vehicle, specifically including speed information, acceleration information, etc.

[0032] It should be noted that in order to collect external environmental sounds in 360° without blind spots, the sound receiving device should be installed in the center of the roof, such as Figure 3 The intersection of the horizontal and vertical dotted lines shown. The main function of the device is to collect sound signals from external sound sources, and to perform noise reduction processing and comparative analysis on them. The sound signal after noise reduction processing will be compared with the sound signal sample in the cloud; if the similarity reaches above the corresponding threshold, the system will judge the specific direction of the sound source with the installation position of the sound receiving device as the center, and continue to receive sound in that direction, while continuously processing and comparing the collected sound signals. Once a specific sound event is identified, the system will display a warning message through the car model in the vehicle dashboard and provide corresponding voice prompts to inform the driver of the direction of the sound source.

[0033] Before starting to acquire the initial ambient sound signal around the vehicle, the system first detects the switch status of all windows, including fully closed, fully open, and partially closed. Only when all windows are fully closed will the vehicle's radio device be activated. In addition, the system will also monitor the sound volume of the current environment. If it is found that the ambient sound volume exceeds the preset threshold, the radio device will be immediately triggered to start collecting the initial ambient sound signal around the vehicle. This ensures that even if the interior of the car is well soundproofed, it can effectively capture and respond to important external sound signals, thereby improving driving safety.

[0034] In a specific embodiment of the present application, take a car driving on a city road with all windows closed as an example. When an ambulance approaches from the side and sounds an emergency alarm, the driver may not be able to hear the alarm due to the good sound insulation inside the car. At this time, the sound receiving device on the roof detects that the volume of the external sound exceeds the preset threshold, and immediately starts to collect environmental sound signals. After noise reduction processing and comparison analysis, the system recognizes that this is the alarm of the ambulance and determines the direction of the sound source. Subsequently, the system displays a red warning icon through the car model in the vehicle dashboard and plays a voice prompt: "There is an emergency vehicle on the left, please pay attention to avoid it." In this way, the driver can respond quickly, make way for the ambulance, and avoid potential traffic accidents.

[0035] In one embodiment of the present application, the initial ambient sound signal is subjected to noise reduction processing, including: dividing the initial ambient sound signal into multiple single-frame audios, and performing frequency domain transformation on each single-frame audio to obtain an amplitude spectrum and a phase spectrum of each single-frame audio; selecting a leading silent segment data from the initial ambient sound signal, and calculating the average energy of the frequency domain spectrum lines of the leading silent segment data as a noise estimation value; adjusting the amplitude spectrum of each single-frame audio based on the noise estimation value, and performing an inverse frequency domain transformation on the adjusted amplitude spectrum and the original phase spectrum to obtain a standard ambient sound signal after noise reduction.

[0036] In one embodiment of the present application, in order to improve the processing effect of the ambient sound signal, the system first performs noise reduction processing on the initial ambient sound signal. The specific process is as follows: the system divides the collected initial ambient sound signal into multiple single-frame audios, each frame usually ranging from 20 milliseconds to 50 milliseconds. Next, each single-frame audio is transformed in the frequency domain, usually by fast Fourier transform (FFT), so as to obtain the amplitude spectrum and phase spectrum of each single-frame audio. In order to estimate the noise level, the system selects a leading silent segment data from the initial ambient sound signal, which is usually a period of silence before the sound signal starts. Then, the average energy of the frequency domain spectrum line of this leading silent segment data is calculated as the noise estimation value. This noise estimation value reflects the energy level of the background noise. Based on the above noise estimation value, the system adjusts the amplitude spectrum of each single-frame audio. The specific adjustment method is to subtract the noise estimation value from the amplitude spectrum of each single-frame audio to remove the influence of background noise. It should be noted that when subtracting the noise estimation value, it should be ensured that the value in the amplitude spectrum does not become negative to avoid distortion. The adjusted amplitude spectrum is transformed back to the time domain together with the original phase spectrum through an inverse frequency domain transform (usually an inverse FFT) to obtain a standard ambient sound signal after noise reduction.

[0037] Specifically, suppose a car is driving on a busy city street, surrounded by various background noises, such as the sounds of other vehicles and pedestrians talking. The vehicle's sound receiver captures these complex environmental sound signals. The system first divides these sound signals into multiple short-time frames and performs FFT transformation on each frame to obtain the amplitude spectrum and phase spectrum; then, the system identifies a silent segment without obvious sound and calculates the average energy of its frequency domain spectrum as the noise estimation value. Using this noise estimation value, the system adjusts the amplitude spectrum of each frame to remove background noise; finally, the adjusted amplitude spectrum and original phase spectrum are converted back to the time domain through inverse FFT to generate a clear noise-reduced sound signal.

[0038] In a specific embodiment of the present application, considering that the non-pure audio signal contains noise, and the noise usually interferes with the result of endpoint detection, the audio signal needs to be subjected to noise reduction processing before feature extraction to reduce the impact of noise. Therefore, the main thing to be processed is the environmental noise on the road, tire noise and wind noise during driving. Among them, tire noise and wind noise are directly related to the degree of subtraction of the driving speed spectrum. The higher the speed of the vehicle, the greater the corresponding tire noise and wind noise. The noise reduction processing needs to effectively remove the above recording to obtain effective sound information. Considering that the sound signal can obtain the amplitude spectrum and phase spectrum after Fourier transformation, the human ear's perception of sound is mainly obtained from the amplitude spectrum of the sound signal, and the sensitivity to the amplitude spectrum is much higher than the phase spectrum. Therefore, this type of noise reduction method targets the short-time spectrum amplitude. Therefore, in this embodiment, spectral subtraction is used to perform noise reduction processing on its initial ambient sound. The specific steps are as follows:

[0039] First, for the i-th frame audio signal x i (m) is subjected to discrete Fourier transform to obtain x i (k) Calculate its amplitude and phase angle respectively. i (k) can be modulo to obtain its amplitude |x i (k)|, and the phase angle θ i (k), which is calculated as follows:

[0040]

[0041] Among them, θ i (k) represents the phase angle, x i (m) represents the i-th frame audio signal.

[0042] Then, taking the audio start position as the starting point, intercept the leading silent segment data of the NIS frame, and calculate the average energy of the frequency domain spectrum of the NIS frame by the following formula: The calculation formula is as follows:

[0043]

[0044] in, represents the average energy of the frequency domain spectrum of the NIS frame, |x i (k)| represents the amplitude.

[0045] Therefore, the spectral subtraction rule can be expressed as follows:

[0046]

[0047] Among them, α is the over-reduction factor, β is the gain compensation factor, |x i (k)| represents the amplitude.

[0048] It should be noted that the value of α has the most direct impact on the result of spectral subtraction. If its value is too large, it may cause distortion of the audio signal. In the experiment, α=3; and β=0.001. According to the amplitude and phase angle after spectral subtraction, the inverse discrete Fourier transform can be performed to obtain the denoised time domain audio signal, that is, the standard environmental sound signal after denoising.

[0049] Step S220, extracting the sound features of the standard environmental sound signal, and identifying the sound features based on the trained sound recognition model to obtain the sound events represented by the standard environmental sound signal, wherein the sound events are used to represent the human voice information, environmental sound information, and alarm sound information of the vehicle's surrounding environment.

[0050] In one embodiment of the present application, extracting the sound features of the standard environmental sound signal includes: identifying the audio data of the standard environmental sound signal, and dividing the audio data into multiple data categories, the data categories at least including human voice, environmental sound, and alarm sound; extracting a first sound feature from the human voice, extracting a second sound feature from the background noise, extracting a third sound feature from the alarm sound, and determining the first sound feature, the second sound feature, and the third sound feature as the sound features of the standard environmental sound signal. In addition, before identifying the sound features based on the trained sound recognition model, it also includes: annotating the historical environmental sound signal based on the sound features, and dividing the annotated information to at least obtain a training data set and a verification data set; training the initial neural network model based on the training data set to learn the initial mapping relationship between the input environmental sound signal and the sound event; verifying the initial mapping relationship based on the verification data set, and optimizing the initial mapping relationship based on the verification result to obtain an optimized neural network model, and determining the optimized neural network model as the sound recognition model.

[0051] Generally speaking, in practical applications, in order to accurately extract the sound features of standard ambient sound signals, the system first processes the standard ambient sound signals after noise reduction. The specific process is as follows: the system identifies the audio data of the standard ambient sound signals and divides the audio data into multiple data categories, including at least human voices, ambient sounds, and alarm sounds. For each type of sound, the system extracts specific sound features:

[0052] Human voice: Extract primary sound characteristics from human voice, such as speech frequency range, pitch variation, etc.

[0053] Ambient sound: Extract secondary sound characteristics from background noise, such as noise spectrum distribution, energy level, etc.

[0054] Siren sound: Extract third-party sound characteristics from the siren sound, such as frequency pattern, duration, etc.

[0055] The extracted first sound characteristics, second sound characteristics and third sound characteristics together constitute the sound features of the standard environmental sound signal.

[0056] In addition, before the above sound features are recognized based on the trained sound recognition model, a series of preparatory steps are required. First, the system annotates the historical environmental sound signals based on the extracted sound features, and the annotated information includes but is not limited to the type of sound events (such as emergency shouts, vehicle horns, alarms, etc.). After the annotation is completed, the information is divided into a training data set and a verification data set; then, the initial neural network model is trained using the training data set to learn the initial mapping relationship between the input environmental sound signal and the corresponding sound event; during the training process, the model optimizes its prediction performance by continuously adjusting parameters. After completing the preliminary training, the performance of the model is evaluated using the verification data set; based on the verification results, the model parameters are further adjusted and optimized until a satisfactory recognition accuracy is achieved; finally, the neural network model obtained after multiple iterative optimizations is determined as the sound recognition model.

[0057] Specifically, in a specific implementation scenario, a car is driving on a city road, surrounded by various sounds, including the conversations of pedestrians, the driving sounds of other vehicles, and the alarm sounds in the distance. The system first performs noise reduction processing on the collected sound signals, and then divides them into three categories: human voice, environmental sound, and alarm sound. Then, the system extracts corresponding features from these three types of sounds, such as the voice frequency in human voice, the noise spectrum in environmental sound, and the frequency pattern in the alarm sound. Subsequently, the system uses a large amount of annotated historical sound data and divides these data into training sets and validation sets. Among them, the training set is used to train the initial neural network model so that it can learn to distinguish different types of environmental sounds, and the validation set is used to test the accuracy of the model and optimize it according to the validation results. After multiple rounds of training and verification, the system finally obtained a high-precision sound recognition model that can effectively identify various sound events in the real-time environment, thereby providing timely safety warnings for drivers.

[0058] In a specific embodiment of the present application, in the process of verifying the sound recognition model, the hit rate (HR), over segmentation rate (OS), and precision (PRC) are used to measure the accuracy of the system's sound information comparison processing. Among them, the hit rate is the ratio of the number of correctly detected ambient sound events to the total number of detections; the over segmentation rate is the ratio of the difference between the number of detected ambient sound events and the total number of annotations to the total number of annotations; the precision is the ratio of the number of correctly detected ambient sound events to the number of detected ambient sound events. The corresponding calculation formulas are as follows:

[0059]

[0060]

[0061]

[0062] Among them, N ref Indicates the number of ambient sound events actually included and marked in the experimental audio signal, N hit Indicates the number of consistent results between the algorithm detection results and the annotation results, N alg It represents the number of ambient sound events detected by the algorithm.

[0063] It should be noted that, in this embodiment, the overall accuracy requirements for the sound signal comparison are that the hit rate HR and the precision PRC are greater than 95%, and the over-segmentation rate is a positive number (indicating that the algorithm detects more ambient sound events than expected). The higher the over-segmentation rate, the more priority ambient sound events are detected during the sound recording process.

[0064] Step S230, determining the current warning level of the vehicle's surrounding environment based on the vehicle driving information and the sound event, triggering the vehicle alarm system to execute a corresponding warning plan according to the current warning level to warn the vehicle driver.

[0065] In one embodiment of the present application, the current warning level of the vehicle's surrounding environment is determined based on vehicle driving information and sound events, including: determining human voice information, environmental sound information, and alarm sound information of the vehicle's surrounding environment based on sound events; identifying keywords of the human voice information to obtain keyword information, determining the road type and climate conditions of the vehicle's surrounding environment based on the environmental sound information, determining the alarm category, alarm volume, and alarm duration based on the alarm sound information, and obtaining the vehicle's current speed based on the driving information; and assigning scores to the keyword information, road type, climate condition, alarm category, alarm volume, and alarm duration based on preset scoring rules to obtain a comprehensive score, so as to determine the current warning level of the vehicle's surrounding environment based on the comprehensive score.

[0066] In one embodiment of the present application, in order to determine the current warning level of the vehicle's surrounding environment, the system first extracts relevant information based on the sound event and performs a comprehensive evaluation in combination with the vehicle driving information. The specific process is as follows:

[0067] The system determines the human voice information, environmental sound information and alarm sound information of the vehicle's surrounding environment based on the sound events. For human voice information, the system uses speech recognition technology to identify key words, such as emergency words such as "help" and "get out of the way", to obtain keyword information. For environmental sound information, the system analyzes the characteristics of background noise to determine the type of road (such as urban roads, highways or rural roads) and climate conditions (such as sunny, rainy or snowy days) where the vehicle is located. For alarm sound information, the system identifies the specific category of the alarm (such as ambulance alarms, fire truck alarms, etc.) and measures the alarm volume and duration.

[0068] At the same time, the system obtains driving information such as the current speed of the vehicle. Next, the system assigns points to the above information based on the preset scoring rules. The scoring rules may include but are not limited to:

[0069] Keyword information: Score keywords based on their urgency, for example, “help” gets a higher score.

[0070] Road type: Different road types have different risk factors. For example, the risk factor of expressways is higher than that of urban roads.

[0071] Weather conditions: Score higher for adverse weather conditions, such as rain or snow.

[0072] Alert categories: Different types of alerts have different urgency levels, for example ambulance alerts have higher scores.

[0073] Alert volume: The louder the volume, the higher the score.

[0074] Alert duration: The longer the duration, the higher the score.

[0075] The system weights and sums these individual scores to get a comprehensive score. The comprehensive score reflects the severity of potential risks in the current environment. Based on this comprehensive score, the system determines the current warning level of the vehicle's surrounding environment. The warning level can be divided into multiple levels, such as low risk, medium risk and high risk.

[0076] In a specific embodiment of the present application, a car is driving on a city road, surrounded by the sounds of pedestrians talking, other vehicles driving, and ambulance sirens from a distance. The system first processes the collected sound signals, identifies the keyword "help" in the human voice, and determines that the road type is a city road and the weather condition is sunny. At the same time, the system detects the ambulance sirens, measuring that their volume is loud and their duration is long. In addition, the system obtains the current speed of the vehicle as 60 kilometers per hour.

[0077] Based on the preset scoring rules, the system scores each factor:

[0078] The keyword “help” scored 8 points;

[0079] Urban roads are scored 5 points;

[0080] A sunny day is scored as 3 points;

[0081] The ambulance siren scored 9 points;

[0082] The score for loud alarm volume was 7 points;

[0083] A longer duration of alarm is worth 6 points.

[0084] The system weights and sums these individual scores to obtain a comprehensive score of 38 points. According to the preset threshold range, this comprehensive score corresponds to a high-risk warning level. Therefore, the system triggers the vehicle alarm system, issues a high-risk warning to the driver, and recommends taking corresponding safety measures, such as slowing down, avoiding ambulances, etc., to improve driving safety.

[0085] It should be noted that the preset scoring rules can be based on actual conditions, and this application does not impose any restrictions on them. These rules can be flexibly adjusted and optimized according to specific application scenarios, driving environments, and safety requirements. For example, different road types may require different scoring criteria, and urban roads and highways may handle emergencies differently, so different weights can be set for each road type. In addition, changes in climatic conditions can also affect driving safety, and the scoring rules for sunny and rainy days may be different.

[0086] In general, when setting scoring rules, you can consider the following aspects:

[0087] Keyword information: Keywords are scored based on their urgency and commonness. For example, words like "help" and "get out of the way" have higher scores, while some common words in daily conversations have lower scores.

[0088] Road type: Different road types have different risk factors. For example, highways have higher risk factors than urban roads due to high speeds and dense traffic; rural roads may have higher risks due to poor visibility or poor road conditions.

[0089] Climate conditions: Severe weather conditions (such as rain and snow) should be scored higher than good weather conditions (such as sunny days). For example, when driving in the rain, the slippery road surface will increase the braking distance, thereby increasing the risk level.

[0090] Alarm category: Different types of alarms have different urgency levels. For example, the sirens of ambulances, fire trucks, and police cars usually indicate a very urgent situation and have a higher score, while the sirens of ordinary vehicles have a lower score.

[0091] Alarm volume: The louder the sound, the closer or more urgent the source of the sound, and thus the higher the score. The system can determine the volume by measuring the strength of the sound signal.

[0092] Alarm duration: The longer the duration, the more likely the emergency is to persist and the higher the risk. The system can record the duration of the alarm and assign a score based on that.

[0093] Vehicle speed: The speed of the vehicle is also an important consideration. When driving at high speeds, the driver has less time to react to unexpected situations, so the score should be higher when driving at high speeds.

[0094] Other factors: Other factors that affect driving safety may also be considered, such as traffic flow, visibility, driver fatigue, etc., which can all be part of the scoring rules.

[0095] It should be noted that by flexibly setting and adjusting these scoring rules, the system can more accurately assess the risk level of the current driving environment, thereby providing more effective warning information and helping the driver to take appropriate safety measures in a timely manner to ensure driving safety. This flexibility enables the system to adapt to various complex driving environments and improve overall safety and user experience.

[0096] Preferably, the current warning level of the vehicle's surrounding environment is determined based on the comprehensive score, and the vehicle alarm system is triggered to execute a corresponding warning plan according to the current warning level, including: when the comprehensive score is greater than or equal to the first scoring threshold and less than the second scoring threshold, the current warning level of the vehicle's surrounding environment is determined to be low risk, and a first-level warning plan is executed, and the first-level warning plan includes a visual warning; when the comprehensive score is greater than or equal to the second scoring threshold and less than the third scoring threshold, the current warning level of the vehicle's surrounding environment is determined to be medium risk, and a second-level warning plan is executed, and the second-level warning plan includes a visual warning and a sound warning; when the comprehensive score is greater than or equal to the third scoring threshold, the current warning level of the vehicle's surrounding environment is determined to be high risk, and a third-level warning plan is executed, and the third-level warning plan includes a visual warning, a sound warning, and a somatosensory warning; the first scoring threshold is less than the second scoring threshold, and the second scoring threshold is less than the third scoring threshold, the visual warning at least includes displaying the warning information in the vehicle instrument, the sound warning at least includes issuing a warning broadcast, and the somatosensory warning at least includes adjusting the seat status and tightening the seat belt.

[0097] In one embodiment of the present application, the current warning level of the vehicle's surrounding environment is determined based on the comprehensive score, and the corresponding warning plan is triggered according to the warning level. The specific process is as follows:

[0098] The system first calculates a comprehensive score based on the sound events and driving information. Based on this comprehensive score, the system divides the risk of the vehicle's surrounding environment into three levels: low risk, medium risk and high risk, and executes different levels of warning plans accordingly.

[0099] Low risk: When the comprehensive score is greater than or equal to the first scoring threshold and less than the second scoring threshold, the system determines that the current warning level of the vehicle's surrounding environment is low risk. At this time, the system implements the first-level warning plan, which includes visual warning. Specifically, the vehicle dashboard will display warning information to remind the driver to pay attention to potential risks in the current environment.

[0100] Medium risk: When the comprehensive score is greater than or equal to the second scoring threshold and less than the third scoring threshold, the system determines that the current warning level of the vehicle's surrounding environment is medium risk. At this time, the system implements the secondary warning plan, which includes visual warning and sound warning. In addition to displaying the warning information on the vehicle dashboard, the system will also issue a warning broadcast through the vehicle audio to remind the driver to take appropriate measures to deal with possible dangerous situations.

[0101] High risk: When the comprehensive score is greater than or equal to the third scoring threshold, the system determines that the current warning level of the vehicle's surrounding environment is high risk. At this time, the system implements a three-level warning plan, which includes visual warning, sound warning, and somatosensory warning. In terms of visual warning, the vehicle dashboard will display more obvious warning information; in terms of sound warning, the system will issue an emergency broadcast through the car audio; in terms of somatosensory warning, the system will adjust the seat status (such as slight vibration) and tighten the seat belt to enhance the driver's perception of potential dangers.

[0102] In a specific embodiment of the present application, take "a car is driving on a city road, surrounded by the sound of pedestrians talking, the sound of other vehicles driving, and the sound of ambulance sirens in the distance" as an example. After processing, the system obtains a comprehensive score of 38 points. Assume that the preset first score threshold is 20 points, the second score threshold is 40 points, and the third score threshold is 60 points.

[0103] Since the comprehensive score is 38 points, which is between the second scoring threshold (40 points) and the first scoring threshold (20 points), the system determines that the current warning level is medium risk.

[0104] The system then implements the secondary warning plan, including:

[0105] Visual warning: The vehicle dashboard displays a yellow warning icon to remind the driver to pay attention to the surrounding environment.

[0106] Voice warning: The car audio system issues a voice prompt: "Please note, there is an emergency vehicle ahead, please slow down and avoid it."

[0107] If the comprehensive score reaches 50 points, the system will determine it as high risk and implement a three-level warning plan, including:

[0108] Visual warning: The vehicle dashboard displays a red warning icon accompanied by a flashing effect.

[0109] Voice warning: The car audio system issues an emergency voice prompt: "Emergency! Please slow down and evade immediately!"

[0110] Somatosensory warning: The seat vibrates slightly and the seat belt tightens automatically to further remind the driver to pay attention to safety.

[0111] It should be noted that the above-mentioned threshold determination method and specific data, as well as the specific implementation methods of visual warning, sound warning, and somatosensory warning are not specifically limited in this application. These parameters and implementation methods can be flexibly set and adjusted according to the needs of actual applications, the specific configuration of the vehicle, and the characteristics of the driving environment. Specifically, the determination of the threshold can be based on a variety of methods such as analysis of historical data, expert opinions, and simulation test results. For example, through statistical analysis of a large amount of actual driving data, the distribution of comprehensive scores under different risk levels can be found, so as to set a reasonable threshold. In addition, it can also be adjusted according to factors such as traffic conditions and climatic conditions in a specific area. The specific threshold data can be set according to actual conditions. For example, the first scoring threshold can be set to 20 points, the second scoring threshold can be set to 40 points, and the third scoring threshold can be set to 60 points. These values ​​are not fixed and can be optimized according to the performance of the system, the driver's habits and other relevant factors. The specific implementation method of the warning can also be diverse, as follows:

[0112] Specific implementation methods of visual warning include but are not limited to instrument panel display, HUD head-up display, and navigation system integration.

[0113] Among them, the dashboard display can display warning icons of different colors on the vehicle dashboard, such as yellow icons for low risk, orange icons for medium risk, and red icons for high risk. HUD head-up display uses HUD (Head-Up Display) technology to project warning information on the front windshield, so that the driver can get warning information without looking down at the dashboard. Navigation system integration integrates warning information into the vehicle navigation system, displays potential danger areas through a map interface, and provides route suggestions.

[0114] The specific implementation methods of sound warnings include but are not limited to voice prompts, alarms, and other personalized settings. Specifically, voice prompts refer to pre-recorded voice prompts played through the vehicle audio system to inform the driver of the current risk level and recommended measures. Alarms refer to playing different types of alarms, such as short beeps or continuous sirens, to attract the driver's attention. Personalized settings allow drivers to choose different voice prompt styles according to their personal preferences, such as male voices, female voices or specific languages.

[0115] The specific implementation methods of somatosensory warning include but are not limited to seat vibration, seat belt tightening, steering wheel vibration, and tactile feedback. Specifically, seat vibration refers to the use of a vibration device built into the seat to transmit a vibration signal to the driver to enhance his perception of the warning. Seat belt tightening refers to automatically tightening the seat belt to alert the driver to the current emergency. Steering wheel vibration refers to the ability to alert the driver through slight vibrations of the steering wheel in some models. Haptic feedback refers to the fact that in some high-end models, it may also include changes in seat back pressure or other forms of tactile feedback.

[0116] Finally, it should be emphasized that the vehicle warning method based on environmental sound signal perception proposed in this application significantly improves driving safety by comprehensively using advanced sound processing technology and intelligent warning mechanism. The method first collects external environmental sound signals 360° without dead angles through a sound receiving device installed at the center of the roof, and performs noise reduction processing on these signals to generate clear standard environmental sound signals. Then, the system extracts different types of sound features such as human voice, environmental sound and alarm sound from the standard environmental sound signal, and uses a pre-trained sound recognition model to identify these features to determine the specific type of sound event. Then, based on the identified sound event, the system further analyzes and extracts detailed information such as keyword information, road type, climate status, alarm category, alarm volume and duration. Combined with driving information such as the current speed of the vehicle, the system scores these factors based on the preset scoring rules to obtain a comprehensive score, which reflects the severity of potential risks in the current environment, and then determines the current warning level of the vehicle's surrounding environment. Finally, based on different warning levels, the system triggers the corresponding warning plan. For example, when the comprehensive score is in the low-risk range, the first-level warning plan is implemented, including displaying visual warning information on the vehicle dashboard; when the risk is medium, the second-level warning plan is implemented, adding sound warnings and issuing voice prompts through the vehicle audio; and when the risk is high, the third-level warning plan is implemented. In addition to visual and sound warnings, it also includes somatosensory warnings, such as seat vibrations and tightening of seat belts, to remind the driver of potential dangers in a multi-sensory way.

[0117] This multi-level warning mechanism not only ensures that the driver can obtain important information in a timely manner, but also enhances the warning effect through multiple perception methods, avoiding the situation where key external sounds are ignored due to good sound insulation in the car; in addition, the flexibility of the system allows the threshold setting and warning method to be adjusted according to actual needs, so as to better adapt to different driving environments and user preferences. In short, the vehicle warning method based on environmental sound signal perception effectively improves driving safety and comfort through accurate sound recognition and intelligent warning, and provides all-round safety protection for drivers.

[0118] In one embodiment of the present application, after the vehicle driver is warned, the method further includes: monitoring the driver's reaction after the warning, at least including monitoring the driver's braking action, steering operation and acceleration behavior to determine whether the driver has implemented countermeasures; if the driver is not detected to implement countermeasures within a preset time, the current warning level is upgraded, and the warning is re-issued based on the upgraded warning level until the driver implements countermeasures, and the countermeasures are determined based on a warning measure mapping table, and the warning measure mapping table is used to characterize the mapping relationship between the countermeasures of the warning levels.

[0119] In a specific implementation of the present application, monitoring the driver's reaction after the warning includes monitoring the driver's actions through the vehicle's built-in sensors, such as monitoring whether the driver has braked through the pedal position sensor, monitoring whether there is steering operation through the steering wheel angle sensor, and monitoring whether there is acceleration through the throttle position sensor, so as to determine whether the driver has executed countermeasures according to the warning information; if the driver is not detected to perform any countermeasures within a preset time (for example, within 5 seconds), the current warning level is upgraded. For example, from a level 1 warning to a level 2 warning. Based on the upgraded warning level, the system will re-issue the warning according to the warning measure mapping table. The warning measure mapping table defines the countermeasures corresponding to different warning levels, such as a level 1 warning prompt tone, and a level 2 warning may include a louder prompt tone, a flashing dashboard prompt, etc.; until the driver executes the countermeasures, the system will continue to monitor and adjust the warning level according to the actual situation. Once an effective countermeasure is detected, the system will stop issuing a warning and record relevant information about the warning event, such as the time, location, warning level, and the final countermeasures taken, so as to facilitate later analysis and system improvement. "

[0120] Specifically, in this embodiment, if the vehicle's warning system detects a pedestrian suddenly appearing in front of it, the system will first issue a level 1 warning to remind the driver to pay attention to the front. If the driver does not respond within 5 seconds (such as braking or turning the steering wheel to avoid), the system will automatically upgrade the warning level to level 2 and add more warning measures until the driver takes the correct response measures.

[0121] It can be understood that the method proposed based on this embodiment not only enhances the effectiveness of the warning system and ensures that the driver can take timely action in emergency situations, but also provides a basis for system self-learning and improvement. By recording the situation of each warning, the warning measure mapping table can be continuously optimized to make it more in line with actual needs.

[0122] Figure 4 is a block diagram of a vehicle warning device based on environmental sound signal perception, which is shown in an exemplary embodiment of the present application. The device can be applied to Figure 1 The device may also be applicable to other exemplary implementation environments and specifically configured in other devices, and this embodiment does not limit the implementation environment to which the device is applicable.

[0123] like Figure 4 As shown, the exemplary vehicle warning device based on environmental sound signal perception includes: a sound signal acquisition and preprocessing module 410, a sound event recognition module 420, and a warning execution module 430.

[0124] Among them, the sound signal acquisition and preprocessing module 410 is used to obtain vehicle driving information and the initial environmental sound signal around the vehicle, and perform noise reduction on the initial environmental sound signal to obtain a standard environmental sound signal after noise reduction; the sound event recognition module 420 is used to extract the sound features of the standard environmental sound signal, and recognize the sound features based on the trained sound recognition model to obtain the sound event represented by the standard environmental sound signal; the early warning execution module 430 is used to determine the current early warning level of the vehicle's surrounding environment based on the vehicle driving information and sound events, and trigger the vehicle alarm system to execute the corresponding early warning plan according to the current early warning level to warn the vehicle driver.

[0125] It should be noted that the vehicle warning device based on environmental sound signal perception provided in the above embodiment and the vehicle warning method based on environmental sound signal perception provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment and will not be repeated here. In actual application, the vehicle warning device based on environmental sound signal perception provided in the above embodiment can distribute the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0126] An embodiment of the present application also provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the electronic device implements the vehicle warning method based on environmental sound signal perception provided in the above-mentioned embodiments.

[0127] Figure 5 The structure diagram of the computer system suitable for implementing the electronic device of the embodiment of the present application is shown. It should be noted that: Figure 5 The computer system 500 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0128] like Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage part 508 to the random access memory (RAM) 503, such as executing the method described in the above embodiment. In the RAM 503, various programs and data required for system operation are also stored. The CPU 501, the ROM 502 and the RAM 503 are connected to each other through the bus 504. The input / output (I / O) interface 505 is also connected to the bus 504.

[0129] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed so that a computer program read therefrom is installed into the storage section 508 as needed.

[0130] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 509, and / or installed from a removable medium 511. When the computer program is executed by a central processing unit (CPU) 501, various functions defined in the system of the present application are executed.

[0131] It should be noted that the computer-readable medium shown in the embodiment of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. This propagated data signal can take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. A computer program contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0132] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0133] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0134] Another aspect of the present application further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor of a computer, the computer executes the vehicle warning method based on environmental sound signal perception as described above. The computer-readable storage medium may be included in the electronic device described in the above embodiment, or may exist independently without being assembled into the electronic device.

[0135] Another aspect of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the vehicle warning method based on environmental sound signal perception provided in each of the above embodiments.

[0136] The above embodiments are merely illustrative of the principles and effects of the present application, and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A vehicle warning method based on environmental sound signal perception, characterized in that: The method comprises: Acquiring vehicle driving information and an initial ambient sound signal around the vehicle, and performing noise reduction processing on the initial ambient sound signal to obtain a standard ambient sound signal after noise reduction, wherein the vehicle driving information is parameter information used to describe the current motion state of the vehicle; Extracting sound features of the standard environmental sound signal, and recognizing the sound features based on the trained sound recognition model to obtain the sound events represented by the standard environmental sound signal, wherein the sound events are used to represent human voice information, environmental sound information, and alarm sound information of the surrounding environment of the vehicle; The current warning level of the vehicle's surrounding environment is determined based on the vehicle driving information and the sound event, and the vehicle alarm system is triggered to execute a corresponding warning plan according to the current warning level to warn the vehicle driver.

2. The vehicle warning method based on environmental sound signal perception according to claim 1 is characterized in that: The initial ambient sound signal is subjected to noise reduction processing, comprising: The initial ambient sound signal is divided into a plurality of single-frame audios, and each single-frame audio is transformed in the frequency domain to obtain an amplitude spectrum and a phase spectrum of each single-frame audio; Selecting a leading silent segment data from the initial ambient sound signal, and calculating the average energy of the frequency domain spectrum line of the leading silent segment data as a noise estimation value; The amplitude spectrum of each single frame of audio is adjusted based on the noise estimation value, and the adjusted amplitude spectrum and the original phase spectrum are subjected to inverse frequency domain transformation to obtain a standard ambient sound signal after noise reduction.

3. The vehicle warning method based on environmental sound signal perception according to claim 1 is characterized in that: Extracting the sound features of the standard environmental sound signal includes: Identifying audio data of the standard ambient sound signal, and dividing the audio data into a plurality of data categories, the data categories including at least human voice, ambient sound, and alarm sound; A first sound characteristic is extracted from human voice, a second sound characteristic is extracted from background noise, and a third sound characteristic is extracted from an alarm sound. The first sound characteristic, the second sound characteristic and the third sound characteristic are determined as sound characteristics of the standard environmental sound signal.

4. The vehicle warning method based on environmental sound signal perception according to claim 3 is characterized in that: Before recognizing the sound feature based on the trained sound recognition model, the method further includes: Annotating the historical environmental sound signals according to the sound features, and dividing the annotated information to obtain at least a training data set and a verification data set; Using the training data set to train the initial neural network model to learn the initial mapping relationship between the input environmental sound signal and the sound event; The initial mapping relationship is verified by the verification data set, and the initial mapping relationship is optimized based on the verification result to obtain an optimized neural network model, and the optimized neural network model is determined as a sound recognition model.

5. The vehicle warning method based on environmental sound signal perception according to claim 1 is characterized in that: Determining a current warning level of the vehicle's surrounding environment based on the vehicle driving information and the sound event includes: Identify the sound event to obtain a recognition result; the recognition result includes: keyword information obtained according to the keyword of the human voice information, road type and climate conditions of the vehicle's surrounding environment determined according to the environmental sound information, alarm type, alarm volume, and alarm duration determined according to the alarm sound information, and current speed of the vehicle according to the driving information; Based on the preset scoring rules, each of the recognition results is scored to obtain a comprehensive score, so as to determine the current warning level of the vehicle's surrounding environment based on the comprehensive score.

6. The vehicle warning method based on environmental sound signal perception according to any one of claims 1 to 5, characterized in that: After warning the vehicle driver, the method further includes: Monitor the driver's response after the warning, including at least monitoring the driver's braking action, steering operation and acceleration behavior to determine whether the driver has implemented the response measures; If the driver is not detected to have taken countermeasures within a preset time, the current warning level will be upgraded, and a warning will be reissued based on the upgraded warning level until the driver takes countermeasures. The countermeasures are determined based on a warning measure mapping table, and the warning measure mapping table is used to characterize the mapping relationship between the countermeasures of each warning level.

7. The vehicle warning method based on environmental sound signal perception according to any one of claims 1 to 5, characterized in that: Before obtaining the initial environmental sound signal around the vehicle, it also includes: Detecting the switch status of all vehicle windows, wherein the switch status of the vehicle windows includes fully closed, fully open and partially closed states; Activate the vehicle's radio when all windows are detected to be fully closed; The sound volume of the current environment is monitored. If the monitored ambient sound volume exceeds a preset threshold, the sound receiving device is triggered to start collecting initial ambient sound signals around the vehicle.

8. A vehicle warning device based on environmental sound signal perception, characterized in that: The device comprises: A sound signal acquisition and preprocessing module, used to obtain vehicle driving information and initial environmental sound signals around the vehicle, and perform noise reduction processing on the initial environmental sound signals to obtain standard environmental sound signals after noise reduction, wherein the vehicle driving information is parameter information used to describe the current motion state of the vehicle; A sound event recognition module, used to extract sound features of the standard environmental sound signal, and recognize the sound features based on the trained sound recognition model to obtain the sound event represented by the standard environmental sound signal, wherein the sound event is used to represent human voice information, environmental sound information, and alarm sound information of the vehicle surrounding environment; The warning execution module is used to determine the current warning level of the vehicle's surrounding environment based on the vehicle driving information and the sound event, and trigger the vehicle alarm system to execute a corresponding warning plan according to the current warning level to warn the vehicle driver.

9. An electronic device, characterized in that: It includes a processor, a memory and a communication bus; the communication bus is used to connect the processor and the memory; the processor is used to execute a computer program stored in the memory to implement the vehicle warning method based on environmental sound signal perception as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is used to enable a computer to execute the vehicle warning method based on environmental sound signal perception as described in any one of claims 1-7.

Images