Driving assistance prompting method and device, vehicle and electronic device

Abstract

The present disclosure relates to the technical field of driving assistance, in particular to a driving assistance prompting method and device, a vehicle and an electronic device. The present disclosure determines whether the intention of the rear driver to sound the horn is to send a prompt signal through the sound signal behind the vehicle, determines whether the external environment of the current vehicle meets the passing condition according to the environmental information in the driving direction of the vehicle, determines whether the vehicle itself and the driver have the driving condition based on the vehicle fault information and the state information of the driver, and gives the driver the corresponding prompt in time when the rear driver sounds the horn to prompt and the external environment and internal condition of the vehicle meet the passing condition, thereby improving the sensitivity of the driving assistance strategy to the traffic state change under complex scenes, giving the driver the corresponding prompt in time, avoiding further pressure on the driver's psychology caused by the false prompt when the vehicle cannot move temporarily due to vehicle failure and the like, and thereby improving the flexibility of the driving assistance prompting method.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of driving assistance, and in particular to a driving assistance prompting method and device, a vehicle and an electronic device. BACKGROUND

[0002] In daily driving, especially in the case of urban traffic congestion, drivers will face frequent parking and starting operations.

[0003] When waiting for a long time in traffic jams or red lights, the driver may be distracted by the long waiting time. At this time, if the driver fails to notice that the front vehicle has started moving in time, and still stays in the original position waiting, it will affect the traffic efficiency, and even cause dissatisfaction of the rear vehicle. In the prior art, the driver can know the congestion situation of the front road and the countdown of the red light waiting by using navigation software, which can help the driver to estimate the road conditions and start the vehicle in time.

[0004] However, the navigation software has certain limitations in collecting and feedback information of the front road situation. For example, in the case of traffic jams, the navigation software can only inform the user that there is congestion in the front road, but cannot prompt the user in time when the front vehicle has moved forward, and lacks flexibility. SUMMARY

[0005] In order to solve the above technical problems, the present disclosure provides a driving assistance prompting method, device, vehicle and electronic device to improve the flexibility of the driving assistance prompting method.

[0006] In a first aspect, the embodiments of the present disclosure provide a driving assistance prompting method, comprising:

[0007] When the vehicle is in a slow driving section, acquiring a sound signal behind the vehicle, environment information in the driving direction of the vehicle and state information of the driver of the vehicle;

[0008] Determining whether the sound signal meets a preset prompting condition and whether the environment in the driving direction of the vehicle meets a traffic condition;

[0009] According to the running state information of the vehicle and / or the state information of the driver of the vehicle, determining whether the vehicle meets a normal driving condition;

[0010] If the sound signal meets the preset prompting condition, the environment in the driving direction of the vehicle meets the traffic condition, and the vehicle meets the normal driving condition, a prompting message is generated.

[0011] In a second aspect, the embodiments of the present disclosure provide a driving assistance prompting device, comprising:

[0012] The acquisition module is configured to acquire a sound signal behind the vehicle, environment information in a driving direction of the vehicle, and state information of a driver of the vehicle when the vehicle is on a slow lane;

[0013] The first determination module is configured to determine whether the sound signal meets a preset prompt condition and whether the environment in the driving direction of the vehicle meets a passing condition.

[0014] The second determination module is configured to determine whether the vehicle meets a normal driving condition according to the running state information of the vehicle and / or the state information of the driver of the vehicle.

[0015] The generation module is configured to generate a prompt message if the sound signal meets the preset prompt condition, the environment in the driving direction of the vehicle meets the passing condition, and the vehicle meets the normal driving condition.

[0016] In a third aspect, an embodiment of the present disclosure provides a vehicle, comprising:

[0017] a memory;

[0018] a processor;

[0019] The memory stores executable program code, and the processor is configured to invoke and execute the executable program code to execute the method of the first aspect.

[0020] In a fourth aspect, an embodiment of the present disclosure provides an electronic device, comprising:

[0021] a memory;

[0022] a processor; and

[0023] a computer program;

[0024] The computer program is stored in the memory and is configured to be executed by the processor to implement the method of the first aspect.

[0025] The driving assistance prompting method, device, vehicle and electronic device provided by the embodiments of the present disclosure determine whether the intention of the rear driver to sound the horn is to send a prompt signal through the sound signal behind the vehicle, determine whether the external environment of the current vehicle meets the passing condition according to the environmental information in the direction in which the vehicle travels, determine whether the vehicle itself and the driver have the driving condition based on the vehicle fault information and the state information of the driver of the vehicle, and give the driver a corresponding reminder in time when the rear driver sounds the horn to prompt and the external environment and internal condition of the vehicle both meet the passing condition, thereby improving the sensitivity of the driving assistance strategy to the traffic state change in a complex scene, giving the driver a corresponding reminder in time, avoiding further pressure on the driver caused by the false reminder when the vehicle cannot move temporarily due to vehicle failure and the like, and thereby improving the flexibility of the driving assistance prompting method. BRIEF DESCRIPTION OF DRAWINGS

[0026] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure.

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the accompanying drawings required to be used in the embodiments or prior art description will be briefly introduced. Obviously, those skilled in the art can obtain other drawings according to these drawings without any creative effort.

[0028] Figure 1 A driving assistance prompting method flowchart provided by an embodiment of the present disclosure;

[0029] Figure 2 A road scene schematic diagram provided by an embodiment of the present disclosure;

[0030] Figure 3 A driving assistance prompting method flowchart provided by another embodiment of the present disclosure;

[0031] Figure 4 A structural schematic diagram of a driving assistance prompting device provided by an embodiment of the present disclosure;

[0032] Figure 5 A structural schematic diagram of a vehicle provided by an embodiment of the present disclosure;

[0033] Figure 6 A structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] In order to more clearly understand the above-mentioned purposes, features and advantages of the present disclosure, the solutions of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0035] In the following description, a large number of specific details are set forth in order to facilitate a thorough understanding of the present disclosure, but the present disclosure can also be implemented in other manners different from those described herein; obviously, the embodiments described in the specification are only a part of the embodiments of the present disclosure, and not all the embodiments.

[0036] During the driving of the driver, especially in the city environment, traffic jams and waiting for traffic lights are common driving scenarios.

[0037] For example, due to serious congestion, or long red light time, and many queued vehicles, the vehicle will be parked for a long time, at this time the driver may be distracted, such as operating the car screen to switch the song played, and unable to timely notice that the front vehicle has started to move. At this time, the driver of the rear vehicle will honk frequently due to impatience, which not only increases noise pollution but also brings pressure to the driver.

[0038] Some existing navigation software and other driving assistance software can already make early announcements about road conditions for various scenarios, such as prompting the driver to enter a congested road section ahead and drive carefully; or showing the driver a red light waiting countdown. However, this way cannot specifically prompt the driver to start in time in the above scenarios.

[0039] For example, the driving assistance software cannot obtain the movement state of each vehicle in the traffic jam scenario, so it cannot determine whether the vehicle needs to start and follow at this time; in the red light waiting scenario, the red light waiting countdown announcement is only accurate for vehicles close to the red light, and for vehicles far from the red light, they cannot pass immediately after the green light, at this time the red light waiting countdown announcement has little significance for the driver.

[0040] To solve the above problems, the present disclosure provides a driving assistance prompting method, which will be introduced below in combination with specific embodiments.

[0041] Figure 1 The driving assistance prompting method provided by the present disclosure is shown in the flowchart. The method can be applied in a vehicle-mounted device, such as a car machine. It can be understood that the driving assistance prompting method provided by the present disclosure can also be applied in other scenarios.

[0042] The driving assistance prompting method provided by the present disclosure is shown in the flowchart. The method can be applied in a vehicle-mounted device, such as a car machine. It can be understood that the driving assistance prompting method provided by the present disclosure can also be applied in other scenarios. Figure 1 The driving assistance prompting method provided by the present disclosure is shown in the flowchart. The method can be applied in a vehicle-mounted device, such as a car machine. It can be understood that the driving assistance prompting method provided by the present disclosure can also be applied in other scenarios.

[0043] S101, when the vehicle is in a slow lane, acquiring a sound signal behind the vehicle, environment information in the driving direction of the vehicle, and state information of the driver of the vehicle.

[0044] Specifically, the sound signal behind the vehicle refers to the horn sound from the rear of the vehicle, i.e. the horn sound of the rear vehicle in the same lane.

[0045] The sound signal generally refers to various sounds generated in the environment around the vehicle, such as horn sounds of other vehicles, voices of pedestrians, road construction sounds, etc. These sound signals can be collected by microphones and other sensors and converted into electrical signals, which are then analyzed and processed by the system.

[0046] The vehicle is equipped with a microphone for collecting sound signals from the external environment of the vehicle. Specifically, using multiple microphones to form an array can improve the accuracy and directivity of sound signal collection. Through beamforming technology, the sound signal in a specific direction can be enhanced while suppressing noise in other directions. For example, installing a microphone array at the tail of the vehicle can more accurately capture the horn sound of the rear vehicle.

[0047] Optionally, the horn sound from the rear of the vehicle is obtained through the microphone array to obtain the sound signal.

[0048] The environment information in the driving direction of the vehicle is the environment information in front of the vehicle. The environment information in the driving direction of the vehicle is obtained through the camera, laser radar, etc. environment information collection device carried on the vehicle. Alternatively, the environment information in the driving direction of the vehicle is obtained by cutting from the environment information of the environment where the vehicle is located according to the orientation of the vehicle.

[0049] Optionally, in the stopped state of the vehicle, the sound signal behind the vehicle and the environment information in the driving direction of the vehicle are acquired.

[0050] The driver state information of the vehicle refers to a collection of data that determines whether the driver has the ability to drive safely by monitoring the driver's physiological, behavioral and cognitive characteristics in real time, including physiological indicators, behavioral characteristics, operation performance, and emotional and health status.

[0051] The driver state information can be obtained by capturing the driver's facial features, eye movements and head posture in real time based on the in-vehicle camera (such as infrared or RGB camera), or by collecting physiological data through sensors on the steering wheel, seat or wearable device, or by analyzing driving operation patterns in combination with vehicle CAN bus data.

[0052] S102, determining whether the sound signal meets the preset prompt condition and whether the environment in the driving direction of the vehicle meets the passing condition.

[0053] The sound signal conforms to the preset prompt condition represents that the rear vehicle emits a horn sound with urging nature.

[0054] After obtaining the sound signal, the interactive information expressed by the sound signal is determined according to the basic characteristics of the sound signal, such as frequency, amplitude, wavelength and phase. When the sound signal conforms to the preset prompt condition, it represents that the rear vehicle emits a horn sound with urging nature.

[0055] When the vehicle is stuck in traffic or waiting for a traffic light, the vehicle driver may be distracted due to the long waiting time and fail to timely notice that the front vehicle has moved forward. The rear vehicle may honk due to impatience. At this time, the sound signal from the rear vehicle is different from the honk prompt in normal driving, but has certain characteristics, such as volume, duration, sound frequency and change frequency, which conform to certain preset conditions, such as long duration or multiple sharp honks. At this time, the sound signal is determined to conform to the preset prompt condition according to the characteristics of the sound signal.

[0056] The volume of the honk can help determine the intensity or distance of the honk. The volume information can be used to assess the urgency or distance of the honk, so as to better understand the intention or source of the honk.

[0057] The acquisition of sound frequency and its change frequency can be used to more accurately identify the type or mode of the honk. Different honk sounds may have different frequency characteristics, which can be used to distinguish different types of honks (such as warning honks, emergency honks, etc.). In addition, the acquisition of change frequency can be used to identify the mode of the honk (such as continuous honk, intermittent honk, etc.), so as to better understand the intention of the honk.

[0058] By acquiring the volume, sound frequency and its change frequency, the present application can perform more comprehensive acoustic feature analysis on the honk sound. Such analysis can not only determine the existence and frequency of the honk, but also further identify the type, mode and urgency of the honk, thereby improving the accuracy and intelligence level of honk recognition.

[0059] In some embodiments, after obtaining the sound signal of the vehicle external environment, the sound signal is preprocessed to improve the quality and availability of the sound signal.

[0060] Optionally, the sound signal is filtered in the frequency domain. The sound signal is converted from time domain to frequency domain by Fourier transform, and high-frequency noise such as wind noise and low-frequency noise such as engine noise are removed. Further, the filter parameters are dynamically adjusted according to the characteristics of the environmental noise to achieve better noise reduction effect.

[0061] Optionally, the sound signal is gain controlled. The gain is automatically adjusted according to the intensity of the sound signal, ensuring that the sound signal is controlled within a suitable range, avoiding excessively strong or weak sound signals affecting subsequent processing.

[0062] Optionally, whether the environment in the direction of the vehicle meets the passing condition is determined by collecting a road image in front of the vehicle through a vehicle-mounted camera and analyzing information in the image to determine whether the environment in the direction of the vehicle meets the passing condition.

[0063] The information in the image includes, but is not limited to, front vehicles and obstacles, traffic signs and signal lights, road conditions, and the like.

[0064] The environmental information in the direction of the vehicle reflects whether the road environment in front of the vehicle is suitable for passing. If the road environment in front of the vehicle is suitable for passing, it is determined that the environment in the direction of the vehicle meets the passing condition; if the road environment in front of the vehicle is not suitable for passing, it is determined that the environment in the direction of the vehicle does not meet the passing condition.

[0065] S103, according to the running state information of the vehicle and / or the state information of the driver of the vehicle, determine whether the vehicle meets the normal driving condition.

[0066] The vehicle meets the normal driving condition includes that the vehicle does not have a fault affecting driving, and the driver of the vehicle has driving ability.

[0067] According to the vehicle, the real-time data stream of the vehicle-mounted electronic control unit is continuously collected, the key parameters of the vehicle power system, the braking system and the steering system are obtained for threshold analysis, for example, when it is detected that the brake pressure is continuously lower than the warning value for a preset time, or the power battery temperature exceeds the preset temperature, or there is a serious fault code, it is determined that the current vehicle has a fault affecting driving, causing the vehicle to temporarily unable to move.

[0068] And the less serious faults such as the electric window cannot be closed, the air conditioning temperature control system in the vehicle fails, etc. It is determined that the current vehicle does not have a fault affecting the driving of the vehicle.

[0069] At the same time, the driver's facial features are captured through visual sensors in the cabin of the vehicle, for example, based on the proportion of eyelid closure time in unit time, based on gaze tracking, the proportion of gaze deviation time in unit time of the driver of the vehicle is detected, and the fatigue driving and distraction state of the driver of the vehicle are comprehensively determined; the biological sensors on the seat and steering wheel are used to monitor the driver's physical condition and steering wheel grip distribution in real time, to determine whether the current driver of the vehicle is temporarily distracted or has persistent loss of driving ability due to sudden illness and the like.

[0070] If the state of the driver of the vehicle belongs to the persistent driving ability loss, it is determined that the driver of the vehicle does not have driving ability at present.

[0071] In S104, if the sound signal meets the preset prompting condition, the environment in the driving direction of the vehicle meets the passing condition, and the vehicle meets the normal driving condition, a prompting message is generated.

[0072] Specifically, the environment in the driving direction of the vehicle meeting the passing condition means that the road environment in the driving direction of the vehicle allows the vehicle to pass. For example, the distance between the vehicle and the preceding vehicle is greater than a preset distance.

[0073] Meanwhile, the vehicle meeting the normal driving condition means that the current vehicle does not have a fault affecting driving, and the driver of the vehicle has driving ability and is not in a persistent driving ability loss state.

[0074] The sound signal meeting the preset prompting condition means that the following vehicle sends a prompting intention to the current vehicle (the vehicle). The environment in the driving direction of the vehicle meeting the passing condition means that the environment around the current vehicle meets the condition for the vehicle to move forward. The vehicle meeting the normal driving condition means that the current vehicle has driving ability, and the driver has driving ability. At this time, it is determined that the driver may not drive the vehicle to start moving in time due to distraction or the like, causing unnecessary waiting of the following vehicle and dissatisfaction of the driver of the following vehicle, and the driver needs to be reminded by the prompting message.

[0075] The prompting message is used to prompt the driver to start in time and / or to pay attention to the road environment in front.

[0076] Optionally, the prompting content of the prompting message is determined according to the current state of the vehicle. Specifically, if the current vehicle is in a parking state, the prompting message is used to prompt the driver to start in time. If the current vehicle is in a driving state, the prompting message is used to prompt the driver to pay attention to the road environment in front.

[0077] For example, when the sound signal meets the preset prompting condition, it means that the driver of the following vehicle sends a prompting signal. If the environment in the driving direction of the vehicle meets the passing condition and the vehicle meets the normal driving condition at the same time, it means that the road environment in the driving direction of the vehicle allows the vehicle to pass, and the driver of the vehicle should start moving in time. Therefore, a prompting message is generated to prompt the driver of the vehicle to start.

[0078] Alternatively, when the distance between the vehicle and the preceding vehicle is too large due to slow driving speed, the driver of the following vehicle honks to urge, at this time, the sound signal meets the preset prompting condition, the environment in the driving direction of the vehicle meets the passing condition, and the vehicle meets the normal driving condition. Therefore, a prompting message is generated to prompt the driver of the vehicle to pay attention to the road environment in front and accelerate in time.

[0079] Optionally, the display mode of the prompt message includes, but is not limited to, voice broadcast, central control screen image display, steering wheel slight vibration, automatic tightening of safety belt, etc., and the present disclosure does not limit this.

[0080] Figure 2 A road scene schematic diagram provided by the present disclosure is shown in FIG. 1. Figure 2 As shown in FIG. 1, the environment in front of the vehicle 21 meets the passing condition, and the vehicle 22 emits a horn sound meeting the preset prompt condition. The sound signal of the vehicle exterior environment acquired by the vehicle 21 includes the horn sound emitted by the vehicle 22 meeting the preset prompt condition, and the vehicle 21 determines through visual detection that the environment in front meets the passing condition, and the vehicle meets the normal driving condition, and then a prompt message is generated to prompt the driver.

[0081] The present disclosure determines whether the sound signal meets the preset prompt condition, whether the environment in the driving direction of the vehicle meets the passing condition, and whether the vehicle meets the normal driving condition according to the running state information of the vehicle and / or the state information of the driver of the vehicle. If the sound signal meets the preset prompt condition, the environment in the driving direction of the vehicle meets the passing condition, and the vehicle meets the normal driving condition, a prompt message is generated. The intention of the rear driver to sound the horn is determined through the sound signal behind the vehicle, whether the external environment of the current vehicle meets the passing condition is determined according to the environmental information in the driving direction of the vehicle, and whether the vehicle itself and the driver have the driving condition is determined based on the vehicle fault information and the state information of the driver of the vehicle. When the rear driver sounds the horn and the external environment and internal condition of the vehicle meet the passing condition, the driver is timely reminded, the sensitivity of the driving assistance strategy to the traffic state change under complex scenes is improved, the driver is timely reminded, and the further psychological pressure on the driver caused by the false reminder when the vehicle cannot temporarily move due to vehicle failure is avoided, thereby improving the flexibility of the driving assistance prompt method.

[0082] In addition, in the long waiting scene such as traffic jam, the present disclosure can effectively reduce the risk of traffic accidents caused by the negligence of the driver, and also relieve the psychological pressure caused by long stagnation.

[0083] On the basis of the above-mentioned embodiments, before the sound signal behind the vehicle, the environmental information in the driving direction of the vehicle, and the state information of the driver of the vehicle are acquired when the vehicle is in the slow driving section, the method further includes determining whether the vehicle is in the slow driving section based on the running state information of the vehicle and / or the environmental information.

[0084] The running state information includes a vehicle speed, and the environment information includes a rear vehicle speed. The determination of whether the vehicle is in the slow driving section based on the running state information and / or the environment information includes: when the running state information and / or the environment information of the vehicle satisfy one or more of the following conditions, determining whether the vehicle is in the slow driving section.

[0085] Optionally, the vehicle speed is lower than a preset speed threshold, and the duration exceeds a preset time threshold.

[0086] In a traffic jam or slow driving section, the vehicle keeps low speed for a long time. When the vehicle speed is lower than a preset speed threshold (for example, 20 km / h), and the low speed state lasts for more than a preset time threshold (for example, 5 minutes), it can be determined that the vehicle is in the slow driving section.

[0087] Alternatively, in a first preset time period, the change amount of the vehicle speed is greater than a preset speed change amount threshold.

[0088] Due to the sudden deceleration of the vehicle caused by the traffic jam or accident in front, it may represent that the slow driving section will be entered in front. In a first preset time period (for example, 10 minutes), if the speed change amount of the vehicle is too large, for example, from 50 km / h to 10 km / h, which exceeds the preset speed change amount threshold (for example, 30 km / h), it can be determined that the vehicle is in the slow driving section.

[0089] Alternatively, in a second preset time period, the relative speed between the vehicle speed and the rear vehicle speed is less than a relative speed threshold.

[0090] In the slow driving section, the vehicle and the rear vehicle may keep similar speed. In a second preset time period (for example, 5 minutes), if the relative speed between the speed of the vehicle and the speed of the rear vehicle, that is, the speed difference between the two vehicles, is less than a preset relative speed threshold (for example, 5 km / h), it can be determined that the vehicle is in the slow driving section.

[0091] Alternatively, according to the environment information, it is determined that the number of vehicles in a preset road section length is greater than a preset number threshold.

[0092] In a vehicle dense section caused by traffic jam or peak period, slow driving phenomenon may occur. According to the camera or sensor data and environment information, if the number of vehicles in a preset road section length exceeds a preset number threshold, it can be determined that the vehicle is in the slow driving section.

[0093] Alternatively, according to the environment information, it is determined that the distance between the adjacent two vehicles is less than a preset interval, and the duration exceeds a preset time threshold.

[0094] wherein the two adjacent vehicles can be the current vehicle and the rear vehicle, or other two adjacent vehicles. When the distance between the two adjacent vehicles is less than the preset distance, and the close distance state lasts more than the preset time threshold, that is, the two adjacent vehicles are in a close distance following state for a long time, it can be judged that the vehicle can be in a slow driving section.

[0095] The embodiments of the present disclosure can comprehensively determine whether the vehicle is in a slow driving section by combining the running state information of the vehicle (such as speed, speed change) and the environmental information (such as rear vehicle speed, vehicle density). The setting of these conditions helps to more accurately identify traffic congestion or slow driving conditions, so as to timely give the driver a corresponding reminder in the slow driving scenario, thereby providing support for the driving assistance prompt method.

[0096] In some embodiments, the determining whether the sound signal meets the preset prompt condition comprises: extracting a sound feature of the sound signal, the sound feature comprising average volume, duration, and change frequency of sound frequency; and determining whether the sound signal meets the preset prompt condition according to the sound feature.

[0097] wherein the volume refers to the loudness of the sound signal. The sound signal of the surrounding environment is collected by the microphone array carried on the vehicle, and through the beamforming technology, the array can directionally enhance the sound from a specific direction (such as the horn sound from the rear vehicle), while suppressing the interference noise from other directions (such as the side wind noise), so as to focus on the sound behind the vehicle. The sound is essentially an air pressure fluctuation signal, and the root mean square (RMS) of the sound emission signal amplitude is a statistical quantity used to describe the intensity of the sound signal, which represents the average energy size of the sound signal in a period of time, that is, the average volume.

[0098] The duration is the duration of the sound signal. In the embodiments of the present disclosure, the duration is used to judge the length of the horn sound of the rear vehicle.

[0099] The change frequency of the sound frequency is the change frequency of the sound signal frequency. The sound frequency is one of the attributes of the sound signal, and the change frequency of the sound frequency reflects the change trend of the sound signal, such as the high-low fluctuation, so as to reflect the meaning of the sound signal. For example, the rear vehicle driver performs multiple short honks in a short time, which can represent the urging intention, which is reflected in the sound signal with frequent changes in high and low frequencies, wherein the high-frequency signal is the horn sound of the rear vehicle, and the low-frequency signal is the environmental signal. The sound signal frequently switches between the high-frequency signal and the low-frequency signal in a short time.

[0100] Specifically, the determining whether the sound signal meets the preset prompt condition according to the sound feature comprises: determining an average volume and a duration of the sound signal; and if the average volume is greater than a preset volume threshold and the duration is greater than a preset time threshold, it is determined that the sound signal meets the preset prompt condition.

[0101] If the average volume of the sound signal is greater than the preset volume threshold and the duration reaches the preset time threshold, it is considered that the sound signal meets the preset prompt condition. For example, a rear vehicle continuously honks to express the intention of urging, at this time, the average volume of the sound signal in a period of time is greater than the preset volume threshold, and the duration of the sound signal reaches the preset time threshold, so it is determined that the sound signal is the horn sound from the rear vehicle in the form of urging, which meets the preset prompt condition.

[0102] A possible case is that the rear vehicle urges by continuously honking, at this time, the duration of the sound signal is greater than the preset time threshold, and the average volume is greater than the preset volume threshold, which meets the preset prompt condition.

[0103] Another possible case is that the rear vehicle is a special vehicle performing a task, such as a police car, an ambulance, etc., continuously emits a special alarm sound, at this time, the duration of the sound signal is greater than the preset time threshold, and the average volume is greater than the preset volume threshold, which meets the preset prompt condition.

[0104] Optionally, whether the duration meets the preset prompt condition is determined by using a preset time interval, if the duration of the sound signal is less than a minimum value of the preset time interval, it represents that the sound signal is too short; if the duration of the sound signal is greater than a maximum value of the preset time interval, it represents that the sound signal is too long, and it is determined that the sound signal does not meet the preset prompt condition.

[0105] Specifically, the determining whether the sound signal meets the preset prompt condition according to the sound feature comprises: determining a variation frequency of the sound frequency of the sound signal in a preset sound frequency interval according to the variation information of the sound frequency; and if the variation frequency of the sound frequency is greater than a preset variation frequency threshold, it is determined that the sound signal meets the preset prompt condition.

[0106] The sound signal is subjected to frequency spectrum analysis, and the main frequency component is extracted. Usually, the horn sound representing the urging nature will have obvious frequency variation, such as frequently switching between high and low frequency signals.

[0107] Meanwhile, a preset sound frequency range needs to be determined to remove frequency bands that are too high or too low. In actual situations, the horn sound of a car is usually in a relatively fixed frequency range, and the sound frequency range can effectively filter other interference sounds. For example, a rear driver performs multiple short honks in a short time, which can represent the intention of urging, which is reflected in the sound signal of frequent changes in high and low frequencies. The high frequency signal is the horn sound of the rear car, and the low frequency signal is the environmental signal. The sound signal frequently switches between high frequency signals and low frequency signals in a short time, which meets the preset prompt condition.

[0108] The embodiments of the present disclosure determine the intention of a sound signal by comprehensively determining the sound features of the sound signal, such as the average volume, duration, and change frequency of the sound frequency. When it is determined according to the sound features that the sound signal represents the intention of urging, it is determined that the sound signal meets the preset prompt condition, which can cover various sound scenarios and ensure the comprehensiveness of sound signal detection.

[0109] On the basis of the above-mentioned embodiments, a sound classification model is trained based on the above-mentioned preset prompt condition, and whether the collected sound signal meets the preset prompt condition is determined by the trained sound classification model.

[0110] Specifically, a sound classification model to be trained is trained according to a sample sound signal and a mode label of the sample sound signal, so that the sound classification model to be trained learns the sound features of the sound signal, and a trained sound classification model is obtained. The trained sound classification model includes a sound classification threshold, which is used to determine whether the sound signal meets the preset prompt condition according to the sound features of the sound signal.

[0111] A large number of sample sound signals with labeled mode labels are obtained, and the mode label of each sample sound signal represents whether the sample sound signal meets the preset prompt condition. The sample sound signal is input into the sound classification model to be trained, so that the sound classification model to be trained learns the sound features of the sample sound signal, including but not limited to the average volume, duration, and change frequency of the sound frequency.

[0112] The mode label of the sample sound signal is used as the output of the sound classification model to be trained, so that the sound classification model to be trained further learns the logical relationship of obtaining the mode label of the sample sound signal based on the sound features learned by the sound classification model to be trained, and finally obtains the trained sound classification model.

[0113] The trained sound classification model includes a sound classification threshold value for distinguishing sound signals meeting the preset prompt condition and sound signals not meeting the preset prompt condition. The sound classification threshold value is obtained by cross-validation and other methods during model training. For example, when the sound feature evaluation value of a sound signal exceeds the sound classification threshold value, the sound signal is considered to be a sound signal meeting the preset prompt condition. Correspondingly, when the sound feature evaluation value of a sound signal does not exceed the sound classification threshold value, the sound signal is considered to be a sound signal not meeting the preset prompt condition.

[0114] The implementation of the sound classification threshold value obtained by the cross-validation method is to divide the sample sound signals into a training set and a test set, train the sound classification model to be trained through the training set, test the sound classification model to be trained through the test set to determine the classification effect of the sound classification model, and adjust the parameters (such as the sound classification threshold value) of the trained sound classification model according to the classification effect until the classification effect of the sound classification model meets the expected standard.

[0115] In actual application, the vehicle-mounted device is equipped with a trained sound classification model. After the microphone array collects the sound signal of the external environment of the vehicle, the sound signal is input into the trained sound classification model to obtain a judgment result of whether the sound signal meets the preset prompt condition, and further participates in subsequent logical decision-making.

[0116] The embodiments of the present disclosure realize the judgment logic of whether the sound signal meets the preset prompt condition by using the pre-trained sound classification model, without performing complex multi-dimensional algorithms every time the sound signal is judged, thereby improving the analysis and classification efficiency of the sound signal.

[0117] On the basis of any one of the above embodiments, the determination of whether the sound signal meets the preset prompt condition includes: determining the size of the obstacle-free space in the vehicle driving direction according to the environmental information; and determining that the environment in the vehicle driving direction meets the passing condition if the obstacle-free space in the vehicle driving direction is greater than a preset space size.

[0118] Specifically, the environmental information is visually recognized to determine whether the environment in the vehicle driving direction meets the passing condition.

[0119] For example, the environmental information of the vehicle driving direction includes the size of the obstacle-free space in the vehicle driving direction. When the size of the obstacle-free space in the vehicle driving direction is greater than a preset space size, it means that the vehicle has the condition to drive forward, and it is determined that the environment in the vehicle driving direction meets the passing condition.

[0120] Alternatively, the environment information of the vehicle driving direction comprises a distance between the vehicle and a nearest obstacle in front of the vehicle, and when the distance between the vehicle and the nearest obstacle in front of the vehicle is greater than a preset distance, it is determined that the environment in the vehicle driving direction meets the passing condition.

[0121] Optionally, if the environment information comprises traffic indication information, it is determined whether the vehicle is allowed to pass according to the traffic indication information, the traffic indication information comprising at least one of a traffic signal, a traffic sign, and a lane guide arrow mark; if it is determined that the vehicle is allowed to pass and there is an obstacle-free space greater than a preset space size in the vehicle driving direction, it is determined that the environment in the vehicle driving direction meets the passing condition.

[0122] The traffic indication information can be visually recognized and analyzed from the environment information of the vehicle driving direction. According to the traffic indication information, it is determined whether the space in front of the vehicle is allowed to pass, for example, when the environment information of the vehicle driving direction comprises traffic signal information, it is determined whether the traffic signal is red or green by distinguishing the color of the traffic signal, and when the traffic signal is red, it means that the space in front of the current vehicle is not allowed to pass, even if there is an obstacle-free space greater than a preset space size in the vehicle driving direction, it cannot be determined that the environment in the vehicle driving direction meets the passing condition.

[0123] For example, when the vehicle is waiting for a red light at an intersection and is the first in the waiting line, the driver of the vehicle behind may start honking in advance to urge when the countdown of the red light is about to end, at this time, there is an obstacle-free space greater than a preset space size in the vehicle driving direction, but since the traffic indication information determines that the space in front of the vehicle is not allowed to pass at present, it still cannot be determined that the environment in the vehicle driving direction meets the passing condition.

[0124] For another example, when the traffic indication information comprises a traffic signal and a lane guide arrow mark, it is assumed that the current vehicle is located in the first position of the straight and right-turn lanes, and the traffic signal in the straight direction in front is red, at this time, the driver of the vehicle behind honks urgently to urge, and the sound signal behind the vehicle meets a preset condition; at this time, there is an obstacle-free space greater than a preset space size in the vehicle driving direction, but since the traffic indication information determines that the space in front of the vehicle is not allowed to pass at present, even if the vehicle meets the normal driving condition, it cannot pass, at this time, it still cannot be determined that the environment in the vehicle driving direction meets the passing condition.

[0125] The embodiments of the present disclosure can determine whether the vehicle is allowed to pass in the vehicle driving direction through visual information, give a determination result that the environment meets the passing condition when the road condition in front is suitable for passing when the driver fails to timely notice the change of the traffic condition in front, and further give a prompt to the driver in combination with the sound signal meeting the preset prompt condition, thereby improving the sensitivity of the identification of the change of the traffic condition and giving a corresponding prompt in time.

[0126] In addition, the embodiment of the present disclosure can make more accurate judgment on whether the vehicle is allowed to pass in the driving direction by combining the front traffic indication information, and give a judgment result meeting the passing condition when the traffic indication information allows the vehicle to pass, so as to avoid the driver from violating the traffic rules due to psychological tension when being urged, and ensure the safety of the driver.

[0127] In some embodiments, the method further comprises: acquiring light information of the rear vehicle; and generating a prompt message when the high beam and low beam switching frequency of the rear vehicle is greater than a preset light frequency threshold and the environment in the driving direction of the vehicle meets the passing condition according to the light information.

[0128] In the urban road, there may be a section where honking is prohibited, or the urging signal of the driver of the rear vehicle may be emitted by frequently switching the high beam and low beam due to different habits of the driver.

[0129] The light information of the rear vehicle is acquired by a camera or a light sensor installed at the tail of the vehicle, and when it is determined according to the light information that the high beam and low beam switching frequency of the rear vehicle is greater than a preset light frequency threshold, it represents that the rear vehicle is frequently switching the high beam and low beam, which is a potential urging signal.

[0130] Meanwhile, it is determined whether the environment in the driving direction of the vehicle meets the passing condition, for example, whether the unobstructed space in the driving direction of the vehicle is greater than a preset space size according to the environment information of the driving direction of the vehicle. When the high beam and low beam switching frequency of the rear vehicle is greater than a preset light frequency threshold and the environment in the driving direction of the vehicle meets the passing condition, it is considered that the rear vehicle emits an urging signal, and a prompt message is generated to prompt the driver to pay attention to the traffic condition in front in time.

[0131] The embodiment of the present disclosure determines whether the rear vehicle emits an urging signal by combining the light information of the rear vehicle and the road environment in front of the vehicle, and reminds the driver to pay attention to the traffic condition in front in time, which further improves the sensitivity of recognizing the change of the traffic condition and gives the corresponding prompt in time.

[0132] In some embodiments, the running state information of the vehicle includes vehicle fault information and turn signal working information; and the determination of whether the vehicle meets the normal driving condition according to the running state information of the vehicle and / or the state information of the driver of the vehicle comprises: determining whether the vehicle has a fault affecting driving according to the vehicle fault information; determining whether the vehicle has a lane changing intention according to the turn signal working information; determining whether the driver of the vehicle has driving ability according to the state information of the driver of the vehicle; and determining that the vehicle does not meet the normal driving condition if the vehicle has a fault affecting driving, or the vehicle has a lane changing intention, or the driver of the vehicle does not have driving ability.

[0133] In determining whether the vehicle meets the normal driving condition, in addition to the objective existence of the vehicle itself affecting the vehicle driving failure, it is judged whether there is a temporary vehicle unable to move caused by the subjective driving intention of the driver.

[0134] For example, when the driver of the vehicle intends to drive the vehicle to change lanes, it is necessary to observe the traffic state of the adjacent lane to find a suitable lane changing opportunity, which may cause the vehicle to drive slowly and cause the rear vehicle to urge. Based on the working information of the vehicle turn signal, it is judged whether the current vehicle has a lane changing intention; when the vehicle has a lane changing intention, it is judged that the vehicle does not meet the normal driving condition.

[0135] Optionally, according to the working information of the turn signal, it is determined whether the vehicle has a lane changing intention, comprising: when the vehicle turn signal is turned on, monitoring the working time of the vehicle turn signal; when the working time of the vehicle turn signal is less than the first preset time, it is determined that the vehicle has a lane changing intention. In order to avoid the vehicle changing lane for a long time without success, or the driver of the vehicle mistakenly opening the turn signal for a long time without closing, when the working time of the turn signal reaches the first preset time, it is determined that the vehicle does not have a lane changing intention, and at this time, when other conditions are met (such as the sound signal meets the preset prompt condition, the environment in the driving direction of the vehicle meets the passing condition, etc.), a prompt message is generated.

[0136] Optionally, according to the working information of the turn signal, it is determined whether the vehicle has a lane changing intention, comprising: when the vehicle turn signal is turned on, acquiring the traffic state information of the adjacent lane on the side where the turn signal is located, and judging whether there is a lane changing space in the adjacent lane; if there is no lane changing space in the adjacent lane, the first preset time is adjusted to the second preset time, and the second preset time is longer than the first preset time. When the current vehicle has a lane changing intention, but the adjacent lane does not have a lane changing condition, the first preset time is appropriately extended to enable the driver of the vehicle to have more opportunities to change lanes.

[0137] Correspondingly, if there is no lane changing space in the adjacent lane, when the working time of the vehicle turn signal is less than the second preset time, it is determined that the vehicle has a lane changing intention; when the working time of the turn signal reaches the second preset time, it is determined that the vehicle does not have a lane changing intention.

[0138] Optionally, the state information of the driver of the vehicle is quantitatively analyzed by fusing biological signals and behavior characteristics. The facial features of the driver are captured by a visual sensor, and an improved PERCLOS algorithm is used to calculate the proportion of eyelid closure time in a unit of time. When the value exceeds the preset proportion and the head posture continuously deviates from the driving axis by a preset angle, a fatigue driving warning is triggered, and at this time, if other conditions are met (such as the sound signal meets the preset prompt condition, the environment in the driving direction of the vehicle meets the passing condition, etc.), a prompt message is generated.

[0139] The biosensor monitors heart rate variability and steering wheel grip distribution in real time. If a sudden physiological disability risk is detected, the heart rate is too low or drops suddenly, and the grip standard deviation increases by more than a preset threshold, the sudden physiological disability risk is determined.

[0140] Optionally, to distinguish between temporary distraction and persistent lack of ability, the system introduces a time sequence behavior model to analyze the operation characteristics (such as steering wheel turning entropy, accelerator pedal fluctuation frequency) in a window within a certain time to confirm whether the driver of the vehicle is a persistent lack of driving ability.

[0141] When it is determined that the driver of the vehicle is in a non-persistent driving ability loss state such as distraction, prompting the driver of the vehicle can achieve a reminding effect. If other conditions are met (such as the sound signal meets the preset prompt condition, the environment in the direction of travel of the vehicle meets the traffic condition, etc.), a prompt message is generated.

[0142] When it is determined that the driver of the vehicle is in a persistent driving ability loss state, prompting the driver of the vehicle not only fails to achieve a prompting effect, but may further stimulate the driver of the vehicle. At this time, even if other conditions are met (such as the sound signal meets the preset prompt condition, the environment in the direction of travel of the vehicle meets the traffic condition, etc.), no prompt message is generated.

[0143] The embodiments of the present disclosure jointly determine whether the current vehicle and the driver of the vehicle themselves meet the normal driving condition by the running state information of the vehicle and the state information of the driver of the vehicle, and then only consider generating a prompt message when the vehicle and the driver of the vehicle themselves meet the normal driving condition, thereby avoiding generating a prompt message that fails to achieve a prompting effect in the case where the vehicle is affected by objective factors such as vehicle failure, there is a lane changing intention but has not successfully changed lanes, or the driver of the vehicle has a sudden condition, and avoiding the driver of the vehicle being more nervous due to the generation of the prompt message when the driver is affected by the above objective factors, thereby improving the humanization degree of the driving assistance prompting method.

[0144] Figure 3 The driving assistance prompting method flowchart provided by another embodiment of the present disclosure is shown in FIG. 2, and the method comprises the following steps. Figure 3

[0145] S301, determining whether the vehicle is in a slow driving section based on the running state information of the vehicle and / or the environment information.

[0146] ​The running state information includes a vehicle speed, and the environment information includes a rear vehicle speed. The determination of whether the vehicle is in the slow driving section based on the running state information and / or the environment information of the vehicle includes: when the running state information and / or the environment information of the vehicle satisfies one or more of the following conditions, determining whether the vehicle is in the slow driving section: the vehicle speed is lower than a preset speed threshold, and the duration exceeds a preset time threshold; or, in a first preset time period, the change amount of the vehicle speed is greater than a preset speed change amount threshold; or, in a second preset time period, the relative speed of the vehicle speed and the rear vehicle speed is less than a relative speed threshold; or, according to the environment information, it is determined that the number of vehicles in a preset road section length is greater than a preset number threshold.

[0147] S302, when the vehicle is in the slow driving section, acquiring a sound signal behind the vehicle, environment information in the driving direction of the vehicle, and state information of the driver of the vehicle.

[0148] S303, extracting sound features of the sound signal, the sound features including average volume, duration, and change frequency of sound frequency.

[0149] S304, determining whether the sound signal meets a preset prompt condition according to the sound features.

[0150] Specifically, the determination of whether the sound signal meets the preset prompt condition according to the sound features includes: determining the average volume and the duration of the sound signal; if the average volume is greater than a preset volume threshold and the duration is greater than a preset time threshold, it is determined that the sound signal meets the preset prompt condition.

[0151] Specifically, the determination of whether the sound signal meets the preset prompt condition according to the sound features includes: determining the change frequency of the sound frequency of the sound signal in a preset sound frequency interval according to the change information of the sound frequency; if the change frequency of the sound frequency is greater than a preset change frequency threshold, it is determined that the sound signal meets the preset prompt condition.

[0152] S305, determining the size of the unobstructed space in the driving direction of the vehicle according to the environment information.

[0153] S306, if the unobstructed space in the driving direction of the vehicle is greater than a preset space size, it is determined that the environment in the driving direction of the vehicle meets the passing condition.

[0154] Specifically, if the traffic indication information is included in the environment information, it is determined whether the vehicle is allowed to pass according to the traffic indication information; if it is determined that the vehicle is allowed to pass, and there is an unobstructed space greater than a preset space size in the driving direction of the vehicle, it is determined that the environment in the driving direction of the vehicle meets the passing condition.

[0155] In S307, it is determined whether the vehicle meets the normal driving condition according to the running state information of the vehicle and / or the state information of the driver of the vehicle.

[0156] Specifically, the running state information of the vehicle includes vehicle fault information and turn signal working information.

[0157] According to the vehicle fault information, it is determined whether the vehicle has a fault affecting driving; according to the turn signal working information, it is determined whether the vehicle has a lane changing intention; according to the state information of the driver of the vehicle, it is determined whether the driver of the vehicle has driving ability; if the vehicle has a fault affecting driving, or the vehicle has a lane changing intention, or the driver of the vehicle does not have driving ability, it is determined that the vehicle does not meet the normal driving condition.

[0158] In S308, if the sound signal meets the preset prompt condition, the environment in the driving direction of the vehicle meets the passing condition, and the vehicle meets the normal driving condition, a prompt message is generated.

[0159] The embodiments of the present disclosure comprehensively evaluate the external environment in which the vehicle is currently located through the sound signal, which is auditory information, and the environment in the driving direction of the vehicle, which is visual information, and determine whether the vehicle itself has driving conditions based on vehicle fault information and the state information of the driver of the vehicle. When the driver behind honks and the external environment and the internal environment of the vehicle both meet the passing conditions, the driver is timely reminded accordingly, the sensitivity to traffic state changes in complex scenarios is improved, the driver is timely reminded accordingly, and further psychological pressure on the driver caused by false reminders when the vehicle cannot temporarily move due to vehicle faults and the like is avoided, thereby improving the flexibility of the driving assistance prompt method.

[0160] In addition, in a long waiting scenario such as traffic jam, the embodiments of the present disclosure can effectively reduce the risk of traffic accidents caused by driver negligence, and also relieve the psychological pressure caused by long stagnation.

[0161] In some embodiments, the method determines whether the current external environment is suitable for driving by combining visual perception (front-facing camera) and auditory perception (analysis of rear vehicle horn sound), while determining whether the vehicle itself and the driver are in a condition to drive based on sensors such as in-vehicle cameras, vehicle control units, biological signal acquisition devices, etc., and reminding the driver to pay attention to driving at the appropriate time using physical feedback methods (steering wheel vibration / seat belt tightening).

[0162] For the visual perception part, the front camera continuously monitors the road conditions in front, and uses computer vision algorithms to identify the relative distance between the front and rear vehicles. If it is found that there is space for at least two vehicles in front, it is considered that there is enough space for the vehicle to drive. That is, if there is space for at least two vehicles in front of the vehicle, it is considered that the unobstructed space in the driving direction of the vehicle is greater than the preset space size, and the environment in the driving direction of the vehicle meets the passing condition.

[0163] For the auditory perception part, the microphone array on the vehicle collects ambient sound, especially focusing on sound signals from the rear. The collected data is preprocessed, including noise reduction, gain control, etc., to ensure the accuracy of subsequent analysis. Based on the sound features (average volume, duration, frequency variation frequency) of the sound signal, the sound signal is classified to determine whether it is a "prompt" horn sound.

[0164] Specifically, the sound signal of the surrounding environment is collected by the microphone array. Then, the average volume of the sound is calculated using RMS. A preset volume threshold (such as 60 decibels) is defined, and if the detected sound volume exceeds the preset volume threshold and lasts for a period of time (exceeds the preset time threshold), then the sound signal is considered to be a possible horn sound that meets the preset prompt condition. It can be understood that the preset volume threshold and the preset time threshold can be adjusted according to actual conditions, such as setting the preset volume threshold higher in the city to distinguish from the ambient volume to avoid misjudgment.

[0165] The frequency variation frequency of the sound frequency determines whether the sound signal has ups and downs. The sound signal is subjected to frequency spectrum analysis, and the main frequency components are extracted. Usually, the sound of a car horn will have a significant frequency change, such as from low frequency to high frequency or vice versa. Assuming that the preset sound frequency range is defined as 500 Hz to 3000 Hz, if the detected sound signal has a significant frequency change within this range, and the frequency variation frequency is greater than the preset variation frequency threshold, then it is considered to be a possible horn sound that meets the preset prompt condition. Specifically, the preset sound frequency range and the preset variation frequency threshold can be fine-tuned according to actual test results.

[0166] For the duration, the length of time that the average volume of the sound signal exceeds the preset volume threshold is recorded. For example, if a sound exceeds 60 decibels and lasts at least 0.5 seconds, it is marked as a potential horn sound, which may meet the preset prompt condition. Further, the preset time interval is defined as between 0.5 seconds and 2 seconds. If the detected sound signal continues to sound within the preset time interval, it is considered to be a horn sound that meets the preset prompt condition. Too short or too long sounds do not meet the characteristics of the preset prompt condition.

[0167] Based on the above sound signal feature judgment logic, a machine learning method (such as support vector machine SVM, neural network, etc.) is used to train a large number of known horn sound samples, so as to learn to identify specific patterns of horn sounds that meet the preset prompt condition. These patterns include the change rule of frequency, the change rule of volume, etc. In the training process, a classification threshold is automatically determined to distinguish horn sounds that meet the preset prompt condition from other sounds. This threshold is usually optimized by cross-validation and other methods, and has no fixed value. In actual application, whether the current sound is a horn sound that meets the preset prompt condition is determined according to the classification threshold.

[0168] For the judgment of whether the vehicle itself and the driver of the vehicle have the driving condition, the vehicle continuously collects real-time data streams of the vehicle electronic control unit to obtain key parameters of the vehicle power system, braking system and steering system for threshold analysis. If the current vehicle has a fault that affects driving, causing the vehicle to temporarily be unable to move, it is determined that the vehicle itself does not have the driving condition.

[0169] By capturing the facial features of the driver in the cabin of the vehicle through visual sensors, and analyzing the driving behavior pattern, it is determined whether the current driver of the vehicle belongs to temporary distraction or belongs to persistent lack of driving ability caused by sudden illness, etc. When the current driver of the vehicle belongs to persistent lack of driving ability, it is determined that the driver of the vehicle does not have the driving ability to meet the normal driving condition.

[0170] Further, the above multi-dimensional information such as hearing, vision, biological signal, etc. is combined to comprehensively evaluate whether it is suitable to start the vehicle at present. If the conditions of "there is enough space in front, there is strong urging behind, the vehicle itself and the driver meet the normal driving condition" are met at the same time, the warning mechanism is triggered to generate prompt information to notify the driver.

[0171] Optionally, real-time position information is directly obtained from other vehicles to determine whether there is enough space in front of the vehicle.

[0172] Among them, the prompt mode of the prompt information is determined according to the preference setting of the driver, including the modes of slight vibration of the steering wheel, automatic tightening of the safety belt, etc.

[0173] The embodiments of the present disclosure improve the sensitivity of the system to traffic state changes in complex scenarios by comprehensively utilizing different perception means such as vision and hearing, combining the vehicle external environment, the vehicle itself and the driver's condition of the vehicle to comprehensively evaluate whether the driver needs to be reminded, and providing various optional notification forms, so that users with different habits can find the most suitable alarm mode, effectively reducing the risk of traffic accidents caused by driver negligence, and also relieving the psychological pressure caused by long stagnation.

[0174] Figure 4 A structural schematic diagram of a driving assistance prompting device provided by the embodiments of the present disclosure is provided. The driving assistance prompting device can be a vehicle-mounted device as described in the above embodiments, or the driving assistance prompting device can be a component or assembly in the vehicle-mounted device. The driving assistance prompting device provided by the embodiments of the present disclosure can perform the processing flow provided by the driving assistance prompting method embodiments, as shown in Figure 4 The driving assistance prompting device 40 includes an acquisition module 41, a first determination module 42, a second determination module 43, and a generation module 44. The acquisition module 41 is configured to acquire a sound signal behind the vehicle, environment information in the driving direction of the vehicle, and state information of the driver of the vehicle when the vehicle is in a slow-moving road section. The first determination module 42 is configured to determine whether the sound signal meets a preset prompting condition and whether the environment in the driving direction of the vehicle meets a passing condition. The second determination module 43 is configured to determine whether the vehicle meets a normal driving condition according to the running state information of the vehicle and / or the state information of the driver of the vehicle. The generation module 44 is configured to generate a prompt message if the sound signal meets the preset prompting condition, the environment in the driving direction of the vehicle meets the passing condition, and the vehicle meets the normal driving condition.

[0175] Optionally, the first determination module 42 includes a feature extraction unit 421 and a first determination unit 422. The feature extraction unit 421 is configured to extract a sound feature of the sound signal, and the sound feature includes an average volume, a duration, and a change frequency of a sound frequency. The first determination unit 422 is configured to determine whether the sound signal meets the preset prompting condition according to the sound feature.

[0176] Optionally, the first determination unit 422 is specifically configured to determine a change frequency of the sound frequency of the sound signal in a preset sound frequency interval according to the change information of the sound frequency. If the change frequency of the sound frequency is greater than a preset change frequency threshold, it is determined that the sound signal meets the preset prompting condition.

[0177] Optionally, the first determining module 42 comprises a second determining unit 423 and a third determining unit 424; the second determining unit 423 is configured to determine the size of the unobstructed space in the vehicle driving direction according to the environment information; and the third determining unit 424 is configured to determine that the environment in the vehicle driving direction meets the passing condition if the unobstructed space in the vehicle driving direction is greater than a preset space size.

[0178] Optionally, the second determining unit 423 is further configured to determine whether the vehicle is allowed to pass according to the traffic indication information if the traffic indication information is included in the environment information, the traffic indication information comprising at least one of a traffic signal, a traffic sign, and a lane guide arrow mark; and the third determining unit 424 is further configured to determine that the environment in the vehicle driving direction meets the passing condition if it is determined that the vehicle is allowed to pass and there is an unobstructed space greater than the preset space size in the vehicle driving direction.

[0179] Optionally, the operation state information of the vehicle comprises vehicle fault information and steering light working information; the second determining module 43 is configured to determine whether the vehicle has a fault affecting driving according to the vehicle fault information, determine whether the vehicle has a lane changing intention according to the steering light working information, and determine whether the driver of the vehicle has driving ability according to the state information of the driver of the vehicle; and it is determined that the vehicle does not meet the normal driving condition if the vehicle has a fault affecting driving, or the vehicle has a lane changing intention, or the driver of the vehicle does not have driving ability.

[0180] Optionally, the driving assistance prompting device 40 further comprises a third determining module 45 configured to determine whether the vehicle is in a slow driving section when the operation state information and / or the environment information of the vehicle meets one or more of the following conditions: the vehicle driving speed is lower than a preset speed threshold and the duration exceeds a preset time threshold; or, the change amount of the vehicle driving speed in a first preset time period is greater than a preset speed change amount threshold; or, the relative speed of the vehicle driving speed and the speed of a rear vehicle in a second preset time period is less than a relative speed threshold; or, according to the environment information, it is determined that the number of vehicles in a preset road section length is greater than a preset number threshold.

[0181] Figure 4 The driving assistance prompting device of the illustrated embodiment can be used to execute the technical solutions of the above-mentioned method embodiments, and has similar implementation principles and technical effects, which will not be described here in detail.

[0182] Figure 5 A structural schematic diagram of a vehicle is provided for the embodiments of the present disclosure. For example, Figure 5As shown, the vehicle includes a memory 501 and a processor 502, wherein the memory 501 stores executable program code 5011, and the processor 502 is configured to invoke and execute the executable program code 5011 to execute the driving assistance prompting method.

[0183] The embodiment can divide the functional modules of the vehicle according to the method examples described above, for example, each functional module can be divided, or two or more functions can be integrated into one processing module, and the integrated module can be realized in the form of hardware. It should be noted that the division of the modules in the embodiment is illustrative, and is only a logical function division, and another division mode can be used in actual implementation.

[0184] In the case of dividing each functional module corresponding to each function, the vehicle can include a first determination module, an acquisition module, a second determination module, a third determination module, a fourth determination module, a generation module, and the like.

[0185] The vehicle provided by the embodiment is used to execute the driving assistance prompting method described above, and therefore the same effect as the implementation method described above can be achieved.

[0186] In the case of using an integrated unit, the vehicle can include a processing module and a storage module. The processing module can be used to control and manage the actions of the vehicle. The storage module can be used to support the vehicle to execute program codes and data, and the like.

[0187] The processing module can be a processor or a controller, which can realize or execute various exemplary logical blocks, modules and circuits described in combination with the disclosure. The processor can also be a combination of computing functions, such as one or more microprocessor combinations, digital signal processing (DSP) and microprocessor combinations, and the like. The storage module can be a memory.

[0188] Figure 6 A structural schematic diagram of an electronic device provided by an embodiment of the disclosure is shown. The electronic device can be a vehicle-mounted device as described in the above embodiment. The electronic device provided by the embodiment of the disclosure can execute the processing flow provided by the driving assistance prompting method embodiment, as shown in Figure 6 As shown, the electronic device 60 includes a memory 61, a processor 62, a computer program and a communication interface 63; wherein the computer program is stored in the memory 61 and is configured to be executed by the processor 62 to execute the driving assistance prompting method as described above.

[0189] The memory 61 can be used to store software programs, vehicle executable instructions and modules, such as program instructions / modules corresponding to the driving assistance prompting method in the embodiments of the present disclosure. The processor 62 executes various functions of the server and data processing by running the software programs, instructions and modules stored in the memory 61, that is, implements the driving assistance prompting method of the above method embodiments.

[0190] The memory 61 can include a program storage area and a data storage area, wherein the program storage area can store an operating system and application programs required by at least one function; and the data storage area can store data created according to the use of the vehicle and the like. In addition, the memory 61 can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory 61 can optionally include a memory disposed remotely with respect to the processor 62, and these remote memories can be connected to the terminal device through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0191] The embodiments also provide a computer readable storage medium (including but not limited to a disk storage, a CD-ROM, an optical storage, etc.) having computer program codes stored therein, when the computer program codes are run on a computer, the computer is caused to execute the above related method steps to implement the driving assistance prompting method provided by the above embodiments.

[0192] In the present disclosure, the computer readable storage medium can be any tangible medium containing or storing a program, which can be used by or in conjunction with an instruction execution system, device or apparatus.

[0193] The embodiments also provide a computer program product, when the computer program product is run on a computer, the computer is caused to execute the above related steps to implement the driving assistance prompting method provided by the above embodiments.

[0194] The beneficial effects of the above embodiments can refer to the beneficial effects in the corresponding methods provided above, which will not be repeated here.

[0195] Through the above description of the embodiments, those skilled in the art can understand that, for the convenience and brevity of description, only the above division of functional modules is taken as an example, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions.

[0196] In the embodiments of the present application, it should be understood that the disclosed apparatus and method can be implemented in other manners. For example, the division of the apparatus embodiments is merely logical function division, and there can be other division manners in actual implementation. For example, a plurality of units or components can be combined or integrated into another apparatus, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different apparatuses can be indirect couplings or communication connections through some interfaces, apparatuses or units, and can be in electrical, mechanical or other forms.

[0197] In the description of the present disclosure, it should be understood that the orientation or positional relationship indicated by the terms such as "upper", "lower", "front", "back", "left" and "right" is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the indicated position or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the present disclosure.

[0198] The above description is merely preferred embodiments of the present disclosure and a description of the principles of the technology applied. It should be understood by those skilled in the art that the disclosed scope of the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and also covers other technical solutions formed by the combinations of the above technical features or equivalent features without departing from the above disclosed concept. For example, the technical solutions formed by the mutual replacement of the above features and the technical features disclosed in the present disclosure (but not limited to) having similar functions.

[0199] In addition, although each operation is described in a particular order, this should not be understood as requiring the operations to be performed in the particular order shown or in a sequential order. In certain circumstances, multitasking and parallel processing can be advantageous. Similarly, although several implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments can also be combined in a single embodiment. Conversely, various features described in the context of a single embodiment can also be separated and implemented in multiple embodiments.

[0200] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely illustrative of the example forms of implementing the claims.

[0201] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of present disclosure. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the block can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations of blocks in the block diagrams and / or flow diagrams, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and

[0202] The units described in the embodiments of the present disclosure can be implemented by software, or by hardware. In some cases, the name of the unit does not constitute a limitation on the unit itself.

[0203] It should be noted that the relationship terms, such as first and second, etc., are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. It should also be noted that the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or equipment including the element.

[0204] The above merely provides embodiments of the present disclosure, but does not serve to limit the present disclosure. For those skilled in the art, the present disclosure can have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. within the spirit and principles of the present disclosure shall be included in the scope of the claims of the present disclosure.

Claims

1. A driving assistance prompting method characterized by comprising: The method comprises: determining that the vehicle is in a slow-moving section when the running state information of the vehicle and / or the environmental information satisfies one or more of the following: the change amount of the vehicle travel speed is greater than a preset speed change amount threshold within a first preset time period; the relative speed of the vehicle travel speed and the rear vehicle travel speed is less than a relative speed threshold within a second preset time period; according to the environmental information, the number of vehicles within a preset road section length is greater than a preset number threshold; when the vehicle is in the slow-moving section, obtaining the sound signal behind the vehicle, the environmental information in the travel direction of the vehicle, and the state information of the driver of the vehicle; determining whether the sound signal meets a preset prompt condition and whether the environment in the travel direction of the vehicle meets a passing condition; determining whether the vehicle meets a normal travel condition according to the running state information of the vehicle and / or the state information of the driver of the vehicle; if the sound signal meets the preset prompt condition, the environment in the travel direction of the vehicle meets the passing condition, and the vehicle meets the normal travel condition, a prompt message is generated.

2. The method of claim 1, wherein, The determination of whether the sound signal meets the preset prompt condition comprises: extracting sound features of the sound signal, the sound features comprising average volume, duration, and change frequency of sound frequency; determining whether the sound signal meets the preset prompt condition according to the sound features.

3. The method of claim 2, wherein, The determination of whether the sound signal meets the preset prompt condition comprises: determining the change frequency of the sound frequency of the sound signal within a preset sound frequency interval according to the change information of the sound frequency; if the change frequency of the sound frequency is greater than a preset change frequency threshold, it is determined that the sound signal meets the preset prompt condition.

4. The method of claim 1, wherein, The determination of whether the environment in the travel direction of the vehicle meets the passing condition comprises: determining the size of the unobstructed space in the travel direction of the vehicle according to the environmental information; if the unobstructed space in the travel direction of the vehicle is greater than a preset space size, it is determined that the environment in the travel direction of the vehicle meets the passing condition.

5. The method of claim 1, wherein, The determination of whether the environment in the travel direction of the vehicle meets the passing condition comprises: if the environmental information includes traffic indication information, it is determined whether the vehicle is allowed to pass according to the traffic indication information, the traffic indication information comprising at least one of traffic signal lights, traffic signs, and lane guiding arrow marks; if it is determined that the vehicle is allowed to pass and there is an unobstructed space greater than a preset space size in the travel direction of the vehicle, it is determined that the environment in the travel direction of the vehicle meets the passing condition.

6. The method of claim 1, wherein, The running state information of the vehicle comprises vehicle fault information and turn signal working information; the determination of whether the vehicle meets the normal travel condition according to the running state information of the vehicle and / or the state information of the driver of the vehicle comprises: determining whether the vehicle has a fault affecting travel according to the vehicle fault information; determining whether the vehicle has a lane changing intention according to the turn signal working information; determining whether the driver of the vehicle has driving ability according to the state information of the driver of the vehicle; If the vehicle has a fault affecting driving, or the vehicle has a lane-changing intention, or the driver of the vehicle does not have driving ability, it is determined that the vehicle does not meet the normal driving condition.

7. The method of claim 1, wherein, When the running state information and / or the environment information of the vehicle meet one or more of the following conditions, it is determined that the vehicle is in the slow driving section, further comprising: The vehicle driving speed is lower than a preset speed threshold, and the duration exceeds a preset time threshold.

8. A driving assistance prompting device characterized by comprising: Comprising: A third determination module is configured to determine that the vehicle is in the slow driving section when the running state information and / or the environment information of the vehicle meet one or more of the following conditions: the change amount of the vehicle driving speed is greater than a preset speed change amount threshold within a first preset time period; the relative speed between the vehicle driving speed and the rear vehicle driving speed is less than a relative speed threshold within a second preset time period; and according to the environment information, it is determined that the number of vehicles within a preset road section length is greater than a preset number threshold; An acquisition module is configured to acquire a sound signal behind the vehicle, environment information in the driving direction of the vehicle, and state information of the driver of the vehicle when the vehicle is in the slow driving section; A first determination module is configured to determine whether the sound signal meets a preset prompt condition, and whether the environment in the driving direction of the vehicle meets a passing condition; A second determination module is configured to determine whether the vehicle meets a normal driving condition according to the running state information of the vehicle and / or the state information of the driver of the vehicle; A generation module is configured to generate a prompt message if the sound signal meets the preset prompt condition, the environment in the driving direction of the vehicle meets the passing condition, and the vehicle meets the normal driving condition.

9. A vehicle characterized by comprising: Comprising: A memory; A processor; Wherein the memory stores executable program codes, and the processor is configured to call and execute the executable program codes to execute the method according to any one of claims 1-7.

10. An electronic device, comprising: Comprising: A memory; A processor; And A computer program; Wherein the computer program is stored in the memory and is configured to be executed by the processor to implement the method according to any one of claims 1-7.

Images