Environment sensing system of vehicle, control method of environment sensing system and vehicle

By integrating audio sensors and environmental perception devices on autonomous driving vehicles and combining multiple perception data, the problem of insufficient vehicle perception capabilities is solved, achieving more comprehensive environmental perception and higher safety.

CN119058727APending Publication Date: 2024-12-03ZHEJIANG GEELY HLDG GRP CO LTD +2
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411229591.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-03
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing autonomous driving vehicles have limited environmental perception capabilities, limited camera shooting angles, small perception range of lidar and ultrasonic radars and are susceptible to noise interference, resulting in incomplete perception of the surrounding environment by the vehicle and affecting safe driving.

Method used

Audio sensors and environmental perception equipment are set up on the vehicle, combined with cameras, lidars and ultrasonic radars, collect audible frequency band sound signals and perception information, and determine the surrounding environment status through controller analysis to achieve multi-angle perception and data fusion.

Benefits of technology

It improves the accuracy and perceived range of the vehicle's environmental perception, and improves the accuracy and safety of vehicle control.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119058727A_ABST
    Figure CN119058727A_ABST
Patent Text Reader

Abstract

The invention discloses an environment sensing system of a vehicle, a control method of the environment sensing system and the vehicle. The system comprises a controller, an audio sensor and environment sensing equipment, wherein the audio sensor and the environment sensing equipment are arranged on the vehicle and are in communication connection with the controller; the audio sensor is used for collecting sound signals of audible frequency bands around the vehicle, and the environment sensing device is used for collecting sensing information of the environment around the vehicle. The controller is used for determining the surrounding environment state of the vehicle according to the received sound signals and the perception information and executing corresponding whole vehicle control according to the surrounding environment state. Therefore, the audio sensor is additionally arranged on the basis of the environment sensing equipment, and the sound signal of the audible frequency band collected by the audio sensor and the sensing information collected by the environment sensing equipment are combined to determine the surrounding environment state of the vehicle, namely, the surrounding environment is sensed from multiple angles, and the surrounding environment state is determined through joint analysis of multiple types of data. And the environmental perception accuracy is improved, so that the vehicle control accuracy and safety are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to an environmental perception system for a vehicle, a control method thereof, and a vehicle. Background Art

[0002] With the continuous progress and development of autonomous driving technology, the safety factor of vehicle driving has gradually improved. In the development of autonomous driving technology, the vehicle's comprehensive perception of the surrounding environment and accurate decision-making are crucial to safe driving.

[0003] At present, the comprehensive perception of the surrounding environment of vehicles is mainly achieved by devices such as cameras, laser radars and ultrasonic radars. However, cameras may have problems such as limited shooting angles and blocked lenses, while laser radars and ultrasonic radars have small perception ranges and are easily disturbed by external noise and produce false alarms, resulting in limited perception capabilities and perception scenarios of the vehicle's surrounding environment.

[0004] How to improve vehicle perception accuracy and expand perception range is one of the most important research directions for technical personnel in this field. Summary of the invention

[0005] In view of this, one aspect of the present application provides an environment perception system for a vehicle, the system comprising: a controller, an audio sensor, and an environment perception device;

[0006] The audio sensor and the environment sensing device are both arranged on the vehicle and are both communicatively connected with the controller;

[0007] The audio sensor is used to collect sound signals in the audible frequency band around the vehicle;

[0008] The environmental perception device is used to collect perception information of the vehicle's surrounding environment; wherein the perception information includes at least one of a video image, point cloud data, and an ultrasonic signal;

[0009] The controller is used to determine the surrounding environment state of the vehicle according to the received sound signal and the perception information, and perform corresponding vehicle control according to the surrounding environment state.

[0010] Optionally, the controller is further used for:

[0011] Extracting features from the sound signal to obtain sound feature parameters;

[0012] Determining the specific sound type to which the sound signal belongs according to the sound characteristic parameter;

[0013] The surrounding environment state of the vehicle is determined based on the specific sound type, the sound characteristic parameters and the perception signal.

[0014] Optionally, the audio sensor includes a plurality of first audio sensors;

[0015] The plurality of first audio sensors are respectively arranged at the middle of the axle of the vehicle;

[0016] The controller is used to analyze the sound characteristic parameters corresponding to the first sound type and the perception signal when determining that the specific sound type is the first sound type, so as to determine the road surface information on which the vehicle is currently traveling; wherein the first sound type includes at least one of tire noise and ground feedback noise.

[0017] Optionally, the controller is further used for:

[0018] When the road surface information meets the preset conditions, current driving information is obtained; wherein the current driving information includes current weather information, current vehicle speed and tire pressure; the preset conditions include that the road surface bumpiness is greater than a bumpiness threshold, or the road surface friction coefficient is less than a coefficient threshold;

[0019] The vehicle is controlled to decelerate according to the current driving information.

[0020] Optionally, the audio sensor includes a plurality of second audio sensors;

[0021] The plurality of second audio sensors are respectively arranged on the outer wall of the vehicle on the main driver's side, the outer wall of the vehicle on the co-driver's side and above the windshield;

[0022] The controller is used to analyze the sound characteristic parameters corresponding to the second sound type and the perception signal when determining that the specific sound type is the second sound type, so as to determine whether there is a designated vehicle and / or law enforcement personnel outputting the second sound type within the first designated range of the vehicle; wherein the second sound type includes at least one of a fire truck siren, a police car siren and an ambulance siren.

[0023] Optionally, the controller is further used for:

[0024] When the designated vehicle exists within the first designated range, controlling the vehicle to slow down or pull over to avoid the designated vehicle;

[0025] When the law enforcement officer exists within the first designated range, the command signal of the law enforcement officer is obtained, and the vehicle control corresponding to the command signal is executed; wherein the command signal includes at least one of a voice signal and an action signal.

[0026] Optionally, the audio sensor includes a third audio sensor;

[0027] The third audio sensor is disposed at the front bumper of the vehicle;

[0028] The controller is used to analyze the sound characteristic parameters corresponding to the third sound type and the perception signal when determining that the specific sound type is the third sound type, so as to determine whether the vehicle has a dangerous collision and / or whether other traffic participants have a dangerous collision within a second specified range of the vehicle; wherein the third sound type includes the collision sound of the vehicle colliding with an obstacle.

[0029] Optionally, the controller is further used for:

[0030] When it is determined that the vehicle is in a dangerous collision and / or other traffic participants are in a dangerous collision within the second specified range of the vehicle, the vehicle is controlled to slow down, take a detour, or pull over;

[0031] The collision event information corresponding to the dangerous collision is reported to a remote service platform.

[0032] Optionally, the environment perception device includes: a camera, a laser radar and an ultrasonic radar;

[0033] The camera is arranged in the vehicle and is used to collect the video image;

[0034] The laser radar is arranged at the front bumper of the vehicle and is used to collect the point cloud data;

[0035] The ultrasonic radar is arranged at the rear bumper of the vehicle and is used to collect the ultrasonic signal.

[0036] Another aspect of the present application provides a vehicle control method, the method comprising:

[0037] Receive the sound signal in the audible frequency band around the vehicle collected by the audio sensor and the perception information of the vehicle's surrounding environment collected by the environment perception device; wherein the perception information includes at least one of a video image, point cloud data and an ultrasonic signal; the audio sensor and the environment perception device are both arranged on the vehicle;

[0038] Determining a state of the surrounding environment of the vehicle according to the sound signal and the perception information;

[0039] According to the surrounding environment state, corresponding vehicle control is performed.

[0040] Another aspect of the present application provides a vehicle, including a driving system and the environment perception system;

[0041] The environmental perception system is connected to the driving system;

[0042] The environmental perception system is used to control the operation of the drive system according to the surrounding environment state of the vehicle to perform corresponding vehicle control.

[0043] The present application provides a vehicle environment perception system and its control method and vehicle, which produce the following beneficial effects: on the basis of the environment perception device, an audio sensor that can collect sound signals in the audible frequency band is added to the vehicle, and the sound signals in the audible frequency band collected by the audio sensor are combined with the perception information collected by the environment perception device to determine the surrounding environment state of the vehicle, that is, the surrounding environment is perceived from multiple angles, and multiple types of data are jointly analyzed to determine the surrounding environment state, which improves the perception accuracy while expanding the perception range, thereby improving the vehicle control accuracy and safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A schematic diagram of a structural diagram of an environment perception system for a vehicle provided in an embodiment of the present application;

[0045] Figure 2 A schematic diagram of a signal processing flow of a controller provided in an embodiment of the present application;

[0046] Figure 3 A schematic diagram of the structure of an environment perception system for a vehicle provided by another embodiment of the present application;

[0047] Figure 4 A schematic flow chart of a vehicle control method provided in an embodiment of the present application;

[0048] Figure 5 A schematic diagram of the structure of a vehicle provided in an embodiment of the present application.

[0049] The figure numbers are as follows: 1 is a controller, 2 is an audio sensor, 3 is an environmental perception device, 4 is a remote platform, 5 is a windshield, 6 is a front bumper, 7 is a smart cockpit, 8 is wire-controlled steering, 9 is wire-controlled rotation, 10 is power and gear shifting, 11 is electronic parking, 12 is body electronics, 13 is a vehicle, 20 is a first audio sensor, 21 is a second audio sensor, 22 is a third audio sensor, 30 is a camera, 31 is a lidar, 32 is an ultrasonic radar, 130 is an environmental perception system, and 131 is a drive system. DETAILED DESCRIPTION

[0050] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0051] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0052] Figure 1 A schematic diagram of a vehicle environment perception system provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the system includes: a controller 1, an audio sensor 2 and an environment sensing device 3. The audio sensor 2 and the environment sensing device 3 are both arranged on the vehicle and are both connected to the controller 1 for communication.

[0053] In a specific embodiment, the audio sensor 2 is used to collect sound signals in the audible frequency band around the vehicle, and the environmental perception device 3 is used to collect perception information of the vehicle's surrounding environment, wherein the perception information includes at least one of video images, point cloud data, and ultrasonic signals.

[0054] It is worth noting that the sound signal collected by the audio sensor 2 is different from the ultrasonic signal in the perception signal. The audio sensor 2 collects signals in the audible frequency band, while the ultrasonic signal is a signal in the non-audible frequency band. That is, the embodiment of the present application adds a new sensor on the basis of the environment perception device 3 to perceive the vehicle's surrounding environment information.

[0055] In an optional embodiment, the sound signals collected by the audio sensor 2 may include the police car sirens, ambulance sirens, and fire truck sirens with strong penetrating power, and may also include collision sounds generated when colliding with obstacles, and may also include tire noise emitted by the tires during vehicle driving and road noise fed back from the ground, etc. In fact, the audio sensor 2 can collect sound signals of any audible frequency band during vehicle driving, and this application does not limit the sound signals.

[0056] In order to collect more and more comprehensive sound signals, thereby improving the accuracy of the vehicle's environmental perception, in an optional embodiment, the audio sensor 2 may include multiple ones, which are arranged at different positions on the vehicle, and this application does not limit this.

[0057] In a specific embodiment, Figure 1 As shown, after the audio sensor 2 collects the sound signal, it transmits the collected sound signal to the controller 1. In an optional embodiment, the audio sensor 2 can transmit the collected sound signal to the controller 1 via an A2B (Audio Video Bridging over Ethernet) audio bus. In fact, the present application does not limit the transmission channel for transmitting the sound signal.

[0058] During the driving process of the vehicle, in addition to the audio sensor 2 collecting sound signals in real time, the environmental sensing device 3 also collects and senses the sensing signals of the vehicle's surrounding environment in real time, and similarly transmits the collected sensing signals to the control.

[0059] It can be understood that the perception information includes at least one of a video image, a point cloud data and an ultrasonic signal. Figure 1 As shown, the environment perception device 3 includes at least one of a camera 30 , a laser radar 31 and an ultrasonic radar 32 .

[0060] It should be noted that the controller 1 may include but is not limited to a domain controller, a vehicle controller, etc., which is not limited in this application. In addition, it should be noted that the vehicles to which the environmental perception device 3 provided in the embodiment of the present application can be applied include but are not limited to sedans, sport utility vehicles (SUV), multi-purpose vehicles (MPV), various commercial vehicles, off-road vehicles, pickup trucks or other power-driven non-track-borne vehicles.

[0061] In an optional embodiment, after the controller 1 obtains the sound signal collected by the audio sensor 2 and the perception information collected by the environmental perception device 3, it determines the surrounding environment state of the vehicle based on the sound signal and the perception information, and performs corresponding vehicle operations based on the surrounding environment state.

[0062] For example, if the system senses that an ambulance is approaching with its siren blaring, it will control the vehicle to slow down and give way, or pull over to the side of the road to give way.

[0063] It is understandable that the audio sensor 2 can collect sound signals in any audible frequency band. Therefore, in another optional embodiment, the audio sensor 2 can also collect abnormal sounds emitted by the vehicle due to mechanical failure and transmit the collected abnormal sounds to the controller 1.

[0064] Furthermore, the controller 1 locates the fault of the vehicle according to the abnormal sound and the sensing information, such as Figure 1 As shown, the fault information is reported to the remote platform 4 so that the vehicle can be troubleshooted and maintained in time, thereby improving vehicle safety.

[0065] In a specific embodiment, when controlling the vehicle, Figure 1 As shown, the controller 1 sends control signals to the corresponding wire-controlled steering 8, wire-controlled rotation 9, power and gear shifting 10, electronic parking 11 and body electronic appliances 12 through the CAN bus to achieve corresponding vehicle control.

[0066] Therefore, the vehicle environment perception system provided in the embodiment of the present application, on the basis of the environment perception device 3, is additionally provided with an audio sensor 2 capable of collecting sound signals in the audible frequency band on the vehicle, and the sound signals in the audible frequency band collected by the audio sensor 2 are combined with the perception information collected by the environment perception device 3 to determine the surrounding environment state of the vehicle, that is, the surrounding environment is perceived from multiple angles, and multiple types of data are jointly analyzed to determine the surrounding environment state, thereby improving the perception accuracy and expanding the perception range, thereby improving the vehicle control accuracy and safety.

[0067] Figure 2 A schematic diagram of a signal processing flow of a controller provided in an embodiment of the present application, in an optional embodiment, as Figure 2 As shown, the controller 1 is used to perform the following steps:

[0068] S10: Extract features of the sound signal to obtain sound feature parameters;

[0069] In a specific embodiment, after the controller 1 obtains the sound signal in the audible frequency band collected by the audio sensor 2, it first pre-processes the sound signal so that the sound signal is suitable for subsequent feature extraction and other processing. Specifically, the pre-processing of the sound signal includes framing, window function processing and denoising.

[0070] Further, after the sound signal is preprocessed, feature extraction is performed on the preprocessed sound signal in step S10 to convert the complex sound signal into sound feature parameters, wherein the sound feature parameters include but are not limited to Mel Frequency Cepstral Coefficients (MFCC) and Linear Prediction Cepstral Coefficients (LPCC).

[0071] S11: Determine the specific sound type to which the sound signal belongs according to the sound characteristic parameter;

[0072] After obtaining the sound feature parameters, the specific sound type to which the sound signal belongs is determined through step S11. In an optional embodiment, the extracted sound feature parameters are matched with a pre-trained sound network sound model to determine whether the sound feature parameters are specific sounds, thereby determining the specific sound type to which the corresponding sound signal belongs, and further determining the time type and scene corresponding to the specific sound.

[0073] In another optional embodiment, a specific sound database is pre-stored, which stores sound feature parameters corresponding to various specific sounds. During vehicle driving, the controller 1 processes the currently acquired sound signal to obtain the current sound feature parameters, and then matches and compares them with the sound feature parameters in the specific sound database to determine the specific sound and event scene corresponding to the current sound signal.

[0074] Among them, specific sound types may include but are not limited to police car sirens, ambulance sirens, fire truck sirens, abnormal sounds of vehicle mechanical failures, sounds of vehicle collisions, and tire noises.

[0075] S12: Determine the surrounding environment state of the vehicle based on the specific sound type, sound characteristic parameters and perception signals.

[0076] Further, after determining the specific sound type, the state of the vehicle's surrounding environment can be determined based on the specific sound type, the sound characteristic parameters and the perception signal. It is understandable that after determining the specific sound type, analyzing the frequency, decibel size, and respective change trends of the sound characteristic parameters, and combining the perception signal, the state of the vehicle's surrounding environment can be determined.

[0077] Figure 3 This is a schematic diagram of the structure of a vehicle environment perception system provided by another embodiment of the present application. In order to accurately analyze the road conditions of the vehicle, slow down on dangerous roads, and thus improve vehicle safety, in an optional embodiment, the audio sensor 2 includes a plurality of first audio sensors 20, such as Figure 3 As shown, a plurality of first audio sensors 20 are respectively disposed at the middle of the axles of the vehicle.

[0078] In order to more accurately collect the noise and tire noise fed back from the road surface, in an optional embodiment, the first audio sensor 20 is disposed at the bottom of the vehicle. Specifically, the first audio sensor 20 can be disposed in the middle of the axle. It is understandable that different vehicles have different numbers of axles, and the number of first audio sensors 20 can be the same as the number of axles, that is, a first audio sensor 20 is disposed in the middle of each axle. Of course, from a cost perspective, a first audio sensor 20 can also be disposed in the middle of the axle corresponding to the front wheel and the axle corresponding to the rear wheel, respectively, or a first audio sensor 20 can be disposed in the middle of any two axles, and this application does not limit this.

[0079] It should be noted that the sound signal collected by the first audio sensor 20 is the first sound signal. In a specific embodiment, after the controller 1 obtains the first sound signal, it preprocesses the first sound signal, extracts sound features, and other processes, and when it is determined that the specific sound type of the first sound signal is the first sound type, the sound feature parameters and perception signals corresponding to the first sound type are analyzed to determine the road surface information on which the vehicle is currently traveling, wherein the first sound type includes at least one of tire noise and ground feedback noise.

[0080] The first sound type includes at least one of tire noise and ground feedback noise. That is, when the controller 1 analyzes and determines that the first sound signal collected by the first audio sensor 20 includes at least one of tire noise and ground feedback noise, the first sound signal and the perception signal collected by the environment collection device will be combined to determine the road surface information on the current driving path of the vehicle.

[0081] Specifically, in an optional embodiment, when the controller 1 determines the current road surface information, if the road surface information satisfies that the road surface bumpiness is greater than the bumpiness threshold, or the road surface friction coefficient is less than the coefficient threshold, it indicates that the current road surface driving risk coefficient is high. It can be understood that the road surface bumpiness greater than the bumpiness threshold indicates that the road surface is relatively flat, and there may be road potholes and road surface cracks, and the road surface friction coefficient is less than the coefficient threshold, indicating that the road surface is slippery, for example, the road surface is icy.

[0082] At this time, the current driving information such as the current weather information, the current vehicle speed and the tire pressure is obtained, and the vehicle is controlled to decelerate according to the current driving information. Specifically, the vehicle speed is reduced to a preset speed according to the current vehicle speed and tire pressure, thereby reducing potential safety risks and improving passenger comfort.

[0083] In addition, whether to pull over is determined based on the current weather information. For example, when the vehicle is traveling on a highway and the road friction coefficient is less than the coefficient threshold, and the weather information shows foggy or rainy weather, the vehicle needs to be slowed down to a certain speed, or the vehicle needs to be controlled to leave the highway as soon as possible.

[0084] At the same time, in an optional embodiment, the controller 1 can mark the road surface information with potential dangers, and report the marked road surface information to the remote platform 4, so as to remind other vehicles of the road surface conditions of the current road, thereby improving the overall safety of the vehicle. Of course, the road surface information can also be reported to the road management department so that the road management department can maintain the dangerous road surface in time.

[0085] Therefore, the vehicle environment perception system provided in the embodiment of the present application is used to collect noise and tire noise from road surface feedback by setting a first audio sensor 20 at the bottom of the vehicle. Thus, the road surface conditions, such as road surface quality and obstacles on the road surface, can be perceived by analyzing the sound feedback from the road surface, which helps the vehicle to adjust the driving speed and driving strategy in time to adapt to different road conditions and improve the driving safety factor.

[0086] In an optional embodiment, the audio sensor 2 includes a plurality of second audio sensors 21. Figure 3 As shown, a plurality of second audio sensors 21 are respectively arranged on the outer wall of the vehicle on the main driver's side, the outer wall of the vehicle on the co-driver's side and above the windshield 5 .

[0087] It should be noted that the sound signal collected by the second audio sensor 21 is the second sound signal. In fact, the second audio sensor 21 is mainly used to sense whether there are designated vehicles such as fire trucks, police cars and ambulances around the vehicle, as well as whether there are law enforcement officers, etc.

[0088] Specifically, after the second audio sensor 21 collects the second sound signal, it transmits the second sound signal to the controller 1. At this time, the controller 1 performs preprocessing, sound feature parameter extraction and other processing on the second sound signal, and when it is determined that the specific sound type of the second sound signal is the second sound type, the sound feature parameters corresponding to the second sound type are combined with the perception signal to determine whether there is a designated vehicle and / or law enforcement personnel that outputs the second sound type within the first designated range of the vehicle, wherein the second sound type includes at least one of a fire truck siren, a police car siren and an ambulance siren, and correspondingly, the designated vehicle includes at least one of a fire truck, a police car and an ambulance.

[0089] It can be understood that when the controller 1 determines that there is a second sound signal such as a police car siren, an ambulance siren or a fire truck siren with strong penetrating power within the first specified range of the vehicle, the change trend of the siren sound is analyzed to determine whether the corresponding specified vehicle is approaching. At the same time, the perception information collected by the environmental equipment is used to further determine whether the specified vehicle is approaching, and determine the current road conditions, for example, Figure 1 As shown, the current road condition is determined through the video image collected by the camera 30.

[0090] Based on the change trend of the sound decibel, it is determined that the designated vehicle is close to the own vehicle and the distance is less than the distance threshold, and the camera 30 is used to determine that if the road conditions permit, the vehicle is controlled to slow down or pull over to avoid the vehicle.

[0091] It should be noted that the first designated range can be an area within a circle with a preset radius (e.g., 100 meters) with the vehicle as the center. It can also be set based on the sound decibel. Specifically, the ground area corresponding to the sound decibel within the preset range is taken as the first designated range. In fact, this application does not specifically limit the first designated range.

[0092] In another optional embodiment, when there are law enforcement officers within the first specified range, a command signal from the law enforcement officers is obtained, and vehicle control corresponding to the command signal is executed; wherein the command signal includes at least one of a voice signal and an action signal.

[0093] In a specific embodiment, Figure 1 As shown, the controller 1 determines that there are law enforcement officers within the first range based on the second sound signal and the perception signal, and then uses the camera 30 to identify the law enforcement officer's clothing, and slows down or pulls over. For example, when the law enforcement officer is a traffic police, the camera 30 and the second audio sensor 21 collect and identify the traffic police's verbal command information and gesture information, and analyze and judge the law enforcement officer's intention, so as to make accurate vehicle control. At the same time, the identification information is reported to the remote platform 4 so that the unmanned driving dispatcher can communicate with the law enforcement officer in a timely and effective manner to avoid traffic congestion.

[0094] Therefore, the vehicle environment perception system provided in the embodiment of the present application can realize the perception of the relevant law enforcement behavior of law enforcement personnel (including the perception of voice instructions and action instructions) by setting the second audio sensor 21, or can perceive the scene of fire trucks and ambulances sirens for emergency passage through the sirens. The second sound sensor is innovatively combined with the environment perception device 3 to achieve a more comprehensive perception of the vehicle's surrounding environment and make correct control strategies for special scenarios.

[0095] In order to timely sense whether the vehicle collides with other obstacles, or whether other participants on the vehicle's driving path collide with each other and need to avoid each other in time, in an optional embodiment, the audio sensor 2 includes a third audio sensor 22, such as Figure 3 As shown, the third audio sensor 22 is disposed at the front bumper 6 of the vehicle.

[0096] Specifically, in order to more comprehensively perceive the collision sound, the third audio sensor 22 is disposed in the middle of the front bumper 6. In an optional embodiment, in order to further more comprehensively perceive the collision sound, the third audio sensor 22 may be multiple, and in addition to being disposed in the middle of the front bumper 6, it may also be disposed in the middle of the rear bumper of the vehicle. Of course, considering the cost issue, one third audio sensor 22 may be disposed in the middle of the front bumper 6, and this application does not limit this.

[0097] It should be noted that the sound signal collected by the third audio sensor 22 is a third sound signal. In a specific embodiment, after the third audio sensor 22 collects the third sound signal, it is transmitted to the controller 1. After the controller 1 performs preprocessing and sound feature extraction on the third sound signal, when determining that the specific sound type is the third sound type, the sound feature parameters corresponding to the third sound type and the perception signal are combined to determine whether the vehicle has a dangerous collision and / or whether other traffic participants have a dangerous collision within the second specified range of the vehicle. Among them, the third sound type includes the collision sound of the vehicle colliding with an obstacle, and can also include the collision sound caused by the dangerous collision of other traffic participants.

[0098] In a specific embodiment, in a low-speed driving scenario, the third audio sensor 22 can collect the collision sound of the vehicle colliding with an obstacle, and can also collect the sound of whether other traffic participants have a dangerous collision.

[0099] Furthermore, the collected third sound signal is combined with the perception information collected by the environmental perception device 3 (for example, the perception results of the obstacle by the ultrasonic radar 32 and the lidar 31) to comprehensively judge whether the vehicle has collided. If it is determined that the vehicle has a dangerous collision and / or other traffic participants within the second specified range of the vehicle have a dangerous collision, the vehicle is controlled to slow down, take a detour, or pull over.

[0100] At the same time, if Figure 1 As shown, the controller 1 reports the collision time to the remote platform 4 and the smart cockpit 7, wherein the smart cockpit 7 is used to inform the passengers of the current collision time. After the remote platform 4 obtains the collision time, it can check and maintain the vehicle in time.

[0101] In the scenario of high-speed driving, if there are flying rocks hitting the vehicle, or objects falling from the vehicle in front, the third audio sensor 22 can collect the corresponding third sound signal in time, process the third sound signal, and determine the event information in combination with the perception signal, and control the vehicle to slow down, take a detour, or pull over. At the same time, the smart cockpit 7 informs the passengers and reports the event to the remote platform 4.

[0102] In an optional embodiment, the controller 1 can feedback the execution results of the entire vehicle to the passengers through voice or other means (for example, a display screen), and feedback the operation results to the remote platform 4. The remote platform 4 can record and save the time of the event, event type, event severity level, vehicle location, vehicle fault information, video of the vehicle event, etc.

[0103] Therefore, the vehicle environment perception system provided in the embodiment of the present application is used to sense collision events around the vehicle by setting a third audio sensor 22 at the vehicle bumper. Once an accident occurs, emergency safety strategies are made in time to improve vehicle safety.

[0104] In an optional embodiment, if Figure 1 As shown, the environment perception device 3 includes: a camera 30, a laser radar 31 and an ultrasonic radar 32. The camera 30 is arranged in the vehicle for collecting video images. The laser radar 31 is arranged at the front bumper 6 of the vehicle for collecting point cloud data. The ultrasonic radar 32 is arranged at the rear bumper of the vehicle for collecting ultrasonic signals.

[0105] Among them, the ultrasonic radar 32 can detect obstacles around the vehicle body, which is crucial for avoiding collisions and maintaining road safety. Specifically, the transmitter of the ultrasonic radar 32 will emit an ultrasonic signal, which will be reflected back after hitting an obstacle. When the ultrasonic radar 32 receives the reflected signal, the controller will convert it into an electrical signal that can be processed through an algorithm, and the electrical signal will be further processed and converted into distance information. Therefore, in a specific embodiment, the ultrasonic radar 32 and the audio sensor 2 are combined to improve the accuracy of obstacle detection.

[0106] In an optional embodiment, the camera 30 can transmit the collected video images through the GMSL (Gigabit Multimedia Serial Link) protocol, and the laser radar 31 and the ultrasonic radar 32 can transmit the collected point cloud data and ultrasonic signals through Ethernet. Among them, the ultrasonic radar 32 can be a millimeter wave ultrasonic radar 32, which is not limited in this application.

[0107] In the above embodiments, the vehicle's environmental perception system is described in detail. The present application also provides an embodiment corresponding to a vehicle control method.

[0108] Figure 4 A flow chart of a vehicle control method provided in an embodiment of the present application is shown as follows: Figure 4 As shown, the method includes:

[0109] S20: receiving a sound signal in an audible frequency band around the vehicle collected by an audio sensor and perception information of the vehicle's surrounding environment collected by an environment perception device; wherein the perception information includes at least one of a video image, point cloud data, and an ultrasonic signal; and the audio sensor and the environment perception device are both disposed on the vehicle;

[0110] S21: Determine the state of the surrounding environment of the vehicle according to the sound signal and the perception information;

[0111] S22: Execute corresponding vehicle control according to the surrounding environment status.

[0112] The control method provided in the embodiment of the present application can be executed by the vehicle controller or the domain controller of the vehicle, which is not limited in the present application. In a specific embodiment, the audio sensor collects sound signals in the audible frequency band, and combines the perception information collected by the environment perception device from at least one of the video image, point cloud data and ultrasonic signal to comprehensively judge the state of the vehicle's surrounding environment so as to perform corresponding vehicle control, thereby improving vehicle safety.

[0113] The sound signal may include, but is not limited to, a police car siren, an ambulance siren, a fire truck siren, the collision sound generated when a vehicle collides with an obstacle, tire noise, and road noise fed back from the ground.

[0114] Since the vehicle control method provided in the embodiment of the present application corresponds to the vehicle environment perception system provided in the above embodiment, the specific implementation method refers to the above embodiment and will not be repeated here.

[0115] Therefore, the vehicle control method provided in the embodiment of the present application combines the perceived sound signals in the audible frequency band around the vehicle and the perceived information of the vehicle's surrounding environment, comprehensively judges the state of the vehicle's surrounding environment, and controls the entire vehicle based on the surrounding environment state, thereby realizing the perception of the surrounding environment from multiple angles, that is, multiple types of data are jointly analyzed to determine the surrounding environment state, thereby improving the perception accuracy and expanding the perception range, thereby improving the vehicle control accuracy and safety.

[0116] Figure 5 FIG. 1 is a schematic diagram of a structure of a vehicle provided in an embodiment of the present application. Based on the above embodiment, an embodiment of the present application further provides a vehicle, such as Figure 5 As shown, the vehicle 13 includes a driving system 131 and an environment perception system 130 in the above embodiment. The environment perception system 130 is connected to the driving system 131.

[0117] In a specific embodiment, the environment sensing system 130 senses the state of the surrounding environment around the vehicle 13, and controls the driving system 131 to work according to the state of the surrounding environment, thereby realizing the corresponding vehicle control. It should be noted that the vehicle 13 provided in the embodiment of the present application includes but is not limited to a sedan, a sport utility vehicle (SUV), a multi-purpose vehicle (MPV), a commercial vehicle, an off-road vehicle, a pickup truck or other power-driven non-track-borne vehicle.

[0118] Since the vehicle provided in the embodiment of the present application includes the environmental perception system in the above embodiment, the embodiment of the present application corresponds to the above embodiment. For the specific implementation method, please refer to the description of the above embodiment, which will not be repeated here.

[0119] It should be noted that, although the operations are depicted in a specific order in the accompanying drawings, this should not be understood as requiring these operations to be performed in the specific order shown or to be performed sequentially, or requiring all illustrated operations to be performed to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product, or packaged into multiple software products.

Claims

1. A vehicle environment perception system, It is characterized in that The system includes: a controller, an audio sensor and an environment sensing device; The audio sensor and the environment sensing device are both arranged on the vehicle and are both communicatively connected with the controller; The audio sensor is used to collect sound signals in the audible frequency band around the vehicle; The environmental perception device is used to collect perception information of the vehicle's surrounding environment; wherein the perception information includes at least one of a video image, point cloud data, and an ultrasonic signal; The controller is used to determine the surrounding environment state of the vehicle according to the received sound signal and the perception information, and perform corresponding vehicle control according to the surrounding environment state.

2. The vehicle environment perception system according to claim 1, It is characterized in that The controller is also used for: Extracting features from the sound signal to obtain sound feature parameters; Determining the specific sound type to which the sound signal belongs according to the sound characteristic parameter; The surrounding environment state of the vehicle is determined based on the specific sound type, the sound characteristic parameters and the perception signal.

3. The vehicle environment perception system as claimed in claim 2, It is characterized in that The audio sensor includes a plurality of first audio sensors; The plurality of first audio sensors are respectively arranged at the middle of the axle of the vehicle; The controller is used to analyze the sound characteristic parameters corresponding to the first sound type and the perception signal when determining that the specific sound type is the first sound type, so as to determine the road surface information on which the vehicle is currently traveling; wherein the first sound type includes at least one of tire noise and ground feedback noise.

4. The vehicle environment perception system as claimed in claim 3, It is characterized in that The controller is also used for: When the road surface information meets the preset conditions, current driving information is obtained; wherein the current driving information includes current weather information, current vehicle speed and tire pressure; the preset conditions include that the road surface bumpiness is greater than a bumpiness threshold, or the road surface friction coefficient is less than a coefficient threshold; The vehicle is controlled to decelerate according to the current driving information.

5. The vehicle environment perception system as claimed in claim 2, It is characterized in that The audio sensor includes a plurality of second audio sensors; The plurality of second audio sensors are respectively arranged on the outer wall of the vehicle on the main driver's side, the outer wall of the vehicle on the co-driver's side and above the windshield; The controller is used to analyze the sound characteristic parameters corresponding to the second sound type and the perception signal when determining that the specific sound type is the second sound type, so as to determine whether there is a designated vehicle and / or law enforcement personnel outputting the second sound type within the first designated range of the vehicle; wherein the second sound type includes at least one of a fire truck siren, a police car siren and an ambulance siren.

6. The vehicle environment perception system as claimed in claim 5, It is characterized in that The controller is also used for: When the designated vehicle exists within the first designated range, controlling the vehicle to slow down or pull over to avoid the designated vehicle; When the law enforcement officer exists within the first designated range, the command signal of the law enforcement officer is obtained, and the vehicle control corresponding to the command signal is executed; wherein the command signal includes at least one of a voice signal and an action signal.

7. The vehicle environment perception system as claimed in claim 2, It is characterized in that The audio sensor includes a third audio sensor; The third audio sensor is disposed at the front bumper of the vehicle; The controller is used to analyze the sound characteristic parameters corresponding to the third sound type and the perception signal when determining that the specific sound type is the third sound type, so as to determine whether the vehicle has a dangerous collision and / or whether other traffic participants have a dangerous collision within a second specified range of the vehicle; wherein the third sound type includes the collision sound of the vehicle colliding with an obstacle.

8. The vehicle environment perception system as claimed in claim 7, It is characterized in that The controller is also used for: When it is determined that the vehicle is in a dangerous collision and / or other traffic participants are in a dangerous collision within the second specified range of the vehicle, the vehicle is controlled to slow down, take a detour, or pull over; The collision event information corresponding to the dangerous collision is reported to a remote service platform.

9. The vehicle environment perception system as claimed in claim 1, It is characterized in that The environment perception equipment includes: a camera, a laser radar and an ultrasonic radar; The camera is arranged in the vehicle and is used to collect the video image; The laser radar is arranged at the front bumper of the vehicle and is used to collect the point cloud data; The ultrasonic radar is arranged at the rear bumper of the vehicle and is used to collect the ultrasonic signal.

10. A method for controlling a vehicle, It is characterized in that The method comprises: Receive the sound signal in the audible frequency band around the vehicle collected by the audio sensor and the perception information of the vehicle's surrounding environment collected by the environment perception device; wherein the perception information includes at least one of a video image, point cloud data and an ultrasonic signal; the audio sensor and the environment perception device are both arranged on the vehicle; Determining a state of the surrounding environment of the vehicle according to the sound signal and the perception information; According to the surrounding environment state, corresponding vehicle control is performed.

11. A vehicle, It is characterized in that Comprising a driving system and an environment sensing system according to any one of claims 1 to 9; The environmental perception system is connected to the driving system; The environmental perception system is used to control the operation of the drive system according to the surrounding environment state of the vehicle to perform corresponding vehicle control.

Citation Information

Cited By

  • Vehicle collision detection method and device, vehicle, storage medium and program product

    CN120056977A

  • Vehicle bump detection method and device, vehicle, storage medium and program product

    CN120056977B