Multi-source sensing fusion intelligent driving system
Through a multi-source sensing fusion system, combined with multiple sensor data for environmental perception and driving control, the problem of excessive speaker volume of intelligent driving system in urban roads is solved, safety and environmental perception are improved, and noise pollution is reduced.
Patent Information
- Application Number
- CN202411222424.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-02
- Publication Date
- 2024-12-03
AI Technical Summary
The existing intelligent driving system relies on a single sensor perception, and has weak environmental perception and scene understanding, which leads to excessive speaker volume when traffic is congested in urban roads, which can easily disturb pedestrians and cause noise pollution.
Using a multi-source sensing fusion intelligent driving system, the sensor fusion module combines multiple sensor data to generate the fused environmental data, the environment sensing module performs scene understanding, and the driving control module adjusts the driving speed and speaker volume according to pedestrian detection information to achieve Adaptive adjustment.
It improves the safety of intelligent driving and environmental perception accuracy, reduces speaker volume, reduces noise pollution, and ensures that the volume is appropriately increased when reminding pedestrians or vehicles when needed, and reduces noise pollution at other moments.
Smart Images

Figure CN119058746A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of sensor fusion technology, and in particular to an intelligent driving system with multi-source sensor fusion. Background Art
[0002] With the rapid development of intelligent automobile driving technology, intelligent driving systems use technology to make up for the shortcomings of human factors. During driving, intelligent driving systems can receive and process various sensor information, identify the environment and vehicle status of the vehicle, and thus control the vehicle. Even in very complex road conditions, it can automatically manipulate and drive the vehicle to bypass obstacles and drive along a predetermined road trajectory. However, in the existing technology, intelligent driving relies on a single sensor to perceive the position of surrounding vehicles and obstacles, and has weak environmental perception and scene understanding. In addition, in urban roads, traffic conditions are congested, such as obstacles ahead or pedestrians frequently honk, and the horn volume is too high, which can easily disturb pedestrians and cause noise pollution. Summary of the invention
[0003] The purpose of the present invention is to provide an intelligent driving system with multi-source sensor fusion to solve the above technical problems;
[0004] The technical problem solved by the present invention can be achieved by adopting the following technical solutions:
[0005] A multi-source sensor fusion intelligent driving system, comprising:
[0006] A sensor fusion module, including a plurality of sensors provided on the vehicle, wherein the sensor fusion module combines the data of the sensors by weighted fusion to obtain fused environmental data;
[0007] An environment perception module, connected to the sensor fusion module, for performing environment perception and scene understanding based on the fused environment data, and obtaining driving environment information including pedestrian detection information and road detection information;
[0008] A driving control module, connected to the environment perception module, is used to generate a driving path and a driving speed according to the driving environment information. The driving control module includes:
[0009] a speed adjustment unit, configured to adjust the driving speed according to the pedestrian detection information;
[0010] A volume adjustment unit is used to adjust the horn volume of the vehicle according to the fused environmental data and the pedestrian detection information.
[0011] Preferably, the environmental data acquired by the sensor includes at least image data, distance data and speed data, and the sensor fusion module further includes a weight allocation unit for adjusting the image of the data of each sensor on the fused environmental data through weights, and the weight vector set by the weight allocation unit is,
[0012] W={w I ,w D ,w V}
[0013] where w I represents the weight of the image data, w D represents the weight of the distance data, w V represents the weight of the speed data, w I 、w D 、w V The sum is 1;
[0014] The sensor fusion module also includes a data fusion unit, which is used to obtain the fused environmental data by weighted fusion. The formula is:
[0015] E=w I I+W D ·D+w V ·V
[0016] Wherein, E represents the fused environment data, I represents the image data, D represents the distance data, and V represents the speed data.
[0017] Preferably, the driving control module includes:
[0018] The path planning unit is connected to an external map system to obtain map data, and is used to perform path planning based on the map data and the driving environment information to obtain the driving path. The formula is:
[0019] P = PathPlanning (M, S, G)
[0020] Wherein P represents the driving path, M represents the map data, S represents the starting point, G represents the end point, and PathPlanning represents the path planning method obtained from the repository;
[0021] A speed control unit is connected to the path planning unit and is used to generate the driving speed according to the driving environment information and the driving path. The formula is:
[0022] v target =min(v max ,max(v min ,v+a·Δt))
[0023] where v target represents the driving speed, v max Maximum speed, v min represents the minimum speed, a represents the acceleration, and Δt represents the control period.
[0024] Preferably, the road detection information acquired by the environmental perception module includes at least road surface information, road sign information, lane line information and obstacle information.
[0025] Preferably, the environmental perception module includes a pedestrian perception unit, which is connected to the sensor fusion module to obtain the pedestrian detection information, predict the pedestrian's movement path based on the pedestrian detection information, and the speed adjustment unit adjusts the driving speed according to the pedestrian's movement path.
[0026] Preferably, the volume adjustment unit is further connected to the noise sensor included in the sensor fusion module to obtain noise data. The volume adjustment unit calculates the horn volume according to the speed data and the noise data. The calculation formula is:
[0027] V final =V base ×f speed (V)×f noise (N)
[0028] Among them, V final Indicates the speaker volume, V base Indicates the basic volume, f speed represents the vehicle speed correction coefficient, V represents the speed data, f noise represents the noise correction coefficient, and N represents the noise data.
[0029] Preferably, the vehicle speed correction factor f speed The calculation formula is,
[0030] f speed (V) = 1 + γ·V
[0031] Wherein V represents the speed data, and γ represents the vehicle speed correction parameter;
[0032] The noise correction factor f noise The calculation formula is,
[0033]
[0034] Wherein, β represents the noise correction parameter, N represents the noise data, and N max Indicates the preset maximum noise level.
[0035] Preferably, the pedestrian sensing unit further identifies pedestrian age data according to the pedestrian detection information. When the pedestrian age data exceeds a preset age threshold, the volume adjustment unit adjusts the basic volume according to the pedestrian age data and the distance data. The formula is:
[0036]
[0037] Among them, V base represents the basic volume, D represents the distance data, and k represents the age coefficient.
[0038] Preferably, the pedestrian perception unit is connected to the sensor to receive the image data, and the pedestrian perception unit is provided with an age analysis subunit, and the age analysis subunit identifies pedestrian features in the image data based on a target detection method, and calculates the pedestrian age data according to the pedestrian feature matching, and an age coefficient comparison library is provided in the pedestrian perception unit, and the pedestrian perception unit matches the age coefficient from the age coefficient comparison library according to the pedestrian age data.
[0039] Preferably, the pedestrian feature identified by the age analysis subunit is at least one of a motion feature, a gait feature, a clothing feature, and a speed feature.
[0040] Beneficial effects of the present invention: Due to the adoption of the above technical scheme, the present invention generates a safe driving path based on the environment perception data output by the sensor fusion module, adjusts the driving speed and the volume of the vehicle horn according to the pedestrian detection information, ensures the safety of intelligent driving and realizes adaptive adjustment of the volume of the vehicle horn, and can increase the volume when it is necessary to remind pedestrians or other vehicles, and reduce the volume at other times, always maintaining a moderate volume and reducing noise pollution. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 Schematic diagram of the intelligent driving system with multi-source sensor fusion in an embodiment of the present invention. DETAILED DESCRIPTION
[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0043] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.
[0044] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.
[0045] An intelligent driving system based on multi-source sensor fusion, such as Figure 1 As shown, including,
[0046] The sensor fusion module 1 includes a plurality of sensors 11 provided on the vehicle, and the sensor fusion module 1 combines the data of the sensors 11 by weighted fusion to obtain fused environmental data;
[0047] The environment perception module 2 is connected to the sensor fusion module 1 and is used to perform environment perception and scene understanding based on the fused environment data to obtain driving environment information including pedestrian detection information and road detection information;
[0048] The driving control module 3 is connected to the environment perception module 2 and is used to generate a driving path and a driving speed according to the driving environment information. The driving control module 3 includes:
[0049] A speed adjustment unit 31, used to adjust the driving speed according to the pedestrian detection information;
[0050] The volume adjustment unit 32 is used to adjust the horn volume of the vehicle according to the fused environmental data and pedestrian detection information.
[0051] Specifically, the intelligent driving system of the present invention integrates multiple sensors 11, such as laser radar, camera, ultrasonic sensor, etc., and each sensor 11 is used to collect different information about the vehicle's surrounding environment.
[0052] The sensor fusion module 1 combines the data of each sensor 11 in a weighted fusion manner to reduce data errors and improve the accuracy of environmental data; based on the data output by the sensor fusion module 1, the environmental perception module 2 performs environmental perception and scene understanding, and identifies and analyzes pedestrians, vehicles, road signs and obstacles in the driving environment; through environmental perception, the present invention can obtain detailed driving environment information, including pedestrian detection information and road status information.
[0053] The driving control module 3 generates a safe driving path based on the driving environment information provided by the environmental perception module 2. The speed adjustment unit 31 adjusts the driving speed based on the pedestrian detection information to ensure a safe distance from pedestrians and adapt to the current traffic conditions. The volume adjustment unit 32 adjusts the volume of the vehicle horn based on the fused environmental data and pedestrian detection information. The volume of the vehicle horn is adaptively adjusted. The volume can be increased when it is necessary to remind pedestrians or other vehicles, and lowered at other times. The volume is always maintained at a moderate level to reduce noise pollution.
[0054] In a preferred embodiment, the environmental data acquired by the sensor 11 includes at least image data, distance data and speed data. The sensor fusion module 1 also includes a weight allocation unit 12 for adjusting the image of the data of each sensor 11 on the fused environmental data through weights. The weight vector set by the weight allocation unit 12 is:
[0055] W={w I ,w D ,w V}
[0056] where w I Represents the weight of image data, w D Represents the weight of distance data, w V represents the weight of speed data, w I 、w D 、w V The sum is 1;
[0057] The sensor fusion module 1 also includes a data fusion unit 13, which is used to obtain fused environmental data by weighted fusion. The formula is:
[0058] E=w I I+w D ·D+w V ·V
[0059] Among them, E represents the fused environment data, I represents the image data, D represents the distance data, and V represents the speed data.
[0060] Specifically, the present invention is provided with multiple types of sensors 11, including image sensors for providing image data, distance sensors for providing distance data, and speed sensors for providing speed data. Since the contribution of each sensor 11 data environment representation is different, the present invention adjusts the influence of different sensor 11 data through weights.
[0061] Preferably, the data of different sensors 11 are fused into an overall representation through the sensor fusion module 1, and the Kalman filter and particle filter algorithms can be used to process the time series relationship between the sensor 11 data to ensure the accuracy of the fused environmental data.
[0062] In a preferred embodiment, the driving control module 3 includes:
[0063] The path planning unit 33 is connected to an external map system to obtain map data, and is used to perform path planning based on the map data and driving environment information to obtain a driving path. The formula is:
[0064] P = PathPlanning (M, S, G)
[0065] Where P represents the driving path, M represents the map data, S represents the starting point, G represents the end point, and PathPlanning represents the path planning method obtained from the repository;
[0066] The speed control unit 34 is connected to the path planning unit 33 and is used to generate the driving speed according to the driving environment information and the driving path. The formula is:
[0067] v target =min(v max ,max(v min ,v+a·Δt))
[0068] where v target represents the driving speed, v max Maximum speed, v min represents the minimum speed, a represents the acceleration, and Δt represents the control period.
[0069] Specifically, the goal of path planning is to find the optimal path from the starting point to the target point under a given map and environmental information. The path planning method adopted by the present invention is, for example, A* algorithm, Dijkstra algorithm, RRT (Rapidly-exploring Random Tree). Speed control determines the speed of the vehicle according to the current driving environment and the path planning result.
[0070] More specifically, the driving control module 3 of the present invention also includes an obstacle avoidance unit 35. The obstacle avoidance unit 35 avoids obstacles by dynamically adjusting the trajectory or speed of the vehicle based on the obstacle information in the environmental perception results. The obstacle avoidance method in the present invention adopts a following method, trajectory correction or local path planning.
[0071] In a preferred embodiment, the road detection information acquired by the environment perception module 2 includes at least road surface information, road sign information, lane line information and obstacle information.
[0072] In a preferred embodiment, the environment perception module 2 includes a pedestrian perception unit 21, which is connected to the sensor fusion module 1 to obtain pedestrian detection information, predict the pedestrian's movement path based on the pedestrian detection information, and the speed adjustment unit 31 adjusts the driving speed according to the pedestrian's movement path.
[0073] Specifically, the environment perception module 2 can detect the distance and speed of surrounding objects, including pedestrians, and then identify the pedestrians and estimate their movement trajectories.
[0074] More specifically, in this embodiment, the vehicle-mounted camera can be used for visual perception to detect the appearance, position and motion state of the pedestrian. The radar is used to detect the position and speed of the pedestrian. The advantage of the radar is that it is not limited by lighting and weather conditions; the laser radar can accurately measure the position of the pedestrian and the geometric structure of the surrounding environment.
[0075] The data from different sensors 11 are fused, and the outputs of each sensor 11 are integrated using a filtering algorithm, such as a Kalman filter, to obtain the pedestrian position and speed information. The pedestrian's speed is calculated based on the displacement of the pedestrian in consecutive frames. The speed can be estimated using a simple ratio of displacement divided by time.
[0076] Or estimated action path: Based on the pedestrian’s current position, speed, and possible pedestrian behavior models, such as linear motion, acceleration, deceleration, etc., predict the pedestrian’s possible movement path in the next few seconds.
[0077] The processed pedestrian position, speed and expected movement path information are transmitted to the speed adjustment unit 31 to determine whether the pedestrian's movement path and the planned path will collide, so that the speed adjustment unit 31 adjusts the speed to avoid collision.
[0078] In a preferred embodiment, the volume adjustment unit 32 is also connected to the noise sensor included in the sensor fusion module 1 to obtain noise data. The volume adjustment unit 32 calculates the horn volume according to the speed data and the noise data. The calculation formula is:
[0079] V final =V base ×f speed (V)×f noise (N)
[0080] Among them, V final Indicates the speaker volume, V base Indicates the basic volume, f speed represents the vehicle speed correction coefficient, V represents the speed data, f noise represents the noise correction factor, and N represents the noise data.
[0081] In a preferred embodiment, the vehicle speed correction factor f speed The calculation formula is,
[0082] f speed (V) = 1 + γ·V
[0083] Where V represents speed data, and γ represents vehicle speed correction parameter;
[0084] Noise correction factor f noise The calculation formula is,
[0085]
[0086] Among them, β represents the noise correction parameter, N represents the noise data, and N max Indicates the preset maximum noise level.
[0087] Specifically, the noise sensor of the present invention collects sound data of the surrounding environment through a vehicle-mounted microphone, analyzes and identifies the main noise sources, such as environmental noise, pedestrian voices, etc. The voltage or digital signal output by the noise sensor can be calibrated and analyzed to obtain the sound pressure level (dB) of the environment to obtain noise data.
[0088] Specifically, the distance data in the present invention is obtained by using a distance sensor, and the distance sensor includes an ultrasonic sensor and a radar sensor. When the vehicle is traveling at a high speed or the ambient noise is large, the horn volume is output at a higher volume to ensure that it can be heard, thereby achieving a horn effect; when the vehicle is traveling at a low speed or the surroundings are relatively quiet, the horn volume is output at a lower volume to ensure that people around can hear it while reducing noise pollution to avoid disturbing others.
[0089] In a preferred embodiment, the pedestrian sensing unit 21 further identifies the pedestrian age data according to the pedestrian detection information. When the pedestrian age data exceeds a preset age threshold, the volume adjustment unit 32 adjusts the basic volume according to the pedestrian age data and the distance data. The formula is:
[0090]
[0091] Among them, V base represents the basic volume, D represents the distance data, and k represents the age coefficient.
[0092] Specifically, for elderly pedestrians, especially when they are close to vehicles, the volume of the horn suddenly honking may frighten them. Therefore, the present invention can control the volume of the horn when facing older pedestrians by adjusting the volume of the horn according to the age and distance of the pedestrians, so that the vehicle honks at a smaller decibel to reduce the sudden fright caused to them. In addition, the possibility of accidents is reduced, and civilized driving is promoted in cities or on crowded roads.
[0093] In addition, the invention can reduce the volume and improve the noise pollution problem in urban environment, especially in traffic-intensive areas, and reduce the interference of unnecessary noise to surrounding residents, which helps to improve the quality of urban life.
[0094] In a preferred embodiment, the pedestrian sensing unit 21 is connected to the sensor 11 to receive image data. The pedestrian sensing unit 21 is provided with an age analysis subunit. The age analysis subunit identifies pedestrian features in the image data based on a target detection method, and calculates the pedestrian age data based on pedestrian feature matching. The pedestrian sensing unit 21 is provided with an age coefficient comparison library. The pedestrian sensing unit 21 matches the age coefficient from the age coefficient comparison library based on the pedestrian age data.
[0095] In a preferred embodiment, the pedestrian feature identified by the age analysis subunit is at least one of a motion feature, a gait feature, a clothing feature, and a speed feature.
[0096] Specifically, the present invention adopts an image processing method to capture images of the surrounding environment in real time through an image sensor, such as a camera, and detects pedestrians, especially the characteristics of the elderly, such as walking posture, clothing characteristics, etc. through pattern recognition technology.
[0097] Specifically, the vehicle-mounted camera captures images or video streams of the surrounding environment in real time. The age analysis subunit uses computer vision technology and a trained deep learning-based target detection model to analyze and process the image to detect pedestrians in the image, identify different features of pedestrians in the image, such as body shape, movement features, etc., especially identify features of the elderly, such as gait, clothing (such as crutches, etc.) and walking speed, and output age judgment results.
[0098] Specifically, the target detection algorithm used by the target detection model can be a convolutional neural network (CNN) based on deep learning, such as YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), Faster R-CNN, etc., which can efficiently detect the position and bounding box of pedestrians in the image.
[0099] The age analysis subunit is used to estimate the age of pedestrians from their features. In this embodiment, a convolutional neural network is used to perform age regression or classification tasks. A large number of labeled data sets are used to train the model, and the data sets contain images of pedestrians and their corresponding age labels.
[0100] During operation, the pedestrian perception unit 21 receives and analyzes image data in real time, identifies pedestrians through the target detection model, estimates their age, obtains the age coefficient from the age coefficient reference library, and adjusts the horn volume according to the age coefficient.
[0101] The above description is only a preferred embodiment of the present invention, and does not limit the implementation mode and protection scope of the present invention. For those skilled in the art, it should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.
Claims
1. An intelligent driving system with multi-source sensor fusion, characterized in that: include, A sensor fusion module (1) comprising a plurality of sensors (11) provided on a vehicle, wherein the sensor fusion module (1) combines data of the sensors (11) by weighted fusion to obtain fused environmental data; An environment perception module (2), connected to the sensor fusion module (1), is used to perform environment perception and scene understanding based on the fused environment data, and obtain driving environment information including pedestrian detection information and road detection information; A driving control module (3) is connected to the environment perception module (2) and is used to generate a driving path and a driving speed according to the driving environment information. The driving control module (3) includes: A speed adjustment unit (31), used for adjusting the driving speed according to the pedestrian detection information; A volume adjustment unit (32) is used to adjust the horn volume of the vehicle according to the fused environmental data and the pedestrian detection information.
2. The multi-source sensor fusion intelligent driving system according to claim 1 is characterized in that: The environmental data acquired by the sensor (11) includes at least image data, distance data and speed data. The sensor fusion module (1) further includes a weight allocation unit (12) for adjusting the image of the data of each sensor (11) on the fused environmental data through weights. The weight vector set by the weight allocation unit (12) is: In={in I ,In D ,In V } where w I represents the weight of the image data, w D represents the weight of the distance data, w V represents the weight of the speed data, w I 、w D 、w V The sum is 1; The sensor fusion module (1) further comprises a data fusion unit (13) for obtaining the fused environmental data by weighted fusion, the formula being: E=w I ·I+w D ·D+w V ·V Wherein, E represents the fused environment data, I represents the image data, D represents the distance data, and V represents the speed data.
3. The multi-source sensor fusion intelligent driving system according to claim 1 is characterized in that: The driving control module (3) comprises: A path planning unit (33) is connected to an external map system to obtain map data, and is used to perform path planning based on the map data and the driving environment information to obtain the driving path. The formula is: P = PathPlanning (M, S, G) Wherein P represents the driving path, M represents the map data, S represents the starting point, G represents the end point, and PathPlanning represents the path planning method obtained from the repository; A speed control unit (34) is connected to the path planning unit (33) and is used to generate the driving speed according to the driving environment information and the driving path, and the formula is: v target =min(v max ,max(v min ,v+a·Δt)) where v target represents the driving speed, v max Maximum speed, v min represents the minimum speed, a represents the acceleration, and Δt represents the control period.
4. The multi-source sensor fusion intelligent driving system according to claim 1 is characterized in that: The road detection information acquired by the environment perception module (2) includes at least road surface information, road sign information, lane line information and obstacle information.
5. The multi-source sensor fusion intelligent driving system according to claim 2 is characterized in that: The environment perception module (2) comprises a pedestrian perception unit (21) connected to the sensor fusion module (1) to obtain the pedestrian detection information, predict the pedestrian's movement path based on the pedestrian detection information, and the speed adjustment unit (31) adjusts the driving speed based on the pedestrian's movement path.
6. The multi-source sensor fusion intelligent driving system according to claim 5 is characterized in that: The volume adjustment unit (32) is also connected to the noise sensor (11) included in the sensor fusion module (1) to obtain noise data. The volume adjustment unit (32) calculates the horn volume according to the speed data and the noise data. The calculation formula is: V final =V base ×f speed (V)×f noise (N) Among them, V final Indicates the speaker volume, V base Indicates the basic volume, f speed represents the vehicle speed correction coefficient, V represents the speed data, f noise represents the noise correction coefficient, and N represents the noise data.
7. The multi-source sensor fusion intelligent driving system according to claim 6 is characterized in that: The vehicle speed correction factor f speed The calculation formula is, f speed (V)=1+γ·V Wherein V represents the speed data, and γ represents the vehicle speed correction parameter; The noise correction factor f noise The calculation formula is, Wherein, β represents the noise correction parameter, N represents the noise data, and N max Indicates the preset maximum noise level.
8. The multi-source sensor fusion intelligent driving system according to claim 6, characterized in that: The pedestrian sensing unit (21) further identifies pedestrian age data based on the pedestrian detection information. When the pedestrian age data exceeds a preset age threshold, the volume adjustment unit (32) adjusts the basic volume based on the pedestrian age data and the distance data. The formula is: Among them, V base represents the basic volume, D represents the distance data, and k represents the age coefficient.
9. The multi-source sensor fusion intelligent driving system according to claim 8, characterized in that: The pedestrian sensing unit (21) is connected to the sensor (11) to receive the image data. The pedestrian sensing unit (21) is provided with an age analysis subunit. The age analysis subunit identifies pedestrian features in the image data based on a target detection method, and calculates the pedestrian age data according to the pedestrian feature matching. The pedestrian sensing unit (21) is provided with an age coefficient comparison library. The pedestrian sensing unit (21) matches the age coefficient from the age coefficient comparison library according to the pedestrian age data.
10. The multi-source sensor fusion intelligent driving system according to claim 9, characterized in that: The pedestrian feature identified by the age analysis subunit is at least one of a motion feature, a gait feature, a clothing feature, and a speed feature.
