Vehicle collision early warning analysis system based on multi-source data fusion analysis

Through a multi-source data fusion analysis system, multiple visual cameras are used to collect and analyze road images, extract the moving characteristics of objects, predict vehicle collision risks, solve the problem of mis-association in the existing technology, and improve the safety of autonomous driving.

CN119091685AInactive Publication Date: 2024-12-06WUXI UNIV
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
CN202411289787.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-14
Publication Date
2024-12-06
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In the existing autonomous driving technology, the use of cloud lattice model for road information calculation is prone to incorrect correlation, resulting in inaccurate analysis of vehicle collision warnings.

Method used

Using a multi-source data fusion analysis system, road images are collected through multiple visual cameras, the moving characteristics of objects in the road images are extracted and analyzed, the movement vectors of target objects and vehicles are calculated, the chance of collision is predicted, and the braking process is performed through the vehicle control module. .

Benefits of technology

It improves data reliability and safety during vehicle driving, and accurately predicts the collision risks between cars and other objects through multi-source data fusion and two-dimensional map analysis to ensure driving safety.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091685A_ABST
    Figure CN119091685A_ABST
Patent Text Reader

Abstract

The invention discloses a vehicle collision early warning analysis system based on multi-source data fusion analysis. According to the method, the positions of the same target object under different time sequences of the images collected by the plurality of visual cameras are analyzed to obtain the position information of the plurality of objects under different time sequences, the movement of the plurality of objects is converted into the movement vector, the movement vector distance between the vehicle and the plurality of objects is analyzed in the two-dimensional map, and the distance between the vehicle and the plurality of objects is calculated. Early warning analysis can be effectively carried out on automobile collision in time, and the safety of the automobile in the driving process is guaranteed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of autonomous driving and relates to a vehicle collision warning analysis system based on multi-source data fusion analysis. Background Art

[0002] Currently, autonomous driving has become a hot topic in society, and various companies are increasing their investment to put autonomous driving technology into mass production.

[0003] Currently, car companies use cloud lattice models to calculate road information. Building models to provide autonomous driving data can easily lead to misassociations. Summary of the invention

[0004] In order to solve the problems existing in the background technology, the present invention proposes a vehicle collision warning analysis system based on multi-source data fusion analysis.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] A visual camera, wherein a sensor is installed on the visual camera;

[0007] A data acquisition module, using two or more visual cameras to capture road images, acquires the road images captured by the visual cameras, and acquires sensor information on the visual cameras;

[0008] A data processing module is used to fuse multiple road image data and process the fused data, including extracting the moving features of any object in the road image and analyzing and calculating the moving features of any object in the road image to obtain the intersection of the moving vector of the target object and the driving vector of the vehicle;

[0009] The vehicle control module controls the vehicle according to the data analysis and processing results of the data processing module.

[0010] Furthermore, the plurality of visual cameras are located on both sides of the longitudinal vehicle body and distributed on the same horizontal plane, and are also symmetrically distributed about the central axis of the vehicle body;

[0011] Furthermore, the moving feature extraction of any object in the road image is performed by analyzing the road image through the internal and external parameters when the visual camera collects the road image. The internal and external parameters are:

[0012] The visual camera intrinsic matrix is ​​set as:

[0013]

[0014] where f x and f y is the focal length in different directions, c x and cy is the displacement coordinate of the pixel point in different directions;

[0015] The visual camera extrinsic parameter matrix is ​​set as:

[0016] RT

[0017] Where R is the rotation matrix and T is the displacement vector.

[0018] Furthermore, the specific method of extracting the moving features of any object in the road image is as follows:

[0019] Identify the pixel coordinates (u, v) of a point in the road image and use the intrinsic parameters to convert the pixel coordinates to normalized visual camera coordinates:

[0020]

[0021] Among them, K -1 is the inverse matrix of the internal parameter matrix;

[0022] Get the depth information of the road image, combine the normalized coordinates with the depth value, and get the 3D visual camera coordinates (x, y, z):

[0023]

[0024] Where d is the depth value;

[0025] Convert the 3D coordinates in the visual camera coordinate system to the position in the world coordinate system:

[0026]

[0027] Among them, (X w ,Y w ,Z w ) are coordinates in the world coordinate system, R is the rotation matrix, and T is the displacement vector.

[0028] Furthermore, feature content is extracted through distance features between the same target object and the vehicle and moving distance features of the target object in multiple frames of images in a time series, the moving vector of the target object is analyzed, and the collision probability between the movement of the target object and the movement of the car is predicted.

[0029] Furthermore, the motion vector of the target object is calculated as follows:

[0030] Obtain the target object moving position changes collected in multiple time series, including the distance between the target object and the vehicle and the moving angle of the target object;

[0031] The target object movement vectors under multiple time sequences are added together to obtain the final target object movement vector.

[0032] Furthermore, the method for predicting the collision probability between the movement of the target object and the movement of the car is:

[0033] Set up a two-dimensional plane, with the location of the car as the origin coordinate, the target object movement as the variable, and the distance between the straight line where the final vector of the target object movement is located and the origin as the judgment basis;

[0034] Set a safety threshold. If the distance between the line where the final vector of the target object moves and the origin is less than the safety threshold, send a braking command to the vehicle control module.

[0035] If the distance between the line where the final moving vector of the target object is located and the origin is greater than the safety threshold, no processing is performed.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] The present invention coordinates the work of multiple cameras to reliably guarantee the data source during vehicle driving. At the same time, the present invention constructs a two-dimensional map to analyze the movement vectors of all objects in the road image, and predicts the vehicle's anti-collision through a vector model, thereby ensuring the safety of the vehicle during driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a schematic diagram of the system architecture of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] like Figure 1 As shown, the technical solution adopted by the present invention is as follows: a vehicle collision warning analysis system based on multi-source data fusion analysis, comprising:

[0041] The data acquisition module uses two or more visual cameras to capture road images and acquires the road images captured by the visual cameras.

[0042] The data processing module fuses multiple road image data and processes the fused data, including extracting the moving features of any object in the road image and analyzing and calculating the moving features of any object in the road image to obtain the intersection of the moving vector of the target object and the driving vector of the vehicle.

[0043] The vehicle control module controls the vehicle according to the data analysis and processing results of the data processing module.

[0044] The data acquisition module uses two or more visual cameras to capture road images and obtain road image data captured by the cameras. When the continuity of the road image captured by a single camera cannot meet the system analysis requirements, the images captured by other cameras can be displaced and deflected to achieve image overlap and unify the road images captured by different cameras. This ensures the continuity of road data capture and reduces the situation where a single camera is missing data due to factors such as occlusion by other vehicles.

[0045] Multiple visual cameras are at the same height and on the same horizontal plane. There are an even number of visual cameras, which are symmetrically arranged on both sides of the vehicle body with the center axis of the vehicle body as the center line. The data captured by multiple cameras are guaranteed to be similar, and any data mark can appear in the images captured by at least two cameras at the same time without obstruction.

[0046] The data processing module fuses multiple road image data and processes the fused data. This includes extracting the moving features of any object in the road image and analyzing and calculating the moving features of any object in the road image to obtain the intersection of the moving vector of the target object and the driving vector of the vehicle. Based on the road images obtained by the analysis data acquisition module, early warning of future vehicle collisions is provided.

[0047] The moving feature extraction of any object in the road image is performed by analyzing the road image through the internal and external parameters when the visual camera collects the road image. The matrix of the internal and external parameters of the visual camera is shown below.

[0048] The changes in the internal and external parameters of the visual camera are sent to the data processing module in real time. The external parameters of the visual camera are obtained through sensors, and the internal parameters of the visual camera are obtained through the built-in application of the visual camera.

[0049] The visual camera intrinsic matrix is ​​set as:

[0050]

[0051] where f x is the focal length of the road image in the horizontal direction, f y is the focal length of the road image in the longitudinal direction, c x is the displacement coordinate of the pixel point in the x-axis direction, c y is the displacement coordinate of the pixel point in the y-axis direction.

[0052] The visual camera extrinsic parameter matrix is ​​set as:

[0053] [RT]

[0054] Where R is the rotation matrix and T is the displacement vector.

[0055] The specific method of extracting the moving features of any object in the road image by using the internal and external parameters when the visual camera collects the road image is as follows:

[0056] Identify the pixel coordinates (u, v) of a point in the road image and use the intrinsic parameters to convert the pixel coordinates to normalized visual camera coordinates:

[0057]

[0058] Among them, K -1 is the inverse matrix of the internal parameter matrix;

[0059] It is necessary to obtain the depth information of the road image, combine the normalized coordinates with the depth value, and obtain the three-dimensional visual camera coordinates (x, y, z):

[0060]

[0061] Where d is the depth value;

[0062] Convert the 3D coordinates in the visual camera coordinate system to the position in the world coordinate system:

[0063]

[0064] Among them, (X w ,Y w ,Z w ) is the coordinate in the world coordinate system, R is the rotation matrix, and T is the displacement vector. Based on the above calculation method, the position of a single pixel is obtained. By analyzing multiple road images taken in a certain time sequence, any pixel of an object is used as an anchor point, and the spatial position change of the anchor point is extracted to obtain the distance characteristics of the object from the vehicle in the time sequence and the moving distance characteristics of the target object.

[0065] The feature content is extracted by analyzing the moving vector of the target object through the distance features between the same target object and the vehicle and the moving distance features of the target object in multiple frame images under time series, and predicting the collision probability between the movement of the target object and the movement of the car.

[0066] The motion vector of the target object is calculated as:

[0067] Obtain the target object moving position changes collected in multiple time series, including the distance between the target object and the vehicle and the target object moving angle.

[0068] The target object movement vectors under multiple time sequences are added together to obtain the final target object movement vector.

[0069] The method to predict the collision probability of the target object movement and the car movement is:

[0070] Set up a two-dimensional plane, with the location of the car as the origin coordinate, the target object movement as the variable, and the distance between the straight line where the final vector of the target object movement is located and the origin as the judgment basis;

[0071] Set a safety threshold. If the distance between the line where the final vector of the target object moves and the origin is less than the safety threshold, send a braking command to the vehicle control module.

[0072] If the distance between the line where the final moving vector of the target object is located and the origin is greater than the safety threshold, no processing is performed.

[0073] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A vehicle collision warning analysis system based on multi-source data fusion analysis, It is characterized in that Included are: A visual camera, wherein a sensor is installed on the visual camera; A data acquisition module, using two or more visual cameras to capture road images, acquires the road images captured by the visual cameras, and acquires sensor information on the visual cameras; A data processing module is used to fuse multiple road image data and process the fused data, including extracting the moving features of any object in the road image and analyzing and calculating the moving features of any object in the road image to obtain the intersection of the moving vector of the target object and the driving vector of the vehicle; The vehicle control module controls the vehicle according to the data analysis and processing results of the data processing module.

2. A vehicle collision warning analysis system based on multi-source data fusion analysis according to claim 1, Features ,Multiple visual cameras are located on both sides of the longitudinal vehicle body and are distributed on the same horizontal plane, and at the same time they are symmetrically distributed about the center axis of the vehicle body.

3. A vehicle collision warning analysis system based on multi-source data fusion analysis according to claim 1, Features ,The moving feature extraction of any object in the road image is used to analyze the ,road image through the internal and external parameters when the ,visual camera collects the road image. The internal and external parameters are: The visual camera intrinsic matrix is ​​set as: where f x and f y is the focal length in different directions, c x and c y is the displacement coordinate of the pixel point in different directions; The visual camera extrinsic parameter matrix is ​​set as: [RT] Where R is the rotation matrix and T is the displacement vector.

4. A vehicle collision warning analysis system based on multi-source data fusion analysis according to claim 3, Features ,The specific method of extracting the moving features of any object in the ,road image is as follows: Identify the pixel coordinates (u, v) of a point in the road image and use the intrinsic parameters to convert the pixel coordinates to normalized visual camera coordinates: Among them, K -1 is the inverse matrix of the internal parameter matrix; Get the depth information of the road image, combine the normalized coordinates with the depth value, and get the 3D visual camera coordinates (x, y, z): Where d is the depth value; Convert the 3D coordinates in the visual camera coordinate system to the position in the world coordinate system: Among them, (X w ,Y w ,Z w ) are coordinates in the world coordinate system, R is the rotation matrix, and T is the displacement vector.

5. The vehicle collision warning analysis system based on multi-source data fusion analysis according to claim 1, Features ,The feature content is extracted by analyzing the moving vector of the target object and the ,distance features between the same target object and the vehicle and the moving distance features of ,the target object and the collision probability of the target object and the ,vehicle in multiple frame images under the time series.

6. A vehicle collision warning analysis system based on multi-source data fusion analysis according to claim 1 or 5, Features ,The calculation method of the moving vector of the target object is: Obtain the target object moving position changes collected in multiple time series, including the distance between the target object and the vehicle and the moving angle of the target object; The target object movement vectors under multiple time sequences are added together to obtain the final target object movement vector.

7. A vehicle collision warning analysis system based on multi-source data fusion analysis according to claim 6, Features ,The method to predict the collision probability of the target object movement and the car movement is: Set up a two-dimensional plane, with the location of the car as the origin coordinate, the target object movement as the variable, and the distance between the straight line where the final vector of the target object movement is located and the origin as the judgment basis; Set a safety threshold. If the distance between the line where the final vector of the target object moves and the origin is less than the safety threshold, send a braking command to the vehicle control module. If the distance between the line where the final moving vector of the target object is located and the origin is greater than the safety threshold, no processing is performed.

Citation Information

Patent Citations

  • Multi-view automobile all-around auxiliary driver assistant system and method

    CN108765496A

  • Vehicle anti-collision sensing algorithm based on computer vision

    CN114219837A

  • Collision early warning method, equipment, storage medium and device

    CN114463708A

  • Automobile blind area detection system based on camera

    CN115527184A

  • Driving early warning method, device and equipment and storage medium

    CN116486351A