Road hidden danger positioning method and device based on camera and radar

Abstract

The invention discloses a road hidden danger positioning method and device based on a camera and a radar, and the method comprises the steps: respectively obtaining the road image data collected by the camera and the laser point cloud data collected by the laser radar when an obstacle enters a monitored road region; performing spatial synchronization processing on the road image data and the laser point cloud data to respectively obtain synchronous image data and synchronous point cloud data; and after the synchronous image data and the synchronous point cloud data are fused, hidden danger positioning processing is carried out on the fused data by adopting a DBSCAN algorithm to obtain a road hidden danger position. According to the method, the road image data and the laser point cloud data can be respectively acquired through the camera and the laser radar, the two data are synchronized and then fused, and finally, the fused data are clustered to determine the hidden danger positioning position, so that the method is small in processing data volume, can improve the processing efficiency, and is suitable for popularization and application. And the two kinds of data can be integrated for positioning processing, so that the positioning precision is improved.

Description

Background Art

[0002] Road traffic safety hazards are one of the important risk sources for public safety. In order to reduce the probability of traffic accidents, it is necessary to detect and eliminate safety hazards on the road to strengthen road traffic accident prevention work.

[0003] At present, the commonly used method for detecting road safety hazards is to conduct manual inspections first. After detecting places where hidden dangers may exist, lidar is used to obtain the spatial position coordinates of the hidden dangers, thereby locating the exact location of the hidden dangers.

[0004] However, the commonly used methods currently have the following technical problems: manual patrol operations are time-consuming and inefficient; and the use of a single laser radar for positioning is easily affected by various reflected light, thereby reducing the accuracy of positioning and resulting in poor positioning results. Summary of the invention

[0005] The present invention proposes a road hidden danger positioning method and device based on camera and radar. The method can obtain road image data and laser point cloud data respectively through a camera and a laser radar, perform data fusion after the two data are synchronized, and then determine the hidden danger positioning position based on the fused data, which can improve both processing efficiency and positioning accuracy.

[0006] A first aspect of an embodiment of the present invention provides a method for locating road hazards based on a camera and a radar, the method comprising:

[0007] When it is determined that an obstacle has entered the monitored road area, the road image data collected by the camera and the laser point cloud data collected by the laser radar are obtained respectively;

[0008] Performing spatial synchronization processing on the road image data and the laser point cloud data to obtain synchronized image data and synchronized point cloud data respectively;

[0009] After fusing the synchronized image data and the synchronized point cloud data, the DBSCAN algorithm is used to perform hidden danger location processing on the fused data to obtain the road hidden danger position.

[0010] In a possible implementation manner of the first aspect, the fusing the synchronized image data and the synchronized point cloud data includes:

[0011] Converting the spatial coordinates of the synchronized point cloud data into image coordinates to obtain converted point cloud data;

[0012] According to the bounding box of the obstacle contained in the synchronous image data, the point cloud data within the bounding box is filtered from the converted point cloud data to complete the fusion.

[0013] In a possible implementation manner of the first aspect, after the step of respectively obtaining the synchronized image data and the synchronized point cloud data, the method further includes:

[0014] Clustering the synchronized point cloud data using the DBSCAN algorithm to obtain obstacle coordinates;

[0015] Clustering the synchronized image data according to the obstacle coordinates to obtain detection data of the obstacles in the image;

[0016] After fusing the detection data with the synchronized point cloud data, the DBSCAN algorithm is used to perform hidden danger location processing on the fused data to obtain the road hidden danger location.

[0017] In a possible implementation manner of the first aspect, the hidden danger location processing includes:

[0018] The DBSCAN algorithm is used to cluster the point cloud of the fused data according to the preset clustering conditions to obtain multiple clusters, wherein each cluster represents a hidden danger target, and the preset clustering conditions include the field radius and density threshold of the hidden danger target;

[0019] Selecting a cluster with the largest number of point clouds from the multiple clusters as a target cluster;

[0020] The spatial position of the target cluster is obtained to obtain the road hazard position.

[0021] In a possible implementation manner of the first aspect, the density threshold of the target cluster is expressed as follows:

[0022]

[0023] Among them, ε s The density threshold of the target cluster, α is the vertical angle of the laser radar to the object S corresponding to the target cluster, β is the angle interval of the laser radar in the horizontal direction, D is the number of point clouds of the target cluster, L s is the distance from point S to the laser radar.

[0024] In a possible implementation manner of the first aspect, after the step of respectively obtaining the synchronized image data and the synchronized point cloud data, the method further includes:

[0025] Respectively acquiring coordinate systems corresponding to the synchronized image data and the synchronized point cloud data, and jointly calibrating the two coordinate systems to obtain a calibrated coordinate system;

[0026] Converting the data about the obstacle in the synchronized point cloud data into an image according to the calibrated coordinate system to obtain an obstacle image;

[0027] Extracting frame parameters from the obstacle image and calculating frame information matching values ​​using the frame parameters, the frame parameters including: frame upper left corner coordinates and frame size;

[0028] If the frame information matching value is less than a preset matching value, it is determined that the synchronous image data and the synchronous point cloud data are synchronously matched successfully;

[0029] Otherwise, it is determined that the synchronization matching between the synchronized image data and the synchronized point cloud data fails.

[0030] In a possible implementation manner of the first aspect, determining that an obstacle enters the monitored road area includes:

[0031] Using a camera to collect two frames of continuous monitoring image data; and calculating the difference between the pixel values ​​corresponding to the two frames of monitoring image data to obtain an image difference;

[0032] If the image difference is greater than the preset difference, it is determined that there is an obstacle and the laser radar is called to detect the obstacle distance;

[0033] When the obstacle distance is within a preset safety distance threshold interval, it is determined that the obstacle has entered the monitored road area.

[0034] In a possible implementation manner of the first aspect, respectively acquiring the road image data collected by the camera and the laser point cloud data collected by the laser radar includes:

[0035] When the laser point cloud data is collected by the laser radar, the scanning time node of the laser radar is obtained;

[0036] Taking the scanning time node as the time reference, the camera is triggered to capture obstacle images to obtain road image data.

[0037] A second aspect of an embodiment of the present invention provides a road hazard positioning device based on a camera and a radar, the device comprising:

[0038] The data acquisition module is used to obtain the road image data collected by the camera and the laser point cloud data collected by the laser radar when it is determined that there is an obstacle entering the monitored road area;

[0039] A synchronization processing module, used for performing spatial synchronization processing on the road image data and the laser point cloud data to obtain synchronized image data and synchronized point cloud data respectively;

[0040] The fusion positioning module is used to fuse the synchronized image data and the synchronized point cloud data, and then use the DBSCAN algorithm to perform hidden danger positioning processing on the fused data to obtain the road hidden danger position.

[0041] Compared with the prior art, the embodiment of the present invention provides a road hazard positioning method and device based on camera and radar, and its beneficial effect is: the present invention can obtain road image data and laser point cloud data through a camera and a laser radar respectively, synchronously process the two data, and then fuse the two data after the synchronization is completed. Finally, the fused data is clustered to determine the hidden danger location. The entire operation process not only processes a small amount of data, which can improve processing efficiency, but also can integrate the two types of data for positioning processing to improve positioning accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a flowchart of a road hidden danger location method based on a camera and a radar provided by an embodiment of the present invention;

[0043] Figure 2 This is an operation flow chart of a road hidden danger location method based on a camera and a radar provided by an embodiment of the present invention;

[0044] Figure 3 It is a structural schematic diagram of a road hazard locating device based on a camera and a radar provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] In order to solve the above problems, a road hazard positioning method based on camera and radar provided in an embodiment of the present application will be introduced and explained in detail through the following specific embodiments.

[0047] Reference Figure 1 , which shows a flow chart of a method for locating road hazards based on a camera and a radar provided in one embodiment of the present invention.

[0048] In one embodiment, the method is applicable to a processing system that can communicate with a camera and a lidar respectively. The processing system can be set in an intelligent terminal of a road monitoring personnel so that the monitoring personnel can refer to the location of the hidden danger source.

[0049] Wherein, as an example, the road hidden danger positioning method based on camera and radar may include:

[0050] S11. When it is determined that an obstacle has entered the monitored road area, the road image data collected by the camera and the laser point cloud data collected by the laser radar are obtained respectively.

[0051] In one embodiment, it can be determined whether there are obstacles entering the road area monitored by the camera and the laser radar. Therefore, hidden dangers on the road are generally caused by obstacles. When there are no obstacles on the road, the road is flat and generally has no hidden dangers; when there are obstacles on the road, it may hinder vehicles from driving on the road, thereby causing safety hazards.

[0052] After determining that an obstacle has entered the monitored road area, the camera can be controlled to shoot the road to obtain road image data containing the obstacle, and the lidar can be controlled to emit laser to the obstacle to receive laser reflected by the obstacle to obtain laser point cloud data.

[0053] It should be noted that the image data collected by the camera includes image data of obstacles on the traffic route, for example, obstacles such as pedestrians, vehicles, and unidentified objects, as well as point cloud data of obstacles on the traffic route obtained by the laser radar.

[0054] By combining the road image data collected by the camera and the laser point cloud data collected by the lidar for comprehensive analysis and judgment, it is possible to accurately determine whether there are obstacles on the road and whether they cause hidden dangers, and at the same time accurately locate the specific location of the source of the hidden danger.

[0055] In order to accurately determine whether there is an obstacle entering the road or whether there is an obstacle on the road, as an example, step S11 may include the following sub-steps:

[0056] S111. Collect two frames of continuous monitoring image data using a camera; and calculate the difference between the pixel values ​​corresponding to the two frames of monitoring image data to obtain an image difference.

[0057] S112: If the image difference is greater than a preset difference, it is determined that there is an obstacle and the laser radar is called to detect the distance of the obstacle.

[0058] S113: When the obstacle distance is within a preset safety distance threshold interval, it is determined that the obstacle enters the monitored road area.

[0059] In one implementation, image data collected before and after a certain moment can be used to determine whether there is an external destructive target entering the traffic route. Specifically, a camera can be used to collect two frames of continuous monitoring image data, and then the image difference of the two frames of monitoring image data can be calculated. The image difference refers to the difference between two images collected before and after a certain moment, specifically, the value obtained by subtracting the corresponding pixel values ​​of the same band image. If the image difference is greater than the preset difference, it means that the two frames of images are different, and there may be an obstacle entering; if the image difference is less than the preset difference, it means that the two frames of images are the same, and there may be no obstacle entering.

[0060] Although an obstacle enters the monitored road area, the location of the obstacle may not affect the road, but the location of the obstacle may affect the road.

[0061] For accurate detection, the laser radar can also be called to emit laser to the obstacle, and then receive the laser reflected by the obstacle. The distance to the obstacle can be calculated by the time difference between emitting the laser and receiving the laser.

[0062] If the obstacle distance is within the preset safety distance threshold range, it is determined that the obstacle has entered the monitored road area, and the location of the obstacle is determined to be likely to affect the road.

[0063] On the contrary, if the obstacle distance is not within the preset safety distance threshold interval, it is determined that the obstacle has mistakenly entered the monitored road area, and it is determined that the location of the obstacle may not affect the road.

[0064] For example, you can set 0-2 meters as the safety distance threshold interval. If the obstacle distance is 5 meters, the obstacle distance is not within the preset safety distance threshold interval, and it is determined that the location of the obstacle may not affect the road. If the obstacle distance is 0.8 meters, the obstacle distance is within the preset safety distance threshold interval, and it is determined that the location of the obstacle may affect the road, and it can be determined that the obstacle enters the monitored road area.

[0065] When it is determined that an obstacle has entered the monitoring area, the camera and the laser radar need to be called to collect data. However, if the time points of the two trigger operations are different, the collected data may be biased, which will lead to errors in subsequent positioning detection. In order to allow the camera and the laser radar to perform data collection actions synchronously, the camera and the laser radar can be used at different times. As an example, step S11 may include the following sub-steps:

[0066] S114. When the laser radar collects laser point cloud data, obtain the scanning time node of the laser radar.

[0067] S115 , using the scanning time node as a time reference, triggering a camera to capture an image of an obstacle to obtain road image data.

[0068] In one implementation, since the frequency of laser radar scanning is relatively stable, the camera and laser radar can be controlled to operate synchronously based on the time information of the detected obstacle, that is, based on the scanning time of the laser radar.

[0069] Specifically, after determining that an obstacle has entered the monitored road area, the laser radar can be called to collect laser point cloud data about the obstacle, and the scanning time node when the laser radar performs this operation can be obtained. The scanning time node is used as the trigger time node to simultaneously trigger the camera to collect image data, thereby ensuring the consistency of the time of the two.

[0070] Synchronizing the specific time frames acquired by the camera and the lidar can avoid the problem of time difference in data caused by the time when the camera and the lidar collect data is not synchronized, thereby improving the accuracy of subsequent calculations and detection.

[0071] S12, performing spatial synchronization processing on the road image data and the laser point cloud data to obtain synchronized image data and synchronized point cloud data respectively.

[0072] After completing the time synchronization, the road image data and the laser point cloud data can be processed for spatial synchronization. The spatial synchronization is to allow the two data to locate hidden dangers at the same latitude to improve the positioning accuracy.

[0073] After synchronous processing of the two data, the road image data can be converted into synchronized image data, and the laser point cloud data can be converted into synchronized point cloud data.

[0074] In one embodiment, the synchronization method may be to match the spatial coordinates of two data.

[0075] Specifically, the coordinate systems of the road image data and the laser point cloud data may be jointly calibrated so that the coordinates of the two data are the same.

[0076] After completing the joint calibration of the coordinate system, it is necessary to detect whether the two data have completed spatial synchronization and determine whether there is a deviation to avoid using the deviated data for positioning. In an optional embodiment, after step S12, the method may further include:

[0077] S21. Obtain coordinate systems corresponding to the synchronized image data and the synchronized point cloud data respectively, and jointly calibrate the two coordinate systems to obtain a calibrated coordinate system.

[0078] S22. Convert the data about the obstacle in the synchronized point cloud data into an image according to the calibrated coordinate system to obtain an obstacle image.

[0079] S23, extracting frame parameters from the obstacle image, and calculating frame information matching values ​​using the frame parameters, wherein the frame parameters include: coordinates of the upper left corner of the frame and the size of the frame.

[0080] S24: If the frame information matching value is less than a preset matching value, it is determined that the synchronization image data and the synchronization point cloud data are successfully synchronized and matched.

[0081] S25. Otherwise, it is determined that the synchronization matching between the synchronized image data and the synchronized point cloud data fails.

[0082] Specifically, after completing the spatial synchronization, the coordinates of the synchronized image data and the synchronized point cloud data can be assumed to be the same, and then the coordinates corresponding to the synchronized point cloud data can be obtained as the laser radar coordinate system, and then the obstacle data in the laser radar coordinate system is obtained and converted to the image to obtain the obstacle image. The bounding box parameters of the converted obstacle are then extracted from the obstacle image, where the bounding box parameters may include: the coordinates of the upper left corner of the bounding box and the size of the bounding box.

[0083] Then, the frame information matching value is calculated using the frame parameters. If the frame information matching value is less than the preset matching value, it is determined that the spatial synchronization of the synchronized image data and the synchronized point cloud data is completed and the synchronization matching is successful. If the frame information matching value is greater than or equal to the preset matching value, it is determined that the spatial synchronization of the synchronized image data and the synchronized point cloud data is not completed and the synchronization matching fails.

[0084] Optionally, the border information matching value may be calculated using a preset formula. For example, the distance from the coordinates of the upper left corner of the border to a vertex of the border may be calculated, and then the ratio of the distance to the border size may be calculated to obtain the border information matching value.

[0085] Alternatively, the distance from the coordinates of the upper left corner of the frame to the midpoint of the frame may be calculated, and then the ratio of the distance to the frame size may be calculated to obtain the frame information matching value. The specific calculation method may be adjusted according to actual needs.

[0086] In addition, there may be deviations in both time synchronization and space synchronization, indicating that there may be problems with the hardware of the camera and the lidar, and there may be a certain delay in the hardware when collecting data. In order to correct the delay of the hardware, the present invention may optionally further include the following operations:

[0087] First, the trigger time of each frame of the camera is recorded, and a certain correction is made to its hardware delay. At the same time, the difference algorithm is used to compare the image difference between the previous and next frames to detect whether the difference value is greater than the set threshold. If it is determined that a hidden danger source has appeared, the time at this time is recorded; the time base of the laser radar is provided by the host computer, and the time of each frame of the laser point cloud is recorded in a time indexed manner. The data of the point cloud frame is located by detecting the image time of the hidden danger source; spatial synchronization involves the conversion of the laser radar coordinate system and the image coordinate system. The specific conversion is: assuming that the coordinates of a target point in the environment in the laser radar are J L , the coordinate in the camera is J C , in the image it is J S , Z is the z-axis coordinate in the camera, K is the camera internal parameter matrix, then the conversion formula is:

[0088] In the process of spatial synchronization of road image data and laser point cloud data, the key point is to obtain the camera's intrinsic parameter matrix K and the external parameter conversion matrix from the laser radar to the camera coordinates. The camera's intrinsic parameter matrix K is calibrated using the plane calibration mode. The principle is to use a controllable movement method to move the target in the workspace. The external parameter matrix K of the laser radar to the camera is It is calculated by selecting common viewpoints in the image and point cloud and by optimally estimating the matrix.

[0089] S13. After fusing the synchronized image data and the synchronized point cloud data, a DBSCAN algorithm is used to perform hidden danger location processing on the fused data to obtain the road hidden danger location.

[0090] After the synchronization is completed, the two data can be fused, and then the DBSCAN algorithm can be used to locate hidden dangers on the fused data, and the exact location of the road hidden danger can be determined through the two data.

[0091] In order to merge the two data together, in one embodiment, step S13 may include the following sub-steps:

[0092] S131 . Convert the spatial coordinates of the synchronized point cloud data into image coordinates to obtain converted point cloud data.

[0093] S132: Filter point cloud data within the frame of the obstacle contained in the synchronized image data from the converted point cloud data to complete fusion.

[0094] Specifically, in one implementation, the fusion operation may refer to fusing the synchronized point cloud data and multiple obstacle data in the synchronized image data, eliminating unnecessary point cloud data, and thus improving data processing efficiency.

[0095] It should be noted that if there are multiple obstacles entering the area, the collected road image data may contain multiple obstacles. At the same time, the point cloud data collected by the lidar may contain multiple obstacle targets. The regional center point cloud data set of the extracted multiple targets can be fused with the multiple obstacle data in the image data.

[0096] Specifically, the spatial coordinates of the synchronized point cloud data set at the center of the region can be transformed into image coordinates through the conversion of spatial coordinates. Point clouds outside the target frame are removed according to the image, and only point clouds within the frame are retained, and the retained point cloud data is used as the fused data. By screening and extracting, the amount of point cloud data that needs to be processed can be greatly reduced, thereby completing the fusion of point cloud and image data.

[0097] The present invention reduces the amount of point cloud data calculation by fusing image data with point cloud data, thereby achieving better real-time positioning effect.

[0098] The hidden danger location processing includes:

[0099] S31. Cluster the point cloud of the fused data according to preset clustering conditions using the DBSCAN algorithm to obtain multiple clusters, wherein each cluster represents a hidden danger target, and the preset clustering conditions include the area radius and density threshold of the hidden danger target.

[0100] S32: Filter the cluster with the largest number of point clouds from the multiple clusters as the target cluster.

[0101] S33: Obtain the spatial position of the target cluster to obtain the road hazard position.

[0102] In one implementation, since the fused data only contains the point cloud of the obstacle border, the obstacle can be used as the target border, and the DBSCAN algorithm can be used to cluster the point cloud within the target border. By setting the field radius and density threshold of the hidden danger source target, a sufficient number of point clouds in the area can be clustered into clusters. Each cluster represents an obstacle hidden danger target, thereby obtaining multiple clustering clusters.

[0103] It should be noted that the advantage of this algorithm is that it can cluster data of any shape and is insensitive to noise points. Each cluster is clustered into one result.

[0104] After obtaining multiple clusters, the clustering result with the largest number of point clouds within the target border can be selected as the true target point cloud, and then the point cloud cluster center is taken as the spatial position of the target, thereby realizing the positioning of the target.

[0105] In addition, since the collected data is determined when it is collected, it can be combined with a time reference to achieve real-time accurate positioning.

[0106] In one embodiment, the density threshold of the target cluster is as follows:

[0107]

[0108] Among them, ε S The density threshold of the target cluster, α is the vertical angle of the laser radar to the object S corresponding to the target cluster, β is the angle interval of the laser radar in the horizontal direction, D is the number of point clouds of the target cluster, L s is the distance from point S to the laser radar.

[0109] In actual application, the road image data includes both image data of obstacles and image data of non-obstacles. In addition, the road image data may include image data of multiple obstacles, which results in a very large data capacity of the road image data and a long time for subsequent processing, thereby reducing processing efficiency.

[0110] In order to reduce the capacity of the image data, in an optional embodiment, after step S12, the method may further include:

[0111] S41. Clustering the synchronized point cloud data using the DBSCAN algorithm to obtain obstacle coordinates.

[0112] S42: Clustering the synchronized image data according to the obstacle coordinates to obtain detection data of the obstacles in the image.

[0113] S43, after fusing the detection data with the synchronized point cloud data, using the DBSCAN algorithm to perform hidden danger location processing on the fused data to obtain the road hidden danger location.

[0114] Specifically, in one implementation, when an obstacle enters the monitored road area, and the distance of the obstacle is within the preset safety distance threshold interval, the DBSCAN algorithm can be used to cluster the point cloud data of the obstacle within the safety distance threshold interval, and the midpoint coordinates of the horizontal line of the obstacle closest to the laser radar are extracted as the coordinates (x, y) of the obstacle. Then, the coordinate system is jointly calibrated and converted to the image, the point cloud in the image area is clustered, and the hidden danger source target in the image is determined, so that the detection data of the obstacle in the image can be obtained. At this time, the image data only retains the data of the obstacle. According to the operation method of step S13, after fusing the detection data with the synchronized point cloud data, the DBSCAN algorithm can be used to perform hidden danger location processing on the fused data to obtain the road hidden danger location. The above-mentioned fusion processing and hidden danger location processing can refer to the contents disclosed in the above-mentioned embodiments. In order to avoid repetition, they will not be repeated here.

[0115] Since the amount of data that needs to be processed for fusion and hidden danger location processing is greatly reduced, the processing time can be effectively shortened and data efficiency can be improved.

[0116] In addition, while determining the hidden danger source target in the image, the distance between the obstacle and the laser radar is also determined.

[0117] Then obtain the time when the data was collected, and perform accurate real-time positioning through time and distance.

[0118] Reference Figure 2 , shows an operation flow chart of a road hazard positioning method based on camera and radar provided by an embodiment of the present invention.

[0119] Specifically, the operation of the road hidden danger location method based on camera and radar may include the following steps:

[0120] The first step is to call the camera to collect image data and the lidar to collect point cloud data.

[0121] The second step is to perform time and space synchronization processing on the image data and point cloud data.

[0122] The third step is to fuse the image data and the point cloud data; alternatively, perform target image detection on the image data to obtain detection data, and then fuse the detection data and the point cloud data.

[0123] The fourth step is to perform point cloud clustering on the fused data within the target range to obtain the real-time positioning of the target obstacle.

[0124] The above operation is based on the fusion of 3D laser radar point cloud data and camera image information. Through the real-time positioning of the laser radar with adaptive spatial clustering algorithm, it can quickly and accurately identify the external targets entering the area, locate the targets in real time, and realize multi-target real-time tracking. It can effectively track multiple targets in a cluttered environment.

[0125] In this embodiment, the embodiment of the present invention provides a road hazard positioning method based on camera and radar, and its beneficial effects are: the present invention can obtain road image data and laser point cloud data through a camera and a lidar respectively, synchronously process the two data, fuse the two data after the synchronization is completed, and finally cluster the fused data to determine the hidden danger location. The entire operation process not only processes a small amount of data, which can improve processing efficiency, but also can integrate the two types of data for positioning processing to improve positioning accuracy.

[0126] The embodiment of the present invention also provides a road hidden danger positioning device based on a camera and a radar, see Figure 3 , showing a schematic structural diagram of a road hazard positioning device based on a camera and a radar provided in one embodiment of the present invention.

[0127] Wherein, as an example, the road hidden danger positioning device based on camera and radar may include:

[0128] The data acquisition module 301 is used to acquire the road image data acquired by the camera and the laser point cloud data acquired by the laser radar when it is determined that an obstacle has entered the monitored road area;

[0129] A synchronization processing module 302 is used to perform spatial synchronization processing on the road image data and the laser point cloud data to obtain synchronized image data and synchronized point cloud data respectively;

[0130] The fusion positioning module 303 is used to fuse the synchronized image data and the synchronized point cloud data, and then use the DBSCAN algorithm to perform hidden danger positioning processing on the fused data to obtain the road hidden danger position.

[0131] Optionally, the fusion positioning module is further used to:

[0132] Converting the spatial coordinates of the synchronized point cloud data into image coordinates to obtain converted point cloud data;

[0133] According to the bounding box of the obstacle contained in the synchronous image data, the point cloud data within the bounding box is filtered from the converted point cloud data to complete the fusion.

[0134] Optionally, the device further comprises:

[0135] A point cloud clustering module, used to cluster the synchronized point cloud data using a DBSCAN algorithm to obtain obstacle coordinates;

[0136] An obstacle detection module, used to cluster the synchronized image data according to the obstacle coordinates to obtain detection data of obstacles in the image;

[0137] The target detection module is used to fuse the detection data with the synchronized point cloud data, and then use the DBSCAN algorithm to perform hidden danger location processing on the fused data to obtain the road hidden danger location.

[0138] Optionally, the hidden danger location processing includes:

[0139] The DBSCAN algorithm is used to cluster the point cloud of the fused data according to the preset clustering conditions to obtain multiple clusters, wherein each cluster represents a hidden danger target, and the preset clustering conditions include the field radius and density threshold of the hidden danger target;

[0140] Selecting a cluster with the largest number of point clouds from the multiple clusters as a target cluster;

[0141] The spatial position of the target cluster is obtained to obtain the road hazard position.

[0142] Optionally, the density threshold of the target cluster is as follows:

[0143]

[0144] Among them, ε S The density threshold of the target cluster, α is the vertical angle of the laser radar to the object S corresponding to the target cluster, β is the angle interval of the laser radar in the horizontal direction, D is the number of point clouds of the target cluster, L s is the distance from point S to the laser radar.

[0145] Optionally, the device further comprises:

[0146] A coordinate system calibration module, used to respectively obtain the coordinate systems corresponding to the synchronized image data and the synchronized point cloud data, and jointly calibrate the two coordinate systems to obtain a calibrated coordinate system;

[0147] An image conversion module, used to convert the data about obstacles in the synchronized point cloud data into an image according to the calibration coordinate system to obtain an obstacle image;

[0148] A parameter extraction module is used to extract frame parameters from the obstacle image and calculate the frame information matching value using the frame parameters, wherein the frame parameters include: the coordinates of the upper left corner of the frame and the size of the frame;

[0149] A matching success determination module, configured to determine that the synchronous image data and the synchronous point cloud data are successfully matched if the frame information matching value is less than a preset matching value;

[0150] The matching failure determination module is used to determine, on the contrary, that the synchronous matching of the synchronous image data and the synchronous point cloud data has failed.

[0151] Optionally, the determining that an obstacle enters the monitored road area includes:

[0152] Using a camera to collect two frames of continuous monitoring image data; and calculating the difference between the pixel values ​​corresponding to the two frames of monitoring image data to obtain an image difference;

[0153] If the image difference is greater than the preset difference, it is determined that there is an obstacle and the laser radar is called to detect the obstacle distance;

[0154] When the obstacle distance is within a preset safety distance threshold interval, it is determined that the obstacle has entered the monitored road area.

[0155] Optionally, the data collection module is further used to:

[0156] When the laser point cloud data is collected by the laser radar, the scanning time node of the laser radar is obtained;

[0157] Taking the scanning time node as the time reference, the camera is triggered to capture obstacle images to obtain road image data.

[0158] Those skilled in the art can clearly understand that, for the sake of convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0159] Furthermore, an embodiment of the present application also provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the road hazard positioning method based on a camera and a radar as described in the above embodiment is implemented.

[0160] Furthermore, an embodiment of the present application also provides a computer-readable storage medium, which stores a computer-executable program, and the computer-executable program is used to enable a computer to execute the road hazard positioning method based on cameras and radars as described in the above embodiments.

[0161] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A road hidden danger location method based on camera and radar, characterized in that: The method comprises: When it is determined that an obstacle has entered the monitored road area, the road image data collected by the camera and the laser point cloud data collected by the laser radar are obtained respectively; Performing spatial synchronization processing on the road image data and the laser point cloud data to obtain synchronized image data and synchronized point cloud data respectively; After fusing the synchronized image data and the synchronized point cloud data, the DBSCAN algorithm is used to perform hidden danger location processing on the fused data to obtain the road hidden danger position.

2. The method for locating hidden dangers on roads based on camera and radar according to claim 1, characterized in that: The fusing of the synchronized image data and the synchronized point cloud data comprises: Converting the spatial coordinates of the synchronized point cloud data into image coordinates to obtain converted point cloud data; According to the bounding box of the obstacle contained in the synchronous image data, the point cloud data within the bounding box is filtered from the converted point cloud data to complete the fusion.

3. The method for locating hidden dangers on roads based on camera and radar according to claim 1, characterized in that: After the steps of respectively obtaining the synchronized image data and the synchronized point cloud data, the method further comprises: Clustering the synchronized point cloud data using the DBSCAN algorithm to obtain obstacle coordinates; Clustering the synchronized image data according to the obstacle coordinates to obtain detection data of the obstacles in the image; After fusing the detection data with the synchronized point cloud data, the DBSCAN algorithm is used to perform hidden danger location processing on the fused data to obtain the road hidden danger location.

4. The camera and radar based road hazard location method according to any one of claims 1 to 3, characterized in that: The hidden danger location processing includes: The DBSCAN algorithm is used to cluster the point cloud of the fused data according to the preset clustering conditions to obtain multiple clusters, wherein each cluster represents a hidden danger target, and the preset clustering conditions include the field radius and density threshold of the hidden danger target; Selecting a cluster with the largest number of point clouds from the multiple clusters as a target cluster; The spatial position of the target cluster is obtained to obtain the road hazard position.

5. The camera and radar based road hazard location method according to claim 4, characterized in that: The density threshold of the target cluster is as follows: Among them, ε s The density threshold of the target cluster, α is the vertical angle of the laser radar to the object S corresponding to the target cluster, β is the angle interval of the laser radar in the horizontal direction, D is the number of point clouds of the target cluster, L s is the distance from point S to the laser radar.

6. The camera and radar based road hazard location method according to claim 1, characterized in that: After the steps of respectively obtaining the synchronized image data and the synchronized point cloud data, the method further comprises: Respectively acquiring coordinate systems corresponding to the synchronized image data and the synchronized point cloud data, and jointly calibrating the two coordinate systems to obtain a calibrated coordinate system; Converting the data about the obstacle in the synchronized point cloud data into an image according to the calibrated coordinate system to obtain an obstacle image; Extracting frame parameters from the obstacle image and calculating frame information matching values ​​using the frame parameters, the frame parameters including: frame upper left corner coordinates and frame size; If the frame information matching value is less than a preset matching value, it is determined that the synchronous image data and the synchronous point cloud data are synchronously matched successfully; Otherwise, it is determined that the synchronization matching between the synchronized image data and the synchronized point cloud data fails.

7. The method for locating hidden dangers on roads based on camera and radar according to claim 1, characterized in that: The determining that an obstacle has entered the monitored road area includes: Using a camera to collect two frames of continuous monitoring image data; and calculating the difference between the pixel values ​​corresponding to the two frames of monitoring image data to obtain an image difference; If the image difference is greater than the preset difference, it is determined that there is an obstacle and the laser radar is called to detect the obstacle distance; When the obstacle distance is within a preset safety distance threshold interval, it is determined that the obstacle has entered the monitored road area.

8. The camera and radar based road hazard location method according to claim 1, characterized in that: The method of respectively acquiring the road image data collected by the camera and the laser point cloud data collected by the laser radar includes: When the laser point cloud data is collected by the laser radar, the scanning time node of the laser radar is obtained; Taking the scanning time node as the time reference, the camera is triggered to capture obstacle images to obtain road image data.

9. A road hidden danger positioning device based on camera and radar, characterized in that: The device comprises: The data acquisition module is used to obtain the road image data collected by the camera and the laser point cloud data collected by the laser radar when it is determined that there is an obstacle entering the monitored road area; A synchronization processing module, used for performing spatial synchronization processing on the road image data and the laser point cloud data to obtain synchronized image data and synchronized point cloud data respectively; The fusion positioning module is used to fuse the synchronized image data and the synchronized point cloud data, and then use the DBSCAN algorithm to perform hidden danger positioning processing on the fused data to obtain the road hidden danger position.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer-executable program, and the computer-executable program is used to enable a computer to execute the road hazard positioning method based on camera and radar as described in any one of claims 1-8.

Images