Tunnel obstacle detection method and apparatus

Abstract

The application discloses a tunnel obstacle detection method and device, and belongs to the technical field of safety detection. The method comprises the following steps: acquiring first point cloud data frames and inertial measurement data of a tunnel running area; correcting the motion distortion of the first point cloud data frames based on the inertial measurement data to obtain second point cloud data frames; removing the point cloud data corresponding to a target vehicle in the second point cloud data frames to obtain a tunnel wall point cloud data set of the tunnel running area; constructing a dense point cloud map of the tunnel running area based on the tunnel wall point cloud data set; and determining obstacle detection information of the tunnel running area based on the dense point cloud map. The method corrects the motion distortion of the point cloud data to ensure the data accuracy, removes the interference point cloud data of the vehicle body, reduces the amount of point cloud data, does not need to be pre-mapped, directly realizes real-time dense mapping, can effectively increase the point cloud density, improves the obstacle detection precision and accuracy, and reduces the small-volume obstacle missing detection rate.

Description

Technical Field

[0001] This application belongs to the field of safety detection technology, and in particular relates to a method and device for detecting obstacles in tunnels. Background Technology

[0002] With rapid economic development, investment in rail transit infrastructure construction has increased, leading to a rise in the total number of tunnels in the rail transit sector. Currently, in tasks involving imaging the inner surface of tunnel walls, the working distance between the detection equipment and the tunnel wall is relatively close (generally around 10cm). The inner wall of the tunnel is not completely smooth, and foreign objects such as long nails and cables often protrude. To avoid collisions between the detection equipment and these foreign objects (i.e., foreign object intrusion), it is necessary to constantly monitor the space in front of the vehicle.

[0003] Manually monitoring for foreign objects ahead presents challenges such as missed detections, high workload for personnel, and low detection efficiency due to limitations in human vision and vehicle speed. While some 3D LiDAR-based solutions can reduce manpower, the small size and distance of foreign objects detected from tunnel walls, coupled with sparse point clouds, result in low obstacle detection accuracy and precision. This can easily lead to missed detections of small objects, potentially causing collisions. Summary of the Invention

[0004] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a tunnel obstacle detection method and apparatus, which can effectively increase the density of point clouds, improve the accuracy and precision of obstacle detection, and reduce the rate of missed obstacle detection.

[0005] In a first aspect, this application provides a tunnel obstacle detection method, the method comprising: Acquire the first point cloud data frame and inertial measurement data of the tunnel travel area. The target vehicle is traveling in the tunnel travel area. The first point cloud data frame is collected by the radar equipment of the target vehicle. Based on the inertial measurement data, motion distortion correction is performed on the first point cloud data frame to obtain the second point cloud data frame. Remove the point cloud data corresponding to the target vehicle from the second point cloud data frame to obtain the tunnel wall point cloud dataset of the tunnel travel area; Based on the tunnel wall point cloud dataset, a dense point cloud map of the tunnel travel area is constructed; Based on the dense point cloud map, obstacle detection information for the tunnel travel area is determined.

[0006] According to the tunnel obstacle detection method of this application, the accuracy of point cloud data is ensured by correcting motion distortion of point cloud data, and the interference point cloud data of vehicle body is removed to reduce the amount of point cloud data. There is no need to perform the tunnel pre-mapping process, and real-time dense mapping can be performed directly, which can effectively increase the density of point cloud, improve the accuracy and precision of obstacle detection, and reduce the missed detection rate of small-volume obstacles.

[0007] According to one embodiment of this application, the step of performing motion distortion correction on the first point cloud data frame based on the inertial measurement data to obtain a second point cloud data frame includes: Based on the first point cloud data frame and the inertial measurement data, the device pose information of the radar device is determined; Based on the device pose information, the motion distortion of the first point cloud data frame is corrected to obtain the second point cloud data frame.

[0008] According to one embodiment of this application, determining the device pose information of the radar device based on the first point cloud data frame and the inertial measurement data includes: Obtain the timestamp information of the first point cloud data frame; Based on the timestamp information, the data acquisition time of each point cloud point in the first point cloud data frame is determined; Based on the data acquisition time, the first inertial data adjacent to the acquisition time of each point cloud point is found in the inertial measurement data, and the first inertial data and the point cloud point correspond one-to-one; Based on the first inertial data, the device pose information is determined.

[0009] According to one embodiment of this application, the step of removing the point cloud data corresponding to the target vehicle from the second point cloud data frame to obtain the tunnel wall point cloud dataset of the tunnel travel area includes: Determine the preset vehicle position in the second point cloud data frame; Based on the preset vehicle position, a plane fitting is performed in the second point cloud data frame to obtain the vehicle point cloud dataset of the target vehicle; The vehicle point cloud dataset is removed from the second point cloud data frame to obtain the tunnel wall point cloud dataset.

[0010] According to one embodiment of this application, the step of performing planar fitting in the second point cloud data frame based on the preset vehicle position to obtain the vehicle point cloud dataset of the target vehicle includes: Based on the second point cloud data frame and the preset vehicle position, a plane fitting is performed on the top surface of the target vehicle to obtain the top surface point cloud dataset of the target vehicle. If the number of points in the top surface point cloud dataset is determined to be greater than the point cloud number threshold, the vehicle point cloud dataset is obtained.

[0011] According to one embodiment of this application, the step of performing planar fitting in the second point cloud data frame based on the preset vehicle position to obtain the vehicle point cloud dataset of the target vehicle includes: Based on the second point cloud data frame and the preset vehicle position, a plane fitting is performed on the top surface of the target vehicle to obtain the top surface point cloud dataset of the target vehicle. If the number of points in the top point cloud dataset is less than or equal to the number of points, the overall point cloud space of the second point cloud data frame is fitted with a plane, and the planar point cloud dataset of the bottom spatial position in the second point cloud data frame is extracted. Based on the point cloud quantity threshold, invalid planar spatial point clouds in the planar point cloud dataset are filtered out to obtain the vehicle point cloud dataset.

[0012] According to one embodiment of this application, after obtaining the tunnel wall point cloud dataset of the tunnel travel area and before constructing the dense point cloud map of the tunnel travel area, the method further includes: The tunnel wall point cloud dataset is cropped according to the interest space algorithm.

[0013] According to one embodiment of this application, constructing a dense point cloud map of the tunnel travel area based on the tunnel wall point cloud dataset includes: Inter-frame point cloud registration is performed on the tunnel wall point cloud dataset to obtain the inter-frame point cloud transformation matrix; The dense point cloud map is constructed based on the tunnel wall point cloud dataset and the inter-frame point cloud transformation matrix.

[0014] According to one embodiment of this application, constructing the dense point cloud map based on the tunnel wall point cloud dataset and the inter-frame point cloud transformation matrix includes: Construct the first point cloud map corresponding to the current frame point cloud data of the tunnel wall point cloud dataset; Based on the inter-frame point cloud transformation matrix, the point cloud data of the next frame is accumulated into the point cloud data of the current frame to construct a second point cloud map; If the number of frames of the second point cloud map is greater than the threshold for the number of mapping frames, the dense point cloud map is obtained.

[0015] According to one embodiment of this application, determining obstacle detection information for the tunnel travel area based on the dense point cloud map includes: Construct a target space model of the target vehicle traveling in the tunnel area; Based on the dense point cloud map and the target space model, boundary intrusion detection is performed to determine the obstacle detection information.

[0016] Secondly, this application provides a tunnel obstacle detection device, the device comprising: The acquisition module is used to acquire the first point cloud data frame and inertial measurement data of the tunnel travel area. The target vehicle travels in the tunnel travel area, and the first point cloud data frame is collected by the radar equipment of the target vehicle. The first processing module is used to perform motion distortion correction on the first point cloud data frame based on the inertial measurement data to obtain the second point cloud data frame. The second processing module is used to remove the point cloud data corresponding to the target vehicle in the second point cloud data frame to obtain the tunnel wall point cloud dataset of the tunnel travel area. The third processing module is used to construct a dense point cloud map of the tunnel travel area based on the tunnel wall point cloud dataset. The fourth processing module is used to determine obstacle detection information in the tunnel travel area based on the dense point cloud map.

[0017] The tunnel obstacle detection device according to this application ensures the accuracy of point cloud data by correcting motion distortion of point cloud data, and then removes interfering point cloud data of vehicle body to reduce the amount of point cloud data. It can directly perform real-time dense mapping without performing the tunnel pre-mapping process, which can effectively increase the density of point cloud, improve the accuracy and precision of obstacle detection, and reduce the missed detection rate of small-volume obstacles.

[0018] Thirdly, this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the tunnel obstacle detection method as described in the first aspect above.

[0019] Fourthly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the tunnel obstacle detection method as described in the first aspect above.

[0020] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the tunnel obstacle detection method as described in the first aspect above.

[0021] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0022] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is one of the flowcharts illustrating the tunnel obstacle detection method provided in the embodiments of this application; Figure 2 This is a second schematic flowchart of the tunnel obstacle detection method provided in the embodiments of this application; Figure 3 This is a schematic diagram of a radar device acquiring a first point cloud data frame according to an embodiment of this application; Figure 4 This is a schematic diagram of the second point cloud data frame for motion distortion correction provided in an embodiment of this application; Figure 5 This is a schematic diagram of the tunnel obstacle detection device provided in the embodiments of this application; Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0024] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0025] The tunnel obstacle detection method, tunnel obstacle detection device, electronic device, and readable storage medium provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0026] Among them, the tunnel obstacle detection method can be applied to the terminal, specifically executed by the hardware or software in the terminal.

[0027] The terminal includes, but is not limited to, portable communication devices such as mobile phones or tablets with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that, in some embodiments, the terminal may not be a portable communication device, but rather a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads).

[0028] The following embodiments describe a terminal including a display and a touch-sensitive surface. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, mouse, and joystick.

[0029] The tunnel obstacle detection method provided in this application embodiment can be executed by an electronic device or a functional module or entity in an electronic device that can implement the tunnel obstacle detection method. The electronic devices mentioned in this application embodiment include, but are not limited to, mobile phones, tablets, computers, cameras, and wearable devices. The tunnel obstacle detection method provided in this application embodiment will be described below using an electronic device as the execution subject as an example.

[0030] like Figure 1 As shown, the tunnel obstacle detection method includes steps 110 to 150.

[0031] Step 110: Obtain the first point cloud data frame and inertial measurement data of the tunnel travel area.

[0032] In this scenario, the target vehicle is traveling in the tunnel area. The first point cloud data frame is collected by the radar equipment of the target vehicle, and the inertial measurement data is collected by the inertial measurement unit of the target vehicle.

[0033] The tunnel travel area can be the area inside the tunnel located in front of the target vehicle, that is, the forward travel space of the target vehicle.

[0034] In this embodiment, the radar device is installed on the target vehicle, and the first point cloud data frame collected by the radar device includes point cloud data of the tunnel travel area and point cloud data of the target vehicle.

[0035] It is understandable that when the target vehicle travels inside the tunnel, the tunnel walls are not completely smooth and there may be obstacles such as long nails or cables protruding. These obstacles on the tunnel walls may collide with the target vehicle or the equipment on the target vehicle, causing damage.

[0036] The target vehicle is equipped with radar equipment and an inertial measurement unit (IMU). The inertial measurement unit is a device that measures the three-axis attitude angles (or angular rates) and acceleration of the object.

[0037] In this embodiment, the radar device is used to acquire the first point cloud data frame of the tunnel travel area, and the inertial measurement unit is used to acquire the inertial measurement data of the target vehicle.

[0038] In practice, radar equipment can be lidar, which detects targets by emitting laser beams, receives signals reflected from targets, compares them with the emitted signals, and then processes them to obtain parameters such as the target's distance, azimuth, altitude, speed, attitude, and shape.

[0039] Step 120: Based on the inertial measurement data, perform motion distortion correction on the first point cloud data frame to obtain the second point cloud data frame.

[0040] It is understandable that during the movement of the target vehicle, the target vehicle is in motion. The first point cloud data frame is dynamically collected real-time point cloud data. Each point cloud data in each frame is collected by the radar equipment at different times, that is, at different reference poses. The movement of the target vehicle body during the collection process will cause motion distortion in the point cloud data.

[0041] Taking lidar as an example of radar equipment, such as Figure 3 As shown, during the operation, the target vehicle and the lidar move along the direction of motion. Figure 3 The plane shown represents a single frame of point cloud data from a lidar radar.

[0042] In this step, the motion distortion of the first point cloud data frame collected by the radar equipment is corrected based on the inertial measurement data of the target vehicle, resulting in a second point cloud data frame after distortion correction. The point cloud data is more accurate, which can improve the accuracy and precision of subsequent mapping using the point cloud data.

[0043] For example, such as Figure 4 As shown, the unprocessed motion distortion data has a certain tilt angle, while the distortion-free data corrected by inertial measurement data does not have a tilt angle and is closer to the true shape and pose of the scanned object.

[0044] In actual implementation, the first point cloud data frame collected by the radar equipment can be acquired in real time. The data of each point in the first point cloud data frame can be traversed, and the point cloud with the value of NAN in the point cloud data can be output. After filtering, the first point cloud data frame with only valid point cloud data frames can be obtained, and then motion distortion correction can be performed through inertial measurement data.

[0045] Step 130: Remove the point cloud data corresponding to the target vehicle from the second point cloud data frame to obtain the tunnel wall point cloud dataset of the tunnel travel area.

[0046] It is understandable that the imaging field of the radar equipment includes point cloud data corresponding to the tunnel scene and point cloud data corresponding to the target vehicle (mainly point cloud data corresponding to the target vehicle body). During actual travel, the radar equipment on the target vehicle and the target vehicle body are in a relatively static state, that is, the point cloud data corresponding to the target vehicle is a static point cloud in the point cloud collected by the radar equipment.

[0047] In this step, based on the second point cloud data frame after removing motion distortion, the point cloud data corresponding to the target vehicle in the second point cloud data frame is removed, while the point cloud data corresponding to the tunnel scene in the second point cloud data frame is retained, resulting in a tunnel wall point cloud dataset with interference removed. This reduces the amount of point cloud data and helps improve the efficiency of tunnel obstacle detection.

[0048] Step 140: Based on the tunnel wall point cloud dataset, construct a dense point cloud map of the tunnel travel area.

[0049] In this step, based on the tunnel wall point cloud dataset, a dense point cloud map of the tunnel travel area is constructed in real time to obtain the dense point cloud map of the tunnel travel area.

[0050] In this embodiment, the motion distortion of the first point cloud data frame is corrected by inertial measurement data to improve the accuracy of the point cloud data and ensure the accuracy of subsequent mapping. The point cloud data corresponding to the target vehicle in the second point cloud data frame is removed to obtain an interference-free tunnel wall point cloud dataset. This can reduce the amount of interference point cloud data during dense mapping and improve the speed of dense mapping.

[0051] Step 150: Based on the dense point cloud map, determine the obstacle detection information of the tunnel travel area.

[0052] In this step, based on the dense point cloud map of the tunnel travel area, obstacle detection information of the tunnel travel area can be obtained. The obstacle detection information includes whether there are obstacles such as long nails and cables on the inner wall of the tunnel travel area.

[0053] In practice, the movement of the target vehicle can be adjusted based on obstacle detection information in the tunnel travel area, and obstacles such as long nails and cables on the tunnel walls in a timely manner can be dealt with.

[0054] Simultaneous Localization and Mapping (SLAM) can be applied to tunnel detection.

[0055] In related technologies, to reduce manpower, 3D LiDAR is used to detect tunnels, and the collected tunnel data is processed offline. The offline data is used for simultaneous localization and map building to determine the presence of obstacles in the tunnel. However, the foreign objects protruding from the tunnel wall are small in size and far away, and the point cloud is too sparse, resulting in low accuracy and precision in obstacle detection. Small foreign objects are prone to being missed, which can lead to collision accidents.

[0056] Furthermore, the existing mapping architecture has high requirements for computing equipment performance during actual deployment, consumes a lot of computing time, and has poor mapping accuracy. It also requires the collection of rail transit data in advance for pre-mapping, which is time-consuming and labor-intensive.

[0057] In this embodiment, motion distortion of point cloud data is corrected by inertial measurement data to ensure the accuracy of point cloud data, and interference point cloud data of the target vehicle body is removed to reduce the amount of point cloud data. There is no need to perform the tunnel pre-mapping process, and real-time dense mapping is achieved, which effectively improves detection efficiency, reduces the missed detection rate of small-volume obstacles, and has low requirements for computing device performance, so mapping can be performed on lightweight computing devices.

[0058] According to the tunnel obstacle detection method provided in the embodiments of this application, the accuracy of point cloud data is ensured by correcting motion distortion of point cloud data, and the interference point cloud data of vehicle body is removed to reduce the amount of point cloud data. There is no need to perform the tunnel pre-mapping process, and real-time dense mapping can be performed directly, which can effectively increase the density of point cloud, improve the accuracy and precision of obstacle detection, and reduce the missed detection rate of small-volume obstacles.

[0059] In some embodiments, step 120, correcting motion distortion of the first point cloud data frame based on inertial measurement data to obtain a second point cloud data frame, may include: Based on the first point cloud data frame and inertial measurement data, the device pose information of the radar equipment is determined; Based on the device pose information, the motion distortion of the first point cloud data frame is corrected to obtain the second point cloud data frame.

[0060] In this embodiment, the device pose information of the radar device can be determined by the first point cloud data frame collected by the radar device and the inertial measurement data collected by the inertial measurement unit. Then, based on the device pose information of the radar device, the motion distortion of the radar device in the first point cloud data frame due to motion is corrected to obtain the second point cloud data frame with motion distortion removed.

[0061] In some embodiments, determining the device pose information of the radar device based on the first point cloud data frame and inertial measurement data may include: Obtain the timestamp information of the first point cloud data frame; Based on the timestamp information, determine the data acquisition time of each point cloud point in the first point cloud data frame; Based on the data acquisition time, the first inertial data adjacent to the acquisition time of each point cloud point is found in the inertial measurement data, and the first inertial data and the point cloud point correspond one-to-one. Based on the initial inertial data, the device's pose information is determined.

[0062] It should be noted that the target vehicle is in motion, and the first point cloud data frame is dynamically acquired real-time point cloud data. The point cloud data in each frame are acquired by the radar equipment at different times, that is, at different reference poses.

[0063] In this embodiment, the radar device pose is different at different data acquisition times. The data acquisition time of each point cloud point is determined according to the timestamp information. The first inertial data adjacent to the acquisition time of each point cloud point is found one by one in the inertial measurement data. Based on the first inertial data corresponding to all point cloud points in the first point cloud data frame, the device pose information of the radar device when the first point cloud data frame is acquired can be determined.

[0064] The following is a specific example of correcting motion distortion.

[0065] In this embodiment, the first point cloud data frame collected by the radar device is acquired, and the inertial measurement data collected by the inertial measurement unit is acquired simultaneously.

[0066] Extract the timestamp information of the start and end points of the first point cloud data frame, and calculate the difference T between the start and end timestamps. diff .

[0067] The point cloud data size of the first point cloud data frame is N. point Based on the difference between the start and end timestamps of the first point cloud data frame and the amount of point cloud data, the time consumed by a single point in the frame can be calculated using the following formula: P evr =T diff / N point Among them, P evr This represents the time consumed by a single point in the intra-frame point cloud.

[0068] Based on the time consumed by a single point in the point cloud within a frame and the distance between the point cloud point and the starting point, the data acquisition time corresponding to each point cloud point can be obtained.

[0069] Based on the data acquisition time, the system searches for the first inertial data adjacent to the time in the inertial measurement data queue that is cached inertial measurement data. Each point cloud point corresponds to a first inertial data adjacent to the time.

[0070] Based on the first inertial data corresponding to the point cloud points in the first point cloud data frame, the radar device pose of the current point cloud points can be calculated using spherical linear interpolation.

[0071] Homogeneous matrix transformation calculation can transform the current point cloud points to the unified pose of the starting point. By traversing all points in the frame and performing pose transformation calculation, a single frame of point cloud data after distortion removal can be obtained, which is the second point cloud data frame.

[0072] In some embodiments, step 130, removing the point cloud data corresponding to the target vehicle from the second point cloud data frame to obtain the tunnel wall point cloud dataset of the tunnel travel area, may include: Determine the preset vehicle position in the second point cloud data frame; Based on the preset vehicle position, plane fitting is performed in the second point cloud data frame to obtain the vehicle point cloud dataset of the target vehicle. Remove the vehicle point cloud dataset from the second point cloud data frame to obtain the tunnel wall point cloud dataset.

[0073] The preset vehicle position is the vehicle position pre-set in the second point cloud data frame.

[0074] For example, the lower left corner of the radar device's imaging field of view can be pre-set as the preset vehicle position. After obtaining the second point cloud data frame with motion distortion removed, the preset vehicle position in the second point cloud data frame can be determined.

[0075] In this embodiment, the preset vehicle position of the target vehicle is first preset in the second point cloud data frame, and plane fitting is performed to determine the true position of the target vehicle, thereby obtaining the vehicle point cloud dataset of the target vehicle. The vehicle point cloud dataset is then removed to obtain the interference-free tunnel inner wall point cloud dataset within the tunnel travel area.

[0076] In actual implementation, the top surface of the target vehicle can be fitted with a plane based on the second point cloud data frame and the preset vehicle position to obtain the top surface point cloud dataset of the target vehicle. The number of points in the top surface point cloud dataset is compared with the point cloud number threshold to determine whether a valid vehicle point cloud dataset has been obtained. If a valid vehicle point cloud dataset has not been obtained, a second fitting is required.

[0077] The point cloud quantity threshold can be a value designed based on experience and can be adjusted according to actual application needs, making it convenient for actual field deployment and application.

[0078] In some embodiments, based on a preset vehicle position, performing plane fitting in the second point cloud data frame to obtain a vehicle point cloud dataset of the target vehicle may include: Based on the second point cloud data frame and the preset vehicle position, a plane fitting is performed on the top surface of the target vehicle to obtain the top surface point cloud dataset of the target vehicle. If the number of points in the top surface point cloud dataset is greater than the point cloud number threshold, then obtain the vehicle point cloud dataset.

[0079] In this embodiment, the number of point clouds in the top surface point cloud dataset of the target vehicle is greater than the point cloud number threshold, indicating that a valid vehicle point cloud dataset can be obtained by performing planar fitting on the top surface of the target vehicle based on the second point cloud data frame and the preset vehicle position.

[0080] It should be noted that by pre-setting the preset vehicle position corresponding to the target vehicle and performing plane fitting to obtain an effective vehicle point cloud dataset, the computation time for identifying the point cloud data corresponding to the target vehicle in the second point cloud data frame can be reduced.

[0081] In some embodiments, based on a preset vehicle position, performing plane fitting in the second point cloud data frame to obtain a vehicle point cloud dataset of the target vehicle may include: Based on the second point cloud data frame and the preset vehicle position, a plane fitting is performed on the top surface of the target vehicle to obtain the top surface point cloud dataset of the target vehicle. If the number of points in the top point cloud dataset is less than or equal to the number of points, perform plane fitting on the overall point cloud space of the second point cloud data frame and extract the planar point cloud dataset of the bottom spatial position in the second point cloud data frame. Based on a point cloud quantity threshold, invalid planar spatial point clouds in the planar point cloud dataset are filtered out to obtain the vehicle point cloud dataset.

[0082] In this embodiment, if the number of points in the top surface point cloud dataset of the target vehicle is less than or equal to the number of points, it means that the planar fitting of the top surface of the target vehicle body based on the second point cloud data frame and the preset vehicle position cannot obtain a valid vehicle point cloud dataset. Therefore, it is necessary to perform planar fitting on the overall point cloud space of the second point cloud data frame in order to obtain a valid vehicle point cloud dataset.

[0083] The following describes a specific implementation example for extracting vehicle point cloud datasets.

[0084] The preset vehicle position corresponding to the target vehicle is set in the imaging field of the radar equipment. The preset point of the preset vehicle position is P1.

[0085] Based on the point cloud data information at point P1 in the second point cloud data frame, the RANSAC plane fitting algorithm is used to obtain the top surface point cloud dataset Q in real time. Train Calculate Q Train Number of point clouds N t .

[0086] The threshold for the number of point clouds is N. conf When N t >N confWhen the time is right, it means that a valid vehicle point cloud dataset has been obtained by performing planar fitting on the top surface of the target vehicle.

[0087] When N t ≤N conf If the condition is met, it indicates that a valid vehicle point cloud dataset has not been obtained. It is necessary to perform planar fitting on the overall point cloud space of the second point cloud data frame, extract the planar point cloud set located at the bottom spatial position, filter invalid planar spatial points of the planar point cloud set according to the point cloud quantity threshold, and finally extract a valid vehicle point cloud dataset.

[0088] In this embodiment, by pre-setting the preset vehicle position corresponding to the target vehicle in the imaging field of the radar device, and performing planar fitting on the second point cloud data frame to obtain an effective vehicle point cloud dataset, the computation time for identifying the point cloud data corresponding to the target vehicle in the second point cloud data frame can be reduced.

[0089] In some embodiments, after obtaining the tunnel wall point cloud dataset of the tunnel travel area in step 130 and before constructing the dense point cloud map of the tunnel travel area in step 140, the tunnel obstacle detection method may further include: Based on the interest space algorithm, the tunnel wall point cloud dataset is cropped.

[0090] In actual implementation, when acquiring a dense point cloud map in real time, certain spatial regions in the field of view are areas that do not need to be detected, and spatial point cloud datasets with excessive forward distance in the tunnel travel area also do not need to be detected.

[0091] In this embodiment, based on the space of interest algorithm, the regions that do not need to be detected in the tunnel travel area are determined. The point cloud data corresponding to the regions that do not need to be detected in the tunnel wall point cloud dataset are then cropped and removed, effectively reducing the amount of point cloud data and improving the mapping efficiency and accuracy of subsequent real-time dense mapping.

[0092] In some embodiments, step 140, constructing a dense point cloud map of the tunnel travel area based on the tunnel wall point cloud dataset, may include: Inter-frame point cloud registration was performed on the tunnel wall point cloud dataset to obtain the inter-frame point cloud transformation matrix; A dense point cloud map is constructed based on the tunnel wall point cloud dataset and the inter-frame point cloud transformation matrix.

[0093] In this embodiment, by registering the point clouds between frames in the tunnel wall point cloud dataset, an inter-frame point cloud transformation matrix is ​​obtained between point cloud frames in the tunnel wall point cloud dataset. The point cloud frames are then transformed using the inter-frame point cloud transformation matrix, and the point cloud frames are accumulated to achieve real-time dense mapping.

[0094] In practice, the inter-frame point cloud transformation matrix of the tunnel wall point cloud dataset can be calculated using a registration algorithm. For example, the inter-frame point cloud transformation matrix can be obtained using the Normal-Distributions Transform point cloud registration algorithm.

[0095] Before performing inter-frame registration, the tunnel wall point cloud dataset can be filtered and sparsed. A raster data structure can be used to represent the effective point cloud data information in space. This reduces the amount of target point cloud data in the tunnel wall point cloud dataset while ensuring the effective point cloud data information, thereby improving the calculation speed of the registration algorithm.

[0096] In some embodiments, a dense point cloud map is constructed based on the tunnel wall point cloud dataset and the inter-frame point cloud transformation matrix, including: Construct the first point cloud map corresponding to the current frame point cloud data of the tunnel wall point cloud dataset; Based on the inter-frame point cloud transformation matrix, the point cloud data of the next frame is accumulated into the point cloud data of the current frame to construct a second point cloud map; A dense point cloud map is obtained when the number of frames in the second point cloud map is greater than the threshold for the number of frames in the map construction.

[0097] In this embodiment, a first point cloud map corresponding to the current frame point cloud data is constructed. The point cloud frame is transformed by the inter-frame point cloud transformation matrix. The point cloud data of the next frame is accumulated into the current frame point cloud data to densify the point cloud data and construct a second point cloud map.

[0098] When the number of frames of the second point cloud map is determined to be greater than the threshold for the number of mapping frames, it is determined that the accumulation of point cloud frames meets the densification requirement, and the second point cloud map is used as a dense point cloud map.

[0099] When the number of frames of the second point cloud map is determined to be less than or equal to the threshold for the number of mapping frames, it is determined that the accumulation of point cloud frames has not yet met the densification requirements. The new point cloud data of the next frame is then added to the new current frame point cloud data. Based on the original point cloud map, a new point cloud map is constructed, and finally a dense point cloud map is obtained.

[0100] The following is a specific implementation example of real-time dense mapping.

[0101] Based on the requirements for dense mapping, set the mapping frame number threshold N. Map N Map This indicates the number of data frames contained in the real-time mapping.

[0102] For the current frame point cloud data of the tunnel wall point cloud dataset, a first point cloud map Map1 is constructed. The point cloud data of the next frame is then accumulated in real time into the current frame point cloud data using an inter-frame transformation matrix to construct a second point cloud map Map2.

[0103] If the number of frames in the newly created map Map2 is less than or equal to the map creation frame threshold N Map Then, continue calculating the inter-frame transformation matrix for the next frame of point cloud data, repeating the calculation and accumulation process until the threshold N for the mapping frame data is met. Map Obtain a dense point cloud map.

[0104] In some embodiments, step 150, determining obstacle detection information for the tunnel travel area based on a dense point cloud map, may include: Construct a target space model of the target vehicle's movement within the tunnel area; Based on dense point cloud maps and target spatial models, boundary intrusion detection is performed to determine obstacle detection information.

[0105] In this embodiment, a target space model is obtained by modeling the space in which the target vehicle travels in the tunnel area and meets the minimum passage boundary. By performing boundary intrusion detection based on the target space model and the dense point cloud map, obstacle detection information is determined, thereby realizing the detection of small-volume foreign objects in the tunnel.

[0106] The following is a specific example.

[0107] like Figure 2 As shown, the radar equipment of the target vehicle collects raw point cloud frame data (first point cloud data frame), and the inertial measurement unit collects IMU data (inertial measurement data).

[0108] The original point cloud frame data is preprocessed to extract and remove invalid point cloud data. Then, motion compensation is performed on a single frame of point cloud using IMU data to remove motion distortion in the point cloud data frame.

[0109] To remove dynamic interference from the point cloud dataset (vehicle point cloud dataset), first set a preset vehicle position, extract the vehicle point cloud dataset through plane fitting, and then remove the vehicle point cloud dataset from the point cloud data after removing motion distortion to obtain an interference-free tunnel wall point cloud dataset.

[0110] Based on the tunnel wall point cloud dataset, the inter-frame point cloud transformation matrix is ​​calculated. The point cloud dataset is then densed using the inter-frame point cloud transformation matrix, and a dense point cloud map is constructed through real-time integral mapping.

[0111] By utilizing the target space model corresponding to the space where the target vehicle travels in the tunnel area and satisfies the minimum passage boundary, boundary intrusion detection is performed to determine the obstacle detection information within the tunnel.

[0112] In this embodiment, motion distortion of point cloud data is corrected by IMU data to ensure the accuracy of point cloud data, dynamic interference point cloud datasets are removed, the amount of point cloud data is reduced, and the tunnel pre-mapping process is not required to achieve real-time dense mapping, which effectively improves detection efficiency and reduces the missed detection rate of small-volume obstacles.

[0113] The tunnel obstacle detection method provided in this application can be executed by a tunnel obstacle detection device. This application uses the example of a tunnel obstacle detection device executing the tunnel obstacle detection method to illustrate the tunnel obstacle detection device provided in this application.

[0114] This application also provides a tunnel obstacle detection device.

[0115] like Figure 5 As shown, the tunnel obstacle detection device includes: The acquisition module 510 is used to acquire the first point cloud data frame and inertial measurement data of the tunnel travel area. The target vehicle travels in the tunnel travel area, and the first point cloud data frame is collected by the radar equipment of the target vehicle. The first processing module 520 is used to perform motion distortion correction on the first point cloud data frame based on inertial measurement data to obtain the second point cloud data frame. The second processing module 530 is used to remove the point cloud data corresponding to the target vehicle in the second point cloud data frame to obtain the tunnel wall point cloud dataset of the tunnel travel area. The third processing module 540 is used to construct a dense point cloud map of the tunnel travel area based on the tunnel wall point cloud dataset; The fourth processing module 550 is used to determine obstacle detection information in the tunnel travel area based on a dense point cloud map.

[0116] The tunnel obstacle detection device provided in this application corrects motion distortion of point cloud data to ensure its accuracy, and removes interfering point cloud data from the vehicle body to reduce the amount of point cloud data. It eliminates the need for a pre-mapping process and directly performs real-time dense mapping, effectively increasing point cloud density, improving obstacle detection accuracy, and reducing the missed detection rate of small obstacles. In some embodiments, the first processing module 520 is used to determine the device pose information of the radar equipment based on the first point cloud data frame and inertial measurement data. Based on the device pose information, the motion distortion of the first point cloud data frame is corrected to obtain the second point cloud data frame.

[0117] In some embodiments, the first processing module 520 is used to obtain the timestamp information of the first point cloud data frame; Based on the timestamp information, determine the data acquisition time of each point cloud point in the first point cloud data frame; Based on the data acquisition time, the first inertial data adjacent to the acquisition time of each point cloud point is found in the inertial measurement data, and the first inertial data and the point cloud point correspond one-to-one. Based on the initial inertial data, the device's pose information is determined.

[0118] In some embodiments, the second processing module 530 is used to determine the preset vehicle position in the second point cloud data frame; Based on the preset vehicle position, plane fitting is performed in the second point cloud data frame to obtain the vehicle point cloud dataset of the target vehicle. Remove the vehicle point cloud dataset from the second point cloud data frame to obtain the tunnel wall point cloud dataset.

[0119] In some embodiments, the second processing module 530 is used to perform planar fitting on the top surface of the target vehicle based on the second point cloud data frame and the preset vehicle position to obtain a top surface point cloud dataset of the target vehicle. If the number of points in the top surface point cloud dataset is greater than the point cloud number threshold, then obtain the vehicle point cloud dataset.

[0120] In some embodiments, the second processing module 530 is used to perform planar fitting on the top surface of the target vehicle based on the second point cloud data frame and the preset vehicle position to obtain a top surface point cloud dataset of the target vehicle. If the number of points in the top point cloud dataset is less than or equal to the number of points, perform plane fitting on the overall point cloud space of the second point cloud data frame and extract the planar point cloud dataset of the bottom spatial position in the second point cloud data frame. Based on a point cloud quantity threshold, invalid planar spatial point clouds in the planar point cloud dataset are filtered out to obtain the vehicle point cloud dataset.

[0121] In some embodiments, the second processing module 530 is further configured to crop the tunnel wall point cloud dataset according to the space of interest algorithm.

[0122] In some embodiments, the third processing module 540 is used to perform inter-frame point cloud registration on the tunnel wall point cloud dataset to obtain an inter-frame point cloud transformation matrix. A dense point cloud map is constructed based on the tunnel wall point cloud dataset and the inter-frame point cloud transformation matrix.

[0123] In some embodiments, the third processing module 540 is used to construct a first point cloud map corresponding to the current frame point cloud data of the tunnel wall point cloud dataset; Based on the inter-frame point cloud transformation matrix, the point cloud data of the next frame is accumulated into the point cloud data of the current frame to construct a second point cloud map; A dense point cloud map is obtained when the number of frames in the second point cloud map is greater than the threshold for the number of frames in the map construction.

[0124] In some embodiments, the fourth processing module 550 is used to construct a target space model of the target vehicle traveling in the tunnel area. Based on dense point cloud maps and target spatial models, boundary intrusion detection is performed to determine obstacle detection information.

[0125] The tunnel obstacle detection device in this application embodiment can be an electronic device or a component of an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the specific device.

[0126] The tunnel obstacle detection device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0127] The tunnel obstacle detection device provided in this application embodiment can achieve... Figures 1 to 4 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0128] In some embodiments, such as Figure 6 As shown, this application embodiment also provides an electronic device 600, including a processor 601, a memory 602, and a computer program stored in the memory 602 and executable on the processor 601. When the program is executed by the processor 601, it implements the various processes of the above-described tunnel obstacle detection method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0129] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0130] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described tunnel obstacle detection method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0131] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0132] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described tunnel obstacle detection method.

[0133] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0134] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described tunnel obstacle detection method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0135] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0136] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0137] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0138] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0139] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0140] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for detecting obstacles in a tunnel, characterized in that, include: Acquire the first point cloud data frame and inertial measurement data of the tunnel travel area. The target vehicle is traveling in the tunnel travel area. The first point cloud data frame is collected by the radar equipment of the target vehicle. Based on the inertial measurement data, motion distortion correction is performed on the first point cloud data frame to obtain the second point cloud data frame. Remove the point cloud data corresponding to the target vehicle from the second point cloud data frame to obtain the tunnel wall point cloud dataset of the tunnel travel area; Based on the tunnel wall point cloud dataset, a dense point cloud map of the tunnel travel area is constructed; Based on the dense point cloud map, obstacle detection information for the tunnel travel area is determined; The construction of a dense point cloud map of the tunnel travel area based on the tunnel wall point cloud dataset includes: Inter-frame point cloud registration is performed on the tunnel wall point cloud dataset to obtain the inter-frame point cloud transformation matrix; Based on the tunnel wall point cloud dataset and the inter-frame point cloud transformation matrix, the dense point cloud map is constructed. The construction of the dense point cloud map based on the tunnel wall point cloud dataset and the inter-frame point cloud transformation matrix includes: Construct the first point cloud map corresponding to the current frame point cloud data of the tunnel wall point cloud dataset; Based on the inter-frame point cloud transformation matrix, the point cloud data of the next frame is accumulated into the point cloud data of the current frame to construct a second point cloud map; If the number of frames of the second point cloud map is greater than the threshold for the number of mapping frames, the dense point cloud map is obtained.

2. The tunnel obstacle detection method according to claim 1, characterized in that, The step of correcting motion distortion in the first point cloud data frame based on the inertial measurement data to obtain the second point cloud data frame includes: Based on the first point cloud data frame and the inertial measurement data, the device pose information of the radar device is determined; Based on the device pose information, the motion distortion of the first point cloud data frame is corrected to obtain the second point cloud data frame.

3. The tunnel obstacle detection method according to claim 2, characterized in that, The step of determining the device pose information of the radar equipment based on the first point cloud data frame and the inertial measurement data includes: Obtain the timestamp information of the first point cloud data frame; Based on the timestamp information, the data acquisition time of each point cloud point in the first point cloud data frame is determined; Based on the data acquisition time, the first inertial data adjacent to the acquisition time of each point cloud point is found in the inertial measurement data, and the first inertial data and the point cloud point correspond one-to-one; Based on the first inertial data, the device pose information is determined.

4. The tunnel obstacle detection method according to claim 1, characterized in that, The step of removing the point cloud data corresponding to the target vehicle from the second point cloud data frame to obtain the tunnel wall point cloud dataset of the tunnel travel area includes: Determine the preset vehicle position in the second point cloud data frame; Based on the preset vehicle position, a plane fitting is performed in the second point cloud data frame to obtain the vehicle point cloud dataset of the target vehicle; The vehicle point cloud dataset is removed from the second point cloud data frame to obtain the tunnel wall point cloud dataset.

5. The tunnel obstacle detection method according to claim 4, characterized in that, The step of performing planar fitting in the second point cloud data frame based on the preset vehicle position to obtain the vehicle point cloud dataset of the target vehicle includes: Based on the second point cloud data frame and the preset vehicle position, a plane fitting is performed on the top surface of the target vehicle to obtain the top surface point cloud dataset of the target vehicle. If the number of points in the top surface point cloud dataset is determined to be greater than the point cloud number threshold, the vehicle point cloud dataset is obtained.

6. The tunnel obstacle detection method according to claim 4, characterized in that, The step of performing planar fitting in the second point cloud data frame based on the preset vehicle position to obtain the vehicle point cloud dataset of the target vehicle includes: Based on the second point cloud data frame and the preset vehicle position, a plane fitting is performed on the top surface of the target vehicle to obtain the top surface point cloud dataset of the target vehicle. If the number of points in the top point cloud dataset is less than or equal to the number of points, the overall point cloud space of the second point cloud data frame is fitted with a plane, and the planar point cloud dataset of the bottom spatial position in the second point cloud data frame is extracted. Based on the point cloud quantity threshold, invalid planar spatial point clouds in the planar point cloud dataset are filtered out to obtain the vehicle point cloud dataset.

7. The tunnel obstacle detection method according to claim 1, characterized in that, After obtaining the tunnel wall point cloud dataset of the tunnel travel area, and before constructing the dense point cloud map of the tunnel travel area, the method further includes: The tunnel wall point cloud dataset is cropped according to the interest space algorithm.

8. The tunnel obstacle detection method according to any one of claims 1-6, characterized in that, The process of determining obstacle detection information for the tunnel travel area based on the dense point cloud map includes: Construct a target space model of the target vehicle traveling in the tunnel area; Based on the dense point cloud map and the target space model, boundary intrusion detection is performed to determine the obstacle detection information.

9. A tunnel obstacle detection device, characterized in that, The tunnel obstacle detection device is used to implement the tunnel obstacle detection method as described in any one of claims 1-8, and the tunnel obstacle detection device comprises: The acquisition module is used to acquire the first point cloud data frame and inertial measurement data of the tunnel travel area. The target vehicle travels in the tunnel travel area, and the first point cloud data frame is collected by the radar equipment of the target vehicle. The first processing module is used to perform motion distortion correction on the first point cloud data frame based on the inertial measurement data to obtain the second point cloud data frame. The second processing module is used to remove the point cloud data corresponding to the target vehicle in the second point cloud data frame to obtain the tunnel wall point cloud dataset of the tunnel travel area. The third processing module is used to construct a dense point cloud map of the tunnel travel area based on the tunnel wall point cloud dataset. The fourth processing module is used to determine obstacle detection information in the tunnel travel area based on the dense point cloud map.

Images