A mine area road surface water detection method based on fusion of vision and laser radar

Abstract

The application belongs to the technical field of unmanned driving, and specifically discloses a mine area road waterlogging detection method based on fusion of vision and laser radar, which comprises the following steps: acquiring image data and performing waterlogging segmentation; acquiring point cloud data, removing non-waterlogging point cloud to obtain a first segmentation result, and performing segmentation according to waterlogging semantic segmentation labeling to obtain a second segmentation result, and fusing to obtain a waterlogging recognition result; judging whether the image confidence degree is qualified or not; extracting ground point cloud in the point cloud data and modeling to obtain a fitting plane, projecting the edge contour pixel data to the fitting plane to construct a point cloud convex hull contour, and recognizing the point cloud convex hull contour to obtain the waterlogging recognition result; and has the following advantages: the multi-sensor fusion sensing mode can effectively deal with the problems existing in the single-sensor sensing, and has higher real-time performance, robustness and high confidence degree.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and more specifically, to a method for detecting water accumulation on roads in mining areas based on the fusion of vision and lidar. Background Technology

[0002] In the field of autonomous driving, comprehensive, accurate, and real-time perception of the attributes of obstacles such as water accumulation is a key technology. It is crucial to the planning, decision-making, control modules, and safety of autonomous driving, and relying on a single sensor for perception presents significant challenges.

[0003] LiDAR can perform 3D point cloud modeling of the surrounding environment. The number of radar beams determines the hardware cost and the sparsity level of the point cloud. Its measurement range can reach 200m. The point cloud becomes sparser with increasing distance. At the same time, it lacks semantic information and color and texture information, making it difficult to accurately identify the outline and category of obstacles. The data is easily affected by slight dust, which increases the difficulty of obstacle identification.

[0004] Visual cameras can capture information rich in color and texture, and their ability to identify obstacles is related to their focal length and the density of optical imaging pixels. They cannot directly perceive distance and depth information; relying on assumptions such as distance calculations to solve for depth information often results in high computational complexity and low confidence. Furthermore, they are susceptible to environmental factors, such as rain, fog, and incident light, which can degrade image quality and consequently reduce perception confidence. For example, patent CN114758139A, titled "Method for Detecting Water Accumulation in Foundation Pit," proposes acquiring and annotating images of water accumulation in foundation pits, then using a deep learning algorithm for training to segment the water-filled areas. This invention achieves accurate identification and detection of water accumulation in construction site images, but lacks depth information about the water accumulation in foundation pits. Similarly, patent CN114419381B, titled "A Semantic Segmentation Method and a Road Water Accumulation Detection Method and Device Using It," proposes an image semantic segmentation method that can identify the presence of road water accumulation, but also lacks depth information.

[0005] Multi-sensor fusion improves the stability and robustness of the algorithm, and has good detection accuracy under complex environmental conditions such as rain, fog, strong light, weak light, and sandstorms. The multi-sensor system effectively ensures the stability of perception redundancy and accuracy. Summary of the Invention

[0006] The present invention aims to provide a method, device and storage medium for detecting surface water on mining roads based on the fusion of vision and lidar, so as to solve or improve at least one of the above-mentioned technical problems.

[0007] In view of this, the first aspect of the present invention is to provide a method for detecting surface water on roads in mining areas based on the fusion of vision and lidar.

[0008] The first aspect of the present invention provides a method for detecting water accumulation on mining roads based on the fusion of vision and lidar, comprising the following steps: S1: acquiring road image data through an onboard camera sensor and performing water accumulation segmentation to obtain edge contour pixel data of the water accumulation and its image confidence score; S2: acquiring point cloud data through an onboard lidar and determining whether the image confidence score is qualified. If it is not qualified, removing non-water accumulation point clouds based on the reflection threshold and plane height variance to obtain a first segmentation result, performing segmentation based on water accumulation semantic segmentation annotation to obtain a second segmentation result, and fusing the first segmentation result and the second segmentation result as the water accumulation recognition result; if qualified, extracting the ground point cloud from the point cloud data and modeling to obtain a fitting plane, projecting the edge contour pixel data of the water accumulation onto the fitting plane to construct the point cloud convex hull contour, and recognizing the point cloud convex hull contour to obtain the water accumulation recognition result; S3: extracting the water accumulation area and the relative height of the water accumulation to the ground from the water accumulation recognition result in S2 and sending them to the vehicle control module.

[0009] This invention provides a method for detecting water accumulation on mining roads based on the fusion of vision and lidar. Currently, water accumulation detection and identification tasks in mining areas face significant challenges due to the difficulty in effectively separating the road surface from the water. Furthermore, while traditional visual methods based on single sensors such as cameras can identify the outline of water accumulation, their accuracy and stability are poor, and they struggle to obtain key attribute information such as the water's height relative to the ground, the depth of puddles, and the area of ​​the water accumulation, making it difficult to support vehicle decision-making and control. The multi-sensor fusion perception approach, which incorporates a deep learning-based water accumulation semantic segmentation algorithm, effectively addresses the problems of single-sensor perception and offers higher real-time performance, robustness, and confidence.

[0010] The image data is segmented to identify the waterlogged areas; then, a fusion algorithm is used in conjunction with LiDAR to further output information about the waterlogged areas.

[0011] Based on the confidence level of the image semantic segmentation results, the method of projecting the image semantic contour pixels onto the ground plane has the advantages of low computational cost and reduced hardware costs. Only when the image semantic segmentation confidence level is low, the segmentation results obtained from two different algorithms are fused to give the water accumulation contour. The use of multiple sensors ensures the redundancy and complementarity of the results and further reduces hardware computing costs.

[0012] By combining road surface slope (road gradient) to improve the accuracy of point cloud extraction, the accuracy of point cloud contours projected from image semantic contour pixels onto the ground plane is also improved, making the extracted water accumulation point cloud within the contour more accurate.

[0013] In addition, the technical solutions provided by embodiments of the present invention may also have the following additional technical features:

[0014] In any of the above technical solutions, before step S1, the detection and recognition method further includes: installing a lidar for acquiring point cloud data and a camera sensor for acquiring image data on the vehicle and calibrating them; creating a point cloud water semantic segmentation model for obtaining a second segmentation result based on the water accumulation semantic segmentation annotation; and creating a ground water accumulation point cloud extraction algorithm for obtaining the first segmentation result.

[0015] In this technical solution, since the subsequent water accumulation identification results require data from different devices such as camera sensors and lidar, joint calibration is necessary to reduce bias.

[0016] Using data acquisition from multiple sensors can address the challenges of water accumulation identification applications using a single sensor such as radar or images. The technical difficulty lies in effectively separating the road surface from the water accumulation.

[0017] In any of the above technical solutions, the step of segmenting based on water accumulation semantic segmentation annotations to obtain a second segmentation result specifically includes: acquiring historical point cloud data from different environments as training data; inputting the training data into a water accumulation semantic segmentation model to obtain water accumulation semantic segmentation annotations; the water accumulation semantic segmentation model segmenting the point cloud data based on the water accumulation semantic segmentation annotations to obtain a second segmentation result and its point cloud confidence score; setting a first threshold to determine whether the point cloud confidence score is greater than the first threshold; if so, outputting the second segmentation result; otherwise, the water accumulation segmentation result is unusable; wherein, the first threshold is set to 0.5.

[0018] In this technical solution, when image data is unavailable, a deep learning algorithm for extracting water accumulation point clouds is used. Based on the collected point cloud data, water accumulation semantic segmentation and annotation are performed to train a water accumulation point cloud semantic segmentation model. The model is then directly used to extract water accumulation point cloud semantic segmentation. Furthermore, data from different environmental scenarios are collected to collect water accumulation semantic segmentation annotation training data to train the water accumulation semantic segmentation model.

[0019] In any of the above technical solutions, the determination of whether the image confidence level is qualified is made by the following method: setting a second threshold for determining whether it is qualified; when the image confidence level of the edge contour pixel data is less than the second threshold, the edge contour pixel data is determined to be unqualified; when the image confidence level of the edge contour pixel data is not less than the second threshold, the edge contour pixel data is determined to be qualified; wherein, the second threshold is set to 0.2.

[0020] In this technical solution, when the image semantic segmentation category confidence is less than or equal to 0.2, the image semantic segmentation accuracy is poor, and the relevant results no longer participate in subsequent algorithm modules. At this time, the water accumulation is identified and its attributes are extracted by fusing the results of traditional algorithms and deep learning algorithms for extracting ground water point clouds.

[0021] In any of the above technical solutions, the step of obtaining water accumulation recognition results by recognizing the convex hull contour of the point cloud specifically includes: determining the segmentation accuracy of the edge contour pixel data constituting the convex hull contour of the point cloud; setting different extension boundary widths according to the determination result to expand the contour of the edge contour pixel data to obtain the point cloud within the contour; and filtering and target detection on the point cloud within the contour to obtain the water accumulation recognition results.

[0022] In this technical solution, before projecting the image contour points onto the ground fitting plane to construct the point cloud convex hull contour, it is necessary to determine the confidence level of the edge contour pixel data, which represents the segmentation accuracy, so as to expand the boundary width by different sizes and make the final water accumulation attributes more accurate.

[0023] In any of the above technical solutions, the segmentation accuracy is determined by the following steps: setting a third threshold for judging the segmentation accuracy as high or low and the value being higher than the second threshold, and obtaining the image confidence level corresponding to the edge contour pixel data; when the image confidence level is greater than the third threshold, the segmentation accuracy of the point cloud convex hull contour formed by the edge contour pixel data is determined to be high; when the image confidence level is not greater than the third threshold but greater than the first threshold, the segmentation accuracy of the point cloud convex hull contour formed by the edge contour pixel data is determined to be low; wherein, the third threshold is set to 0.5.

[0024] In this technical solution, the following rules are specifically adopted: when the image semantic segmentation category confidence is greater than 0.5, the image contour points are projected onto the ground fitting plane to construct the point cloud convex hull contour. The contour is expanded outward by the expansion boundary width WE1 to obtain the point cloud within the contour. The point cloud is filtered and the target is detected. It is determined whether the obstacle is water accumulation and the attributes are extracted as area and relative depth.

[0025] When the image semantic segmentation category confidence is less than or equal to 0.5 and greater than 0.2, the image contour points are projected onto the ground fitting plane to construct the point cloud convex hull contour. The contour is expanded outward by the extended boundary width WE2 to obtain the point cloud within the contour. The point cloud is filtered and the target is detected. It is determined whether the obstacle is water accumulation and the attributes are extracted as area and relative depth.

[0026] In any of the above technical solutions, the setting of different extended boundary widths based on the judgment result adopts the following rule: when the judgment segmentation accuracy is high, the extended boundary width of the contour used to extend the edge contour pixel data is WE1; when the judgment segmentation accuracy is low, the extended boundary width of the contour used to extend the edge contour pixel data is WE2; wherein, WE1 is the width of 5 pixel values ​​and WE2 is the width of 10 pixel values.

[0027] In this technical solution.

[0028] In any of the above technical solutions, the coordinate system of the fitting plane is the same as the coordinate system of the ground where the point cloud data is located.

[0029] In this technical solution, the coordinate system of the fitting plane is set to be the same as the coordinate system of the ground where the point cloud data is located. This makes it easier to obtain the water accumulation point cloud, as well as the area and depth of the water accumulation relative to the ground after constructing the convex hull contour of the point cloud.

[0030] In any of the above technical solutions, the step of acquiring image data and performing water accumulation segmentation includes: correcting the distortion of the image data to obtain a corrected image; inputting the corrected image into an image water accumulation semantic segmentation model to obtain a semantic segmentation result; and sequentially performing binarization, erosion and dilation, region connectivity, pixel area filtering, and edge contour detection on the semantic segmentation result to obtain the number of pixels of the edge convex hull contour.

[0031] The beneficial effects of this invention compared to the prior art are as follows:

[0032] Based on image data, a deep learning semantic segmentation model is used to initially divide the waterlogged area; then, a fusion algorithm combined with LiDAR is used to further output information about the waterlogged area.

[0033] Based on the confidence level of the image semantic segmentation results, the method of projecting the image semantic contour pixels onto the ground plane has the advantages of low computational cost and reduced hardware equipment costs. Only when the image semantic segmentation confidence level is low, the water accumulation point cloud fusion module is activated and the water accumulation contour is given. The redundancy and complementarity of the results are guaranteed by multiple sensors and the hardware computing cost is further reduced.

[0034] By combining the changes in road slope, the accuracy of point cloud extraction is improved. At the same time, the accuracy of the point cloud contour of the image semantic contour pixels projected onto the ground plane is improved, making the extracted water accumulation point cloud within the contour more accurate.

[0035] Additional aspects and advantages of embodiments of the invention will become apparent in the following description or may be learned by practice of embodiments of the invention. Attached Figure Description

[0036] The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of the invention.

[0037] Figure 1 This is a flowchart of the method of the present invention;

[0038] Figure 2 This is a diagram illustrating the recognition effect of the present invention. Detailed Implementation

[0039] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0040] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0041] Some embodiments of the present invention provide a method, apparatus, and storage medium for detecting surface water on mining roads based on the fusion of vision and lidar. Please refer to [link / reference]. Figure 1-2 This invention provides a method for detecting water accumulation on roads in mining areas based on the fusion of vision and lidar. The method includes:

[0042] I. Overall Solution Framework

[0043] It consists of two parts: hardware and software. The hardware part includes installing lidar and video cameras, and performing individual and joint calibrations, while the video camera performs intrinsic parameter calibration. The software part includes deep learning model training and algorithm framework. The deep learning model training includes image deep learning model training and point cloud deep learning model training. The algorithm framework includes deep image algorithm, point cloud algorithm, and image-point cloud fusion algorithm.

[0044] II. Hardware Installation and Calibration

[0045] The vehicle is equipped with a forward-facing camera and a lidar. The transfer matrix between the radar coordinate system and the camera coordinate system is calibrated, and the camera intrinsic parameters are calibrated.

[0046] III. Model Training

[0047] Collect on-site water accumulation image data and perform semantic annotation to train an image semantic segmentation model; collect on-site water accumulation point cloud data and perform semantic annotation to train a 3D point cloud semantic segmentation model.

[0048] IV. Algorithm Framework

[0049] 1 Image Algorithm

[0050] The sensor acquires images for image correction. After deploying a semantic segmentation inference model, the corrected image is subjected to water accumulation semantic segmentation. The semantic segmentation results are then subjected to binarization, erosion and dilation, region connectivity, pixel area filtering, and edge contour detection to obtain and store edge contour pixel data.

[0051] 2-point cloud algorithm

[0052] First, the point cloud between the left and right retaining walls is extracted and filtered. The ground point cloud is extracted and a plane is fitted to the ground point cloud. The transfer matrix T2 of the fitted ground plane relative to the radar system is obtained. The fitted ground plane maintains the same X and Y coordinates as the original ground point cloud.

[0053] Traditional algorithm for extracting point clouds of ground water accumulation: First, potential water accumulation areas are initially delineated based on the point cloud reflection intensity being less than a threshold. Then, regional filtering is performed using the plane height variance to extract non-water accumulation areas.

[0054] Deep learning algorithm for extracting point clouds of ground water accumulation: Based on the collected point cloud data, perform semantic segmentation and annotation of water accumulation, train a semantic segmentation model for water accumulation point clouds, and directly use the model to extract semantic segmentation from water accumulation point clouds.

[0055] A fusion algorithm for extracting surface water point clouds using traditional and deep learning algorithms: The union of the segmentation results from the two algorithms is taken as the fused semantic segmentation result of the surface water point clouds.

[0056] 3 Fusion Algorithm

[0057] Given the radar's transfer matrix T1 in camera coordinates, and the ground's transfer matrix T2 in radar coordinates. Specifically, this includes:

[0058] 1) When the image semantic segmentation category confidence is greater than 0.5, the image contour points are projected onto the ground fitting plane to construct the point cloud convex hull contour. The contour is expanded outward by the expansion boundary width WE1 to obtain the point cloud within the contour. The point cloud is then filtered and target detection is performed to extract attributes, specifically including area and relative depth.

[0059] 2) When the image semantic segmentation category confidence is less than or equal to 0.5 and greater than 0.2, the image contour points are projected onto the ground fitting plane to construct the point cloud convex hull contour. The contour is expanded by the extended boundary width WE2 to obtain the point cloud inside the contour. The point cloud is then filtered and the target is detected to extract attributes.

[0060] 3) When the image semantic segmentation category confidence is less than or equal to 0.2, the image semantic segmentation accuracy is poor and the category confidence is low. The image segmentation related results no longer participate in the subsequent algorithm modules. At this time, the water accumulation is identified and the attributes are extracted according to the point cloud slurry detection method.

[0061] V. Decision Control:

[0062] The results of water accumulation and attribute identification are sent to the decision control module, and the vehicle performs relevant operations.

[0063] This invention provides a method for detecting water accumulation on mining roads based on the fusion of vision and lidar. For the task of detecting and identifying water accumulation in mining areas, current methods based on single sensors such as radar or images face significant challenges. The technical difficulty lies in effectively separating the road surface from the water accumulation. Furthermore, while traditional visual methods based on single sensors such as cameras can identify the outline of the water accumulation, their accuracy and stability are poor. They also struggle to obtain key attribute information such as the water's height relative to the ground, the depth of the puddles, and the area of ​​the water accumulation, making it difficult to support vehicle decision-making and control. The multi-sensor fusion perception approach, which introduces a deep learning-based water accumulation semantic segmentation algorithm, effectively addresses the problems of single-sensor perception and offers higher real-time performance, robustness, and confidence.

[0064] Specific implementation steps:

[0065] Step 1: Vehicle hardware installation;

[0066] S101, a lidar and video camera are installed in front of the vehicle, with the installation angle fixed and locked.

[0067] Step 2: Perform joint calibration of the lidar and camera;

[0068] S201: Perform intrinsic and extrinsic parameter calibration and joint calibration on the lidar and video camera. The calibration will obtain the camera intrinsic parameter matrix, distortion correction coefficient, translation matrix of the radar relative to the camera imaging coordinate system, and rotation vector of the radar relative to the vehicle center coordinate system.

[0069] S202: After acquiring video camera image data, perform distortion correction;

[0070] Step 3: Image deep learning algorithm training;

[0071] S301: Collect data from different environmental scenarios to train a water accumulation semantic segmentation model. Before inference of the water accumulation semantic segmentation model, perform distortion correction preprocessing on the real-time acquired images.

[0072] Step 4: Model Deployment;

[0073] S401: In the image algorithm deployment, after being fed into the inference model, the semantic segmentation result is obtained. The post-processing of the semantic segmentation result includes binarization, erosion and dilation, region connectivity, pixel area filtering, and edge contour detection operations, and the edge convex hull contour pixel data is obtained and stored.

[0074] S402: During radar data acquisition and preprocessing, dust filtering, drivable area segmentation, and ground segmentation are performed on the raw point cloud data. Point cloud data at different levels is then fed in, and the point cloud algorithm is deployed to perform corresponding data processing. Ground point cloud planar modeling enables planar fitting modeling of the extracted ground point cloud, obtaining the optimal fitting plane for the ground point cloud. This facilitates the projection of image pixels onto the ground in the point cloud coordinate system during sensor fusion.

[0075] Step 5: Extraction of water point clouds;

[0076] S501: The results of traditional and deep learning algorithms for extracting water accumulation point clouds are fused. If the confidence level of the deep learning algorithm for extracting water accumulation point clouds is greater than 0.5, the union of the segmentation results of the two algorithms is taken as the fused semantic segmentation result of the water accumulation point cloud; otherwise, the "water accumulation detection module unavailable" status information is sent directly to the decision control module.

[0077] Step 6: Deployment of point cloud fusion algorithm for water accumulation image

[0078] S601: The deployment of the water accumulation image point cloud fusion algorithm mainly includes fitting the ground projection of the water accumulation contour of the image to the point cloud coordinate system and projecting the point cloud to the pixel coordinate system.

[0079] S602: Image radar fusion algorithm logic flow: When the image semantic segmentation category confidence is greater than 0.5, the image contour points are projected onto the ground fitting plane to construct the point cloud convex hull contour. The contour is expanded outward according to the extended boundary width WE1 to obtain the point cloud inside the contour. The point cloud is filtered and the target is detected. It is determined whether the obstacle is water accumulation and the attributes are extracted.

[0080] When the image semantic segmentation category confidence is less than or equal to 0.5 and greater than 0.2, the image contour points are projected onto the ground fitting plane to construct the point cloud convex hull contour. The contour is expanded outward by the extended boundary width WE2 to obtain the point cloud inside the contour. The point cloud is filtered and the target is detected. It is determined whether the obstacle is water accumulation and the attributes are extracted.

[0081] When the confidence level of the image semantic segmentation category is less than or equal to 0.2, the image semantic segmentation accuracy is poor, and the relevant results are no longer used in subsequent algorithm modules. At this time, the water accumulation is identified and its attributes are extracted by fusing the results of traditional algorithms and deep learning algorithms for ground water accumulation point cloud extraction.

[0082] Specifically, the attributes include the area and relative depth of the water.

[0083] Step 7: Identification results and attribute output;

[0084] S701: Extract the identification and attributes of the accumulated water and send them to the decision control module, so that the vehicle can perform relevant operations.

[0085] In the description of this invention, it should be understood that the terms "longitudinal", "lateral", "up", "down", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this invention, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention.

[0086] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A mine area road surface water detection method based on fusion of vision and laser radar, characterized in that, The method comprises the following steps: S1: obtaining road surface image data through a vehicle-mounted camera sensor and performing water accumulation segmentation to obtain edge profile pixel data of water accumulation and image confidence thereof; S2: obtaining point cloud data through a vehicle-mounted laser radar and determining whether the image confidence is qualified, if not, removing non-water accumulation point cloud according to a reflection threshold and plane height variance to obtain a first segmentation result, performing segmentation according to water accumulation semantic segmentation annotation to obtain a second segmentation result, and fusing the first segmentation result and the second segmentation result as a water accumulation recognition result; if qualified, extracting ground point cloud in the point cloud data and modeling to obtain a fitting plane, projecting the edge profile pixel data of the water accumulation to the fitting plane to construct a point cloud convex hull profile, and recognizing the point cloud convex hull profile to obtain the water accumulation recognition result; S3: extracting water accumulation area and relative height of water accumulation and ground in the water accumulation recognition result in S2 and sending to a vehicle control module; The method for determining whether the image confidence is qualified comprises the following steps: setting a second threshold for determining whether the image confidence is qualified; when the image confidence of the edge profile pixel data is less than the second threshold, determining that the edge profile pixel data is unqualified; when the image confidence of the edge profile pixel data is not less than the second threshold, determining that the edge profile pixel data is qualified; wherein the second threshold is set to 0.2; The step of recognizing the point cloud convex hull profile to obtain the water accumulation recognition result comprises the following steps: determining segmentation accuracy of edge profile pixel data constituting the point cloud convex hull profile; according to the determination result, setting different extension boundary widths to expand the profile of the edge profile pixel data to obtain point cloud in the profile; filtering and target detecting the point cloud in the profile to obtain the water accumulation recognition result; The segmentation accuracy is determined by the following steps: setting a third threshold for determining segmentation accuracy and higher than the second threshold, and obtaining image confidence corresponding to the edge profile pixel data; when the image confidence is greater than the third threshold, determining that the segmentation accuracy of the point cloud convex hull profile constituted by the edge profile pixel data is high; when the image confidence is not greater than the third threshold and greater than a first threshold, determining that the segmentation accuracy of the point cloud convex hull profile constituted by the edge profile pixel data is low; wherein the third threshold is set to 0.5; The different extension boundary widths set according to the determination result adopt the following rules: when the segmentation accuracy is determined to be high, the extension boundary width for expanding the profile of the edge profile pixel data is WE1; when the segmentation accuracy is determined to be low, the extension boundary width for expanding the profile of the edge profile pixel data is WE2; wherein WE1 is a width of 5 pixel values, and WE2 is a width of 10 pixel values.

2. The mine area road surface water detection method based on fusion of vision and laser radar according to claim 1, characterized in that, Before step S1, the detection method further comprises the following steps: installing a laser radar for obtaining point cloud data and a camera sensor for obtaining image data on a vehicle and calibrating; creating a point cloud water accumulation semantic segmentation model for obtaining the second segmentation result according to water accumulation semantic segmentation annotation; creating a ground water accumulation point cloud extraction algorithm for obtaining the first segmentation result.

3. The mine area road surface water detection method based on fusion of vision and laser radar according to claim 2, characterized in that, The step of segmenting according to the waterlogging semantic segmentation label to obtain a second segmentation result specifically comprises: acquiring historical point cloud data of different environments as training data; inputting the training data into a point cloud waterlogging semantic segmentation model to obtain waterlogging semantic segmentation labels; the point cloud waterlogging semantic segmentation model segments the point cloud data according to the waterlogging semantic segmentation labels to obtain a second segmentation result and a point cloud confidence thereof; setting a first threshold to determine whether the point cloud confidence is greater than the first threshold; if yes, outputting the second segmentation result, and if no, the waterlogging segmentation result is unavailable; wherein the first threshold is set to 0.

5.

4. The mine area road surface water detection method based on fusion of vision and laser radar according to claim 1, characterized in that, The coordinate system of the fitted plane is the same as that of the ground surface of the point cloud data.

5. The mine area road surface water detection method based on fusion of vision and laser radar according to claim 2, characterized in that, The step of acquiring road surface image data and performing waterlogging segmentation comprises: correcting the distortion of the image data to obtain a corrected image; inputting the corrected image into an image waterlogging semantic segmentation model to obtain a semantic segmentation result; sequentially performing binarization processing, erosion and expansion, region connection, pixel area filtering and edge contour detection on the semantic segmentation result to obtain the number of edge convex hull contour pixels.

Images