Point cloud reconstruction method and system for irregular coal pile surface in complex environment

By collecting lidar point cloud and multi-view image data in complex environments, performing time synchronization and extrinsic parameter calibration, removing outliers, and using deep learning models and prior data on coal pile structures, a virtual point cloud is generated, solving the problems of local distortion and data loss in the three-dimensional reconstruction of the coal pile surface, and achieving complete point cloud reconstruction.

CN122866504BActive Publication Date: 2026-11-10JIANGSU HAOHAN INFORMATION TECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202611317812.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-08-28
Publication Date
2026-11-10
Estimated Expiration
2046-08-28

AI Technical Summary

Technical Problem

Existing technologies suffer from local distortion and missing regions in the 3D reconstruction of coal pile surfaces under complex environments. In particular, laser scanning methods struggle to acquire complete point cloud data under humid and dusty conditions.

Method used

By collecting lidar point cloud data and multi-view image data, time synchronization and extrinsic parameter calibration are performed to establish a unified spatial coordinate system, outliers are removed, and a dense depth map is extracted using a deep learning visual depth estimation model. Combined with multi-source depth correlation fields and prior data of coal pile structure, a virtual point cloud is generated to fill in the missing data areas. Multi-source weighted fusion is then performed to generate a complete point cloud on the surface of the coal pile.

Benefits of technology

It suppresses local distortions in the measurement data, fills in missing data areas, and generates a complete point cloud of the coal pile surface without distortion or voids, thereby improving the accuracy and completeness of the three-dimensional reconstruction of the coal pile surface.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122866504B_ABST
    Figure CN122866504B_ABST
Patent Text Reader

Abstract

The application discloses a complex environment under irregular coal pile surface point cloud reconstruction method and system, and relates to the field of data processing, which comprises the following steps: acquiring laser radar point cloud data and multi-view image data of the coal pile area, performing time synchronization processing and external parameter calibration to establish a data mapping relationship; performing statistical filtering and outlier rejection on the point cloud data, identifying an abnormal area to establish an initial effective point cloud set; inputting the image data into a visual depth estimation model to extract coal pile surface features and output a dense depth map; after spatial registration of the dense depth map and the initial effective point cloud set, a multi-source depth correlation field is established, depth continuity constraints of the edge of the missing area and coal pile structure priori data are extracted, and a virtual point cloud set is constructed; and multi-source weighted fusion processing is performed to generate a point cloud reconstruction result. The application solves the problems of local distortion and area loss of the measurement data in the existing coal pile surface reconstruction, and achieves the effect of suppressing local distortion in the data and filling the missing area of the data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of data processing, and in particular to a method and system for reconstructing point clouds on the surface of irregular coal piles in complex environments. Background Technology

[0002] Accurate measurement of coal pile volume and morphology plays a crucial role in coal mine inventory management, production scheduling, and transportation loading, directly impacting resource accounting and operational efficiency. Current methods for acquiring geometric information about coal pile surfaces primarily employ 3D laser scanning and structured light measurement. Specifically, this includes multi-station stitched scanning of coal piles using stationary ground-based lidar scanners, and line-by-line acquisition of point cloud data from the coal pile surface using portable 3D scanning equipment based on time-of-flight or phase-based ranging principles. However, in humid, dusty coal working environments, lidar beams are prone to refraction in areas of localized water accumulation on the coal pile surface, causing echo signals to deviate from the true reflection position and resulting in depth measurement errors. Simultaneously, the scattering and absorption of the laser beam by suspended coal dust particles drastically attenuates or even completely eliminates the echo intensity, creating large areas of voids in the point cloud data. Furthermore, areas covered by high-carbon, fine coal ash have extremely low reflectivity, causing echo signals to be submerged in system noise, making it difficult to obtain effective ranging values. These factors collectively lead to significant local gaps and measurement distortions in the point cloud data acquired by existing laser scanning methods in complex coal pile environments.

[0003] At present, the three-dimensional reconstruction of coal pile surfaces in complex environments faces technical problems such as local distortion and missing regions of measurement data. Summary of the Invention

[0004] This application provides a method and system for reconstructing point clouds on the surface of irregular coal piles in complex environments. It collects lidar point cloud data and multi-view image data of the target coal pile area, unifying the two types of data into the same spatial coordinate system through time synchronization and extrinsic parameter calibration. Statistical filtering is applied to the lidar point cloud to remove outliers, and abnormal regions such as dust obstruction and water refraction are identified based on local density, curvature changes, and echo intensity fluctuations, resulting in an initial effective point cloud set. Simultaneously, the multi-view images are input into a deep learning visual depth estimation model, outputting dense depth maps corresponding to each viewpoint. The dense depth maps are registered and aligned with the initial effective point cloud to construct a multi-source depth correlation field that integrates the information from both sources, from which the depth variation patterns at the edges of missing regions are extracted. In addition to the prior knowledge of the structural morphology of the coal pile surface, based on the aforementioned depth continuity constraints and structural prior knowledge, virtual point clouds are generated within the data-missing areas to fill the gaps. Finally, the virtual point cloud, the initial valid point cloud, and the dense depth map are mapped onto a unified spatial grid. The fusion weights are dynamically allocated according to the local confidence of each source data. After weighted fusion and surface continuity correction, a complete, continuous point cloud reconstruction result that conforms to the real morphology of the coal pile is output. These technical means solve the technical problems of local distortion and regional missing data in the existing three-dimensional reconstruction of the coal pile surface under complex environments. The technical effect of suppressing local distortion in the measurement data and filling the data-missing areas is achieved, generating a complete coal pile surface point cloud without distortion and voids.

[0005] This application provides a method for reconstructing point clouds on the surface of irregular coal piles in complex environments, including: acquiring lidar point cloud data and multi-view image data of the target coal pile area; performing time synchronization processing and extrinsic parameter calibration on the lidar point cloud data and multi-view image data to establish a multi-source sensing data mapping relationship under a unified spatial coordinate system; performing statistical filtering and outlier removal on the lidar point cloud data; identifying abnormal regions based on the local density distribution characteristics, spatial curvature change characteristics, and echo intensity fluctuation characteristics of the point cloud; and establishing an initial effective point cloud set; and inputting the multi-view image data into deep learning. A visual depth estimation model extracts texture continuity features, shadow variation features, and edge morphology features from the surface of the coal pile, outputting dense depth maps corresponding to images from each viewpoint. After spatial registration of the dense depth maps with an initial valid point cloud set, a multi-source depth correlation field is established. This field is then used to extract depth continuity constraints at the edges of missing regions and prior data on the coal pile structure. A virtual point cloud set is constructed using the depth continuity constraints and prior data. Finally, a multi-source weighted fusion process is performed on the virtual point cloud set, the initial valid point cloud set, and the dense depth map to generate the point cloud reconstruction result.

[0006] In a possible implementation, the depth continuity constraints of the missing region edges and prior data of the coal pile structure are extracted using the multi-source depth correlation field, and the following processing is performed: the spatially registered dense depth map and the initial effective point cloud set are mapped to a unified voxel space, and a multi-source depth correlation field is constructed based on the real depth distribution, visual depth change trend, and spatial neighborhood connectivity within each voxel; a depth continuity propagation path corresponding to each spatial location is established using the multi-source depth correlation field, and gradient expansion analysis is performed on the edges of each missing region along the depth continuity propagation path to extract the depth decay trend, surface tilt change trend, and local curvature continuity trend of the missing region edges in different directions, generating depth continuity constraints; based on the point cloud accumulation morphology, slope extension direction, and elevation distribution law of adjacent regions in the multi-source depth correlation field, the natural accumulation boundary features and particle flow evolution features of the coal pile surface are extracted to construct prior data of the coal pile structure.

[0007] In a possible implementation, a virtual point cloud set is constructed using depth continuity constraints and prior coal pile structure data, and the following processing is performed: the depth extension direction of the missing region edge is located using depth continuity constraints, and spatial interpolation extension processing is performed on the interior of the missing region along the depth extension direction to generate the initial virtual point distribution of the missing region; the initial virtual point distribution is subjected to morphological constraint processing based on the prior coal pile structure data, and the spatial position and spatial distribution density of each virtual point are adjusted according to the natural accumulation slope variation law of the coal pile, the continuous undulation law of the local surface, and the particle flow accumulation direction to generate a virtual point cloud set that conforms to the real accumulation morphology of the coal pile.

[0008] In a possible implementation, a multi-source weighted fusion process is performed on the virtual point cloud set, the initial valid point cloud set, and the dense depth map to generate a point cloud reconstruction result. The following processing is performed: the virtual point cloud set, the initial valid point cloud set, and the dense depth map are mapped to a unified spatial grid. A multi-source correspondence is established based on the depth consistency, spatial distribution continuity, and neighborhood density correlation of different source points within each spatial grid. Based on the multi-source correspondence and the point cloud echo intensity stability of the initial valid point cloud set, the depth estimation confidence in the dense depth map, and the virtual point confidence parameters in the virtual point cloud set, dynamic fusion weights for point clouds from different sources are generated. Multi-source weighted fusion processing is performed according to the dynamic fusion weights to generate the point cloud reconstruction result.

[0009] In a possible implementation, multi-source weighted fusion processing is performed according to the dynamic fusion weights to generate point cloud reconstruction results, and the following processing is performed: local surface continuity constraint analysis is performed on adjacent point clouds after weighted fusion; iterative smoothing correction and outlier backoff processing are performed using the results of the local surface continuity constraint analysis to generate point cloud reconstruction results.

[0010] In a possible implementation, statistical filtering and outlier removal are performed on the lidar point cloud data. The following processing is performed: the average distance deviation feature and spatial dispersion feature of adjacent points in the lidar point cloud data are calculated; statistical filtering and outlier removal are performed based on the average distance deviation feature and spatial dispersion feature to establish a preprocessed point cloud set.

[0011] In a possible implementation, abnormal regions are identified based on the local density distribution characteristics, spatial curvature variation characteristics, and echo intensity fluctuation characteristics of the point cloud. The following processing is performed: the preprocessed point cloud set is divided into multiple continuous spatial neighborhoods, and the mean local point spacing, variance of point spacing, and rate of change of the number of neighborhood points in each spatial neighborhood are calculated. Low-density abnormal regions are marked according to the calculation results. The echo intensity sequence corresponding to the low-density abnormal regions is extracted along the laser scanning direction, and the echo intensity attenuation rate and attenuation fluctuation frequency between adjacent scanning points are calculated. Regions with continuously attenuating echo intensity and attenuation fluctuation frequency higher than a preset threshold are marked as dust-covered regions. Local surface fitting processing is performed on the point cloud surface in the low-density abnormal regions. The change in normal angle and curvature jump amplitude between adjacent surfaces are calculated. Combined with the offset angle between the laser incident direction and the echo direction, regions with abrupt normal changes and echo direction offset exceeding a preset angle threshold are marked as water surface refraction regions. Dust-covered regions and water surface refraction regions are output as abnormal regions.

[0012] In a possible implementation, the following processing is performed: the abnormal area includes a dust-covered area, a water surface refraction abnormal area, and a low-reflectivity coal ash covered area.

[0013] In a possible implementation, the following processing is performed: the multi-view image data includes a sequence of coal pile surface images acquired at different observation angles, different exposure conditions, and different times, which is used to enhance the reconstruction integrity of the dense depth map.

[0014] This application also provides a point cloud reconstruction system for irregular coal pile surfaces under complex environments, including: a multi-source sensing data mapping relationship establishment module, used to acquire lidar point cloud data and multi-view image data of the target coal pile area, perform time synchronization processing and extrinsic parameter calibration on the lidar point cloud data and multi-view image data, and establish a multi-source sensing data mapping relationship under a unified spatial coordinate system; an initial effective point cloud set establishment module, used to perform statistical filtering and outlier removal on the lidar point cloud data, identify abnormal areas based on the local density distribution characteristics, spatial curvature change characteristics and echo intensity fluctuation characteristics of the point cloud, and establish an initial effective point cloud set; and a dense depth map output module, used to input the multi-view image data into a deep depth map. A depth estimation model is implemented to extract texture continuity features, shadow variation features, and edge morphology features from the surface of the coal pile, and output dense depth maps corresponding to images from various viewpoints. A multi-source depth correlation field establishment module is used to establish a multi-source depth correlation field by spatially registering the dense depth map with the initial effective point cloud set, and then using the multi-source depth correlation field to extract depth continuity constraints of missing region edges and prior data of the coal pile structure. A virtual point cloud set construction module is used to construct a virtual point cloud set using depth continuity constraints and prior data of the coal pile structure. A point cloud reconstruction result generation module is used to perform multi-source weighted fusion processing on the virtual point cloud set, the initial effective point cloud set, and the dense depth map to generate point cloud reconstruction results.

[0015] The proposed method and system for reconstructing point clouds on irregular coal pile surfaces in complex environments involves, firstly, acquiring lidar point cloud data and multi-view image data of the target coal pile area. Time synchronization processing and extrinsic parameter calibration are then performed on the lidar point cloud data and multi-view image data to establish a multi-source sensing data mapping relationship under a unified spatial coordinate system. Next, statistical filtering and outlier removal are performed on the lidar point cloud data. Based on the local density distribution characteristics, spatial curvature variation characteristics, and echo intensity fluctuation characteristics of the point cloud, abnormal regions are identified, and an initial effective point cloud set is established. Finally, the multi-view image data is input into the depth... A visual depth estimation model is learned to extract texture continuity features, shadow variation features, and edge morphology features of the coal pile surface, outputting dense depth maps corresponding to images from various viewpoints. Then, the dense depth maps are spatially registered with an initial valid point cloud set to establish a multi-source depth correlation field. This field is used to extract depth continuity constraints at the edges of missing regions and prior data on the coal pile structure. A virtual point cloud set is then constructed using these constraints and prior data. Finally, the virtual point cloud set, the initial valid point cloud set, and the dense depth map are subjected to multi-source weighted fusion processing to generate a point cloud reconstruction result. Through this process, the method and system proposed in this application achieve the technical effect of suppressing local distortions in measurement data and filling in missing data regions, generating a distortion-free and void-free complete coal pile surface point cloud. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings of the embodiments of the present invention will be briefly described below. Flowcharts are used in this application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed precisely in sequence. Instead, various steps can be processed in reverse order or simultaneously as needed. Furthermore, other operations can be added to these processes, or one or more steps can be removed from these processes.

[0017] Figure 1 This is a flowchart illustrating the point cloud reconstruction method for irregular coal pile surfaces in complex environments, as provided in the embodiments of this application.

[0018] Figure 2 A schematic diagram of the structure of the point cloud reconstruction system for irregular coal pile surfaces in a complex environment provided in this application embodiment.

[0019] Figure labeling: Module 10 for establishing multi-source sensing data mapping relationship, Module 20 for establishing initial effective point cloud set, Module 30 for outputting dense depth map, Module 40 for establishing multi-source depth correlation field, Module 50 for constructing virtual point cloud set, and Module 60 for generating point cloud reconstruction result. Detailed Implementation

[0020] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0021] This application provides a method for reconstructing point clouds on the surface of irregular coal piles in complex environments, such as... Figure 1 As shown, the method includes: Step S100: Acquire lidar point cloud data and multi-view image data of the target coal pile area; perform time synchronization processing and extrinsic parameter calibration on the lidar point cloud data and multi-view image data; establish a multi-source sensing data mapping relationship under a unified spatial coordinate system; wherein, the multi-view image data includes a sequence of coal pile surface images acquired at different observation angles, different exposure conditions and different times, which is used to enhance the reconstruction integrity of dense depth maps.

[0022] Specifically, LiDAR point cloud data is acquired using a 3D laser scanner, while multi-view image data is acquired using high-resolution industrial cameras positioned at different locations. During acquisition, the exposure parameters, gain parameters, and acquisition timestamps of each camera are recorded simultaneously. Time synchronization processing is performed on the LiDAR point cloud data and multi-view image data: using the time reference provided by the Global Positioning System or a timing module, the echo timestamps of each scanning beam of the LiDAR are aligned with the image exposure start timestamps of each camera. Cubic spline interpolation is used to adjust the image sampling time to be consistent with the acquisition time of each scanning point within the LiDAR scanning cycle, with the interpolation time error controlled within ±1ms. Then, external parameter calibration is performed: multiple checkerboard calibration plates or spherical calibration objects are set up around the target coal pile area. The point cloud and image data of the calibration objects are collected simultaneously by the lidar and each camera. Based on the correspondence between the corner points or centers of the calibration objects in the point cloud and the image, the PnP (Perspective-n-Point) algorithm is used to solve the rotation matrix and translation vector between the lidar coordinate system and the coordinate systems of each camera. The calibration accuracy of the rotation matrix is ​​better than 0.5° and the calibration accuracy of the translation vector is better than 2cm. Then, all sensor data are unified into the preset world coordinate system to establish a multi-source sensing data mapping relationship under a unified spatial coordinate system. The multi-view image data includes a sequence of coal pile surface images acquired at different observation angles, exposure conditions, and times. The different observation angles are used to provide appearance information of the same spatial location on the coal pile surface under different line-of-sight directions to eliminate occlusion and perspective ambiguity under a single viewpoint. The different exposure conditions are used to fuse high dynamic range textures to enhance the image detail discernibility of low-light or high-reflectivity coal dust areas. The image sequences acquired at different times are used to filter out local texture flickering anomalies caused by dynamic dust using temporal consistency constraints. The joint input of the above multi-dimensional information is used to provide complementary visual features for the deep learning visual depth estimation model in step S300, thereby generating a dense depth map with continuous depth and matching texture in the area where the lidar data is missing, achieving the purpose of enhancing the integrity of dense depth map reconstruction.

[0023] Step S200: Perform statistical filtering and outlier removal on the lidar point cloud data. Identify abnormal regions based on the local density distribution characteristics, spatial curvature change characteristics, and echo intensity fluctuation characteristics of the point cloud, and establish an initial effective point cloud set. The abnormal regions include dust-covered regions, water surface refraction abnormal regions, and low-reflectivity coal ash covered regions.

[0024] Specifically, firstly, by calculating the average distance and distance variance between each point and its neighboring points, spatially isolated points that significantly deviate from the overall distribution are filtered out. Then, based on this, the point cloud is divided into multiple local regions, and the density of points, surface curvature, and the degree of change in echo signal intensity within each region are analyzed. This allows for the identification of regions where echoes are abnormally attenuated due to dust obstruction, regions where point cloud positions are shifted due to water refraction, and regions where echoes are too weak or noise is increased due to low-reflectivity coal ash coverage. This process eliminates invalid measurement points caused by environmental interference and accurately locates unreliable or missing spatial positions in the original data, establishing a clear spatial range for subsequent compensation of these regions using image data.

[0025] In one possible implementation, statistical filtering and outlier removal are performed on the lidar point cloud data. Step S200 further includes step S210, which calculates the average distance deviation characteristic and spatial dispersion characteristic of adjacent points in the lidar point cloud data. Specifically, for the input 3D point cloud set, a K-tree is used to establish a point cloud spatial index. The leaf node capacity of the K-tree is set to 32. For each query point, all neighboring points within its neighborhood radius are searched. The neighborhood radius is calculated based on the lidar angular resolution and scanning distance. For example, it can be set to one-thousandth of the scanning distance, i.e., neighborhood radius = 0.001 × scanning distance. The scanning distance ranges from 10m to 100m, and the corresponding neighborhood radius ranges from 1cm to 10cm. The Euclidean distance between the query point and each of its neighboring points is calculated. The sum of all Euclidean distances is divided by the total number of neighboring points to obtain the local average distance of the query point. Calculate the difference between the Euclidean distance and the local average distance between each neighboring point and the query point. Sum the squares of each difference and divide by the total number of neighboring points to obtain the local distance variance of the query point, which serves as a feature of spatial dispersion.

[0026] Step S220: Statistical filtering and outlier removal are performed based on the average distance deviation characteristics and spatial dispersion characteristics to establish a preprocessed point cloud set. Specifically, the global average distance of the entire point cloud set is calculated by summing the local average distances of all query points and dividing by the total number of query points. Simultaneously, the global distance variance of the entire point cloud set is calculated by summing the local distance variances of all query points and dividing by the total number of query points. Then, a distance threshold multiple and a preset multiple are set. The distance threshold multiple is a scaling factor used to determine whether the local average distance is too large. Its physical meaning is: when the local average distance of a point exceeds this multiple of the global average distance, it indicates that the distance between that point and other points in its neighborhood significantly deviates from the overall distribution and should be identified as an outlier. The preset multiple is a scaling factor used to determine whether the local distance variance is too large. Its physical meaning is: when the local distance variance of a point exceeds this multiple of the global distance variance, it indicates that the dispersion of the distance between that point and its neighboring points is abnormal and should be identified as an outlier. In this embodiment, the distance threshold multiple is set to 2.5, and the preset multiple is set to 2.0. During engineering deployment, the values ​​of these two multiples range from 2.0 to 3.5 and from 1.5 to 3.0, respectively. The specific values ​​are adjusted based on the actual angular resolution of the lidar scan line, the scanning distance of the target coal pile, and the on-site dust concentration: the higher the angular resolution, the closer the scanning distance, and the lower the dust concentration, the lower the value should be; conversely, the lower the angular resolution, the farther the scanning distance, and the higher the dust concentration, the higher the value should be. For each query point, if its local average distance is greater than the product of the global average distance and the distance threshold multiple, or its local distance variance is greater than the product of the global distance variance and the preset multiple, then the query point is determined to be an outlier and removed from the point cloud set. After traversing all points, the remaining points constitute a preprocessed point cloud set.

[0027] In one possible implementation, based on the local density distribution characteristics, spatial curvature variation characteristics, and echo intensity fluctuation characteristics of the point cloud, abnormal regions are identified. Step S200 further includes step S230, which divides the preprocessed point cloud set into multiple continuous spatial neighborhoods, calculates the mean local point spacing, variance of point spacing, and rate of change of the number of neighborhood points in each spatial neighborhood, and marks low-density abnormal regions based on the calculation results. Specifically, a three-dimensional mesh is established using a voxel downsampling method, and the voxel side length is set according to the LiDAR scan line spacing. The LiDAR scan line spacing refers to the physical interval between two adjacent scan lines of the LiDAR at the same distance, perpendicular to the scanning direction. This interval is calculated by multiplying the angular resolution of the LiDAR by the scanning distance. The specific method for setting the voxel side length according to the scan line spacing is as follows: the voxel side length should not be less than the scan line spacing to ensure that each voxel contains point cloud data from at least one scan line, and should not be greater than twice the scan line spacing to avoid excessive smoothing of local density features due to too many scan lines contained in a single voxel. In this embodiment, the voxel side length = 1.5 × LiDAR scan line spacing, and the LiDAR scan line spacing = LiDAR angular resolution (angle values ​​converted to radians for calculation) × scanning distance. The LiDAR angular resolution ranges from 0.01° to 0.1°, and the scanning distance ranges from 10m to 100m, corresponding to a voxel side length range of 0.26cm to 26.2cm. During engineering deployment, the voxel side length is dynamically calculated based on the actual angular resolution of the LiDAR and the average scanning distance of the target coal pile: the larger the product of the angular resolution and the scanning distance, the larger the voxel side length increases proportionally; conversely, it decreases. For each voxel grid and its twenty-six adjacent neighboring voxels, all points falling within these voxels form a continuous spatial neighborhood. For each point within this continuous spatial neighborhood, a K-dimensional tree is used to search for its nearest neighbor within this neighborhood, and the Euclidean distance between them is calculated to obtain the nearest neighbor distance of that point. The mean local nearest neighbor distance is obtained by summing the nearest neighbor distances of all points within the neighborhood and dividing by the total number of points in the neighborhood. This is the mean local point spacing. The mean local nearest neighbor distance is then subtracted from the mean local nearest neighbor distance of each point, and the squares are summed and divided by the total number of points in the neighborhood. This yields the variance of the local nearest neighbor distance, which is the variance of the point spacing. The rate of change in the number of neighborhood points is calculated by dividing the difference between the number of points in the current spatial neighborhood and the number of points in adjacent spatial neighborhoods by the number of points in adjacent spatial neighborhoods. If the mean local nearest neighbor distance is greater than a set multiple (1.5 times) of the global average nearest neighbor distance, or the variance of the local nearest neighbor distance is greater than a set multiple (2.0 times) of the global nearest neighbor distance variance, or the rate of change in the number of neighborhood points is greater than a preset threshold (50%), then the spatial neighborhood is marked as a low-density anomaly region. Among them, the global average nearest neighbor distance is the average of the mean local nearest neighbor distances of all continuous spatial neighborhoods, and the global nearest neighbor distance variance is the average of the variances of the local nearest neighbor distances of all continuous spatial neighborhoods.

[0028] Step S240: Extract the echo intensity sequence corresponding to the low-density anomaly region along the laser scanning direction, calculate the echo intensity attenuation rate and attenuation fluctuation frequency between adjacent scanning points, and mark the region with continuously attenuating echo intensity and attenuation fluctuation frequency higher than a preset threshold as a dust-covered region. Specifically, for each point marked as a low-density anomaly region, arrange these points in ascending order of time according to the scan line number and scan timestamp recorded during lidar acquisition to obtain the echo intensity sequence along the scanning direction. Calculate the echo intensity attenuation rate between adjacent scanning points: divide the difference in echo intensity between two adjacent points by the spatial distance between the two points. Calculate the attenuation fluctuation frequency: count the number of attenuation rate sign changes in the attenuation rate sequence of multiple consecutive adjacent scanning point pairs, and divide the number of changes by the number of scanning points to obtain the attenuation fluctuation frequency. Set a negative threshold for the attenuation rate and a threshold for the fluctuation frequency. The physical meaning of the negative attenuation rate threshold is that when the attenuation rate of the echo intensity between adjacent scanning points is lower than this negative threshold, it indicates that the echo intensity has decreased significantly within a unit distance. This decrease is usually caused by the scattering and absorption of the laser signal by dust particles present on the scanning path, and is a quantitative basis for identifying dust-obstructed areas. The principle behind setting the negative attenuation rate threshold is as follows: In clean air without dust interference, the LiDAR echo intensity decreases slowly and steadily with increasing distance, and the absolute value of the attenuation rate is small; when dust obstruction exists, the echo intensity decreases sharply within a very short distance passing through the dust area, and the absolute value of the attenuation rate increases significantly. Therefore, the negative attenuation rate threshold is determined by multiplying the normal attenuation rate of the echo intensity under clean air conditions by a safety margin factor. The normal attenuation rate under clean air conditions is approximately -0.1 dB / m to -0.2 dB / m. After multiplying by a safety margin factor of 3 to 5, the negative attenuation rate threshold is obtained as -1.0 dB / m to -0.3 dB / m. Considering the risk of false triggering caused by measurement noise, this embodiment sets it to -0.5 dB / m. The physical meaning of the fluctuation frequency threshold is: when the frequency of sign changes in the attenuation rate sequence is higher than this threshold, it indicates that the echo intensity exhibits violent oscillations rather than a monotonically decreasing trend in the scanning direction. This oscillation is usually caused by the random distribution of dust particles, resulting in alternating changes in the laser signal between obstruction and transmission. This is a key characteristic that distinguishes between dust obstruction and simple distance attenuation. The principle behind setting the fluctuation frequency threshold is as follows: In signal changes caused by simple distance attenuation or water surface refraction, the echo intensity exhibits a monotonous or slow trend change along the scanning direction, with fewer sign changes in the attenuation rate. However, in dust-covered areas, due to the non-uniform distribution of dust particles, the echo intensity fluctuates frequently within short distances, and the sign of the attenuation rate alternates rapidly. The fluctuation frequency threshold is determined based on the scanning point density of the lidar and the spatial distribution scale of dust particles. In this embodiment, it is set to 0.3, representing at least 3 sign changes within every 10 consecutive scanning points.If the echo intensity attenuation rate of more than 60% of adjacent point pairs in a continuous scan point is less than the negative attenuation rate threshold, and the attenuation fluctuation frequency is higher than the fluctuation frequency threshold, then the area is marked as a dust-covered area.

[0029] Step S250: Perform local surface fitting processing on the point cloud surface in the low-density anomaly region, calculate the change in the normal angle and the curvature jump amplitude between adjacent surfaces, and combine the offset angle between the laser incident direction and the echo direction to identify regions with abrupt normal changes and echo direction offsets exceeding a preset angle threshold as water surface refraction regions. Output the dust-covered region and the water surface refraction region as anomaly regions. Specifically, for each point in the low-density anomaly region and its neighboring points, use the least squares method to fit a quadratic surface z=ax. 2 +by 2 The formula is: +cxy+dx+ey+f, where x and y are the projected coordinates of the point on the horizontal plane, z is the elevation of the point, and a, b, c, d, e, and f are the undetermined coefficients of the quadratic surface obtained by least-squares fitting. After fitting, the normal vector of the local surface is obtained. The change in the normal angle between adjacent surfaces is calculated: the dot product of the normal vectors of two adjacent local surfaces is taken, the result is divided by the product of the magnitudes of the two normal vectors, and the inverse cosine is taken to obtain the normal angle. The absolute value of the difference in curvature values ​​between two adjacent local surfaces is taken as the curvature jump amplitude. The offset angle between the laser incident direction and the sensor received echo direction recorded during lidar scanning is obtained.

[0030] Thresholds for normal angle variation, curvature jump, and offset angle are set. The physical meaning of the normal angle variation threshold is: when the normal angle between adjacent local curved surfaces exceeds this threshold, it indicates an unnatural abrupt change in normal direction on the coal pile surface at that location. This abrupt change is usually caused by the refraction of the laser point cloud position due to water surface refraction, and is a core geometric feature for identifying water surface refraction areas. The principle behind setting the normal angle variation threshold is: on a natural coal pile surface, due to the packing characteristics of coal particles, the normal angle between adjacent areas is usually small, and the surface exhibits a continuous gradual change; when a laser beam passes through the water surface, because the refractive index of water is different from that of air, the laser propagation direction is deflected, causing the calculated point cloud position to shift towards the bottom of the water, thus forming a steep normal jump between the actual water surface position and the bottom position. The normal angle variation threshold is determined based on the statistical distribution of natural normal changes on the coal pile surface under conditions without refraction interference. Under natural conditions, the normal angle between adjacent voxels is usually less than 15°, therefore, in this embodiment, the threshold is set to 25° (including a safety margin).

[0031] The physical meaning of the curvature jump threshold is as follows: when the curvature jump amplitude between adjacent local curved surfaces exceeds this threshold, it indicates that the curvature of the coal pile surface has changed drastically at that location. This drastic change is usually associated with surface irregularities caused by the sinking of the point cloud position due to water refraction. The principle for setting the curvature jump threshold is as follows: due to the continuity of particle accumulation, the curvature of a natural coal pile surface gradually changes along the surface, and the curvature difference between adjacent areas is small; water refraction makes the originally flat water surface appear as a concave curved surface in the point cloud, while the actual coal pile surface at the water surface edge maintains normal curvature, resulting in a significant curvature jump at the interface between the two. The curvature jump threshold is determined jointly based on the natural curvature change range of the coal pile surface under non-refraction conditions and the ranging accuracy of the lidar. Under natural conditions, the curvature change between adjacent areas is usually less than 0.2m. -1 Therefore, in this embodiment, the threshold is set to 0.5m. -1 (Including safety margin).

[0032] The physical meaning of the offset angle threshold is as follows: when the offset angle between the laser incident direction and the sensor's received echo direction exceeds this threshold, it indicates that the reflection path of the laser beam after irradiating the target surface deviates from the specular reflection direction. This deviation is particularly significant in water surface refraction scenarios because, in addition to causing the point cloud position to sink due to refraction, the smooth surface of the water surface also causes a concentrated shift in the echo direction. The principle for setting the offset angle threshold is as follows: on the rough surface of a diffusely reflecting coal pile, the offset angle between the laser incident direction and the echo direction is relatively dispersed and has no obvious pattern; while in the water surface region, due to the specular reflection characteristics of the water surface, an effective echo can only be received when the sensor is located near the specular reflection direction. At this time, the offset angle between the incident direction and the echo direction satisfies the relationship that the incident angle equals the reflection angle. This offset angle is usually large and deterministic. The offset angle threshold is determined based on the sensor layout geometry parameters of the lidar and the optical characteristics of water surface specular reflection. The offset angle generated by water surface specular reflection is usually greater than 12°, so this embodiment sets the threshold to 15° (including a safety margin). If the angle between the normals of adjacent surfaces exceeds the normal angle change threshold and the curvature jump amplitude exceeds the curvature jump threshold, and the offset angle between the corresponding laser incident direction and the echo direction exceeds the offset angle threshold, then this region is marked as a water surface refraction region. Dust-covered regions and water surface refraction regions are output as abnormal regions.

[0033] Step S300: Input the multi-view image data into the deep learning visual depth estimation model, extract the texture continuity features, shadow change features and edge morphology features of the coal pile surface, and output the dense depth map corresponding to each view image.

[0034] Specifically, the deep learning visual depth estimation model adopts an encoder-decoder structure. The encoder consists of multiple convolutional and pooling layers stacked together to extract multi-scale feature maps from the input image. The decoder consists of multiple upsampling and convolutional layers, which gradually restore the feature maps output by the encoder to the original image resolution, and fuse the high-resolution features of the corresponding layers of the encoder through skip connections at each upsampling stage. The specific parameter configuration of the model is as follows: The encoder uses ResNet-50 as the backbone network, which contains 5 stages. The output feature map sizes are 1 / 2, 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the input image, and the number of channels are 64, 256, 512, 1024 and 2048, respectively. The decoder contains 4 upsampling modules. Each upsampling module consists of a bilinear upsampling layer (upsampling factor of 2), a 3×3 convolutional layer (stride of 1, padding of 1), a batch normalization layer and a ReLU activation function. Skip connections concatenate the feature map of the corresponding layer of the encoder with the upsampling result of the decoder along the channel dimension after adjusting the number of channels through a 1×1 convolution. The output layer contains a 3×3 convolutional layer (stride of 1, padding of 1), which maps the feature map to a single-channel depth map. The convolutional layers inside the network use the rectified linear unit (ReLU) as the activation function, and the output layer uses the hyperbolic tangent function (Tanh) to map the depth values ​​to a preset depth range. During training, supervised learning was performed using the NYU Depth V2 and KITTI Depth datasets. The loss function was the mean absolute error between the predicted and true depths. The initial learning rate was set to 1e-4, the batch size to 8, and the training epochs to 50. The Adam optimizer was used, with a weight decay coefficient of 1e-5. During inference, coal pile surface images from each viewpoint were adjusted to a resolution of 640×480 and input into the model one by one. The model output a single-channel dense depth map with the same resolution as the input images. Each pixel value represents the depth estimate of the corresponding spatial point in the world coordinate system. For multi-view inputs, the model infers independently for each viewpoint image. The output dense depth maps are projected onto a unified world coordinate system through the extrinsic parameter calibration relationship in step S100. The median of the depth estimates from all views at the same spatial location is taken as the final depth estimate for that location to enhance the robustness of the depth map.

[0035] Step S400: After spatial registration of the dense depth map with the initial effective point cloud set, a multi-source depth correlation field is established. The depth continuity constraints of the missing region edge and the prior data of the coal pile structure are extracted using the multi-source depth correlation field.

[0036] Specifically, the dense depth map generated in step S300 and the initial effective point cloud set obtained in step S200 are aligned to the same three-dimensional voxel grid using the previously calibrated spatial transformation relationship. Within each voxel, the actual depth value from the lidar and the depth value estimated by vision are fused to form a depth correlation field containing multi-source information. Then, starting from the boundary of the voxel in the missing region of this correlation field, that is, the boundary of the voxel that cannot be covered by either type of data, the depth attenuation trend, the surface tilt angle change trend, and the local curvature continuity trend are analyzed along the change direction of the depth value of adjacent voxels. This reveals the law of how the depth at the boundary of the missing region extends smoothly outward. At the same time, information reflecting the coal pile formation law, such as the natural accumulation boundary, slope extension direction, and particle flow direction of the coal pile surface, is extracted from the region with existing data in the correlation field. This provides two dimensions of basis for the subsequent generation of virtual points inside the cavity: first, the constraint of how the depth transitions continuously at the boundary; and second, the inherent morphological and structural laws of the coal pile surface.

[0037] In one possible implementation, the multi-source depth correlation field is used to extract the depth continuity constraints of the missing region edges and the prior data of the coal pile structure. Step S400 further includes step S410, which maps the spatially registered dense depth map and the initial effective point cloud set to a unified voxel space. A multi-source depth correlation field is constructed based on the true depth distribution within each voxel, the visual depth change trend, and the spatial neighborhood connectivity. Specifically, firstly, according to the extrinsic parameter calibration relationship established in step S100, the spatial coordinates of each point in the initial effective point cloud set are transformed to the camera coordinate system. Then, the points in the camera coordinate system are projected onto the image plane through camera intrinsic parameters to obtain the depth values ​​of the corresponding pixels in the dense depth map. A three-dimensional voxel mesh is established, and the voxel side length is set according to the ratio of the average spacing of the lidar point cloud to the image pixel resolution. The average spacing of the lidar point cloud refers to the average Euclidean distance between each point and its nearest neighbor in the initial effective point cloud set, reflecting the spatial sampling density of the lidar point cloud. The image pixel resolution refers to the physical size of a single pixel in actual three-dimensional space. It is calculated by multiplying the field of view of a single pixel at the average scanning distance of the target coal pile by that distance to obtain the spatial width corresponding to the pixel. The specific principle behind setting the voxel side length based on the ratio of the above two is as follows: As the basic unit for multi-source data fusion, a voxel should simultaneously accommodate both LiDAR point cloud and image depth information. If the voxel side length is too small, a single voxel may not contain any LiDAR points or effective image pixel projections, resulting in a large number of empty voxels. If the voxel side length is too large, a single voxel will mix multiple LiDAR points and image pixels at different depth levels, causing excessive smoothing of depth information and loss of local geometric details. Therefore, the voxel side length should strike a balance between the average spacing of the LiDAR point cloud and the image pixel resolution. The specific method for setting the voxel side length based on the ratio of the two is as follows: First, calculate the ratio of the average spacing of the LiDAR point cloud to the image pixel resolution. If this ratio is greater than 1, it indicates that the LiDAR point cloud density is lower than the image resolution. The voxel side length should be based primarily on the average spacing of the LiDAR point cloud, set to one to two times the average spacing to ensure that there is at least one LiDAR point within each voxel. If this ratio is less than 1, it indicates that the image resolution is lower than the LiDAR point cloud density. The voxel side length should be based primarily on the image pixel resolution, set to one to two times the spatial size corresponding to the pixel to ensure that there is at least one effective pixel projection within each voxel. If this ratio is close to 1, the voxel side length can be set to the larger of the two values. During engineering deployment, the value of the voxel side length is dynamically adjusted according to the actual sensor configuration: the higher the LiDAR angular resolution, the higher the camera resolution, and the closer the scanning distance, the smaller the voxel side length should be; conversely, the smaller the resolution, the larger the voxel side length should be.For example, when the scanning distance increases, leading to an increase in both the average spacing of the point cloud and the pixel resolution, the voxel side length increases proportionally. When only a higher resolution camera is used, the pixel resolution decreases, and the voxel side length should be recalculated based on the new ratio, potentially adjusted to a smaller value to match higher image details. For each voxel, the depth value and variance of the LiDAR point cloud falling within that voxel are statistically analyzed as the true depth distribution. The mean and variance of the visual depth estimates within the corresponding pixel region of that voxel in the image space are statistically analyzed as the visual depth variation trend. The connectivity between that voxel and its neighboring voxels is analyzed; if at least one LiDAR point or at least 50% of the pixels in each neighboring voxel have a valid depth estimate, then the spatial neighborhood is considered connected. The true depth distribution, visual depth variation trend, and spatial neighborhood connectivity are combined to construct a multi-source depth correlation field.

[0038] Step S420: Establish a continuous depth propagation path corresponding to each spatial location using a multi-source depth correlation field. Perform gradient expansion analysis along the edges of each missing region along the continuous depth propagation path to extract the depth attenuation trend, surface tilt angle change trend, and local curvature continuity trend of the missing region edges in different directions, generating depth continuity constraints. Specifically, for voxels marked as missing regions in the multi-source depth correlation field (i.e., voxels without both lidar points and effective visual depth estimates), search for voxels on the boundary of the missing region. Starting from the current boundary voxel, expand outward layer by layer along the positive and negative directions of each coordinate axis, according to the voxel adjacency relationship, with an upper limit of 20 layers. Record the depth value change sequence of each voxel on the expansion path. Gradient expansion analysis is performed on the edges of each missing region along the continuous propagation path: the ratio of the difference in depth values ​​between adjacent voxels along the path to the distance between voxel centers is calculated to obtain a depth gradient sequence; the difference in adjacent gradient values ​​in the depth gradient sequence is divided by the distance between voxel centers to obtain the rate of change of the depth decay trend; a local plane is fitted based on the spatial coordinates of each voxel along the path, and the angle between the normal vector of this plane and the horizontal plane is calculated to obtain the surface tilt angle change trend; the second difference of the depth values ​​of adjacent voxels is calculated along the path, and the second difference is divided by the square of the distance between voxel centers to obtain the local curvature continuity trend. The depth decay trend, the surface tilt angle change trend, and the local curvature continuity trend are combined to generate a depth continuity constraint.

[0039] Step S430: Based on the point cloud accumulation morphology, slope extension direction, and elevation distribution patterns of adjacent regions in the multi-source depth correlation field, extract the natural accumulation boundary features and particle flow evolution features of the coal pile surface to construct prior data for the coal pile structure. Specifically, for the voxel regions with existing point cloud data in the multi-source depth correlation field, a region growing algorithm is used to segment different slope regions: using the voxel with the smallest curvature as the seed point, adjacent voxels with a normal vector angle less than a preset angle threshold and a depth value difference less than a preset depth threshold are merged into the same region. The physical meaning of the preset angle threshold is: when the angle between the normal vectors of the fitting planes of two adjacent voxels is less than this threshold, it indicates that the local surfaces where these two voxels are located have similar orientations and belong to the same continuous slope. The principle behind setting the preset angle threshold is as follows: the surface of a coal pile is formed by the natural accumulation of coal particles. Within the same slope area, the surface orientation changes continuously and gently, and the angle between the normal vectors of adjacent voxels is usually small. However, at the boundaries between different slope areas, such as the transition between the slope top and the slope surface, or where different accumulation directions intersect, the angle between the normal vectors increases significantly. The preset angle threshold is determined based on the statistical distribution of the angle between the normal vectors within the same slope under the natural accumulation state of the coal pile surface. Under natural conditions, the angle between the normal vectors within the same slope is usually less than 25°, so in this embodiment, the threshold is set to 30° (including a safety margin). The physical meaning of the preset depth threshold is: when the absolute value of the difference between the depth values ​​of two adjacent voxels is less than this threshold, it indicates that the two voxels are at similar heights and belong to the same continuous accumulation layer. The principle behind setting the preset depth threshold is as follows: within the same coal pile slope area, the depth value changes continuously along the slope direction. The depth difference between adjacent voxels is determined by the product of the slope gradient and the horizontal distance between voxels, and this difference is relatively small under normal slope conditions; however, there are significant depth jumps between different accumulation layers or between collapsed and non-collapsed areas. The preset depth threshold is set based on the maximum natural slope of the coal pile surface and the voxel side length, ensuring that the depth difference between adjacent voxels is still acceptable as belonging to the same area under the maximum allowable slope conditions. The natural angle of repose of the coal pile is typically 30° to 40°, corresponding to a maximum slope of 0.58 to 0.84. Therefore, in this embodiment, the depth threshold is set to 1.5 times the voxel side length. For each segmented slope area, the spatial coordinates of the boundary voxels are calculated, and the natural accumulation boundary characteristics are fitted. Within each slope area, the direction vector from the highest voxel to the lowest voxel is calculated, and this direction vector is projected onto the horizontal plane to obtain the main direction of particle flow. The cumulative change in voxel depth values ​​along the main direction of particle flow is statistically analyzed to obtain the particle flow evolution characteristics, including the nonlinear decay curve of depth with flow distance. The natural accumulation boundary characteristics and particle flow evolution characteristics are then combined to form the prior data for coal pile structure.

[0040] Step S500: Construct a virtual point cloud set using depth continuity constraints and prior data on coal pile structure.

[0041] Specifically, guided by the depth continuity constraint extracted in step S400, interpolation is performed starting from the edge of the missing region and gradually extending into the missing region according to the trend of depth variation with spatial location, generating an initial virtual point distribution. Then, using the natural accumulation slope variation law, local surface continuous undulation law, and particle flow accumulation direction in the prior data of the coal pile structure, the position and density of the initial virtual points are adjusted so that the spatial arrangement and depth variation of the virtual points conform to the accumulation morphology characteristics of the real coal pile surface. Thus, in the void areas where neither lidar nor visual depth estimation can obtain effective data, a set of physically reasonable supplementary point clouds is generated, providing necessary data support for the subsequent reconstruction of the complete surface.

[0042] In one possible implementation, a virtual point cloud set is constructed using depth continuity constraints and prior data of the coal pile structure. Step S500 further includes step S510, which uses depth continuity constraints to locate the depth extension direction of the missing region's edge, and performs spatial interpolation extension processing along the depth extension direction to generate the initial virtual point distribution of the missing region. Specifically, for each voxel marked as missing, the depth continuity constraints generated in step S420 are obtained, and the depth decay trend terms in each direction are extracted. The direction with the smallest absolute value of the depth decay trend is selected as the main direction of depth extension because the depth change in this direction is the most gradual, which conforms to the natural transition characteristics of the coal pile surface. Along the main direction of depth expansion, spatial interpolation expansion processing is performed on the missing region: starting with the depth value of the voxels at the edge of the missing region, and using the depth decay trend value as the slope, the predicted depth value of each voxel position in the missing region is calculated step by step according to the voxel center spacing. For missing regions with an extension distance of more than 5 voxel side lengths, quadratic polynomial interpolation is used instead of linear interpolation to better fit the nonlinear depth change, generating the initial virtual point distribution of the missing region. Each virtual point is located at the center of the corresponding voxel, and the depth value is the predicted depth at that voxel.

[0043] Step S520 involves morphological constraint processing of the initial virtual point distribution based on the prior structural data of the coal pile. The spatial position and density of each virtual point are adjusted according to the natural slope variation law of the coal pile, the continuous undulation law of the local surface, and the particle flow and accumulation direction, generating a virtual point cloud set that conforms to the actual coal pile accumulation morphology. Specifically, the prior structural data of the coal pile generated in step S430 is obtained, and the set of slope extension directions in the natural accumulation boundary features and the depth decay curve in the particle flow evolution features are extracted. For each virtual point in the initial virtual point distribution, its corresponding slope region is located, and the main particle flow direction of that slope region is projected onto the horizontal plane where the virtual point is located to obtain the local accumulation direction. Based on the natural slope variation law of the coal pile, the normalized distance from the virtual point along the local accumulation direction to the slope region boundary is calculated. The normalized distance is substituted into the depth decay curve in the particle flow evolution features to obtain the theoretical depth correction value. The predicted depth of the initial virtual point and the theoretical depth correction value are weighted at a ratio of 0.6:0.4 to obtain the adjusted depth value. Based on the continuous undulation pattern of the local surface, for adjacent virtual points, if the difference in their adjusted depth values ​​exceeds a preset continuous undulation threshold, the depth values ​​are averaged and smoothed. According to the particle flow and accumulation direction, the spatial distribution density of the virtual points is adjusted: in the main particle flow direction, for every additional voxel spacing along the flow direction, the virtual point spacing increases by a coefficient of 1.05 to simulate the sparse distribution caused by particle flow on the coal pile surface. After the above adjustments, a virtual point cloud set that conforms to the actual accumulation morphology of the coal pile is generated.

[0044] Step S600: Perform multi-source weighted fusion processing on the virtual point cloud set, the initial valid point cloud set, and the dense depth map to generate the point cloud reconstruction result.

[0045] Specifically, point cloud data from the three sources mentioned above are mapped onto a unified spatial grid. For each grid location, the fusion weights of the data from each source are dynamically calculated based on the stability of the lidar point cloud echo intensity, the confidence level of the visual depth estimation, and the credibility of the virtual point cloud. Then, a weighted average is calculated to obtain the final depth value at that location. After the initial fusion is completed, local surface continuity analysis is performed on adjacent point clouds to check whether the normal angle and depth gradient are continuous. Discontinuous locations are iteratively smoothed and corrected, and outliers that still cannot meet the continuity requirements after correction are removed. Through this process, in areas where lidar measurement points are reliable, the measured points dominate the results; in areas where visual depth estimation is reliable, visual results supplement the results; and in areas where both are missing, virtual point clouds fill the gaps. Continuity constraints ensure that the final reconstructed surface is complete, continuous, and conforms to the true geometry of the coal pile.

[0046] In one possible implementation, a multi-source weighted fusion process is performed on the virtual point cloud set, the initial effective point cloud set, and the dense depth map to generate a point cloud reconstruction result. Step S600 further includes step S610, which maps the virtual point cloud set, the initial effective point cloud set, and the dense depth map to a unified spatial grid, and establishes a multi-source correspondence based on the depth consistency, spatial distribution continuity, and neighborhood density correlation of different source points within each spatial grid. Specifically, a three-dimensional voxel grid with the same parameters as in step S410 is established. For each point in the initial effective point cloud set, its corresponding voxel is determined according to its spatial coordinates, and the depth value of that point is stored in the lidar channel of that voxel. For the dense depth map, each pixel is back-projected into three-dimensional space according to the extrinsic parameter calibration relationship in step S100, and its corresponding voxel is determined according to the spatial coordinates of the back-projected point. The depth estimate of that pixel is stored in the visual depth channel of that voxel. For each virtual point in the virtual point cloud set, its corresponding voxel is determined according to its spatial coordinates, and the depth value of that virtual point is stored in the virtual point channel of that voxel. For each voxel, the number of valid data points in the LiDAR channel, visual depth channel, and virtual point channel is checked. Multi-source correspondence is established based on the consistency of depth values ​​within each channel, the continuity of spatial distribution, and the correlation of neighborhood density. The method for determining depth value consistency is as follows: calculate the ratio of the absolute difference between any two channel depth values ​​to the average of the two channel depth values. If this ratio is less than a preset consistency threshold, the depth is considered consistent. The preset consistency threshold measures the relative deviation between depth estimates from two different sources. When the relative deviation is less than this threshold, the measurement results from the two sources are considered consistent within the engineering allowable error range and can be mutually verified. The preset consistency threshold is set based on the larger of the relative accuracy of the LiDAR ranging and the average relative error of the visual depth estimation, ensuring that it does not exceed the typical error range of any sensor. In this embodiment, the larger value is taken from the relative accuracy of the LiDAR ranging (±2% to ±5%) and the average relative error of the visual depth estimation (±8% to ±12%), and a safety margin is added, setting the preset consistency threshold to 15%. The method for determining the continuity of spatial distribution is as follows: The number of voxels with valid data in each channel within a voxel and its 26 neighboring voxels is counted. If this number exceeds a preset threshold, the spatial distribution is considered continuous. The physical meaning of the preset threshold is: within a local spatial range composed of a voxel and its 26 neighbors, at least how many voxels with valid data are needed to consider the spatial distribution of that local area as continuous rather than isolated scattered points. The preset threshold is set based on the following: In a 3D voxel grid, there are 27 positions (26 neighbors plus the central voxel). If the number of valid voxels is too small, it indicates that the data distribution in that area is sparse and does not meet the conditions for forming a continuous surface. In this embodiment, the preset threshold is set to 8, meaning that at least 8 of the 27 voxel positions have valid data.The neighborhood density correlation is calculated as follows: for each valid data point, the correlation coefficient between its voxel and the data density of the same channel within surrounding voxels is calculated. Depth consistency, spatial distribution continuity, and neighborhood density correlation are combined to establish a multi-source correspondence.

[0047] Step S620: Based on the multi-source correspondence and the point cloud echo intensity stability, depth estimation confidence within the dense depth map, and virtual point confidence parameters in the virtual point cloud set of the initial effective point cloud set, dynamic fusion weights for point clouds from different sources are generated. Specifically, for points in the initial effective point cloud set, the point cloud echo intensity stability is determined by the echo intensity attenuation rate calculated in step S240. The stability value = 1 - |echo intensity attenuation rate / maximum allowable attenuation rate|, where the maximum allowable attenuation rate is set to 2.0 dB / m. The smaller the absolute value of the attenuation rate, the higher the stability. For pixels in the dense depth map, the depth estimation confidence is given by the standard deviation estimate output by the deep learning visual depth estimation model. This model adds a variance prediction branch at the end of the decoder, outputting the uncertainty of the depth estimation for each pixel. The confidence value = 1 - uncertainty / uncertainty normalization constant, where the uncertainty normalization constant is set to 0.5m. For virtual points in the virtual point cloud set, the virtual point confidence parameter is calculated based on the gradient smoothness of the depth continuity constraint used in the generation process of the virtual point and the matching degree of the prior data of the coal pile structure: gradient smoothness = 1 ÷ (1 + variance of depth decay trend sequence), matching degree = exp(-|virtual point adjusted depth - theoretical depth correction value| / depth normalization constant), the depth normalization constant is set to 0.2m, confidence = gradient smoothness × matching degree, and normalized to between 0 and 1. The dynamic fusion weights are calculated as follows: The initial weights of each source point are normalized. For example, the weight of the initial effective point cloud set is set to the normalized value of the point cloud echo intensity stability, the weight of the dense depth map is set to the normalized value of the depth estimation confidence, and the weight of the virtual point cloud set is set to the normalized value of the virtual point confidence parameter. Then, within the same voxel, the weights of the three sources are linearly combined: LiDAR normalized weight = LiDAR stability value ÷ (LiDAR stability value + visual depth confidence + virtual point confidence); Visual normalized weight = Visual depth confidence ÷ (LiDAR stability value + Visual depth confidence + Virtual point confidence); Virtual point normalized weight = Virtual point confidence ÷ (LiDAR stability value + Visual depth confidence + Virtual point confidence), ensuring that the sum of these three is 1. If a channel within a voxel has no valid data, the weight of that channel is reset to 0, and the weights of the remaining channels are re-normalized.

[0048] Step S630: Perform multi-source weighted fusion processing according to the dynamic fusion weights to generate point cloud reconstruction results. Specifically, for each voxel, multiply the depth value of the LiDAR point cloud, the visual depth estimate, and the virtual point depth value by their corresponding dynamic fusion weights, and then sum them to obtain the fused depth value. Combine the fused depth value with the center coordinates of the voxel to generate a fused point cloud.

[0049] In one possible implementation, multi-source weighted fusion processing is performed according to the dynamic fusion weights to generate point cloud reconstruction results. Step S630 further includes step S631, performing local surface continuity constraint analysis on the adjacent point clouds after weighted fusion. Specifically, for the fused point cloud generated in step S630, the local surface fitting method in step S250 is used to fit a quadratic surface to each point and its neighboring points, and the local curvature at each point is calculated. The cosine value of the normal angle between two adjacent points is calculated. If the cosine value is less than a preset continuity threshold, it is determined that the adjacent relationship is discontinuous. The preset continuity threshold is the minimum allowable value of the cosine value of the normal angle, used to determine whether the orientation of the local surface where the two adjacent points are located is continuous. The closer the cosine value of the normal angle is to 1, the more consistent the normal directions of the two surfaces are, and the smoother the surface is; when the cosine value is less than the threshold, it indicates that the normal angle exceeds the allowable range, and there is an obvious turn or break on the surface. The preset continuity threshold is set based on the cosine value corresponding to the maximum allowable normal change angle under the natural stacking state of the coal pile surface. In this embodiment, the preset continuity threshold is set to cos(30°)≈0.866, meaning the maximum allowable normal angle is 30°. The ratio of the difference in depth values ​​between two adjacent points to the spatial distance between the two points is calculated. If this ratio is greater than the preset gradient threshold, it is determined that there is an abnormal gradient in the adjacent relationship. The preset gradient threshold is the maximum allowable value of the ratio of the depth change between two adjacent points to the horizontal distance, used to determine the rationality of the surface slope. When the depth gradient exceeds this threshold, it indicates that there is an excessively steep slope change between the two points, exceeding the limit slope that the natural angle of repose of the coal pile can reach, which may be caused by data fusion errors or anomalies. The preset gradient threshold is set based on the tangent value of the natural angle of repose of coal. In this embodiment, the preset gradient threshold is set to tan(40°)≈0.839, meaning the maximum allowable slope is 40° (the natural angle of repose of coal generally does not exceed 40°).

[0050] Step S632 involves performing iterative smoothing correction and outlier rollback processing using the results of the local surface continuity constraint analysis to generate the point cloud reconstruction result. Specifically, for adjacent point pairs identified as having discontinuities or anomalous gradients in step S631, the point farther from the voxel center is marked as a point to be corrected. Smoothing correction is then performed on the point to be corrected, i.e., searching all points within a neighborhood radius (twice the voxel's side length) centered on the point to be corrected, calculating the median depth value of these points, and updating the depth value of the point to be corrected to this median. After completing one smoothing correction, the continuity constraint analysis in step S631 is re-executed. If discontinuities or anomalous gradients still exist, the smoothing correction process is repeated. The maximum number of iterations is set to 10. If the continuity constraint cannot be satisfied after the maximum number of iterations, the point to be corrected is identified as an outlier and removed from the fused point cloud. After iterative smoothing correction and outlier rollback processing, the final point cloud reconstruction result is output.

[0051] This application embodiment collects lidar point cloud data and multi-view image data of the target coal pile area. Through time synchronization and extrinsic parameter calibration, the two types of data are unified to the same spatial coordinate system. Statistical filtering is performed on the lidar point cloud to remove outliers. Based on local density, curvature changes, and echo intensity fluctuations, abnormal areas such as dust obstruction and water surface refraction are identified to obtain an initial effective point cloud set. Simultaneously, the multi-view images are input into a deep learning visual depth estimation model, which outputs dense depth maps corresponding to each viewpoint. The dense depth maps are registered and aligned with the initial effective point cloud to construct a multi-source depth correlation field that integrates the information from both sources. From this field, the depth variation patterns of missing region edges and the structural morphology of the coal pile surface are extracted. Based on the aforementioned depth continuity constraints and structural priors, virtual point clouds are generated within the data-deficient regions to fill the gaps. Finally, the virtual point clouds, the initial valid point clouds, and the dense depth map are mapped onto a unified spatial grid. Fusion weights are dynamically allocated according to the local confidence of each source data. After weighted fusion and surface continuity correction, a complete, continuous point cloud reconstruction result that conforms to the real shape of the coal pile is output. These technical means solve the technical problems of local distortion and regional missing data in the 3D reconstruction of the coal pile surface under complex environments. The technical effect of suppressing local distortion in the measurement data and filling the data-deficient regions is achieved, generating a complete coal pile surface point cloud without distortion or gaps.

[0052] In the above text, refer to Figure 1 A method for reconstructing point clouds on the surface of irregular coal piles in complex environments, according to embodiments of the present invention, is described in detail. Next, reference will be made to... Figure 2 A point cloud reconstruction system for irregular coal pile surfaces in complex environments is described according to an embodiment of the present invention.

[0053] The point cloud reconstruction system for irregular coal pile surfaces under complex environments, according to embodiments of the present invention, addresses the technical problems of local distortion and missing regions in measurement data during existing 3D reconstruction of coal pile surfaces under complex environments. It achieves the technical effect of suppressing local distortion in measurement data and filling in missing data regions, generating a complete coal pile surface point cloud without distortion or voids. The point cloud reconstruction system for irregular coal pile surfaces under complex environments includes: a multi-source sensing data mapping relationship establishment module 10, an initial effective point cloud set establishment module 20, a dense depth map output module 30, a multi-source depth correlation field establishment module 40, a virtual point cloud set construction module 50, and a point cloud reconstruction result generation module 60.

[0054] The multi-source sensing data mapping relationship establishment module 10 is used to acquire lidar point cloud data and multi-view image data of the target coal pile area, perform time synchronization processing and extrinsic parameter calibration on the lidar point cloud data and multi-view image data, and establish a multi-source sensing data mapping relationship under a unified spatial coordinate system; the initial effective point cloud set establishment module 20 is used to perform statistical filtering and outlier removal on the lidar point cloud data, identify abnormal areas based on the local density distribution characteristics, spatial curvature change characteristics and echo intensity fluctuation characteristics of the point cloud, and establish an initial effective point cloud set; the dense depth map output module 30 is used to input the multi-view image data into a deep learning visual depth estimation model to extract coal... The system analyzes the texture continuity features, shadow variation features, and edge morphology features of the pile surface, outputting dense depth maps corresponding to images from various viewpoints. A multi-source depth correlation field establishment module 40 is used to establish a multi-source depth correlation field after spatial registration with the dense depth map and the initial valid point cloud set. This multi-source depth correlation field is then used to extract depth continuity constraints of missing region edges and prior data of the coal pile structure. A virtual point cloud set construction module 50 is used to construct a virtual point cloud set using depth continuity constraints and prior data of the coal pile structure. A point cloud reconstruction result generation module 60 is used to perform multi-source weighted fusion processing on the virtual point cloud set, the initial valid point cloud set, and the dense depth map to generate point cloud reconstruction results.

[0055] The detailed configuration of the multi-source depth correlation field establishment module 40 is explained as follows: As mentioned above, the multi-source depth correlation field is used to extract the depth continuity constraints of the missing region edges and the prior data of the coal pile structure. The multi-source depth correlation field establishment module 40 may further include: a multi-source depth correlation field construction unit for mapping the spatially registered dense depth map and the initial effective point cloud set to a unified voxel space, and constructing a multi-source depth correlation field based on the real depth distribution, visual depth change trend, and spatial neighborhood connectivity within each voxel; a gradient expansion analysis unit for using the multi-source depth correlation field to establish a depth continuity propagation path corresponding to each spatial location, performing gradient expansion analysis on the edges of each missing region along the depth continuity propagation path, extracting the depth decay trend, surface tilt angle change trend, and local curvature continuity trend of the missing region edges in different directions, and generating depth continuity constraints; and a coal pile structure prior data construction unit for extracting the natural accumulation boundary features and particle flow evolution features of the coal pile surface based on the point cloud accumulation morphology, slope extension direction, and elevation distribution law of adjacent regions in the multi-source depth correlation field, and constructing the coal pile structure prior data.

[0056] The specific configuration of the virtual point cloud set construction module 50 is described in detail below: As mentioned above, a virtual point cloud set is constructed using depth continuity constraints and prior data of the coal pile structure. The virtual point cloud set construction module 50 may further include: an initial virtual point distribution generation unit used to locate the depth expansion direction of the edge of the missing region using depth continuity constraints, and to perform spatial interpolation expansion processing on the interior of the missing region along the depth expansion direction to generate the initial virtual point distribution of the missing region; and a virtual point cloud set generation unit used to perform morphological constraint processing on the initial virtual point distribution based on the prior structural data of the coal pile, and to adjust the spatial position and spatial distribution density of each virtual point according to the natural accumulation slope variation law of the coal pile, the continuous undulation law of the local surface, and the particle flow accumulation direction to generate a virtual point cloud set that conforms to the real accumulation morphology of the coal pile.

[0057] The detailed configuration of the point cloud reconstruction result generation module 60 is explained as follows: As described above, the virtual point cloud set, the initial effective point cloud set, and the dense depth map are subjected to multi-source weighted fusion processing to generate point cloud reconstruction results. The point cloud reconstruction result generation module 60 may further include: a multi-source correspondence establishment unit for mapping the virtual point cloud set, the initial effective point cloud set, and the dense depth map to a unified spatial grid, and establishing multi-source correspondence based on the depth consistency, spatial distribution continuity, and neighborhood density correlation of different source points within each spatial grid; a multi-source correspondence generation unit for generating dynamic fusion weights for point clouds from different sources based on the multi-source correspondence and the point cloud echo intensity stability of the initial effective point cloud set, the depth estimation confidence in the dense depth map, and the virtual point confidence parameters in the virtual point cloud set; and a multi-source weighted fusion processing unit for performing multi-source weighted fusion processing according to the dynamic fusion weights to generate point cloud reconstruction results.

[0058] The multi-source weighted fusion process is performed according to the dynamic fusion weights to generate point cloud reconstruction results. The multi-source weighted fusion process unit may further include: a local surface continuity constraint analysis subunit for performing local surface continuity constraint analysis on adjacent point clouds after weighted fusion; and an iterative smoothing correction subunit for performing iterative smoothing correction and outlier backoff processing using the results of the local surface continuity constraint analysis to generate point cloud reconstruction results.

[0059] The detailed description of the initial effective point cloud set establishment module 20 is as follows: As mentioned above, statistical filtering and outlier removal are performed on the lidar point cloud data. The initial effective point cloud set establishment module 20 may further include: a point cloud data neighbor point feature calculation unit for calculating the average distance deviation feature and spatial dispersion feature of neighbor points in the lidar point cloud data; and an outlier removal unit for performing statistical filtering and outlier removal based on the average distance deviation feature and spatial dispersion feature to establish a preprocessed point cloud set.

[0060] The initial effective point cloud set establishment module 20, which identifies abnormal regions based on the local density distribution characteristics, spatial curvature variation characteristics, and echo intensity fluctuation characteristics of the point cloud, may further include: a low-density abnormal region marking unit for dividing the preprocessed point cloud set into multiple continuous spatial neighborhoods, calculating the mean local point spacing, variance of point spacing, and rate of change of the number of neighborhood points in each spatial neighborhood, and marking low-density abnormal regions based on the calculation results; and a dust-obstructed region marking unit for extracting the echo intensity sequence corresponding to the low-density abnormal region along the laser scanning direction and calculating the echo intensity sequence of adjacent scanning points. The echo intensity attenuation rate and attenuation fluctuation frequency are used to identify areas where the echo intensity continuously attenuates and the attenuation fluctuation frequency is higher than a preset threshold as dust-covered areas. The water surface refraction area identification unit is used to perform local surface fitting processing on the point cloud surface in the low-density abnormal area, calculate the change in the normal angle between adjacent surfaces and the curvature jump amplitude, and combine the offset angle between the laser incident direction and the echo direction to identify areas with abrupt normal changes and echo direction offset exceeding a preset angle threshold as water surface refraction areas. The dust-covered area and the water surface refraction area are output as abnormal areas.

[0061] The initial effective point cloud set establishment module 20 may further include: the abnormal region includes a dust-covered region, a water surface refraction abnormal region, and a low-reflectivity coal ash covered region.

[0062] The multi-source sensing data mapping relationship establishment module 10 may further include: the multi-view image data includes a sequence of coal pile surface images collected at different observation angles, different exposure conditions and at different times, which is used to enhance the reconstruction integrity of the dense depth map.

[0063] The point cloud reconstruction system for irregular coal pile surfaces in complex environments provided in this embodiment of the invention can execute the point cloud reconstruction method for irregular coal pile surfaces in complex environments provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0064] Although this application makes various references to certain modules in the system according to the embodiments of this application, any number of different modules can be used and run on user terminals and / or servers. The various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy distinction between each other and are not used to limit the scope of protection of this invention.

[0065] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for reconstructing point clouds on the surface of irregular coal piles under complex environments, characterized in that, The method includes: Acquire lidar point cloud data and multi-view image data of the target coal pile area, perform time synchronization processing and external parameter calibration on the lidar point cloud data and multi-view image data, and establish a multi-source sensing data mapping relationship under a unified spatial coordinate system. Statistical filtering and outlier removal are performed on the lidar point cloud data. Abnormal regions are identified based on the local density distribution characteristics, spatial curvature change characteristics and echo intensity fluctuation characteristics of the point cloud, and an initial effective point cloud set is established. The multi-view image data is input into a deep learning visual depth estimation model to extract the texture continuity features, shadow variation features and edge morphology features of the coal pile surface, and output dense depth maps corresponding to each view image. After spatial registration with the dense depth map and the initial effective point cloud set, a multi-source depth correlation field is established. The depth continuity constraints of the missing region edge and the prior data of the coal pile structure are extracted using the multi-source depth correlation field. A virtual point cloud set is constructed using depth continuity constraints and prior data on coal pile structure; Multi-source weighted fusion processing is performed on the virtual point cloud set, the initial valid point cloud set, and the dense depth map to generate the point cloud reconstruction result.

2. The method for reconstructing point clouds on the surface of irregular coal piles under complex environments as described in claim 1, characterized in that, The depth continuity constraints of the missing region edges and prior data of the coal pile structure are extracted using the multi-source depth correlation field, including: The spatially registered dense depth map and the initial effective point cloud set are mapped to a unified voxel space, and a multi-source depth correlation field is constructed based on the real depth distribution, visual depth change trend and spatial neighborhood connectivity within each voxel. A depth continuity propagation path corresponding to each spatial location is established using a multi-source depth correlation field. Gradient expansion analysis is performed on the edges of each missing region along the depth continuity propagation path to extract the depth decay trend, surface tilt angle change trend and local curvature continuity trend of the missing region edges in different directions, and generate depth continuity constraints. Based on the point cloud accumulation morphology, slope extension direction, and elevation distribution patterns of adjacent regions in the multi-source deep correlation field, the natural accumulation boundary features and particle flow evolution features of the coal pile surface are extracted to construct prior data of the coal pile structure.

3. The method for reconstructing point clouds on the surface of irregular coal piles under complex environments as described in claim 2, characterized in that, A virtual point cloud set is constructed using depth continuity constraints and prior data on coal pile structure, including: The depth extension direction of the missing region edge is located by using depth continuity constraints. Spatial interpolation extension processing is performed on the interior of the missing region along the depth extension direction to generate the initial virtual point distribution of the missing region. Based on the prior structural data of the coal pile, the initial virtual point distribution is subjected to morphological constraint processing. According to the natural accumulation slope variation law of the coal pile, the continuous undulation law of the local surface, and the particle flow and accumulation direction, the spatial position and spatial distribution density of each virtual point are adjusted to generate a virtual point cloud set that conforms to the real accumulation morphology of the coal pile.

4. The method for reconstructing point clouds on the surface of irregular coal piles under complex environments as described in claim 3, characterized in that, Multi-source weighted fusion processing is performed on the virtual point cloud set, the initial valid point cloud set, and the dense depth map to generate point cloud reconstruction results, including: The virtual point cloud set, the initial valid point cloud set, and the dense depth map are mapped to a unified spatial grid. Multi-source correspondence is established based on the depth consistency, spatial distribution continuity, and neighborhood density correlation of different source points within each spatial grid. Based on the stability of point cloud echo intensity, confidence of depth estimation within dense depth maps, and confidence parameters of virtual points in virtual point cloud sets, dynamic fusion weights for point clouds from different sources are generated. Multi-source weighted fusion processing is performed based on the dynamic fusion weights to generate point cloud reconstruction results.

5. The method for reconstructing point clouds on the surface of irregular coal piles in complex environments as described in claim 4, characterized in that, Multi-source weighted fusion processing is performed based on the dynamic fusion weights to generate point cloud reconstruction results, including: Perform local surface continuity constraint analysis on adjacent point clouds after weighted fusion; Iterative smoothing correction and outlier rollback are performed using the results of local surface continuity constraint analysis to generate point cloud reconstruction results.

6. The method for reconstructing point clouds on the surface of irregular coal piles under complex environments as described in claim 1, characterized in that, Perform statistical filtering and outlier removal on the lidar point cloud data, including: Calculate the average distance deviation and spatial dispersion characteristics of adjacent points in lidar point cloud data; Based on the average distance deviation characteristics and spatial dispersion characteristics, statistical filtering and outlier removal are performed to establish a preprocessed point cloud set.

7. The method for reconstructing point clouds on the surface of irregular coal piles under complex environments as described in claim 6, characterized in that, Anomaly regions are identified based on local density distribution characteristics, spatial curvature variation characteristics, and echo intensity fluctuation characteristics of point clouds, including: The preprocessed point cloud set is divided into multiple continuous spatial neighborhoods. The mean local point spacing, point spacing variance, and rate of change of the number of neighborhood points in each spatial neighborhood are calculated. Low-density anomaly regions are marked based on the calculation results. The echo intensity sequence corresponding to the low-density abnormal area is extracted along the laser scanning direction. The echo intensity attenuation rate and attenuation fluctuation frequency between adjacent scanning points are calculated. The area where the echo intensity continuously attenuates and the attenuation fluctuation frequency is higher than a preset threshold is marked as a dust-covered area. Local surface fitting is performed on the point cloud surface in the low-density abnormal region. The change in the normal angle between adjacent surfaces and the curvature jump amplitude are calculated. Combined with the offset angle between the laser incident direction and the echo direction, the region with a sudden change in normal and an echo direction offset exceeding a preset angle threshold is identified as the water surface refraction region. The dust-covered region and the water surface refraction region are output as abnormal regions.

8. The method for reconstructing point clouds on the surface of irregular coal piles under complex environments as described in claim 1, characterized in that, The abnormal areas include dust-covered areas, water surface refraction abnormal areas, and low-reflectivity coal ash-covered areas.

9. The method for reconstructing point clouds on the surface of irregular coal piles under complex environments as described in claim 1, characterized in that, The multi-view image data includes a sequence of coal pile surface images acquired at different observation angles, exposure conditions, and times, which are used to enhance the reconstruction integrity of dense depth maps.

10. A point cloud reconstruction system for irregular coal pile surfaces under complex environments, characterized in that, The system is used to implement the point cloud reconstruction method for irregular coal pile surfaces in complex environments as described in any one of claims 1-9. The system comprises: The multi-source sensing data mapping relationship establishment module is used to acquire lidar point cloud data and multi-view image data of the target coal pile area, perform time synchronization processing and external parameter calibration on the lidar point cloud data and multi-view image data, and establish a multi-source sensing data mapping relationship under a unified spatial coordinate system. The initial effective point cloud set establishment module is used to perform statistical filtering and outlier removal on the lidar point cloud data, and to identify abnormal regions based on the local density distribution characteristics, spatial curvature change characteristics and echo intensity fluctuation characteristics of the point cloud to establish an initial effective point cloud set. The dense depth map output module is used to input the multi-view image data into a deep learning visual depth estimation model, extract the texture continuity features, shadow change features and edge morphology features of the coal pile surface, and output a dense depth map corresponding to each view image. The multi-source depth correlation field establishment module is used to establish a multi-source depth correlation field after spatial registration with the dense depth map and the initial effective point cloud set, and to extract the depth continuity constraints of the missing region edge and the prior data of the coal pile structure using the multi-source depth correlation field. The virtual point cloud set construction module is used to construct virtual point cloud sets using depth continuity constraints and prior data of coal pile structure. The point cloud reconstruction result generation module is used to perform multi-source weighted fusion processing on the virtual point cloud set, the initial valid point cloud set, and the dense depth map to generate the point cloud reconstruction result.

Citation Information

Patent Citations

  • Method and device for measuring volume of material pile in construction scene

    CN121259077A

  • Coal pile form dynamic visualization modeling method based on point cloud fusion and electronic equipment

    CN122618158A