Intelligent deformation monitoring and early warning system for waste dump slope retaining wall based on point cloud analysis

Abstract

The invention relates to the technical field of waste dump monitoring, in particular to an intelligent deformation monitoring and early warning system for a waste dump slope retaining wall based on point cloud analysis, which comprises the following steps: acquiring initial point cloud data in real time through a three-dimensional laser scanner, acquiring real-time image information in combination with a visual monitoring unit, and outputting a three-dimensional model through a system computing unit; the anomaly analysis module is used for performing anomaly analysis on the three-dimensional model according to the target point cloud data and / or the real-time image information to obtain an abnormal point coordinate; and the early warning identification module is used for carrying out distinguishing marking on the three-dimensional model according to the coordinates of the abnormal points and outputting early warning information, so that the problem that potential safety hazards cannot be effectively identified and early warned due to the fact that accurate measurement of the size of the side slope retaining wall cannot be realized in a complex mining area environment by an existing measurement means is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of dump monitoring, in particular to an intelligent deformation monitoring and early warning system for a dump slope retaining wall based on point cloud analysis. BACKGROUND

[0002] A slope retaining wall is an important structure in slope protection engineering, which is usually formed by piling up gravel or soil blocks along the edge of the slope to prevent people or vehicles from sliding. At present, a conical cylinder is mainly used as a height measuring tool for the slope retaining wall in the field, and the conical cylinder is placed on the slope protection at both ends of the dump surface for visual reference by operators and supervisors.

[0003] However, due to factors such as visual angle error, easy burying of the conical cylinder by sand and stone, and manual monitoring, the accuracy of the measured results of the actual height of the slope retaining wall cannot be guaranteed, especially when the height is lower than the radius of the tire of a mining vehicle, which can easily lead to major safety accidents due to height misjudgment.

[0004] Therefore, how to solve the problem that the existing measurement means cannot achieve accurate measurement of the size of the slope retaining wall in a complex mine environment, so that safety hazards cannot be effectively identified and warned, is a problem that needs to be solved by those skilled in the art. SUMMARY

[0005] In order to solve the technical problem that the existing measurement means cannot achieve accurate measurement of the size of the slope retaining wall in a complex mine environment, so that safety hazards cannot be effectively identified and warned, the present application provides an intelligent deformation monitoring and early warning system for a dump slope retaining wall based on point cloud analysis.

[0006] The intelligent deformation monitoring and early warning system for a dump slope retaining wall based on point cloud analysis provided by the present application adopts the following technical solution: An intelligent deformation monitoring and early warning system for a dump slope retaining wall based on point cloud analysis, comprising at least one three-dimensional laser scanner, a visual monitoring unit and a system calculation unit, the system calculation unit comprising a model generation module, an abnormality analysis module and an early warning marking module; The three-dimensional laser scanner and the visual monitoring unit are respectively in communication connection with the system calculation unit; The three-dimensional laser scanner is used to collect initial point cloud data of the slope retaining wall; The visual monitoring unit is used to collect real-time image information of the slope retaining wall; The model generation module is used to process the initial point cloud data and the real-time image information, obtain target point cloud data and image feature points, and generate a three-dimensional model of the slope retaining wall based on the target point cloud data and the image feature points; The abnormality analysis module is used to perform abnormality analysis on the three-dimensional model according to the target point cloud data to obtain abnormal point coordinates; and / or, According to the real-time image information, the three-dimensional model is analyzed for abnormality, and the abnormal point coordinates are obtained; The early warning marking module is configured to mark the abnormal point coordinates on the three-dimensional model and output early warning information.

[0007] Further, the step of collecting the initial point cloud data of the slope retaining wall comprises: firing a laser pulse at the slope retaining wall, and determining distance information between the slope retaining wall based on the round trip time point of the laser pulse; and performing horizontal calibration and vertical calibration on the slope retaining wall by multi-line laser scanning to determine the orientation information of the slope retaining wall; The initial point cloud data includes the distance information and the orientation information.

[0008] Further, after processing the initial point cloud data and the real-time image information to obtain the target point cloud data and the image feature points, the step of generating the three-dimensional model of the slope retaining wall based on the target point cloud data and the image feature points comprises: compensating the initial point cloud data, pre-processing the compensated initial point cloud data to obtain the target point cloud data, and performing feature matching on the real-time image information to output the image feature points; aligning the image feature points with the target point cloud data in space, and generating a texture map by a dense reconstruction algorithm; generating an initial point cloud model based on the target point cloud data, fusing the initial point cloud model and the texture map, and outputting the three-dimensional model.

[0009] Further, the step of compensating the initial point cloud data comprises: establishing a dust compensation relationship based on the real-time dust concentration, the laser pulse reflection intensity of the three-dimensional laser scanner, and the distance information in the initial point cloud data; optimizing and compensating the initial point cloud data based on the dust compensation relationship to obtain the compensated initial point cloud data.

[0010] Further, the step of compensating the initial point cloud data comprises: establishing a light compensation relationship based on the real-time light intensity, the laser pulse reflection intensity of the three-dimensional laser scanner, and the preset laser pulse reflection intensity; optimizing and compensating the initial point cloud data based on the light compensation relationship to obtain the compensated initial point cloud data.

[0011] Further, the step of pre-processing the compensated initial point cloud data to obtain the target point cloud data comprises: performing noise reduction processing on the compensated initial point cloud data, and removing outliers in the compensated initial point cloud data to obtain first point cloud data; According to the real-time dust concentration, the point cloud coordinates of each point cloud in the first point cloud data are corrected to obtain second point cloud data; The second point cloud data is converted into a sparse voxel grid structure, the geometric centers of each voxel unit in the sparse voxel grid structure are obtained, the point cloud data not on the geometric centers in each voxel unit is removed, and third point cloud data is obtained; The third point cloud data is segmented to obtain a plurality of point cloud blocks, a target point cloud block with a geometric feature of a slope retaining wall is extracted from the plurality of point cloud blocks, and each point cloud in the target point cloud block is determined as target point cloud data.

[0012] Further, the step of outputting the image feature points by feature matching the real-time image information includes: Obtaining the geometric feature point positions in the real-time image information; Determining the feature point coordinates and generating the feature descriptors according to the geometric feature point positions, wherein the image feature points include the feature point coordinates and the feature descriptors.

[0013] Further, the step of generating the texture map by the dense reconstruction algorithm after the image feature points are spatially aligned with the target point cloud data includes: Spatially aligning the feature point coordinates, the feature descriptors and the second point cloud data, and outputting a mapping table between the feature point coordinates and the second point cloud data; Converting the mapping table into a dense point cloud model by the dense reconstruction algorithm, and performing surface reconstruction on the dense point cloud model to obtain a triangular mesh model; Projecting and mapping the triangular mesh model, the mapping table and the real-time image information to generate the texture map.

[0014] Further, the step of obtaining the abnormal point coordinates by performing abnormal analysis on the three-dimensional model according to the target point cloud data includes: Converting the target point cloud data from a scanner coordinate system to a slope retaining wall coordinate system with a wall bottom plane as a reference by a coordinate system conversion algorithm; According to the slope retaining wall coordinate system, a geometric feature vector of the slope retaining wall is calculated, and it is determined whether the geometric feature vector is in a preset geometric feature vector interval; If the geometric feature vector is not in the preset geometric feature vector interval, the position of the geometric feature vector on the three-dimensional model is determined, and the output is the abnormal point coordinates.

[0015] Further, the step of obtaining the abnormal point coordinates by performing abnormal analysis on the three-dimensional model according to the real-time image information includes: The three-dimensional model and the real-time image information are input into a convolutional neural network, and an attention mechanism is introduced, and based on a preset risk warning rule, it is determined whether there is risk information on the three-dimensional model; If there is risk information on the three-dimensional model, the position of the risk information on the three-dimensional model is determined, and the output is an abnormal point coordinate.

[0016] The beneficial effects achieved are: The application builds a dump slope retaining wall intelligent deformation monitoring and early warning system based on point cloud analysis, which includes at least one three-dimensional laser scanner, a visual monitoring unit and a system calculation unit, the system calculation unit includes a model generation module, an abnormal analysis module and a warning identification module; the three-dimensional laser scanner and the visual monitoring unit are respectively connected with the system calculation unit; the three-dimensional laser scanner is used for collecting initial point cloud data of the slope retaining wall; the visual monitoring unit is used for collecting real-time image information of the slope retaining wall; the model generation module is used for processing the initial point cloud data and the real-time image information, obtaining target point cloud data and image feature points, and generating a three-dimensional model of the slope retaining wall based on the target point cloud data and the image feature points; the abnormal analysis module is used for performing abnormal analysis on the three-dimensional model according to the target point cloud data to obtain an abnormal point coordinate; and / or performing abnormal analysis on the three-dimensional model according to the real-time image information to obtain an abnormal point coordinate; and the warning identification module is used for distinguishing and marking the abnormal point coordinate on the three-dimensional model and outputting warning information.

[0017] That is, the application builds a dump slope retaining wall intelligent deformation monitoring and early warning system based on point cloud analysis by a three-dimensional laser scanner, a visual monitoring unit and a system calculation unit, real-time high-precision initial point cloud data of the slope retaining wall is collected by the three-dimensional laser scanner, real-time image information such as surface texture and edge features is obtained by the visual monitoring unit, multi-source data fusion and three-dimensional reconstruction are performed by the system calculation unit, a three-dimensional model with texture is finally output, visualization of the slope retaining wall on the system is realized, the corresponding slope retaining wall of the three-dimensional model is quickly monitored by the abnormal analysis module and the warning identification module in the system calculation unit, abnormal point information is quickly located and warned, and effective measurement of the slope retaining wall in the complex mining environment is realized, and then safety hazards can be effectively identified and warned. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 is a module schematic diagram of the dump slope retaining wall intelligent deformation monitoring and early warning system based on point cloud analysis of the application; Figure 2 is a whole structure schematic diagram of the dump slope retaining wall intelligent deformation monitoring and early warning system based on point cloud analysis of the application; Figure 3 is a comparison schematic diagram of the initial point cloud data after compensation before and after noise reduction processing of the application; Figure 4is a comparison schematic diagram of the second point cloud data before and after the point cloud quantity reduction of the present application; Figure 5 is a comparison schematic diagram of the third point cloud data before and after the point cloud segmentation of the present application; Figure 5 (a) in is a schematic diagram of the third point cloud data before segmentation; Figure 5 (b) in is a schematic diagram of the third point cloud data after segmentation.

[0019] Explanation of reference signs: 10, three-dimensional laser scanner; 20, visual monitoring unit; 30, system calculation unit; 40, vehicle-mounted platform; 50, support frame; 60, solar photovoltaic panel; 70, communication unit; 80, storage bin; 90, audible and visual alarm; 100, baffle; 110, spare tire; 120, fire extinguisher. DETAILED DESCRIPTION

[0020] The following will be described in detail in combination with the accompanying Figures 1-5 The present application will be further described in detail.

[0021] In the description of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship shown in the drawings, which is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second", "third" are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0022] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms "mounting", "connection", "connection" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium, or it can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0023] The embodiment of the present application discloses an intelligent deformation monitoring and early warning system for slope retaining wall of dump based on point cloud analysis.

[0024] Referring to Figure 1 The intelligent deformation monitoring and early warning system for slope retaining wall of dump based on point cloud analysis proposed by the present embodiment can include: At least one three-dimensional laser scanner 10, a visual monitoring unit 20 and a system computing unit 30, the system computing unit 30 comprising a model generation module, an anomaly analysis module and a warning identification module; the three-dimensional laser scanner 10 and the visual monitoring unit 20 are respectively in communication connection with the system computing unit 30; the three-dimensional laser scanner 10 is used to collect initial point cloud data of the slope retaining wall; the visual monitoring unit 20 is used to collect real-time image information of the slope retaining wall; the model generation module is used to process the initial point cloud data and the real-time image information, and after obtaining target point cloud data and image feature points, a three-dimensional model of the slope retaining wall is generated based on the target point cloud data and the image feature points; the anomaly analysis module is used to perform anomaly analysis on the three-dimensional model according to the target point cloud data to obtain abnormal point coordinates; and / or, perform anomaly analysis on the three-dimensional model according to the real-time image information to obtain abnormal point coordinates; the warning identification module is used to distinguishably mark the abnormal point coordinates on the three-dimensional model and output warning information.

[0025] According to Figure 1 As shown in the module schematic diagram, the three-dimensional laser scanner 10 can measure the slope retaining wall to be built by measuring light pulses or modulated signals, and can collect initial point cloud data of the slope retaining wall. The target point cloud data obtained by processing the initial point cloud data can directly calculate the actual height of the slope retaining wall, avoiding the measurement error caused by using a conical cylinder for measurement. At least one multi-line laser is arranged in the three-dimensional laser scanner 10, and through the actual application environment, a single multi-line laser or multiple multi-line lasers can be dynamically deployed to realize 360° detection of the slope retaining wall.

[0026] Although the initial point cloud data collected by the three-dimensional laser scanner 10 can accurately represent the three-dimensional geometric shape of the retaining wall, it cannot automatically identify the physical properties such as gravel, soil and cracks, and it is also difficult to judge the gradual risk such as sandstone sliding and water seepage. Therefore, the embodiment also proposes to supplement the surface semantic information of the point cloud data by the visual monitoring unit 20. Specifically, the visual monitoring unit 20 collects real-time image information of the slope retaining wall in real time, and sends the collected real-time image information to the system computing unit 30. The system computing unit 30 processes the real-time image information into image feature points, thereby supplementing the semantic information of the point cloud data of the slope retaining wall framework, giving the point cloud data environmental perception ability, and solving the monitoring blind area based on only point cloud data.

[0027] The system computing unit 30 is mainly used to provide calculation processing operations. The three-dimensional laser scanner 10 and the visual monitoring unit 20 are connected to the system computing unit 30 in communication, and after transmitting the initial point cloud data and real-time image information collected by the three-dimensional laser scanner 10 and the visual monitoring unit 20 to the system computing unit 30, the system computing unit 30 processes the initial point cloud data and real-time image information, and finally outputs a three-dimensional model of the slope retaining wall. According to the target point cloud data and real-time image information, the three-dimensional model can be quickly, conveniently and accurately marked and the on-site personnel can be warned. The on-site personnel can intuitively obtain the geometric information of the slope retaining wall and the abnormal point positioning through the three-dimensional model, and further obtain the existing safety hazard information without manual on-site monitoring, which greatly improves the safety hazard investigation efficiency.

[0028] Further, according to Figure 2 It can be known that the intelligent deformation monitoring and early warning system for the slope retaining wall of the dump site based on point cloud analysis comprises two parts. One part is a three-dimensional laser scanning system for realizing three-dimensional laser-visual fusion monitoring and self-powered of the slope retaining wall, and the other part is a vehicle-mounted platform 40 for realizing platform traction type movement.

[0029] The vehicle-mounted platform 40 comprises a bottom plate and a support frame 50. The system computing unit 30 in the three-dimensional laser scanning system is arranged on the bottom plate. The three-dimensional laser scanner 10 and the visual monitoring unit 20 need to be arranged on the support frame 50 because they need to be at a high angle to avoid a visual monitoring blind area. It should be noted that according to actual environmental measurement needs, a storage bin 80 can be arranged on the bottom plate. The power supply unit and the system computing unit 30 are assembled in the storage bin 80 to avoid damage to the equipment caused by environmental factors such as dust and light.

[0030] Further, because the three-dimensional laser scanning system built in the embodiment is mounted on the vehicle-mounted platform 40, the purpose is to realize the fixed-point detection and mobile detection of the slope retaining wall, so as to avoid the movement limitation of the fixed power supply, the three-dimensional laser scanning system in the embodiment is also provided with a power supply unit, the power supply unit includes a battery pack and a solar photovoltaic panel, during the sunshine, the solar photovoltaic panel converts solar energy into electric energy, and then supplies power to the three-dimensional laser scanning system, which avoids the wiring complexity caused by the power supply wire drawing on the dumping site, and the electric energy converted by the solar energy is also stored in the battery pack, so that the three-dimensional laser scanning system can also normally enter the running state at night, avoids the use limitation caused by the power supply problem, enables the platform to realize the convenience of movement in the complex geographical environment, effectively improves the measurement continuity of the measured slope retaining wall, and further improves the accuracy of the output three-dimensional model, the improvement of the three-dimensional model accuracy is beneficial to the improvement of the visualization, and further enables the safety hidden danger to be quickly and effectively identified and warned.

[0031] It should be noted that, Figure 2 The 60 in the power supply unit is a solar photovoltaic panel 60, and the battery pack is arranged in the storage bin 80. In addition, the support frame 50 is also provided with a communication unit 70 and an audible and visual alarm 90, and in order to reduce the damage degree of the device arranged on the support frame 50 due to environmental factors, the support frame 50 is also provided with a baffle 100 on the top end.

[0032] According to Figure 2 It can also be known that the vehicle-mounted platform 40 is also provided with a spare tire 110 and a fire extinguisher 120.

[0033] Specifically, the specific process that can be realized based on the three-dimensional laser scanner is as shown in step S10: Step S10, the laser pulse is emitted to the slope retaining wall, the distance information between the slope retaining wall and the laser pulse is determined according to the round trip time point, and the horizontal calibration and vertical calibration of the slope retaining wall are carried out through multi-line laser scanning, and the orientation information of the slope retaining wall is determined; wherein the initial point cloud data includes distance information and orientation information.

[0034] Figure 1 The three-dimensional laser scanner in the embodiment obtains the distance information between the slope retaining wall based on the time-of-flight method or the phase modulation method. After the laser pulse is emitted to the surface of the slope retaining wall, the round trip time is calculated by receiving the reflection signal of the laser pulse, and the distance information between the slope retaining wall is determined in combination with the speed of light. The three-dimensional laser scanner covers the horizontal 360° and vertical 40° field of view through high-speed rotation or multi-line laser array, collects more than 1 million point cloud data per second, determines the orientation information of the slope retaining wall, and forms high-density point cloud data.

[0035] Specifically, based on the specific process that the system computing unit can achieve, as shown in steps S20-S40: Step S20, compensating the initial point cloud data, pre-processing the compensated initial point cloud data to obtain target point cloud data, and performing feature matching on real-time image information to output image feature points.

[0036] Considering the influence of the complex mine environment on the initial point cloud data, the embodiment proposes to compensate and pre-process the initial point cloud data before outputting the three-dimensional model based on the initial point cloud data, to obtain high-precision target point cloud data, so that the three-dimensional model structure built based on the target point cloud data can effectively fit the actual slope retaining wall.

[0037] And by performing feature matching on real-time image information, a multi-view image is established to associate pixels at the same physical location, providing a basis for corresponding points for geometric parameter calculation and three-dimensional reconstruction.

[0038] Among them, the step of compensating the initial point cloud data in step S20 can refer to a feasible implementation manner as shown in steps S201-S202: Step S201, according to the real-time dust concentration, the laser pulse reflection intensity of the three-dimensional laser scanner and the distance information in the initial point cloud data, a dust compensation relationship is established.

[0039] Step S202, based on the dust compensation relationship, the initial point cloud data is optimized and compensated to obtain the compensated initial point cloud data.

[0040] Considering the dust problem existing in the dump, which will affect the propagation of the laser beam and the data collection of the three-dimensional laser scanner, resulting in errors in the initial point cloud data, therefore, based on this situation, the embodiment proposes to compensate the initial point cloud data to reduce the influence of dust on the subsequent three-dimensional model built, specifically: According to the real-time dust concentration, the laser pulse emission intensity of the three-dimensional laser scanner and the distance information in the initial point cloud data, a dust compensation relationship for the initial point cloud data is established, as shown in formula 1: Formula 1 Among them, is the corrected laser pulse emission intensity, is the laser pulse emission intensity, is the dust concentration weight, is the real-time dust concentration, is the distance attenuation weight, and d is the distance information.

[0041] The corrected laser pulse emission intensity output by the dust compensation formula can reflect the laser pulse emission intensity after dust interference is eliminated. Therefore, the point cloud attributes of the initial point cloud data are updated and compensated using the corrected laser pulse emission intensity to obtain the dust-resistant compensated initial point cloud data, which provides a reliable input for the construction of the 3D model.

[0042] The dust compensation formula used in this embodiment, compared with the conventional logarithmic function fitting optimization compensation, shows that the measured value k(x) is more effective when the dust concentration is 0 mg / m³. 3 ~400mg / m 3 Within the specified range, the function fitting optimization curve closely matches the dust compensation relationship, while the logarithmic function fitting optimization curve is more effective when the dust concentration is above 150 mg / m³. 3 Afterwards, the value deviated significantly from the measured value. Therefore, the strong robustness of the dust compensation formula proposed in this embodiment can effectively avoid the interference of dust on point cloud data.

[0043] In another feasible implementation, as shown in steps S203-S204: Step S203: Based on the real-time illumination intensity, the laser pulse reflection intensity of the 3D laser scanner, and the preset laser pulse reflection intensity, establish an illumination compensation formula.

[0044] Step S204: Optimize and compensate the initial point cloud data based on the illumination compensation formula to obtain the compensated initial point cloud data.

[0045] Considering that the spoil heap is in an exposed environment, collecting initial point cloud data in excessively bright or dim lighting conditions can affect the propagation of the laser beam and the data acquisition of the 3D laser scanner, leading to errors in the initial point cloud data. Therefore, based on this situation, this embodiment proposes to compensate for the initial point cloud data to reduce the impact of abnormal lighting on the subsequently constructed 3D model. Specifically: Based on the real-time illumination intensity, laser pulse reflection intensity, and preset laser pulse reflection intensity, an illumination compensation formula for the initial point cloud data is established, as shown in Formula 2: ——Formula 2 in, The corrected laser pulse emission intensity. The intensity of the laser pulse emission. This is the illumination weighting coefficient. denoted as the distance attenuation coefficient, d represents the distance information, and I represents the actual illumination intensity.

[0046] The corrected laser pulse emission intensity output through the illumination compensation relationship can reflect the laser pulse emission intensity after the illumination interference is eliminated, and therefore the point cloud data of the initial point cloud data is updated and compensated by using the corrected laser pulse emission intensity, so that the compensated initial point cloud data resistant to illumination interference is obtained, thereby providing reliable input for the establishment of a three-dimensional model.

[0047] It should be noted that the dust compensation and the illumination compensation can be selected or combined according to actual application requirements.

[0048] As to the step S20 of preprocessing the compensated initial point cloud data to obtain target point cloud data, reference can be made to steps S205-S208 as shown in the following: In step S205, the compensated initial point cloud data is subjected to noise reduction processing, and outliers in the compensated initial point cloud data are removed to obtain first point cloud data.

[0049] When the point cloud data of the slope retaining wall is acquired, due to the device accuracy, the dust raised by the mine car, and the diffraction of the electromagnetic wave, the surface properties of the measured slope retaining wall will change. Even the compensated initial point cloud data still has noise, and therefore the compensated initial point cloud data is subjected to noise reduction processing, as shown in the following: Figure 3

[0050] For each compensated initial point cloud data, the average distance Pi of each point to its nearest k points is calculated, that is, the distances of all points in the compensated initial point cloud data should constitute a Gaussian distribution, and the average distance and the standard deviation of all points are calculated If the average distance d of the average distance Pi is greater than the set dynamic threshold, it is regarded as an outlier (i.e., dp in the formula 2) and is removed. Figure 3

[0051] The basic principle of the noise reduction processing shown in the embodiment is that all points in the compensated initial point cloud data field have a local linear relationship before and after smoothing, and by solving the coefficients of each local linear equation set, the smoothed point cloud, that is, the first point cloud data obtained after removing the outliers, can be solved according to the corresponding local linear relationship and the compensated initial point cloud data.

[0052] In step S206, the point cloud coordinates of each point cloud in the first point cloud data are corrected according to the real-time dust concentration to obtain second point cloud data.

[0053] In order to enhance the anti-dust interference characteristics of the first point cloud data, in the preprocessing stage, the point cloud coordinates of each point cloud in the first point cloud data and the real-time dust concentration are acquired, and the point cloud coordinates of each point cloud in the first point cloud data are corrected based on the formula 3. ​​

[0054] Equation 3 wherein, is the corrected point cloud coordinate, is the uncorrected point cloud coordinate, is the real-time dust concentration, a, b and c are pre-calibrated fitting coefficients, and the purpose is to suppress the point cloud coordinate drift caused by dust scattering.

[0055] By performing coordinate correction on each point cloud coordinate contained in the first point cloud data through Equation 3, the second point cloud data resistant to dust interference can be obtained.

[0056] In step S207, the second point cloud data is converted into a sparse voxel grid structure, the geometric center of each voxel unit in the sparse voxel grid structure is obtained, the point cloud data not at the geometric center in each voxel unit is removed, and the third point cloud data is obtained.

[0057] The point cloud space corresponding to the second point cloud data is divided into cubic grids with a side length of L, only the voxel units containing points are retained, and the sparse voxel grid structure is output. For each voxel unit, the geometric center of all the point clouds inside it is calculated, and after the geometric center is determined, the points not at the geometric center are removed, and the third point cloud data is output. The purpose is to reduce the density of the point cloud data to reduce the subsequent calculation amount, and by retaining only the geometric center obtained in the voxel unit, the geometric distribution characteristics of the points in the voxel unit can be determined, the accuracy of subsequent segmentation and feature extraction can be ensured under the condition of reduced calculation amount, and efficient data simplification is realized.

[0058] Specifically, the maximum and minimum values (Xmax, Ymax, Zmax) and (Xmin, Ymin, Zmin) of the second point cloud data in each direction of the coordinate axis are obtained, Lx=max-Xmin, Ly=Ymax-Ymin, and Lz=Zmax-Zmin. The second point cloud data is divided into AxBxC voxel units under the condition that the side length of the cubic grid is L. Among them, A=Ceil(Lx / L+1), B=Ceil(Ly / L+1), and C=Ceil(Lz / L+1), and Ceil(x) is a rounding up function. In order to better retain local feature information while sampling, both different numbers of point clouds and macroscopic regulation of the number of sampled point clouds can be adapted, the geometric center of all points in each voxel unit containing point cloud data is calculated, the geometric center is selected as the final sampling point of the voxel unit, and the simplified point cloud, i.e., the third point cloud data, is formed.

[0059] Reference can be made to Figure 4As shown, after the step shown in step S207, the second point cloud data with a point cloud quantity of 16939328 is converted into third point cloud data with a point cloud quantity of 11620, the point cloud data is greatly optimized to reduce the complexity of subsequent analysis in the case of retaining the shape characteristics of the point cloud.

[0060] In step S208, the third point cloud data is segmented to obtain a plurality of point cloud blocks, a target point cloud block with a point cloud attribute of a geometric feature of a slope retaining wall is extracted from the plurality of point cloud blocks, and each point cloud in the target point cloud block is determined as target point cloud data.

[0061] According to the spatial, geometric and texture feature points, the third point cloud data is segmented, the point cloud blocks in the same division have similar features, and the point cloud blocks with a point cloud attribute of a geometric feature of a slope retaining wall are separately divided out. In view of the intelligent analysis demand of the complex geometric feature of the slope retaining wall, a hierarchical semantic segmentation network is constructed based on the PointTransformer V3 architecture based on the deep learning point cloud segmentation method. An asymmetric encoding and decoding architecture is adopted to realize multi-scale feature learning. A local-global feature aggregation module is constructed through a spatial attention mechanism, wherein the local feature aggregation formula is shown in the following formula 4: Formula 4 Wherein, Q, K and V represent query, key and value vectors respectively, d is the feature dimension, and a dynamic feature sampling strategy is designed for the scene characteristics of the slope retaining wall: Farthest Point Sampling is adopted for multi-resolution down-sampling in the encoding stage, and up-sampling is realized through a feature propagation layer in the decoding stage. In order to enhance the robustness of the model, a multi-modal feature fusion mechanism is introduced to cross-modally correlate the point cloud intensity, geometric features such as normal vector and deep learning features, and a feature similarity matrix is constructed as follows: Wherein, is a learnable parameter, and a conditional random field is adopted in the post-processing stage to optimize the segmentation boundary. The specific implementation includes three key stages: ① an octree index is adopted to divide the point cloud in space to construct a point cloud graph structure containing geometric topological relationship; ② a multi-scale feature pyramid is adopted to extract a geometric feature vector, and a long-range dependence relationship is established by combining a self-attention mechanism; ③ a dynamic convolution module is adopted to realize point cloud semantic labeling, and an energy function minimization is adopted to realize accurate extraction of the retaining wall region.

[0062] The third point cloud data can also be extracted by point cloud FCM (Fuzzy C-means) clustering segmentation, that is, the membership of each point cloud data to each point cloud attribute is calculated, for example, the membership of the geometric characteristics of the slope retaining wall, the pile and the vehicle, and then the point cloud data is segmented according to the maximum membership to determine the point cloud attribute to which each point cloud data belongs, as shown in Figure 5

[0063] Figure 5 (a) in (a) is the third point cloud data before segmentation, Figure 5 (b) in (b) is the third point cloud data after segmentation, wherein the same gray in (b) represents the same point cloud block, and the point cloud block with darker gray represents the non-slope retaining wall, and the point cloud block with lighter gray represents the slope retaining wall.

[0064] After completing the point cloud segmentation of the third point cloud data, a plurality of point cloud blocks are obtained, and a target point cloud block with the geometric characteristics of the slope retaining wall is extracted from the plurality of point cloud blocks, and each point cloud in the target point cloud block is determined as a target point cloud data, so as to avoid the influence of the point cloud data with the geometric characteristics of the non-slope retaining wall on the subsequent three-dimensional model building.

[0065] Regarding the step of performing feature matching on the real-time image information and outputting image feature points in step S20, refer to steps S209-S210 shown in the following: Step S209, obtaining the geometric feature point position in the real-time image information.

[0066] Step S210, determining the feature point coordinates and generating the feature descriptor according to the geometric feature point position, wherein the image feature point includes the feature point coordinates and the feature descriptor.

[0067] The geometric feature point position is monitored by SIFT (Scale-Invariant Feature Transform) or ORB (Oriented Fast and Rotated Brief) algorithm, specifically: the edge, crack intersection, and high-contrast structure of the slope retaining wall are located by using scale space extreme scanning, a multi-scale pyramid is constructed by a Gaussian difference function to capture local extreme values, ORB uses a FAST (Features from Accelerated Segment Test) corner detector to improve real-time performance, and a wide dynamic range is introduced to suppress strong light overexposure and a median filter is introduced to block dust noise to ensure specific stability in complex environments, and the geometric feature point position is obtained.

[0068] ​Subsequently, the direction assignment is performed based on the monitored geometric feature point position, the pixel gradient amplitude and direction in its field are calculated, and a 36-bin histogram is generated to determine the main direction by peak value, which makes the subsequent geometric feature point position have rotation invariance and can adapt to the view angle offset caused by device movement.

[0069] Then, the generation of feature descriptor is performed with the geometric feature point position with main direction as input, specifically: through 4*4 sub-region segmentation, 8-direction gradient histogram statistics, 128-dimensional SIFT vector or 256-bit ORB code is generated, and then L2 normalization is performed to enhance the light robustness, the feature point coordinates and feature descriptor of the geometric feature point position are output.

[0070] In step S30, after the image feature points are aligned with the target point cloud data space, a texture map is generated by a dense reconstruction algorithm.

[0071] Specifically, step S30 includes steps S301-S303: In step S301, the feature point coordinates, feature descriptors and second point cloud data are spatially aligned, and a mapping table between the feature point coordinates and the second point cloud data is output.

[0072] The feature point coordinates, feature descriptors and second point cloud data are input into a spatial alignment module. First, a candidate point pair set is selected based on the cosine similarity of the feature descriptors and the local geometric features (normal vector + curvature) of the point cloud. Then, the RANSAC algorithm is used to iteratively remove outliers with a re-projection error greater than 2 pixels. Finally, a rotation matrix and a translation vector are solved by SVD decomposition, and a mapping table between the feature point coordinates and the second point cloud data is output. This directly eliminates the 20cm-level view angle deviation of traditional manual calibration, and provides a millimeter-level coordinate reference for subsequent geometric reconstruction.

[0073] In step S302, the mapping table is converted into a dense point cloud model by a dense reconstruction algorithm, and a triangular mesh model is obtained by surface reconstruction on the dense point cloud model.

[0074] Then, the dense reconstruction is started with the mapping table as input. A multi-view stereo matching algorithm is used to iteratively propagate the optimal depth value (5 neighborhood points are calculated each time) and cross-view consistency check (multi-view re-projection error threshold is 0.8 pixels) to generate a dense point cloud model with a hole filling rate ≥97% (compared with the original point cloud density, the density is improved by 8 times), which breaks through the bottleneck of sparse point cloud that cannot represent crack details. Then, surface reconstruction is performed on the dense point cloud: an implicit function is solved based on the Poisson surface reconstruction algorithm, and a topologically continuous triangular mesh model is generated by octree adaptive subdivision and isosurface extraction.

[0075] In step S303, the triangular mesh model, the mapping table and the real-time image information are projected and mapped to generate a texture map.

[0076] Finally, the triangular mesh model, mapping table, and real-time image information are input into the projection mapping module. Based on the projection matrix solved in the spatial alignment stage, perspective transformation is performed to achieve sub-pixel-level coordinate binding and output a 24-bit RGB texture map. This process has been verified to achieve a texture matching accuracy of 0.5cm (20 times lower than manual mapping error). The generated 3D model can directly mark areas with excessive slope and drive the crack recognition model to maintain 92% accuracy under dust interference, reducing the vehicle landslide warning response time from the traditional 90 seconds to real-time triggering.

[0077] Step S40: Generate an initial point cloud model based on the target point cloud data, fuse the initial point cloud model and texture map, and output a 3D model.

[0078] First, based on the target point cloud data, the implicit function is solved by the Poisson surface reconstruction algorithm. Then, the spatial topology is constructed by octree adaptive subdivision and isosurfaces are extracted to generate an initial point cloud model with millimeter-level accuracy.

[0079] Next, the initial point cloud model and texture map are input into the fusion module. Perspective mapping is performed based on the projection matrix calculated in the spatial alignment stage. Seam artifacts are eliminated through bilinear filtering, and a texture-geometric aligned 3D model is output.

[0080] It should be noted that the steps that can be achieved based on the anomaly analysis module and the early warning indicator module are as follows: ① After outputting the 3D model of the retaining wall of the slope to be measured, steps S401~S403 can be executed: Step S401: Using a coordinate system transformation algorithm, the target point cloud data is transformed from the scanner coordinate system to the slope retaining wall coordinate system with the bottom plane of the wall as the reference.

[0081] Step S402: Calculate the geometric feature vector of the slope retaining wall based on the coordinate system of the slope retaining wall, and determine whether the geometric feature vector is within the preset geometric feature vector interval.

[0082] Based on the target point cloud data obtained from fine segmentation, a geometric parameter calculation system fused from multi-source data is constructed. First, a coordinate system for the retaining wall slope is established with the wall base plane as the reference. Then, the rigid body transformation matrix of the point cloud is solved through singular value decomposition. in, To find the minimum value of the decision variables R (linear coefficients) and t (constant offset), To Linear prediction model, Let i be the input feature / independent variable for the i-th sample. Let be the target value / true value of the i-th sample.

[0083] The height calculation adopts a multi-modal fusion strategy: ① the highest point statistics method based on the segmentation result, constructing a height histogram to obtain the mode extreme value; ② fitting the retaining wall top plane engineering ax+by+cz+d=0 through the RANSAC (Random Sample Consensus) algorithm, and calculating the vertical distance from the reference surface to the top surface.

[0084] The slope calculation adopts the principal component analysis method to solve the characteristic vector of the point cloud covariance matrix: wherein, is the maximum eigenvalue of the covariance matrix , v is the eigenvector of the covariance matrix , and T is the vector transpose operator.

[0085] The maximum eigenvector is projected onto the vertical plane, and the slope angle is calculated through the inverse tangent function: wherein, is the slope angle, is the vertical height difference between two points on the surface of the slope retaining wall, and is the horizontal displacement.

[0086] Based on the above steps, the height data and slope data (i.e. geometric feature vector) of the slope retaining wall corresponding to the three-dimensional model can be obtained. At this time, the preset geometric feature vector interval corresponding to the slope retaining wall is obtained, and it is judged whether the height data and slope data of the three-dimensional model corresponding to the slope retaining wall are within the preset geometric feature vector interval.

[0087] wherein, the preset geometric feature vector interval is the safe height data interval and the safest slope data interval of the slope retaining wall.

[0088] Step S403, if the geometric feature vector is not on the preset geometric feature vector interval, the position of the geometric feature vector on the three-dimensional model is determined, and the output is the abnormal point coordinates.

[0089] If it is determined that the height data and / or slope data of the slope retaining wall does not fall within the corresponding preset geometric feature vector interval, indicating that the height and / or slope of the slope retaining wall does not meet the expected height and slope, the system calculation unit directly determines the location of the geometric feature vector that does not meet the expected height and slope on the three-dimensional model, and then generates an abnormal point coordinate based on the location and performs a distinctive marking on the three-dimensional model, such as a highlighting operation, a red marking operation, etc., and generates an alarm instruction to remind the on-site personnel to check and maintain, greatly improving the convenience and accuracy of slope retaining wall monitoring.

[0090] After outputting the three-dimensional model of the slope retaining wall to be measured, the intelligent early warning system can perform steps S404-S405: Step S404, the three-dimensional model and real-time image information are transmitted into the convolutional neural network, and the attention mechanism is introduced. Based on the preset risk early warning rule, it is determined whether there is a risk mark on the three-dimensional model.

[0091] Step S406, if there is a risk mark on the three-dimensional model, determine the position of the risk information on the three-dimensional model, and output as an abnormal point coordinate.

[0092] The real-time image information of the dump site is collected by the visual monitoring unit in real time, and the real-time image and the three-dimensional model are transmitted to the backend device through the communication unit. The convolutional neural network fuses multi-modal data: first, the image deep texture features (such as crack direction, pile loose degree) are extracted through the ResNet-50 backbone network, and the three-dimensional model is analyzed at the same time; then the SE (Squeeze-and-Excitation, Squeeze-and-Excitation network) channel attention mechanism is introduced to dynamically enhance the response weight of the risk area, and a joint decision is made according to the preset risk early warning rule; when the real-time image information contains the features included in the preset risk early warning rule, the spatial position relationship between the real-time image information and the three-dimensional model is determined, and the abnormal point coordinate of the risk information on the three-dimensional model is determined. After generating a distinctive mark on the abnormal point coordinate, an alarm instruction is generated, for example, if a vehicle is detected in the warning area, a sound and light alarm can be triggered for 30 seconds, and the hidden danger coordinate is pushed to the backend device. This operation can effectively improve the efficiency of safety hidden danger investigation, and greatly improves the safety early warning effect of the dump site.

[0093] It should be noted that the three-dimensional model is generated on the system calculation unit and transmitted to the backend device for display through the communication unit, so the distinctive marking on the three-dimensional model will also be visually distinguished on the three-dimensional model through the communication unit.

[0094] The above are all preferred embodiments of the present application, and do not limit the protection scope of the present application, so that: all equivalent changes made according to the structure, shape, principle of the present application should be covered in the protection scope of the present application.

Claims

1. A dump slope retaining wall intelligent deformation monitoring and early warning system based on point cloud analysis, characterized in that, The system comprises at least one three-dimensional laser scanner, a visual monitoring unit and a system calculation unit, the system calculation unit comprises a model generation module, an anomaly analysis module and a warning identification module; The three-dimensional laser scanner and the visual monitoring unit are respectively connected with the system calculation unit; The three-dimensional laser scanner is used for collecting initial point cloud data of the slope retaining wall; The visual monitoring unit is used for collecting real-time image information of the slope retaining wall; The model generation module is used for processing the initial point cloud data and the real-time image information, obtaining target point cloud data and image feature points, and generating a three-dimensional model of the slope retaining wall based on the target point cloud data and the image feature points; The anomaly analysis module is used for performing anomaly analysis on the three-dimensional model according to the target point cloud data to obtain abnormal point coordinates; And / or, According to the real-time image information, the three-dimensional model is analyzed to obtain abnormal point coordinates; The warning identification module is used for distinguishing and marking the abnormal point coordinates on the three-dimensional model and outputting warning information.

2. The dump slope retaining wall intelligent deformation monitoring and early warning system based on point cloud analysis of claim 1, characterized in that, The step of collecting the initial point cloud data of the slope retaining wall comprises: emitting a laser pulse to the slope retaining wall, determining distance information between the slope retaining wall and the three-dimensional laser scanner according to the round trip time point of the laser pulse; and performing horizontal calibration and vertical calibration on the slope retaining wall through multi-line laser scanning to determine the orientation information of the slope retaining wall; The initial point cloud data comprises the distance information and the orientation information.

3. The point cloud analysis-based dump slope retaining wall intelligent deformation monitoring and early warning system according to claim 2, characterized in that, The step of processing the initial point cloud data and the real-time image information to obtain target point cloud data and image feature points, and generating a three-dimensional model of the slope retaining wall based on the target point cloud data and the image feature points comprises: compensating the initial point cloud data, pre-processing the compensated initial point cloud data to obtain the target point cloud data, and performing feature matching on the real-time image information to output the image feature points; aligning the image feature points with the target point cloud data in space, and generating a texture map through a dense reconstruction algorithm; generating an initial point cloud model according to the target point cloud data, fusing the initial point cloud model and the texture map, and outputting the three-dimensional model.

4. The dump slope retaining wall intelligent deformation monitoring and early warning system based on point cloud analysis of claim 3, characterized in that, The step of compensating the initial point cloud data comprises: establishing a dust compensation relationship according to real-time dust concentration, laser pulse reflection intensity of the three-dimensional laser scanner and distance information in the initial point cloud data; optimizing and compensating the initial point cloud data based on the dust compensation relationship to obtain compensated initial point cloud data.

5. The dump slope retaining wall intelligent deformation monitoring and early warning system based on point cloud analysis of claim 3, wherein, The step of compensating the initial point cloud data comprises: establishing a light compensation relationship according to real-time light intensity, laser pulse reflection intensity of the three-dimensional laser scanner and a preset laser pulse reflection intensity; optimizing and compensating the initial point cloud data based on the light compensation relationship to obtain compensated initial point cloud data.

6. The dump slope retaining wall intelligent deformation monitoring and early warning system based on point cloud analysis of claim 3, characterized in that, The step of pre-processing the compensated initial point cloud data to obtain the target point cloud data comprises: The compensated initial point cloud data is denoised, outliers in the compensated initial point cloud data are removed, and first point cloud data is obtained; According to the real-time dust concentration, the point cloud coordinates of each point cloud in the first point cloud data are corrected, and second point cloud data is obtained; The second point cloud data is converted into a sparse voxel grid structure, the geometric centers of each voxel unit in the sparse voxel grid structure are obtained, the point cloud data not on the geometric centers in each voxel unit is removed, and third point cloud data is obtained; The third point cloud data is segmented to obtain a plurality of point cloud blocks, a target point cloud block with a geometric feature of a slope retaining wall is extracted from the plurality of point cloud blocks, and each point cloud in the target point cloud block is determined as the target point cloud data.

7. The dump slope retaining wall intelligent deformation monitoring and early warning system based on point cloud analysis of claim 6, characterized in that, The step of performing feature matching on the real-time image information and outputting the image feature points comprises: Obtaining the geometric feature point positions in the real-time image information; Determining feature point coordinates and generating a feature descriptor according to the geometric feature point positions, wherein the image feature points include the feature point coordinates and the feature descriptor.

8. The dump slope retaining wall intelligent deformation monitoring and early warning system based on point cloud analysis of claim 7, characterized in that, The step of generating a texture map through a dense reconstruction algorithm after spatial alignment of the image feature points and the target point cloud data comprises: Spatially aligning the feature point coordinates, the feature descriptor, and the second point cloud data, and outputting a mapping table between the feature point coordinates and the second point cloud data; Converting the mapping table into a dense point cloud model through the dense reconstruction algorithm, and performing surface reconstruction on the dense point cloud model to obtain a triangular mesh model; Projecting and mapping the triangular mesh model, the mapping table, and the real-time image information to generate the texture map.

9. The point cloud analysis-based dump slope retaining wall intelligent deformation monitoring and early warning system according to claim 1, characterized in that, The step of performing anomaly analysis on the three-dimensional model according to the target point cloud data to obtain an abnormal point coordinate comprises: Converting the target point cloud data from a scanner coordinate system to a slope retaining wall coordinate system with a wall bottom plane as a reference by a coordinate system conversion algorithm; According to the slope retaining wall coordinate system, calculating a geometric feature vector of the slope retaining wall, and determining whether the geometric feature vector is within a preset geometric feature vector interval; If the geometric feature vector is not within the preset geometric feature vector interval, determining the position of the geometric feature vector on the three-dimensional model, and outputting the abnormal point coordinate.

10. The point cloud analysis-based dump slope retaining wall intelligent deformation monitoring and early warning system according to claim 1, characterized in that, The step of performing anomaly analysis on the three-dimensional model according to the real-time image information to obtain an abnormal point coordinate comprises: Inputting the three-dimensional model and the real-time image information into a convolutional neural network and introducing an attention mechanism, and determining whether there is risk information on the three-dimensional model based on a preset risk warning rule; If there is risk information on the three-dimensional model, determining the position of the risk information on the three-dimensional model, and outputting the abnormal point coordinate.

Images