A method and system for construction facility positioning and classification based on lidar

Abstract

The application discloses a construction facility positioning and classification method and system based on a laser radar, which comprises arranging each laser radar to obtain a construction environment point cloud and sequencing each point cloud; taking the point cloud with the largest sequence number as a post-order point cloud, and fusing the post-order point cloud into a pre-order point cloud; taking the fused point cloud as a post-order point cloud, and again fusing the post-order point cloud into the pre-order point cloud until the sequence number of the fused point cloud is 1; obtaining each sub-point cloud based on a clustering threshold; determining the coordinates of each sub-point cloud, and constructing a sub-point cloud data set; constructing a construction classification model; training the facility classification model based on the sub-point cloud data set; obtaining a complete point cloud in real time, and separating each sub-point cloud and the coordinates of each sub-point cloud; inputting each sub-point cloud into the trained facility classification model to classify the facilities in real time, and outputting each facility category and coordinates. The application has the outstanding advantages of accurate positioning, accurate identification, high reliability and the like.

Description

Technical Field

[0001] This application relates to the field of positioning and classification, and in particular to a method and system for positioning and classifying construction facilities based on lidar. Background Technology

[0002] Accurate location and scientific classification of construction facilities are crucial for ensuring construction safety. In common construction scenarios, facility location and classification typically rely on image data captured by cameras, identified manually or through machine vision algorithms. However, because camera data consists of two-dimensional images lacking depth information, machines cannot accurately determine the spatial location and accuracy of different types of construction facilities. Furthermore, manual identification is not only costly but also relies on the operator's subjective experience, making consistency and reliability difficult to guarantee. In harsh working conditions or adverse weather, the image quality captured by cameras can be severely affected, further complicating the determination of construction facility location and type, and limiting the applicability and accuracy of existing methods in complex scenarios.

[0003] CN111340145A discloses a point cloud data classification method, apparatus, and classification device. This patent constructs relationships between different point cloud primitives by calculating primitive association information reflecting the correlation between them, and then determines the category label for each point cloud primitive through a primitive classification model. However, this patent suffers from inaccurate type identification due to interference between point clouds at different locations.

[0004] CN116645657A discloses a method, apparatus, computer device, and storage medium for determining voxel semantic categories. The method includes: acquiring image information, the image pixel depth distribution of the image information, and point cloud data of the image information; obtaining voxel features based on the image information and the image pixel depth distribution, and obtaining grid features of a three-dimensional mesh based on the point cloud features, wherein the voxel features are pixel features with height information; acquiring image features from multiple preset viewpoints based on the voxel features, and acquiring point cloud features from multiple preset viewpoints based on the grid features; and obtaining the target voxel semantic category to which the image information belongs based on the image features from the preset viewpoints, the point cloud features from the preset viewpoints, and the target model. However, this patent suffers from the problem of inaccurate type recognition due to mutual interference between point clouds at different locations.

[0005] CN110673183B discloses a container identification and positioning method combining GPS and INS, which involves calibrating a lidar and GPS / INS modules; scanning the entire container yard environment with lidar and modeling the yard environment based on GPS data and point cloud data; segmenting the yard model based on container height information to obtain multiple sub-point clouds; clustering each sub-point cloud using Euclidean distance and point cloud normal vectors as criteria; and projecting, segmenting, and matching each cluster using prior container size information. Combined with GPS data, the scanning and positioning information of containers within the yard environment can be obtained. This patent does not involve the classification of different objects; furthermore, this patent relies on a port crane moving a container spreader around the container yard environment to obtain point clouds, which is a high-risk operation that consumes significant public resources. The location of the point cloud collected during aerial movement is also difficult to accurately determine, resulting in poor data acquisition reliability. After identifying the target container, the positioning information needs continuous correction. Furthermore, this patent can only be applied to containers with known height information, limiting its applicability. Also, the patent requires the pre-determining of the dimensions of each container, which is typically collected manually, introducing subjective factors and resulting in poor accuracy. Consequently, the overall positioning accuracy of this patent is poor. Summary of the Invention

[0006] To address the shortcomings of existing technologies, such as inaccurate positioning, inaccurate identification, and weak reliability, this invention provides a method and system for locating and classifying construction facilities based on lidar, which can achieve excellent positioning and classification results for construction facilities under harsh working conditions and weather conditions.

[0007] The present invention adopts the following technical solution.

[0008] This invention discloses a method for locating and classifying construction facilities based on lidar, comprising:

[0009] S1: Deploy each lidar to acquire point clouds of the construction environment and sort the point clouds.

[0010] S2: Take the point cloud with the largest sequence number as the subsequent point cloud and merge the subsequent point cloud into the previous sequence point cloud;

[0011] S3: Use the fused point cloud as the subsequent point cloud, and according to step S2, fuse the subsequent point cloud into the previous point cloud until the fused point cloud is sorted as 1 to obtain a complete point cloud.

[0012] S4: Separate sub-point clouds from the complete point cloud based on the set clustering threshold; determine the coordinates of each sub-point cloud and construct the sub-point cloud dataset; the clustering threshold is set based on the shortest distance from each point to the construction boundary;

[0013] S5: Construct a facility classification model; train the facility classification model based on the sub-point cloud dataset;

[0014] S6: Obtain the complete point cloud in real time according to steps S1-S4, separate each sub-point cloud and its coordinates, input each sub-point cloud into the facility classification model trained in step S5 to classify the facilities in real time, and use the coordinates of each sub-point cloud as the location of the corresponding facility.

[0015] More preferably,

[0016] In step S1, each lidar is configured based on the construction environment, as shown in the following formula:

[0017] ;

[0018] in, This indicates the maximum detection range of the lidar; This represents the maximum distance between a point on the construction boundary and the geometric center of the outline. This indicates the maximum ground clearance of facilities that need to be located and categorized in the construction environment; This indicates the horizontal field of view of the lidar; This indicates the vertical viewing angle of the lidar.

[0019] More preferably,

[0020] In step S1, the number of lidars, the spacing between adjacent lidars, and the height of each lidar above the ground are determined based on the construction environment.

[0021] The spacing between adjacent lidar units As shown in the following formula:

[0022] ;

[0023] in, L The length of the construction boundary; n This refers to the number of lidar units;

[0024] The lidar is set at a height of 1.2 meters above the ground. h ,in, h This indicates the maximum ground clearance of facilities that need to be located and classified in the construction environment.

[0025] More preferably,

[0026] The number of lidar n As shown in the following formula:

[0027] ;

[0028] in, This represents the maximum distance between a point on the construction boundary and the geometric center of the outline. This indicates rounding up to the nearest integer.

[0029] More preferably,

[0030] In step S1, the lidar farthest from the geometric center of the construction boundary is numbered 1, and the lidar numbers are incremented in a counter-clockwise direction.

[0031] The serial number of the point cloud in the construction environment is the same as that of the lidar.

[0032] More preferably,

[0033] In step S2, fusing the subsequent point cloud to the previous point cloud involves matching and calibrating the subsequent point cloud with the previous point cloud to obtain the transformation matrix between the two point clouds. The transformation matrix is ​​then used to transform the subsequent point cloud to the coordinate system of the previous point cloud and fuse them. Finally, the fused point cloud is downsampled.

[0034] More preferably,

[0035] In step S4, the process of separating sub-point clouds from the complete point cloud based on a set clustering threshold includes:

[0036] Filter out outliers from the complete point cloud;

[0037] For each point in the point cloud, search its neighboring point set within a set clustering threshold to obtain each sub-point cloud.

[0038] More preferably,

[0039] The clustering threshold for each point is set according to the following formula:

[0040] ;

[0041] in, The clustering threshold for the i-th point in the point cloud; The maximum clustering threshold; To find the shortest distance from the point that yields the maximum clustering threshold to the construction boundary; Let be the shortest distance from the i-th point in the point cloud to the construction boundary.

[0042] More preferably,

[0043] In step S4, the coordinates of each sub-point cloud are determined by using the coordinates of the points in each sub-point cloud to determine the depth, horizontal and vertical coordinates of each sub-point cloud in the constructed three-dimensional rectangular coordinate system.

[0044] More preferably,

[0045] Set the depth of each sub-point cloud x and level y Coordinates include:

[0046] Project the separated sub-point clouds onto The plane transforms a three-dimensional point cloud into a two-dimensional contour.

[0047] Find the convex hull of each contour, and use the depth coordinates and horizontal coordinates of the geometric center of the convex hull as the depth coordinates and horizontal coordinates of the corresponding sub-point cloud.

[0048] More preferably,

[0049] The vertical coordinates of each sub-point cloud are set according to the following formula:

[0050] ;

[0051] in, Let be the vertical coordinate of the c-th sub-point cloud; j It is an integer; For the c-th sub-point cloud j Large vertical coordinates; For the c-th sub-point cloud j Small vertical coordinates.

[0052] Another aspect of this invention discloses a construction facility positioning and classification system based on a construction facility positioning and classification method, comprising a point cloud acquisition module, a point cloud fusion module, a sub-point cloud separation module, a facility classification model construction module, a facility classification module, and an output module:

[0053] The point cloud acquisition module deploys various lidars to acquire point clouds of the construction environment and sorts the point clouds.

[0054] The point cloud fusion module takes the point cloud with the largest sequence number as the subsequent point cloud and merges the subsequent point cloud into the previous sequence number point cloud; then takes the merged point cloud as the subsequent point cloud and merges the subsequent point cloud into the previous sequence number point cloud again, until the merged point cloud is sorted into 1 to obtain a complete point cloud.

[0055] The sub-point cloud separation module separates each sub-point cloud from the complete point cloud based on a set clustering threshold; determines the coordinates of each sub-point cloud, and constructs a sub-point cloud dataset; the clustering threshold is set based on the shortest distance from each point to the construction boundary.

[0056] The facility classification model construction module constructs a facility classification model and trains the facility classification model based on the sub-point cloud dataset.

[0057] The facility classification module obtains the complete point cloud in real time, separates each sub-point cloud and its coordinates, inputs each sub-point cloud into the trained facility classification model to classify facilities in real time, and uses the coordinates of each sub-point cloud as the location of the corresponding facility.

[0058] The output module outputs the category and location of each facility in the construction environment.

[0059] Another aspect of this application discloses an electronic device, including a processor and a storage medium; characterized in that:

[0060] The storage medium is used to store instructions;

[0061] The processor is used to operate according to the instructions to execute the aforementioned construction facility positioning and classification method.

[0062] This application also discloses a computer-readable storage medium having a computer program stored thereon, characterized in that the program, when executed by a processor, implements the construction facility positioning and classification method.

[0063] The beneficial effects of this invention are compared with those of the prior art:

[0064] The positioning and classification method of the present invention can fuse the point clouds of multiple lidar sensors and still accurately locate and classify construction facilities even when point cloud data is partially missing due to occlusion.

[0065] This invention uses a clustering method to segment the original point cloud into independent small blocks representing individual objects before feeding them into the network for identification. This minimizes the risk of inaccurate identification due to interference from point clouds in other locations. This invention achieves excellent positioning and classification results for construction facilities even under harsh working conditions and weather conditions.

[0066] This invention also has outstanding advantages such as precise positioning, accurate identification, and high reliability. Attached Figure Description

[0067] The accompanying drawings are provided to further illustrate the present application and form part of the specification. Together with the embodiments of the present application, they serve to explain the present application but do not constitute a limitation thereof. In the drawings:

[0068] Figure 1 This is a flowchart illustrating the lidar-based method for locating and classifying construction facilities according to this application.

[0069] Figure 2 This is a schematic diagram of a simulation scene;

[0070] Figure 3 A schematic diagram of the complete construction scene point cloud reconstructed using the proposed method;

[0071] Figure 4 A schematic diagram of the evolution of classification accuracy of the facility classification model when training it using the proposed method. Detailed Implementation

[0072] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this invention. The embodiments described in this application are merely some embodiments of this invention, and not all embodiments. Based on the spirit of this invention, other embodiments obtained by those skilled in the art without creative effort are all within the protection scope of this invention.

[0073] This application discloses a method for locating and classifying construction facilities based on lidar, see appendix. Figure 1 ,include:

[0074] S1: Deploy each lidar to acquire point clouds of the construction environment and sort the point clouds.

[0075] The construction environment point cloud refers to the point cloud obtained by scanning the area within the construction boundary using lidar; the construction boundary refers to the outer contour of the construction area; the construction environment includes the construction boundary, facilities within the boundary area, and the ground clearance of each facility.

[0076] Each lidar is configured based on the construction environment, and the selected lidar must meet the following constraints:

[0077] ;

[0078] in, This indicates the maximum detection range of the lidar; This represents the maximum distance between a point on the construction boundary and the geometric center of the outline. This indicates the maximum ground clearance of facilities that need to be located and categorized in the construction environment; This indicates the horizontal field of view of the lidar; This indicates the vertical viewing angle of the lidar.

[0079] The number of lidars, the spacing between adjacent lidars, and the height of each lidar above the ground are determined based on the construction environment.

[0080] The spacing between adjacent lidar units As shown in the following formula:

[0081] ;

[0082] in, L The length of the construction boundary; n This refers to the number of lidar units;

[0083] The lidar is set at a height of 1.2 meters above the ground. h ,in, h This indicates the maximum ground clearance of facilities that need to be located and classified in the construction environment.

[0084] The number of lidar n As shown in the following formula:

[0085] ;

[0086] in, This represents the maximum distance between a point on the construction boundary and the geometric center of the outline. This indicates rounding up to the nearest integer.

[0087] The number of any lidar farthest from the geometric center of the construction boundary is set to 1, and the lidar numbers are incremented sequentially in a counterclockwise direction.

[0088] The serial number of the construction environment point cloud is the same as the number of the lidar that collected the construction environment point cloud.

[0089] S2: Take the point cloud with the largest sequence number as the subsequent point cloud, and merge the subsequent point cloud into the point cloud with the previous sequence number of the subsequent point cloud; the previous sequence number of the subsequent point cloud is 1 smaller than the sequence number of the subsequent point cloud.

[0090] Fusing a subsequent point cloud to the preceding point cloud involves transforming the subsequent point cloud to the coordinate system of the preceding point cloud and then fusing them. Specifically, this involves matching and calibrating the subsequent point cloud with the preceding point cloud to obtain the transformation matrix and translation vector between the two point clouds. The transformation matrix and translation vector are then used to transform the subsequent point cloud to the coordinate system of the preceding point cloud and fuse them. Finally, the fused point cloud is downsampled.

[0091] S3: Use the fused point cloud as the subsequent point cloud, and according to step S2, fuse the subsequent point cloud into the previous point cloud until the fused point cloud is sorted as 1 to obtain a complete point cloud.

[0092] S4: Separate sub-point clouds from the complete point cloud based on the set clustering threshold; determine the coordinates of each sub-point cloud and construct the sub-point cloud dataset; the clustering threshold is set based on the shortest distance from each point to the construction boundary and the maximum clustering threshold.

[0093] Those skilled in the art should know that separating sub-point clouds from the complete point cloud obtained in step S3 based on a set clustering threshold can be achieved using methods such as adaptive segmentation based on PCA and KMeans, Euclidean distance-based clustering, region-growing-based segmentation, and density-feature-based clustering (such as DBSCAN). In a preferred embodiment of the present invention, Euclidean clustering is used.

[0094] Based on a set clustering threshold, the sub-point clouds obtained from the complete point cloud include:

[0095] Filtering outliers from a complete point cloud; the filtering outliers refers to identifying and removing abnormal points from the point cloud data that do not conform to the expected distribution or deviate significantly from the main data, and those skilled in the art should know how to achieve this.

[0096] For each point in the point cloud, search its neighboring point set within a set clustering threshold to obtain each sub-point cloud.

[0097] The clustering threshold for each point is set according to the following formula:

[0098] ;

[0099] in, The clustering threshold for the i-th point in the point cloud; The maximum clustering threshold; The goal is to find the shortest distance from the point that yields the maximum clustering threshold within a 1 cubic meter area to the construction boundary. This represents the shortest distance from the i-th point in the point cloud to the construction boundary. The maximum clustering threshold is defined as the distance from the i-th point in the complete point cloud to the outer contour of the construction environment being less than 0.05. l For each point, the maximum number of its nearest neighbors is obtained by searching its set of nearest neighbors within a 1 cubic meter; where, It represents the maximum distance from a point on the outer contour of the construction environment to the geometric center of the contour.

[0100] Determining the coordinates of each sub-point cloud includes:

[0101] Those skilled in the art should know that the point cloud obtained by lidar scanning has its own coordinates, which are obtained based on the lidar's own coordinate system.

[0102] The depth, horizontal, and vertical coordinates of each sub-point cloud are determined in a constructed three-dimensional Cartesian coordinate system based on the coordinates of the midpoints of each sub-point cloud.

[0103] Determine the depth of each sub-point cloud x and level y Coordinates include:

[0104] Project the separated sub-point clouds onto The plane transforms a three-dimensional point cloud into a two-dimensional contour.

[0105] Find the convex hull of each contour, and use the depth coordinates and horizontal coordinates of the geometric center of the convex hull as the depth coordinates and horizontal coordinates of the corresponding sub-point cloud.

[0106] The vertical coordinates of each sub-point cloud are set according to the following formula:

[0107] ;

[0108] in, Let be the vertical coordinate of the c-th sub-point cloud;j It is an integer, and j ∈[1,50]; For the c-th sub-point cloud j Large vertical coordinates; For the c-th sub-point cloud j Small vertical coordinates.

[0109] S5: Construct a facility classification model; train the facility classification model based on the sub-point cloud dataset;

[0110] S6: Obtain the complete point cloud in real time according to steps S1-S4, separate each sub-point cloud and its coordinates, input each sub-point cloud into the facility classification model trained in step S5 to classify the facilities in real time, and use the coordinates of each sub-point cloud as the location of the corresponding facility.

[0111] Example 1

[0112] One embodiment of this application provides a method for locating and classifying construction facilities based on lidar. The lidar-based method for locating and classifying construction facilities of this application will be described in detail below.

[0113] Step S101: Configure the model, deployment strategy and sorting of the LiDAR based on the construction environment, and use the LiDAR to acquire the point cloud of the construction environment and sort it according to the LiDAR number; wherein, using the LiDAR to acquire the point cloud of the construction environment and sorting it according to the LiDAR number means to acquire the LiDAR point cloud at the same time and sort each LiDAR point cloud, and its sequence number is the LiDAR number.

[0114] Determine the maximum ground clearance of facilities that need to be located and classified in the construction environment. The maximum distance between a point on the construction boundary and the geometric center of the outline. The construction boundary refers to the outer contour of the construction environment.

[0115] Select the maximum detection range of the lidar. It should be greater than Horizontal view of lidar It should be greater than 90°; vertical field of view. Should be greater than .

[0116] The lidar units are arranged counter-clockwise along the outline of the construction environment, at a height of [height missing] from the ground. Let the boundary length be The required number of lidar units is: ,in, This indicates rounding up; the distance between two adjacent lidar units is... ;

[0117] The lidar furthest from the geometric center of the construction boundary is designated as lidar number 1, and the lidar numbers are sequentially increased counterclockwise.

[0118] Step S102: Take the point cloud with the largest sequence number as the subsequent point cloud and the point cloud with the previous sequence number of the subsequent point cloud as the preceding point cloud. By matching and calibrating the two adjacent lidar point clouds of the subsequent point cloud and the preceding point cloud, obtain the transformation matrix between the adjacent lidar point clouds. Use the transformation matrix to transform the subsequent point cloud to the coordinate system of the preceding point cloud and fuse them. Downsample the fused point cloud.

[0119] Those skilled in the art should know that point cloud matching calibration is the process of aligning point cloud data collected from different sources (different lidars) or at different times to a unified coordinate system. This can be achieved using methods such as Normal Distribution Transform (NDT) and Implicit Moving Least Squares-ICP. Those skilled in the art can perform point cloud matching calibration according to actual conditions. The point cloud matching calibration method proposed in this embodiment is only a preferred embodiment and is not a necessary limitation on implementing this invention. The specific method for point cloud matching calibration is as follows:

[0120] Registration is performed on the point clouds of adjacent lidar units. Specifically, this involves registering the point clouds of adjacent lidar units. and Where P is the subsequent point cloud in the point cloud of the adjacent lidar, and Q is the preceding point cloud in the point cloud of the adjacent lidar. It is the first point in the subsequent point cloud of the adjacent lidar point cloud, presented in three-dimensional coordinate form. ,in, The depth coordinates are the first point in the subsequent point cloud of the adjacent lidar point cloud; The horizontal coordinates of the first point in the subsequent point cloud of the adjacent lidar; The vertical coordinates of the first point in the subsequent point cloud of the adjacent lidar; It is the nth point in the subsequent point cloud of the point cloud of the adjacent lidar, presented in three-dimensional coordinates, and... Similarly, I will not repeat myself; It is the first point in the preceding point cloud of the point cloud of the adjacent lidar, presented in three-dimensional coordinates; It is the nth point in the preceding point cloud of the point cloud of the adjacent lidar, presented in three-dimensional coordinates; n is the total number of points in the point cloud that this type of lidar can collect at any given moment.

[0121] For each point in the source point cloud, find the point in the adjacent laser point cloud that is closest to it.

[0122] Establish a set of point-to-point correspondences: Take the nearest point pair found as the corresponding point pair to form a set of corresponding point pairs.

[0123] Looking for rotation matrices Translation vector t makes satisfy Where i is an integer and i belongs to [1, n].

[0124] Those skilled in the art should be able to select the required rotation matrix according to the actual situation. The rotation matrix used can be obtained by conversion through Euler angles, axis angles, etc., which will not be elaborated here.

[0125] First, define the error term for the i-th pair of points: Construct a least squares problem to find the solution that minimizes the sum of squared errors. :

[0126]

[0127] in, f(x) represents the input R,t that minimizes the function f(x); This indicates the calculation of the Euclidean norm.

[0128] This problem is solved using Singular Value Decomposition (SVD). The specific steps are as follows:

[0129] First, define the centroids of two sets of points. The centroids are the average x, y, and z coordinates of all points in the point cloud, as shown in the following formula:

[0130]

[0131] in, p It is the centroid of the subsequent point cloud in the point cloud of the adjacent lidar; q It is the centroid of the preceding point cloud in the point cloud of the adjacent lidar.

[0132] The error function is then processed as follows:

[0133]

[0134] Note the cross term part If the summation is zero, the objective function can be optimized. J Simplified to:

[0135]

[0136] in, This indicates that solving for the objective function The minimum values ​​of R and t are calculated.

[0137] At this point, the centroid coordinates of each point are calculated: ,in, The centroid coordinates of the first point in the subsequent point cloud of the adjacent lidar; The coordinates of the centroid of the first point in the preceding point cloud of the adjacent lidar point cloud.

[0138] Solve for the optimized rotation matrix based on the following optimization problem:

[0139]

[0140] in, This represents the optimized rotation matrix; f(x) represents the input R that makes the function f(x) reach its minimum value.

[0141] Expand on The error term is obtained as follows:

[0142]

[0143] Note the first item and Irrelevant, the second item is due to Also with Irrelevant, the actual optimization function becomes:

[0144]

[0145] in, Represents the trace of a matrix;

[0146] Next, singular value decomposition (SVD) is used to solve for the optimal solution. Define a matrix ,right SVD decomposition yields U and V are both orthogonal matrices, denoted as the left singular vector matrix and the right singular vector matrix, respectively; It is a diagonal matrix, and the elements on its diagonal are called singular values. When When the term of office is full According to the above Solve for the translation vector ;

[0147] The rotation matrix calculated using the above algorithm Translation vector The process involves transforming the subsequent point cloud to the coordinate system of the preceding point cloud and then overlaying the two sets of point clouds to create a larger point cloud. Subsequently, voxel filtering is used to optimize the density of the overlaid point cloud, restoring it to a level comparable to the point cloud density directly obtained from the LiDAR, thus ensuring the consistency and comparability of the point cloud data. Here, the subsequent point cloud refers to the point cloud that appears later in the sequence between two adjacent point clouds, and the preceding point cloud refers to the point cloud that appears earlier in the sequence between two adjacent point clouds.

[0148] Step S103: Register, downsample, and forward fuse the point cloud after the initial fusion using the method in S102. Repeat this step until all point clouds are fused into the same coordinate system to construct a complete point cloud of the construction environment.

[0149] For the initially fused lidar point cloud obtained in step S102, since the preceding and subsequent point clouds already contain common parts, the rotation matrix from the subsequent point cloud to the preceding point cloud is further obtained using the method in S102. Translation vector The subsequent point cloud is transformed to the coordinate system of the preceding point cloud, and the two sets of point clouds are superimposed.

[0150] Repeat the above steps until all LiDAR point clouds are restored to the coordinate system of LiDAR point cloud No. 1, thus completing the reconstruction task of the construction scene.

[0151] Step s104: Using Euclidean clustering, different clustering thresholds are set according to the distance between the facility and the lidar to obtain each sub-point cloud, and each sub-point cloud is separated from the complete point cloud.

[0152] Each sub-point cloud can be viewed as a facility.

[0153] Typically, construction facilities are independent of each other, so in a point cloud, the points of each facility are also independent of the points of other facilities. Based on this characteristic, the Euclidean clustering algorithm can be used to process the point cloud, effectively separating the points corresponding to different construction facilities, thereby achieving independent identification and extraction of each facility. To achieve good clustering results, it is first necessary to filter out outliers, that is, to remove outliers from the point cloud.

[0154] Those skilled in the art should know that the outlier removal can be achieved using methods such as multivariate Gaussian distribution outlier detection and the LOF algorithm, and can be selected according to the actual situation. To improve computational accuracy, the outlier removal method proposed in this embodiment is only a preferred embodiment and is not a necessary limitation for implementing this invention's method for locating and classifying construction facilities based on lidar. The method for filtering outliers is constructed based on the characteristics of the outliers.

[0155] Preferably, outlier detection can be performed using a multivariate Gaussian distribution: assuming there is point cloud data. ,in Let represent the first point in the point cloud data; let represent the second point in the point cloud data; let represent the nth point in the point cloud data; each point is represented in three-dimensional coordinate form, as shown below:

[0156]

[0157] Among them, the i-th point in the point cloud data Represented as the three-dimensional coordinates of a point. This represents the depth coordinates of the i-th point in the point cloud data; This represents the horizontal coordinate of the i-th point in the point cloud data; This represents the vertical coordinate of the i-th point in the point cloud data.

[0158] 3D mean vector of point cloud for:

[0159]

[0160] in, This represents the mean depth coordinates of the point cloud. This represents the mean horizontal coordinate of the point cloud; This represents the mean vertical coordinate of the point cloud.

[0161] point set covariance matrix for:

[0162]

[0163] in, This indicates the calculation of the covariance of variables a and b. In this invention, both a and b can represent x / y / z. When a / b is x, / This represents the depth coordinates of the i-th point in the point cloud data; / This represents the mean depth coordinates of the point cloud; when a / b is y, / This represents the horizontal coordinate of the i-th point in the point cloud data; / This represents the mean horizontal coordinate of the point cloud; when a / b is z, / This represents the vertical coordinate of the i-th point in the point cloud data; / This represents the mean vertical coordinate of the point cloud.

[0164] For any data point in the point cloud Probability can be calculated To determine whether a point is an outlier:

[0165]

[0166] Due to the imaging characteristics of LiDAR, the density of a point cloud decreases as its distance from the origin of the LiDAR coordinate system increases. Therefore, in the reconstructed point cloud, the closer to the center of the construction scene, the lower the point cloud density; conversely, the closer to the edge of the construction scene, the higher the point cloud density. To achieve accurate Euclidean clustering, an adaptive threshold needs to be dynamically set based on the distribution characteristics of the point cloud at different locations: the point cloud density of LiDAR typically approximately follows the inverse square law. ,in It is point cloud density. It is the distance between this point and the lidar. The scaling factor K is to be estimated. Five locations are randomly selected from the complete point cloud, and the point cloud density at each of these five locations is measured. The scaling factor K is then calculated using the least squares method to determine the point cloud density. The distance between the point and the lidar (outer contour of the construction environment) The relationship between them;

[0167] For each point in the point cloud, search its set of neighboring points within a set clustering threshold;

[0168] Set a feasible Euclidean clustering threshold for points near the edge of the construction environment. , The selected value is related to the lidar used and can be set by the user. In this invention, the maximum number of point clouds generated by the lidar in a unit space (1 cubic meter) is preferred. Points near the edge of the construction environment that can be searched for with the maximum number of point clouds are generally located 1m away from the lidar. Assuming that the closest distance between the aforementioned points near the edge of the construction environment and the construction outline is... ,but =1; Preferably, a point near the edge of the construction environment can be defined as having a distance of less than 0.05 from the edge. l any point; This represents the maximum distance between a point on the construction boundary and the geometric center of the outline.

[0169] Since there is a linear relationship between point cloud density and Euclidean clustering threshold, the shortest distance between any point cloud element and the construction edge is... One of the places, its Euclidean clustering threshold for:

[0170]

[0171] Step s105: Project the separated sub-point clouds onto... In the plane, the three-dimensional point cloud is transformed into a two-dimensional contour. The convex hull of the contour is obtained, and the coordinates of the geometric center of the convex hull are calculated as the XY coordinates of the facility. The Z-axis coordinate of the facility is determined by calculating the average of the top 50 points with the maximum Z-axis coordinate and the top 50 points with the minimum Z-axis coordinate in the point cloud.

[0172] Those skilled in the art should know that obtaining the convex hull of the contour involves calculating the smallest convex polygon that completely contains all points from a given set of points. This can be achieved using methods such as Graham's scan, QuickHull's algorithm, and Andrew's algorithm. Those skilled in the art can construct the method according to the actual situation. The method for obtaining the convex hull of the contour proposed in this embodiment is only a preferred embodiment and is not a necessary limitation on implementing this invention. The specific implementation of this invention is as follows:

[0173] Suppose that a total of m sub-point clouds are obtained, and let c represent any sub-point cloud, where c∈[1,m] and c is an integer;

[0174] Perform a projection transformation on the point cloud separated by Euclidean clustering in step s104, projecting all points in the point cloud onto... The plane transforms 3D point cloud data into a 2D contour.

[0175] Using Graham's scan method to solve for the convex hull vertices of the contour, first find... The lowest point P in the plane, i.e., the point with the smallest ordinate, is selected. If multiple points with the same ordinate are at the bottom, the leftmost point, i.e., the point with the smallest abscissa, is selected and taken as the starting point. A polar coordinate system is established based on the starting point, with the starting point as the origin and the polar axis and x-axis aligned. All points are then sorted in ascending order of their polar angles relative to the starting point P. Finally, a stack is created to store the current convex hull. Based on the sorting result obtained from the polar angles, points are added to the stack sequentially. If the point under consideration is not rotated to the left by the two points at the top of the stack (i.e., the point at the top of the stack is not located in the counterclockwise direction of its adjacent points within the stack), it indicates that the point at the top of the stack is not on the convex hull, and it needs to be popped from the stack. This process is repeated until the point under consideration is rotated to the left by the two points at the top of the stack.

[0176] Based on the convex hull vertices obtained in the previous step, new points are uniformly inserted into the lines connecting adjacent convex hull vertices. The density of the new points should be consistent with the density of the original point cloud. Assume that the point set consisting of the convex hull vertices and all inserted points is... Let the center point of the convex hull be... Then we have:

[0177]

[0178] At this point, the XY coordinates of the facility can be given by the center point of the convex hull;

[0179] Determining the Z-coordinate of a facility requires using a point cloud after outlier removal and clustering. Assume the Z-coordinates of the 50 points with the largest Z-coordinates in the c-th sub-point cloud are... { ,……, },in, It is the largest vertical coordinate in the c-th sub-point cloud; It is the 50th largest vertical coordinate in the c-th sub-point cloud; the Z-coordinates of the 50 points with the smallest Z-coordinates in the point cloud are... { ,……, },in, It is the smallest vertical coordinate in the c-th sub-point cloud; It is the 50th smallest vertical coordinate in the c-th sub-point cloud; therefore, the Z-axis coordinate of the current facility is:

[0180] ;

[0181] in, Let be the vertical coordinate of the c-th sub-point cloud; j It is an integer, and j ∈[1,50]; For the c-th sub-point cloud j Large vertical coordinates; For the c-th sub-point cloud j Small vertical coordinates.

[0182] Step s106: Construct a sub-point cloud dataset based on the possible occlusion of different types of construction facilities, and train a lightweight PointNet network model based on the dataset;

[0183] The process involves obtaining complete sub-point clouds corresponding to the construction facilities that need to be located and classified. Based on the occlusion that the facilities may encounter, the complete sub-point clouds are processed accordingly. For example, if a sub-point cloud has no front part or the front part is incomplete, the front part of the sub-point cloud is removed or the front part is cut off. If a sub-point cloud has missing corners, the corners of the sub-point cloud are cut off. Then, the point clouds are randomly stretched and rotated, and a small amount of Gaussian noise is added to the transformed point clouds to construct the sub-point cloud dataset.

[0184] Label each sub-point cloud in the dataset;

[0185] Those skilled in the art should know how to set sub-point cloud labels according to actual conditions; the sub-point cloud label setting method proposed in this embodiment is only a preferred embodiment and is not a necessary limitation for implementing the lidar-based construction facility positioning and classification method of this invention. Specifically, sub-point cloud labels can be selected such as 0 for oil drums, 1 for generators, 2 for cement bags, etc.

[0186] A lightweight PoinNet network was trained using the point cloud dataset. Once the network's classification accuracy exceeded 98% and the training epochs exceeded 100, the current checkpoint of the network was saved for subsequent real-time classification.

[0187] S107: Obtain the complete point cloud in real time according to steps S101-S105, separate each sub-point cloud and the coordinates of each sub-point cloud, input each sub-point cloud into the facility classification model trained in step S106 to classify the facilities in real time, and use the coordinates of each sub-point cloud as the location of the corresponding facility.

[0188] In actual operation, the lightweight PointNet network loads the above checkpoint and sends the clustered point cloud from step s104 into the network in the format of a dataset. The network then provides the specific category of the current point cloud.

[0189] The entire system ultimately outputs the specific location and category of each facility in the construction environment.

[0190] Example 2

[0191] A method for locating and classifying construction facilities based on lidar.

[0192] For example, such as Figure 2 This is a simulated construction scenario, with eight LiDAR sensors placed on the outer contour of the scenario.

[0193] When the point cloud obtained directly from the lidar is displayed in the coordinate system of lidar No. 1, the point cloud is not registered and the point clouds of each lidar do not overlap, so it is impossible to reconstruct a complete three-dimensional construction scene.

[0194] See Figure 3 , Figure 3 The paper demonstrates how the proposed method accurately reconstructs the complete construction environment after registering the point cloud.

[0195] Referring to Table 1, which shows the theoretical coordinates of points on the facility and the measured values ​​obtained by registration of the point cloud after reconstructing the complete construction scene using the proposed method (unit: meters), it can be seen that the difference between the theoretical and measured values ​​is small, proving the effectiveness of the proposed method.

[0196] Table 1. Comparison of theoretical and actual coordinates of points on the construction facility obtained using the proposed method.

[0197]

[0198] Referring to Table 2, which shows the theoretical values ​​and measured values ​​(unit: meters) of the coordinates of the construction facilities in the point cloud using the proposed method, it can be seen that the difference between the theoretical and measured values ​​is small, proving the effectiveness of the proposed method.

[0199] Table 2 Comparison of coordinates of construction facilities calculated using the proposed method and theoretical coordinates.

[0200]

[0201] See Figure 4 , Figure 4 To construct a point cloud classification dataset using the proposed method, the dataset was divided into training and test sets in a 7:3 ratio. The classification accuracy evolution curve of the lightweight PointNet network trained on the test set was obtained using this dataset. It can be seen that the classification accuracy of the network is close to 100% after 50 epochs of training, and it can correctly classify point clouds of different types of construction facilities, proving the effectiveness of the proposed method.

[0202] This application also discloses a construction facility positioning and classification system based on a construction facility positioning and classification method, including a point cloud acquisition module, a point cloud fusion module, a sub-point cloud separation module, a facility classification model construction module, a facility classification module, and an output module:

[0203] The point cloud acquisition module deploys various lidars to acquire point clouds of the construction environment and sorts the point clouds.

[0204] The point cloud fusion module takes the point cloud with the largest sequence number as the subsequent point cloud and merges the subsequent point cloud into the previous sequence number point cloud; then takes the merged point cloud as the subsequent point cloud and merges the subsequent point cloud into the previous sequence number point cloud again, until the merged point cloud is sorted into 1 to obtain a complete point cloud.

[0205] The sub-point cloud separation module separates each sub-point cloud from the complete point cloud based on a set clustering threshold; determines the coordinates of each sub-point cloud, and constructs a sub-point cloud dataset; the clustering threshold is set based on the shortest distance from each point to the construction boundary.

[0206] The facility classification model construction module constructs a facility classification model and trains the facility classification model based on the sub-point cloud dataset.

[0207] The facility classification module obtains the complete point cloud in real time, separates each sub-point cloud and its coordinates, inputs each sub-point cloud into the trained facility classification model to classify facilities in real time, and uses the coordinates of each sub-point cloud as the location of the corresponding facility.

[0208] The output module outputs the category and location of each facility in the construction environment.

[0209] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0210] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0211] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0212] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0213] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. A method for locating and classifying construction facilities based on lidar, characterized in that, include: S1: Deploy each lidar to acquire point clouds of the construction environment and sort the point clouds. S2: Take the point cloud with the largest index as the subsequent point cloud, match and calibrate the subsequent point cloud with the previous index point cloud, obtain the transformation matrix between the two point clouds, and use the transformation matrix to transform the subsequent point cloud to the coordinate system of the previous index point cloud for fusion. S3: Use the fused point cloud as the subsequent point cloud, and according to step S2, fuse the subsequent point cloud into the previous point cloud until the fused point cloud is sorted as 1 to obtain a complete point cloud. S4: Separate sub-point clouds from the complete point cloud based on the set clustering threshold; determine the coordinates of each sub-point cloud and construct the sub-point cloud dataset; the clustering threshold is set based on the shortest distance from each point to the construction boundary; S5: Construct a facility classification model; train the facility classification model based on the sub-point cloud dataset; S6: Obtain the complete point cloud in real time according to steps S1-S4, separate each sub-point cloud and its coordinates, input each sub-point cloud into the facility classification model trained in step S5 to classify the facilities in real time, and use the coordinates of each sub-point cloud as the location of the corresponding facility.

2. The method for locating and classifying construction facilities according to claim 1, characterized in that: In step S1, each lidar is configured based on the construction environment, as shown in the following formula: ； in, This indicates the maximum detection range of the lidar; This represents the maximum distance between a point on the construction boundary and the geometric center of the outline. This indicates the maximum ground clearance of facilities that need to be located and categorized in the construction environment; This indicates the horizontal field of view of the lidar; This indicates the vertical viewing angle of the lidar.

3. The method for locating and classifying construction facilities according to claim 1, characterized in that: In step S1, the number of lidars, the spacing between adjacent lidars, and the height of each lidar above the ground are determined based on the construction environment. The spacing between adjacent lidar units As shown in the following formula: ； in, L The length of the construction boundary; n This refers to the number of lidar units; The lidar is set at a height of 1.2 meters above the ground. h ,in, h This indicates the maximum ground clearance of facilities that need to be located and classified in the construction environment.

4. The method for locating and classifying construction facilities according to claim 1 or 3, characterized in that: The number of lidar n As shown in the following formula: ； in, This represents the maximum distance between a point on the construction boundary and the geometric center of the outline. This indicates rounding up to the nearest integer.

5. The method for locating and classifying construction facilities according to claim 1, characterized in that: In step S1, the lidar farthest from the geometric center of the construction boundary is numbered 1, and the lidar numbers are incremented in a counter-clockwise direction. The serial number of the point cloud in the construction environment is the same as that of the lidar.

6. The method for locating and classifying construction facilities according to claim 1, characterized in that: In step S2, the subsequent point cloud is matched and calibrated with the previous point cloud to obtain the transformation matrix between the two point clouds. After the subsequent point cloud is transformed into the coordinate system of the previous point cloud using the transformation matrix and fused, the fused point cloud is downsampled.

7. The method for locating and classifying construction facilities according to claim 1, characterized in that: In step S4, the process of separating sub-point clouds from the complete point cloud based on a set clustering threshold includes: Filter out outliers from the complete point cloud; For each point in the point cloud, search its neighboring point set within a set clustering threshold to obtain each sub-point cloud.

8. The method for locating and classifying construction facilities according to claim 1 or 7, characterized in that: The clustering threshold for each point is set according to the following formula: ； in, The clustering threshold for the i-th point in the point cloud; The maximum clustering threshold; To find the shortest distance from the point that yields the maximum clustering threshold to the construction boundary; Let be the shortest distance from the i-th point in the point cloud to the construction boundary.

9. The method for locating and classifying construction facilities according to claim 1, characterized in that: In step S4, the coordinates of each sub-point cloud are determined by using the coordinates of the points in each sub-point cloud to determine the depth, horizontal and vertical coordinates of each sub-point cloud in the constructed three-dimensional rectangular coordinate system.

10. The method for locating and classifying construction facilities according to claim 1 or 9, characterized in that: Determine the depth of each sub-point cloud x and level y Coordinates include: Project the separated sub-point clouds onto The plane transforms a three-dimensional point cloud into a two-dimensional contour. Find the convex hull of each contour, and use the depth coordinates and horizontal coordinates of the geometric center of the convex hull as the depth coordinates and horizontal coordinates of the corresponding sub-point cloud.

11. The method for locating and classifying construction facilities according to claim 1 or 9, characterized in that: The vertical coordinates of each sub-point cloud are set according to the following formula: ； in, Let be the vertical coordinate of the c-th sub-point cloud; j It is an integer; For the c-th sub-point cloud j Large vertical coordinates; For the c-th sub-point cloud j Small vertical coordinates.

12. A construction facility positioning and classification system utilizing the construction facility positioning and classification method according to any one of claims 1-11, characterized in that, It includes a point cloud acquisition module, a point cloud fusion module, a sub-point cloud separation module, a facility classification model construction module, a facility classification module, and an output module. The point cloud acquisition module deploys various lidars to acquire point clouds of the construction environment and sorts the point clouds. The point cloud fusion module takes the point cloud with the largest sequence number as the subsequent point cloud and merges the subsequent point cloud into the previous sequence number point cloud; then takes the merged point cloud as the subsequent point cloud and merges the subsequent point cloud into the previous sequence number point cloud again, until the merged point cloud is sorted into 1 to obtain a complete point cloud. The sub-point cloud separation module separates each sub-point cloud from the complete point cloud based on a set clustering threshold; determines the coordinates of each sub-point cloud, and constructs a sub-point cloud dataset; the clustering threshold is set based on the shortest distance from each point to the construction boundary. The facility classification model construction module constructs a facility classification model and trains the facility classification model based on the sub-point cloud dataset. The facility classification module obtains the complete point cloud in real time, separates each sub-point cloud and its coordinates, inputs each sub-point cloud into the trained facility classification model to classify facilities in real time, and uses the coordinates of each sub-point cloud as the location of the corresponding facility. The output module outputs the category and location of each facility in the construction environment.

13. A terminal, comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is configured to operate according to the instructions to perform the steps of the construction facility positioning and classification method according to any one of claims 1-11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the construction facility positioning and classification method according to any one of claims 1-11.

Images