Construction facility positioning and classifying method and system based on laser radar

Abstract

The invention discloses a construction facility positioning and classification method and system based on laser radars. The method comprises the steps of arranging the laser radars to obtain construction environment point clouds and sorting the point clouds; taking the point cloud with the maximum serial number as a post-order point cloud, and fusing the post-order point cloud to the pre-order point cloud; taking the fused point cloud as a post-order point cloud, and fusing the post-order point cloud to the pre-order point cloud until the sequence of the fused point cloud is 1; obtaining each sub-point cloud based on a clustering threshold; determining each sub-point cloud coordinate, and constructing a sub-point cloud data set; constructing a facility classification model; training a facility classification model based on the sub-point cloud data set; and a complete point cloud is obtained in real time, sub-point clouds and coordinates of the sub-point clouds are obtained through separation, the sub-point clouds are input into the trained facility classification model to classify the facilities in real time, and categories and coordinates of the facilities are output. The method has the outstanding advantages of accurate positioning, accurate identification, high reliability and the like.

Description

Technical Field

[0001] The present application relates to the field of positioning and classification, and particularly to a method and system for positioning and classifying construction facilities based on lidar. Background Art

[0002] The accurate positioning and scientific classification of construction facilities are of great significance in construction safety assurance. In common construction scenarios, the positioning and classification of facilities usually rely on image data collected by cameras and are identified through manual or machine vision algorithms. However, since the data collected by cameras are two-dimensional images without depth information, the machine cannot accurately judge the spatial positions and their correctness of different types of construction facilities. At the same time, manual identification not only has high costs, but also the identification results depend on the subjective experience of operators, making it difficult to ensure consistency and reliability. Under harsh working conditions or adverse weather conditions, the quality of the images collected by cameras may be severely affected, further increasing the difficulty of determining the positions and types of construction facilities and limiting the applicability and accuracy of existing methods in complex scenarios.

[0003] CN111340145A discloses a point cloud data classification method, device, and classification equipment. This patent constructs the association relationship between different point cloud primitives by calculating the primitive association information used to reflect the association between point cloud primitives, and then determines the class label of each said point cloud primitive through a primitive classification model. However, this patent has the problem that the point clouds at different positions interfere with each other, resulting in inaccurate recognition types.

[0004] CN116645657A discloses a voxel semantic category determination method, device, computer device, and storage medium. The method includes: obtaining image information, the depth distribution of image pixels of the image information, and the point cloud data of the image information; obtaining voxel features based on the image information and the depth distribution of image pixels, and obtaining mesh features of a three-dimensional mesh based on the point cloud features, where the voxel features are pixel features with height information; obtaining image features at multiple preset perspectives based on the voxel features, and obtaining point cloud features at multiple preset perspectives based on the mesh features; obtaining the target voxel semantic category to which the image information belongs based on the image features at the preset perspectives, the point cloud features at the preset perspectives, and a target model. However, this patent has the problem that the point clouds at different positions interfere with each other, resulting in inaccurate recognition types.

[0005] CN110673183B discloses a container identification and positioning method combining GPS and INS, calibrating the lidar with the GPS / INS module; the lidar scans the entire container yard environment, and models the yard environment according to the GPS data and the point cloud data; the yard model is segmented according to the container height information to obtain multiple sub-point clouds; the Euclidean distance and the point cloud normal vector are used as criteria to cluster each sub-point cloud; the prior container size information is used to project, segment and match each cluster, and the scanning and positioning information of the container in the yard environment can be obtained by combining the GPS data. This patent does not involve the classification of different objects; and this patent is based on the gantry crane driving the container spreader to obtain the point cloud around the container yard environment for one week. This operation is relatively dangerous, occupies more public resources, and it is difficult to accurately obtain the position of the point cloud collected in the air, resulting in poor reliability of data collection. After determining the target container, the positioning information needs to be continuously corrected. In addition, this patent can only be applied to containers with determined height information, its scope of application is small, and this patent needs to determine the size information of each container in advance. This information is generally collected manually, with subjective factors and poor accuracy, resulting in poor overall positioning accuracy of this patent. Summary of the Invention

[0006] To solve the deficiencies such as inaccurate positioning, inaccurate identification, and weak reliability in the prior art, the present invention provides a lidar-based construction facility positioning and classification method and system, which can have excellent construction facility positioning and classification effects under harsh working conditions and weather conditions.

[0007] The present invention adopts the following technical solutions.

[0008] On the one hand, the present invention discloses a lidar-based construction facility positioning and classification method, including: S1: Arrange each lidar to obtain the construction environment point cloud and sort each point cloud; S2: Take the point cloud with the largest serial number as the subsequent point cloud, and fuse the subsequent point cloud into the previous serial number point cloud; S3: Take the fused point cloud as the subsequent point cloud, and fuse the subsequent point cloud into the previous serial number point cloud according to step S2 until the fused point cloud is sorted as 1 to obtain the complete point cloud; S4: Separate each sub-point cloud from the complete point cloud based on the set clustering threshold; determine the coordinates of each sub-point cloud, and construct a sub-point cloud data set; the clustering threshold is set based on the shortest distance from each point to the construction boundary; S5: Construct a facility classification model; train the facility classification model based on the sub-point cloud data set; S6: Obtain the complete point cloud in real time according to steps S1 - S4, separate to obtain each sub - point cloud and the coordinates of each sub - point cloud, input each sub - point cloud into the facility classification model trained in step S5 to classify the facilities in real time, and use the coordinates of each sub - point cloud as the positioning of the corresponding facilities.

[0009] Further preferably, In step S1, each lidar is configured based on the construction environment, as shown in the following formula: ; Wherein, represents the maximum detection distance of the lidar; represents the maximum distance between a point on the construction boundary and the geometric center of the contour; represents the maximum height above the ground of the facilities to be positioned and classified in the construction environment; represents the horizontal viewing angle of the lidar; represents the vertical viewing angle of the lidar.

[0010] Further preferably, In step S1, arrange the lidars according to the number of lidars determined based on the construction environment, the arrangement interval between adjacent two lidars, and the height of each lidar from the ground; The arrangement interval between adjacent two lidars is shown in the following formula: ; Wherein, L is the length of the construction boundary; n is the number of lidars; The height of the lidar from the ground is set to 1.2 h Wherein, h represents the maximum height above the ground of the facilities to be positioned and classified in the construction environment.

[0011] Further preferably, The number of lidars n is shown in the following formula: ; Wherein, represents the maximum distance between a point on the construction boundary and the geometric center of the contour; represents rounding up.

[0012] Further preferably, In step S1, set the lidar with the farthest distance from the geometric center of the construction boundary as No. 1, and the lidar numbers increase sequentially in the counter - clockwise direction; The serial number of the construction environment point cloud is the same as the lidar number.

[0013] Further preferably, In step S2, the fusion of the subsequent point cloud to the previous serial number point cloud involves matching and calibrating the subsequent point cloud with its previous serial number point cloud to obtain the transformation matrix between the two point clouds, using the transformation matrix to convert the subsequent point cloud into the coordinate system of the previous serial number point cloud for fusion, and downsampling the fused point cloud.

[0014] Further preferably, In step S4, separating each sub-point cloud from the complete point cloud based on the set clustering threshold includes: Filtering out the outliers in the complete point cloud; For each point in the point cloud, searching for its neighboring point set within the set clustering threshold to obtain each sub-point cloud.

[0015] Further preferably, The clustering threshold for each point is set according to the following formula: ; where, is the clustering threshold for the i-th point in the point cloud; is the maximum clustering threshold; is the shortest distance from the point where the maximum clustering threshold is searched to the construction boundary; is the shortest distance from the i-th point in the point cloud to the construction boundary.

[0016] Further preferably, In step S4, determining the coordinates of each sub-point cloud is to determine the depth, horizontal, and vertical coordinates of each sub-point cloud based on the coordinates of the points in each sub-point cloud in the constructed three-dimensional rectangular coordinate system.

[0017] Further preferably, Setting the depth x and horizontal y coordinates of each sub-point cloud includes: Projecting each separated sub-point cloud onto the plane to convert the three-dimensional point cloud into a two-dimensional contour; Finding the convex hull of each contour and taking the depth coordinate and horizontal coordinate of the geometric center of the convex hull as the depth coordinate and horizontal coordinate of the corresponding sub-point cloud.

[0018] Further preferably, The vertical coordinate of each sub-point cloud is set according to the following formula: ; where, is the vertical coordinate of the c-th sub-point cloud; j is an integer; is the j th largest vertical coordinate in the c-th sub-point cloud; is thej Small vertical coordinate.

[0019] On the other hand, the present invention discloses a construction facility positioning and classification system based on a construction facility positioning and classification method, including a point cloud acquisition module, a point cloud fusion module, a sub-point cloud separation module, a facility classification model construction module, a facility classification module, and an output module: The point cloud acquisition module arranges each lidar to acquire the construction environment point cloud and sort each point cloud; The point cloud fusion module takes the point cloud with the largest serial number as the subsequent point cloud, and fuses the subsequent point cloud into the previous serial number point cloud; takes the fused point cloud as the subsequent point cloud, and then fuses the subsequent point cloud into the previous serial number point cloud until the fused point cloud is sorted to 1 to obtain the complete point cloud; The sub-point cloud separation module separates each sub-point cloud from the complete point cloud based on a set clustering threshold; determines the coordinates of each sub-point cloud, and constructs a sub-point cloud data set; the clustering threshold is set based on the shortest distance from each point to the construction boundary; The facility classification model construction module constructs a facility classification model; trains the facility classification model based on the sub-point cloud data set; The facility classification module obtains the complete point cloud in real time, separates each sub-point cloud and the coordinates of each sub-point cloud, inputs each sub-point cloud into the trained facility classification model to classify the facilities in real time, and uses the coordinates of each sub-point cloud as the positioning of the corresponding facilities; The output module outputs the categories and positions of each facility in the construction environment.

[0020] On the other hand, the present application discloses an electronic device, including a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is used to operate according to the instructions to execute the construction facility positioning and classification method according to the foregoing.

[0021] The present application also discloses a computer-readable storage medium, on which a computer program is stored, characterized in that the program, when executed by a processor, implements the construction facility positioning and classification method.

[0022] The beneficial effect of the present invention is that, compared with the prior art: The positioning and classification method of the present invention can fuse the point clouds of multiple lidars with each other, and can still accurately position and classify construction facilities even when there is partial loss of point cloud data due to occlusion.

[0023] The present invention uses a clustering method to divide the original point cloud into independent small blocks representing single objects, and then sends them into the network for recognition, which maximally avoids inaccurate recognition types caused by the interference of point clouds in other positions. The present invention can have excellent construction facility positioning and classification effects under harsh working conditions and weather conditions.

[0024] The present invention also has prominent advantages such as accurate positioning, accurate recognition, and strong reliability. Brief Description of the Drawings

[0025] The drawings are used to provide a further understanding of the present application, and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation to the present application. In the drawings: Figure 1 is a schematic flowchart of a method for positioning and classifying construction facilities based on lidar according to the present application; Figure 2 is a schematic diagram of a simulation scenario; Figure 3 is a schematic diagram of a complete construction scene point cloud restored using the proposed method; Figure 4 is a schematic diagram of an evolution broken line of the model classification accuracy when training a facility classification model using the proposed method. Detailed Embodiments

[0026] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. The embodiments described in this application are only a part of the embodiments of the present invention, rather than all embodiments. Based on the spirit of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0027] This application discloses a method for positioning and classifying construction facilities based on lidar. Refer to the attached Figure 1 , including: S1: Arrange each lidar to obtain the construction environment point cloud and sort each point cloud; The construction environment point cloud is the point cloud obtained by the lidar scanning the area within the construction boundary; the construction boundary is the outer contour of the construction area; the construction environment includes the construction boundary, the facilities in the internal area of the boundary, the height of each facility from the ground, etc.

[0028] Each lidar is configured based on the construction environment, and the selected lidar needs to meet the constraints of the following formula: ; Wherein, represents the maximum detection distance of the lidar; Represents the maximum distance between a point on the construction boundary and the geometric center of the contour; Represents the maximum ground clearance of the facilities that need to be located and classified in the construction environment; Represents the horizontal viewing angle of the lidar; Represents the vertical viewing angle of the lidar.

[0029] Arrange the lidars according to the number of lidars determined based on the construction environment, the arrangement interval between adjacent two lidars, and the height of each lidar from the ground; The arrangement interval between adjacent two lidars Is shown as the following formula: ; Wherein, L Is the length of the construction boundary; n Is the number of lidars; The height of the lidar from the ground is set to 1.2 h Wherein, h Represents the maximum ground clearance of the facilities that need to be located and classified in the construction environment.

[0030] The number of lidars n Is shown as the following formula: ; Wherein, Represents the maximum distance between a point on the construction boundary and the geometric center of the contour; Represents rounding up.

[0031] Set the number of the lidar farthest from the geometric center of the construction boundary to 1, and the lidar numbers increase sequentially in the counterclockwise direction; The serial number of the construction environment point cloud is the same as the lidar number that collected the construction environment point cloud.

[0032] S2: Take the point cloud with the largest serial number as the subsequent point cloud, and fuse the subsequent point cloud into the point cloud with the previous serial number of the subsequent point cloud; the previous serial number of the subsequent point cloud, that is, the serial number is 1 less than the serial number of the subsequent point cloud; Fusing the subsequent point cloud into the point cloud with the previous serial number of the subsequent point cloud means transforming the subsequent point cloud to the coordinate system of its previous serial number point cloud and fusing; specifically, it is to perform matching and calibration on the subsequent point cloud and its previous serial number point cloud, obtain the transformation matrix and translation vector between the two point clouds, use the transformation matrix and translation vector to transform the subsequent point cloud to the previous serial number point cloud coordinate system for fusion, and downsample the fused point cloud.

[0033] S3: Take the fused point cloud as the subsequent point cloud, and fuse the subsequent point cloud into the point cloud with the previous serial number according to step S2 until the fused point cloud is sorted to 1 to obtain the complete point cloud; S4: Separate each sub - point cloud from the complete point cloud based on the set clustering threshold; determine the coordinates of each sub - point cloud and construct a sub - point cloud data set; the clustering threshold is set based on the shortest distance from each point to the construction boundary and the maximum clustering threshold. Those skilled in the art should know that separating each sub - point cloud from the complete point cloud obtained in step S3 based on the set clustering threshold can be achieved by methods such as adaptive segmentation based on PCA and KMeans, Euclidean distance - based clustering method, region - growing - based segmentation, density - feature - based clustering (such as DBSCAN), etc. In the preferred embodiment of the present invention, the Euclidean clustering method is used to achieve it.

[0034] Separating each sub - point cloud from the complete point cloud based on the set clustering threshold includes: Filter out the outliers in the complete point cloud; the filtering of outliers refers to identifying and removing the abnormal points that do not conform to the expected distribution or are significantly deviated from the main data in the point cloud data. Those skilled in the art should know how to achieve it.

[0035] For each point in the point cloud, search for its set of neighboring points within the set clustering threshold to obtain each sub - point cloud.

[0036] The clustering threshold of each point is set according to the following formula: ; where, is the clustering threshold of the i - th point in the point cloud; is the maximum clustering threshold; is the shortest distance from the point where the maximum clustering threshold is searched within a range of 1 cubic meter to the construction boundary; is the shortest distance from the i - th point in the point cloud to the construction boundary. The maximum clustering threshold is the maximum number of neighboring points obtained by searching for the set of neighboring points within 1 cubic meter for each point in the complete point cloud whose distance from the outer contour of the construction environment is less than 0.05 l ; where, is the maximum distance from the point on the outer contour of the construction environment to the geometric center of the contour.

[0037] Determining the coordinates of each sub - point cloud includes: Those skilled in the art should know that the point cloud obtained by lidar scanning comes with its own coordinates, which are based on the lidar's own coordinate system.

[0038] Based on the coordinates of the points in each sub - point cloud, determine the depth, horizontal, and vertical coordinates of each sub - point cloud in the constructed three - dimensional rectangular coordinate system.

[0039] Determine the depth of each sub - point cloud x and horizontal y coordinates include: Project each separated sub - point cloud onto A plane is used to convert the three-dimensional point cloud into a two-dimensional contour; The convex hull of each contour is obtained, and the depth coordinate and horizontal coordinate of the geometric center of the convex hull are used as the depth coordinate and horizontal coordinate of the corresponding sub-point cloud.

[0040] The vertical coordinates of each sub-point cloud are set according to the following formula: ; where, is the vertical coordinate of the c-th sub-point cloud; j is an integer, and j ∈[1, 50]; is the j -th largest vertical coordinate in the c-th sub-point cloud; is the j -th smallest vertical coordinate in the c-th sub-point cloud.

[0041] S5: Construct a facility classification model; Train the facility classification model based on the sub-point cloud dataset; S6: Obtain the complete point cloud in real time according to steps S1 - S4, separate each sub-point cloud and the coordinates of each sub-point cloud, input each sub-point cloud into the facility classification model trained in step S5 to classify the facilities in real time, and use the coordinates of each sub-point cloud as the positioning of the corresponding facilities.

[0042] Embodiment 1 An embodiment of the present application provides a method for positioning and classifying construction facilities based on lidar. The method for positioning and classifying construction facilities based on lidar of the present application will be described in detail below.

[0043] Step S101: Configure the model type, layout strategy and sorting of the lidar based on the construction environment, and use the lidar to obtain the construction environment point cloud and sort it according to the lidar number; wherein, using the lidar to obtain the construction environment point cloud and sort it according to the lidar number means obtaining the lidar point clouds simultaneously and sorting each lidar point cloud, and its serial number is the lidar number.

[0044] Determine the maximum ground clearance of the facilities that need to be positioned and classified in the construction environment and the maximum distance between the points on the construction boundary and the geometric center of the contour ; The construction boundary is the outer contour of the construction environment; Select the maximum detection distance of the lidar should be greater than , the horizontal viewing angle of the lidar should be greater than 90°; the vertical field of view should be greater than .

[0045] Arrange lidars counterclockwise on the contour line of the construction environment, at a height from the ground , set the boundary length , then the number of lidars to be prepared is , where represents rounding up; the interval distance between adjacent lidars is ; Set the lidar farthest from the geometric center of the construction boundary as the No. 1 lidar, and the lidar numbers increase sequentially counterclockwise.

[0046] Step S102: Take the point cloud with the largest serial number as the subsequent point cloud, take the point cloud with the previous serial number of the subsequent point cloud as the previous point cloud, obtain the transformation matrix between adjacent lidar point clouds by matching and calibrating the point clouds of two adjacent lidars of the subsequent point cloud and the previous point cloud, use the transformation matrix to transform the subsequent point cloud into the coordinate system of the previous point cloud and fuse them, and downsample the fused point cloud; Those skilled in the art should know that matching and calibrating point clouds is a process of aligning point cloud data collected from different sources (different lidars) or at different times to a unified coordinate system, which can be achieved by methods such as Normal Distribution Transform (NDT), Implicit Moving Least Squares - Iterative Closest Point (IMLS - ICP), etc. Those skilled in the art can match and calibrate point clouds according to the actual situation; the method of matching and calibrating point clouds proposed in the embodiments of the present invention is only a preferred embodiment and is not an inevitable limitation for implementing the present invention; the specific method of matching and calibrating point clouds is as follows: Perform registration on the point clouds of adjacent lidars. Specifically, for the point clouds of adjacent lidars and , where P is the subsequent point cloud in the point clouds of adjacent lidars, and Q is the previous point cloud in the point clouds of adjacent lidars; is the first point in the subsequent point cloud of the point clouds of adjacent lidars, presented in three - dimensional coordinate form as , where is the depth coordinate of the first point in the subsequent point cloud of the point clouds of adjacent lidars; is the horizontal coordinate of the first point in the subsequent point cloud of the point clouds of adjacent lidars; is the vertical coordinate of the first point in the subsequent point cloud of the point clouds of adjacent lidars; is the nth point in the subsequent point cloud of the point clouds of adjacent lidars, presented in three - dimensional coordinate form, similar to , not elaborated here; is the first point in the previous point cloud of the point clouds of adjacent lidars, presented in three - dimensional coordinate form; is the nth point in the previous point cloud among the point clouds of adjacent lidars, presented in three-dimensional coordinate form; n is the total number of points in the point cloud that can be collected by each lidar of this model at each moment.

[0047] For each point in the source point cloud, find the point in the adjacent lidar point cloud that is closest in distance.

[0048] Establish a set of point pair correspondences: Take the searched closest point pairs as corresponding point pairs to form a set of corresponding point pairs.

[0049] Hope to find the rotation matrix and the translation vector t such that satisfies ; where i is an integer and i belongs to [1, n]. Those skilled in the art should be able to select the required rotation matrix according to the actual situation. The rotation matrix used can be obtained through conversions such as Euler angles and axis-angle representations, which will not be elaborated here.

[0050] First, define the error term for the i-th pair of points: , construct a least squares problem, and find the that minimizes the sum of squared errors:

[0051] where f(x) represents returning the input R, t that minimizes the function f(x); represents calculating the Euclidean norm.

[0052] This problem is solved based on singular value decomposition (SVD). The specific steps are as follows: First, define the centroids of two sets of points. The centroid is the mean of the xyz coordinates of all points in the point cloud, as shown in the following formula:

[0053] where p is the centroid of the subsequent point cloud among the point clouds of adjacent lidars; q is the centroid of the previous point cloud among the point clouds of adjacent lidars.

[0054] Subsequently, make the following processing in the error function:

[0055] Note that the cross-term part is zero after summation, and the optimization objective function J can be simplified to:

[0056] where It means to solve for R and t that minimize the objective function ; So far, calculate the centroid - removed coordinates of each point: , where is the centroid coordinate of the first point in the subsequent point cloud of the adjacent lidar's point cloud; is the centroid coordinate of the first point in the previous point cloud of the adjacent lidar's point cloud.

[0057] Solve the optimized rotation matrix according to the following optimization problem:

[0058] where represents the optimized rotation matrix; f(x) represents returning the input R that minimizes the function f(x).

[0059] Expand the error term about to get:

[0060] Note that the first term is independent of , and the second term, due to , is also independent of . The actual optimization function becomes:

[0061] where represents the trace of the matrix; Next, use singular value decomposition (SVD) to solve for the optimal : Define the matrix , and perform SVD decomposition on to get , where U and V are both orthogonal matrices, denoted as the left - singular vector matrix and the right - singular vector matrix respectively; is a diagonal matrix, and the elements on its diagonal are called singular values. When is full - rank , according to the above solve for the translation vector ; The rotation matrix and the translation vector , the subsequent point cloud is transformed into the coordinate system of the previous point cloud, and the two groups of point clouds are superimposed, that is, the two point clouds are combined into a larger point cloud. Subsequently, the density of the superimposed point cloud is optimized through voxel filtering to restore it to a level comparable to that of the point cloud directly obtained from the lidar, thereby ensuring the consistency and comparability of the point cloud data. Among them, the subsequent point cloud is the point cloud with a later order among two adjacent point clouds in sorting, and the previous point cloud is the point cloud with an earlier order among two adjacent point clouds in sorting.

[0062] Step S103: The point cloud after the initial fusion is registered, downsampled, and fused forward again using the method in S102. Repeat this step until all point clouds are fused into the same coordinate system to construct a complete point cloud of the construction environment; For the initially fused lidar point cloud obtained in step S102, since the previous point cloud and the subsequent point cloud already contain a common part, on this basis, the rotation matrix of the subsequent point cloud to the previous point cloud is further obtained using the method in S102 and the translation vector , the subsequent point cloud is transformed into the coordinate system of the previous point cloud, and the two groups of point clouds are superimposed; Repeat the above steps until all lidar point clouds are restored to the coordinate system of the No. 1 lidar point cloud to complete the reconstruction task of the construction scene.

[0063] Step s104: Use the Euclidean clustering method to set different clustering thresholds according to the distance between the facilities and the lidar to obtain each sub-point cloud, and separate each sub-point cloud from the complete point cloud; Each sub-point cloud can be regarded as a facility.

[0064] Generally, the construction facilities are independent of each other. Therefore, in the point cloud, the points of each facility are also independent of the points of other facilities. Based on this characteristic, the Euclidean clustering algorithm can be used to process the point cloud to effectively separate the points corresponding to different construction facilities, so as to realize the independent identification and extraction of each facility. To achieve a good clustering effect, it is first necessary to filter out the outlier points, that is, filter out the outlier points in the point cloud.

[0065] Those skilled in the art should know that the outlier points can be filtered out using methods such as outlier detection of multivariate Gaussian distribution and LOF algorithm, and those skilled in the art can choose according to the actual situation; in order to improve the calculation accuracy, the method for filtering out outlier points proposed in the embodiment of the present invention is only a preferred embodiment, and is not an inevitable limitation for implementing a method for positioning and classifying construction facilities based on lidar of the present invention. According to the characteristics of the outlier points, a method for filtering out the outlier points is constructed; Preferably, outlier detection of multivariate Gaussian distribution can be used: Assume there is point cloud data , where Represents the first point in the point cloud data; represents the second point in the point cloud data; represents the nth point in the point cloud data; each point is represented in the form of three-dimensional coordinates as follows:

[0066] Among them, the ith point in the point cloud data Is represented as the three-dimensional coordinates of the point, Represents the depth coordinate of the ith point in the point cloud data; Represents the horizontal coordinate of the ith point in the point cloud data; Represents the vertical coordinate of the ith point in the point cloud data.

[0067] The three-dimensional mean vector of the point cloud Is:

[0068] Among them, Represents the mean of the depth coordinates of the point cloud; Represents the mean of the horizontal coordinates of the point cloud; Represents the mean of the vertical coordinates of the point cloud.

[0069] Of the point set Covariance matrix Is:

[0070] Among them, Represents calculating the covariance of variables a and b. In the present invention, both a and b can represent x / y / z, , when a / b is x, / Represents the depth coordinate of the ith point in the point cloud data; / Represents the mean of the depth coordinates of the point cloud; when a / b is y, / Represents the horizontal coordinate of the ith point in the point cloud data; / Represents the mean of the horizontal coordinates of the point cloud; when a / b is z, / Represents the vertical coordinate of the ith point in the point cloud data; / Represents the mean of the vertical coordinates of the point cloud.

[0071] For any data point in the point cloud The probability can be calculated To determine whether it is an outlier:

[0072] Due to the imaging characteristics of the lidar, the density of the point cloud decreases as the distance from the origin of the lidar coordinate system increases. Therefore, in the reconstructed point cloud, the closer to the center of the construction scene, the lower the point cloud density; while the closer to the edge of the construction scene, the higher the point cloud density. To achieve the precise execution of Euclidean clustering, it is necessary to dynamically set an adaptive threshold according to the distribution characteristics of the point cloud at different positions: the point cloud density of the lidar generally approximately satisfies the inverse square law , where is the point cloud density, is the distance between this point and the lidar, is the proportionality coefficient to be estimated; randomly select 5 positions from the complete point cloud, measure the point cloud density at these 5 positions respectively, and use the least squares method to obtain the proportionality coefficient K, and determine the relationship between the point cloud density and the distance between the point and the lidar (the outer contour of the construction environment); For each point in the point cloud, search for its set of neighboring points within the set clustering threshold; Set the feasible Euclidean clustering threshold for the points near the edge contour of the construction environment , The value selected is related to the lidar used and can be set by itself. In the present invention, it is preferably the maximum number of point clouds generated by the used lidar in the unit space size (1 cubic meter). The points near the edge contour of the construction environment that can search for the maximum number of point clouds are generally located 1 m away from the lidar. Assume that the shortest distance from the aforementioned points near the edge contour of the construction environment to the construction contour is , then = 1; preferably, the points near the edge contour of the construction environment can be defined as any points with a distance from the edge less than 0.05 l ; represents the maximum distance between the points on the construction boundary and the geometric center of the contour; Since there is a linear relationship between the point cloud density and the Euclidean clustering threshold, then for any point in the point cloud with the shortest distance from the construction edge, its Euclidean clustering threshold is:

[0073] Step s105: Project each separated sub-point cloud onto the plane, convert the three-dimensional point cloud into a two-dimensional contour, obtain the convex hull of the contour, calculate the geometric center coordinates of the convex hull as the XY coordinates of the facility, and determine the Z-axis coordinate of the facility by calculating the average value of the first 50 points of the maximum Z-axis coordinate and the first 50 points of the minimum Z-axis coordinate in the point cloud; Those skilled in the art should know that calculating the convex hull of the contour is to calculate the smallest convex polygon that can completely contain all points from a given point set, which can be implemented by Graham scan method, QuickHull algorithm, Andrew algorithm, etc. Those skilled in the art can construct it according to the actual situation. The method for calculating the convex hull of the contour proposed in the embodiment of the present invention is only a preferred embodiment and is not an inevitable limitation for implementing the present invention. The specific implementation manner of the present invention is as follows: It is assumed that a total of m sub-point clouds are obtained. Let c represent any sub-point cloud, c ∈ [1, m], and c is an integer; Perform a projection transformation on the point cloud separated by Euclidean clustering in step s104, and project all points in the point cloud onto a plane to convert the three-dimensional point cloud data into a two-dimensional contour; Use the Graham scan method to solve the convex hull vertices for the contour. First, find the lowest point P in the plane, that is, the point with the smallest ordinate. If there are multiple points with the same ordinate at the bottom, select the leftmost one, that is, select the point with the smallest abscissa, and use this point as the starting point. Establish a polar coordinate system based on the starting point, use the starting point as the origin of the polar coordinate system, and the polar axis is in the same direction as the X-axis. Then sort all points in ascending order of the polar angle relative to the starting point P. Finally, establish a stack to store the current convex hull. According to the sorting result obtained by the polar angle, add points to the stack in turn. If the point being considered and the two points at the top of the stack do not turn left (that is, the point at the top of the stack is not in the counterclockwise direction of its adjacent points in the stack), it means that the current point at the top of the stack is not on the convex hull, and we need to pop it out of the stack. Repeat this process until the point being considered and the two points at the top of the stack turn left; According to the convex hull vertices obtained in the previous step, uniformly insert new points into the connections between adjacent convex hull vertices. The density of the new points should be consistent with the density of the original point cloud. Assume that the point set composed of the convex hull vertices and all inserted points is , and let the center point of the convex hull be , then there is:

[0074] So far, the XY coordinates of the facility can be given by the center point of this convex hull; To solve the Z coordinate of the facility, the point cloud after removing outliers and clustering needs to be used. Assume that the Z coordinates of the 50 points with the largest Z coordinates in the c-th sub-point cloud are { , ……, }, where is the largest vertical coordinate in the c-th sub-point cloud; is the 50th largest vertical coordinate in the c-th sub-point cloud; the Z coordinates of the 50 points with the smallest Z coordinates in the point cloud are { , ……, }, where is the smallest vertical coordinate in the c-th sub-point cloud; is the 50th smallest vertical coordinate in the c-th sub-point cloud; then the Z-axis coordinate of the current facility is: ; where is the vertical coordinate of the c-th sub-point cloud; j is an integer, and j ∈[1, 50]; is the j th largest vertical coordinate in the c-th sub-point cloud; is the j th smallest vertical coordinate in the c-th sub-point cloud.

[0075] Step S106: Construct a sub-point cloud data set according to the possible occlusion situations of different types of construction facilities, and train a lightweight PointNet network model based on this data set; Obtain the complete sub-point cloud corresponding to the construction facility to be located and classified, and perform corresponding defect processing on the complete sub-point cloud according to the possible occlusion situations of the facility. For example, when a certain sub-point cloud has no front part or the front part is incomplete, perform defect processing such as removing the front or digging out the front of the sub-point cloud. When a certain sub-point cloud has corner defects, perform defect processing such as digging out the corners of the sub-point cloud, etc.; then, perform random stretching and rotation transformations on the point cloud, and add a small amount of Gaussian noise to the transformed point cloud to construct a sub-point cloud data set; Set labels for each sub-point cloud in the data set; Those skilled in the art should know how to set the labels of the sub-point clouds according to the actual situation; the method for setting the sub-point cloud labels proposed in the embodiments of the present invention is only a preferred embodiment, and is not an inevitable limitation for implementing a method for positioning and classifying construction facilities based on lidar of the present invention. The specific sub-point cloud labels can be selected to represent an oil barrel with 0, a generator with 1, a cement bag with 2, and so on.

[0076] Use this point cloud data set to train a lightweight PoinNet network. When the classification accuracy of the network is higher than 98% and the training epoch is greater than 100, save the current checkpoint of the network for subsequent real-time classification using the network; S107: Obtain the complete point cloud in real time according to Steps S101 - S105, separate each sub-point cloud and the coordinates of each sub-point cloud, input each sub-point cloud into the facility classification model trained in Step S106 to perform real-time classification on the facility, and use the coordinates of each sub-point cloud as the positioning of the corresponding facility.

[0077] In the actual working condition, the lightweight PointNet network loads the above checkpoint, sends the clustered point cloud in step s104 into the network according to the dataset format, and the network gives the specific category of the current point cloud.

[0078] The entire system finally outputs the specific positions and categories of various facilities in the construction environment.

[0079] Embodiment 2 A method for positioning and classifying construction facilities based on lidar.

[0080] Exemplarily, as Figure 2 is a simulated construction scene, and 8 lidars are placed on the outer contour of the construction scene: When the point cloud directly obtained by the lidar is directly displayed in the coordinate system where the No. 1 lidar is located, since the point cloud is not registered and the point clouds of each lidar do not coincide, a complete three-dimensional construction scene cannot be restored.

[0081] Refer to Figure 3 , Figure 3 shows the complete construction environment accurately restored after registering the point cloud using the proposed method.

[0082] Refer to Table 1. Table 1 shows the specific data (unit: meter) of the theoretical values of the coordinates of the points on the facilities and the measured values obtained from the registered point cloud in the complete construction scene restored using the proposed method. It can be seen that the difference between the theoretical value and the measured value is small, which proves the effectiveness of the proposed method.

[0083] Table 1 Comparison table of the theoretical coordinates and actual coordinates of the points on the construction facilities restored using the proposed method

[0084] Refer to Table 2. Table 2 shows the specific data (unit: meter) of the theoretical values of the coordinates of the construction facilities and the measured values calculated using the proposed positioning method in positioning the construction facilities in the point cloud. It can be seen that the difference between the theoretical value and the measured value is small, which proves the effectiveness of the proposed method.

[0085] Table 2 Comparison table of the coordinates of the construction facilities calculated using the proposed method and the theoretical coordinates

[0086] Refer to Figure 4 , Figure 4To construct a point cloud classification dataset using the proposed method, the dataset is divided into a training set and a test set in a ratio of 7:3. The classification accuracy evolution curve of the network obtained by training a lightweight PointNet network using this dataset on the test set shows that the classification accuracy of the network is close to 100% after 50 epochs of training, and it can correctly classify the point clouds of different types of construction facilities, proving the effectiveness of the proposed method.

[0087] This application also discloses a construction facility positioning and classification system based on a construction facility positioning and classification method, including a point cloud acquisition module, a point cloud fusion module, a sub-point cloud separation module, a facility classification model construction module, a facility classification module, and an output module: The point cloud acquisition module arranges each lidar to acquire the construction environment point cloud and sort each point cloud; The point cloud fusion module takes the point cloud with the largest serial number as the subsequent point cloud, and fuses the subsequent point cloud into the previous serial number point cloud; takes the fused point cloud as the subsequent point cloud, and then fuses the subsequent point cloud into the previous serial number point cloud until the fused point cloud is sorted as 1 to obtain the complete point cloud; The sub-point cloud separation module separates each sub-point cloud from the complete point cloud based on a set clustering threshold; determines the coordinates of each sub-point cloud, and constructs a sub-point cloud dataset; the clustering threshold is set based on the shortest distance from each point to the construction boundary; The facility classification model construction module constructs a facility classification model; trains the facility classification model based on the sub-point cloud dataset; The facility classification module obtains the complete point cloud in real time, separates each sub-point cloud and the coordinates of each sub-point cloud, inputs each sub-point cloud into the trained facility classification model to classify the facilities in real time, and takes the coordinates of each sub-point cloud as the positioning of the corresponding facility; The output module outputs the categories and positions of each facility in the construction environment.

[0088] This disclosure may be a system, method, and / or computer program product. The computer program product may include a computer-readable storage medium having thereon computer-readable program instructions for causing a processor to implement various aspects of this disclosure.

[0089] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example—but not limited to—an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device, such as a punched card or raised structures in grooves having instructions stored thereon, and any suitable combination of the foregoing. The computer-readable storage medium as used herein is not construed as being a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated through a waveguide or other transmission medium (e.g., an optical pulse through an optical fiber cable), or an electrical signal transmitted through a wire.

[0090] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to respective computing / processing devices, or can be downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include a copper transmission cable, an optical fiber transmission, a wireless transmission, a router, a firewall, a switch, a gateway computer, and / or an edge server. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing / processing device.

[0091] Computer program instructions for performing the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine - related instructions, microcode, firmware instructions, state - setting data, or source code or object code written in any combination of one or more programming languages, including object - oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer - readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a stand - alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, by using the state information of the computer - readable program instructions to customize an electronic circuit, such as a programmable logic circuit, a field - programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can execute the computer - readable program instructions to implement various aspects of this disclosure.

[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: still, modifications or equivalent replacements can be made to the specific embodiments of the present invention, and any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the protection scope of the claims of the present invention.

Claims

1. A method for positioning and classifying construction facilities based on lidar, characterized in that, Including: S1: Arrange each lidar to obtain the construction environment point cloud and sort each point cloud; S2: Take the point cloud with the largest serial number as the subsequent point cloud, and fuse the subsequent point cloud into the previous serial number point cloud; S3: Take the fused point cloud as the subsequent point cloud, and fuse the subsequent point cloud into the previous serial number point cloud according to step S2 until the fused point cloud is sorted as 1 to obtain the complete point cloud; S4: Separate each sub-point cloud from the complete point cloud based on the set clustering threshold; determine the coordinates of each sub-point cloud, and construct a sub-point cloud data set; the clustering threshold is set based on the shortest distance from each point to the construction boundary; S5: Construct a facility classification model; train the facility classification model based on the sub-point cloud data set; S6: Obtain the complete point cloud in real time according to steps S1 - S4, separate each sub-point cloud and the coordinates of each sub-point cloud, input each sub-point cloud into the facility classification model trained in step S5 to classify the facilities in real time, and use the coordinates of each sub-point cloud as the positioning of the corresponding facilities.

2. The construction facility positioning and classification method according to claim 1, characterized in that: In step S1, each lidar is configured based on the construction environment, as shown in the following formula: ； Among them, represents the maximum detection distance of the lidar; represents the maximum distance between a point on the construction boundary and the geometric center of the contour; represents the maximum ground clearance of the facilities to be located and classified in the construction environment; represents the horizontal viewing angle of the lidar; represents the vertical viewing angle of the lidar.

3. The construction facility positioning and classification method according to claim 1, characterized in that: In step S1, arrange the lidars by using the number of lidars determined based on the construction environment, the layout interval between adjacent two lidars, and the height of each lidar from the ground; The layout interval between two adjacent lidars is as follows: ； Among them, L is the construction boundary length; n is the number of lidars; The height of the lidar from the ground is set to 1.2 h , where h represents the maximum height from the ground of the facilities that need to be located and classified in the construction environment.

4. The construction facility positioning and classification method according to claim 3, characterized in that: The number of lidars n As shown in the following formula: ； Among them, represents the maximum distance between the points on the construction boundary and the geometric center of the contour; represents rounding up.

5. The construction facility positioning and classification method according to claim 1, characterized in that: In step S1, set the lidar with the farthest distance from the geometric center of the construction boundary as No. 1, and the lidar numbers increase sequentially counterclockwise; The serial number of the construction environment point cloud is the same as the lidar number.

6. The construction facility positioning and classification method according to claim 1, characterized in that: In step S2, when fusing the subsequent point cloud into the previous serial number point cloud, it is to perform matching and calibration on the subsequent point cloud and its previous serial number point cloud, obtain the transformation matrix between the two point clouds, use the transformation matrix to transform the subsequent point cloud into the previous serial number point cloud coordinate system for fusion, and downsample the fused point cloud.

7. The construction facility positioning and classification method according to claim 1, characterized in that: In step S4, separating each sub-point cloud from the complete point cloud based on the set clustering threshold includes: Filter out the outliers in the complete point cloud; For each point in the point cloud, search for its neighboring point set within the set clustering threshold to obtain each sub-point cloud.

8. The construction facility positioning and classification method according to claim 1 or 7, characterized in that: The clustering threshold of each point is set according to the following formula: ； Among them, is the clustering threshold of the i-th point in the point cloud; is the maximum clustering threshold; is the shortest distance from the point where the maximum clustering threshold is obtained by searching to the construction boundary; is the shortest distance from the i-th point in the point cloud to the construction boundary.

9. The construction facility positioning and classification method according to claim 1, characterized in that: In step S4, determining the coordinates of each sub-point cloud is to determine the depth, horizontal and vertical coordinates of each sub-point cloud based on the coordinates of the points in each sub-point cloud in the constructed three-dimensional rectangular coordinate system.

10. The construction facility positioning and classification method according to claim 9, characterized in that: Determine the depth of each sub-point cloud x and horizontal y coordinates include: Project the separated sub-point clouds onto a plane to convert the three-dimensional point cloud into a two-dimensional contour; Find the convex hulls of each contour, and use the depth coordinate and horizontal coordinate of the geometric center of the convex hull as the depth coordinate and horizontal coordinate of the corresponding sub-point cloud.

11. The construction facility positioning and classification method according to claim 9, characterized in that: The vertical coordinates of each sub-point cloud are set according to the following formula: ； Among them, is the vertical coordinate of the c-th sub-point cloud; j is an integer; is the j largest vertical coordinate in the c-th sub-point cloud; is the j smallest vertical coordinate in the c-th sub-point cloud.

12. A construction facility positioning and classification system using the construction facility positioning and classification method according to any one of claims 1-11, characterized in that, It includes a point cloud acquisition module, a point cloud fusion module, a sub-point cloud separation module, a facility classification model construction module, a facility classification module, and an output module: The point cloud acquisition module arranges each lidar to acquire the construction environment point cloud and sort each point cloud; The point cloud fusion module uses the point cloud with the largest serial number as the subsequent point cloud, and fuses the subsequent point cloud into the previous serial number point cloud; uses the fused point cloud as the subsequent point cloud, and then fuses the subsequent point cloud into the previous serial number point cloud until the fused point cloud is sorted as 1 to obtain the complete point cloud; The sub-point cloud separation module separates each sub-point cloud from the complete point cloud based on the set clustering threshold; determines the coordinates of each sub-point cloud and constructs a sub-point cloud data set; the clustering threshold is set based on the shortest distance from each point to the construction boundary; The facility classification model construction module constructs a facility classification model; trains the facility classification model based on the sub-point cloud data set; The facility classification module obtains the complete point cloud in real time, separates each sub-point cloud and the coordinates of each sub-point cloud, inputs each sub-point cloud into the trained facility classification model to classify the facilities in real time, and uses the coordinates of each sub-point cloud as the positioning of the corresponding facility; The output module outputs the categories and positions of each facility in the construction environment.

13. A terminal, comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is used to operate according to the instructions to execute the steps of the construction facility positioning and classification method according to any one of claims 1-11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it realizes the steps of the construction facility positioning and classification method according to any one of claims 1-11.

Images